In our new book ‘What Does This Look Like in the Classroom?’ we interviewed Dylan Wiliam on how to implement research on assessment in the classroom.
A central problem in the area of assessment in the classroom has been in the way we often confuse marking and feedback. As Dylan Wiliam points out in our discussion, there is an extraordinary amount of energy expended by teachers on marking and often very little to show for it in the way of student benefit. Although feedback is one of the most effective drivers of learning, one of the more surprising findings is that a lot of it actually has a negative effect on student achievement.
A set of marked books is traditionally seen as an effective proxy for good teaching but there is a lot of evidence to say that this might not always be the case. This problem is on a scale that might surprise a lot of people:
Dylan: I once estimated that, if you price teacher’s time appropriately, in England we spend about two and a half billion pounds a year on feedback and it has almost no effect on student achievement.
Certainly students need to know where they make misconceptions or spelling errors and correcting those is important. Doing so also provides a useful diagnostic for teachers to inform what they will teach next, but the written comments at the end of a piece of work are often both the most time-consuming and also the most ineffective. For example, taking the following typical comments on a GCSE English essay:
- Try to phrase your analysis of language using more sophisticated vocabulary and phrasing.
- Try to expand on your points with more complex analysis of Macbeth’s character.
This is a good example of certain assessment practices where the feedback mainly focuses on what was deficient about it, which as Douglas Reeve’s notes, is “more like a post-mortem than a medical.” The other thing is that it doesn’t really tell the student what they need to do to improve. What is more useful to the student here? receiving vague comments like these or actually seeing sophisticated vocabulary, phrasing and analysis in action? It’s very difficult to be excellent if you don’t know what excellent looks like.
Often, teachers give both a grade and comments like those above to students, hoping that they somehow improve by the time their next piece of writing comes around a week later and then berate the student when, lo and behold, they make the same mistakes again. Perhaps part of the problem here is that we have very low expectations of what students are willing to do in response to a piece of work and do not afford them the opportunity to engage in the kind of tasks that might really improve their learning.
To address this problem, Dylan advocates a much more streamlined model of marking that is not only more manageable for teachers, but also allows students to have more ownership over the process:
Dylan: I recommend what I call ‘four quarters marking.’ I think that teachers should mark in detail, 25% of what students do, should skim another 25%, students should then self-assess about 25% with teachers monitoring the quality of that and finally, peer assessment should be the other 25%. It’s a sort of balanced diet of different kinds of marking and assessment.
Dylan Wiliam’s Four Quarters Marking (Oliver Caviglioli)
After producing a piece of work, instead of using abstract skills based success criteria, it is probably more powerful for students to have access to a bank of exemplar essays or worked solutions to see concrete examples of success against which to self-assess their own work. Marking everything in sight and leaving detailed comments is an established cultural norm now but this practice doesn’t appear to be based on good evidence. We know for example that many students will look at a grade and not engage with the feedback but is that feedback always useful anyway?
As we discuss in the book, a common issue we see again and again in using research in the classroom is the ‘Chinese whisper effect’ where by the time evidence works its way down to the level of the classroom, it’s a pale imitation of its original form. This is especially prevalent in the area of marking where convoluted policies such as triple marking are enacted as a means of raising pupil achievement whereas all they are doing is often increasing teacher workload. As Dylan Wiliam reminds us, “feedback should be more work for the recipient than the donor,” but how do you change a culture that has traditionally been the opposite?
Dylan: In terms of what we do about this, I would say first of all, headteachers should lay down clear expectation to parents and say things like, “We are not going to give detailed feedback on more than 25% of what your child does. The reason for that is not because we’re lazy. It’s because there are better uses we could make of that time. We could mark everything your child does, but that would lead to lower quality teaching and then your child will learn less.” Heads have to establish those cultural norms. If a teacher is marking everything your child does, it’s bad teaching. It is using time in a way that does not have the greatest benefit for students.
As a profession, we are too some extent, we are our own worst enemy. Using marking policies that have little impact on student achievement and a negative impact on teacher workload and morale makes little sense. By adopting an approach like four quarters marking, we might go some way to address this issue and at the same time, give students more ownership over their own learning.
‘What Does This Look Like in the Classroom?’ is out later this month.
On an indecently hot day in Texas, professor Jerry B. Harvey was visiting his wife’s family when his father-in-law suggested they visit a new restaurant in the town of Abilene to which his wife exclaimed “sounds like a great idea.” Harvey had reservations about this however, as a 53 mile trip in a car with no air-conditioning sounded terrible to him, but not wanting to rock the boat he also proclaimed this a good idea and asked his mother in law if she wanted to go. As she was now the only one in the group who had not yet expressed agreement with this “great idea,” she also said they should go, and so they began their journey to Abilene. However, as Harvey explains, the trip was not a success:
My predictions were fulfilled. The heat was brutal. Perspiration had cemented a fine layer of dust to our skin by the time we had arrived. The cafeteria’s food could serve as a first-rate prop in an antacid commercial.
Some four hours and 106 miles later, we returned to Coleman, hot and exhausted. We silently sat in front of the fan for a long time. Then to be sociable, I dishonestly said, “It was a great trip wasn’t it?”
No one spoke.
After a while, his mother-in-law admitted that she never really wanted to go but only did so because she thought everyone else wanted to and didn’t want to cause a fuss, to which his wife also protested that she never really wanted to go either which then lead to a volley of argument. Eventually his father in law broke the silence and exclaimed in a long Texas drawl: “Shee-it. Listen, I never wanted to go to Abilene. I just thought you might be bored. You visit so seldom I wanted to be sure you enjoyed it. I would have preferred to play another game of dominoes and eat the leftovers in the icebox.” This experience led to Harvey coining the term ‘the Abilene paradox’ to explain a curious aspect of group dynamics in which the opposite of what everyone wants is tacitly created by the group who thinks they are agreeing with what everyone else wants.
After the outburst of recrimination we all sat back in silence. Here we were, four reasonably sensible people who, of our own volition, had just taken a 106-mile trip across a godforsaken desert in a furnace-like temperature through a cloud-like dust storm to eat unpalatable food at a hole-in-the-wall cafeteria in Abilene, when none of us had really wanted to go. In fact, to be more accurate, we’d done just the opposite of what we wanted to do. The whole situation simply didn’t make sense.
The Abeline paradox lies in the fact that we have problems not with disagreement, but rather with agreement. It is characterised by groups of people in an organisation privately agreeing that one course of action makes sense but failing to properly communicate those ideas and then collectively stumbling to what they think is the right course of action or what everyone else wants. Eventually an inaccurate picture of what to do emerges and based on that, the organisation takes steps towards actions that nobody really wants and which is ultimately counterproductive to the aims of the organisation itself.
You can witness the Abilene paradox at work in many schools. Often this takes the form of ill-considered marking policies which increase teacher workload to the point of exhaustion, endless tracking and monitoring of students, behaviour policies which punish the teacher more than the student who misbehaves, and graded lesson observations where teachers abandon what they normally do to put on a one-off, all singing, all dancing lesson for the observer, because that’s what everyone thinks that’s what inspectors want.
A lot of this can be accounted for by innate cognitive biases such as groupthink but it can also be exacerbated by either poor evidence, as in the case of learning styles or a poor understanding and misappropriation of good evidence as in the case of formative assessment. With the emergence of a solid evidence base, we might just be able to defend ourselves from these kind of cognitive biases if they are communicated clearly and appropriated effectively as part of a broader discussion about the values of a school. At it’s best, good evidence can act as a bulwark against the tsunami of nonsense that has so often washed over our schools. If we fail to have these important discussions and simply go with what we think might work, then we are at risk of loading the entire staff onto the school mini-bus and heading off to Abilene.
This is an excerpt taken from the forthcoming book ‘What Does This Look Like in the Classroom? Bridging the Gap Between Research and Practice