We’ve lost Sight of the Point of Testing

Testing in its current form is a relatively new phenomenon. In the Ancient World, Socrates would ‘test’ his students through a dialogue where there were no ‘correct’ responses but simply more questions and answers. The Socratic tradition of dialogue was largely continued in Europe with students being tested through oral responses and then essays until around 150 years where we begin to see the notion of testing as something that can be standardised in a uniform fashion.

In the 19th century, the “father of American public education” Horace Mann advocated testing to provide “objective information about the quality of teaching and learning in urban schools, monitor the quality of instruction, and compare schools and teachers within each school.” At the beginning of the 20th century psychologist Alfred Binet developed a standardized test of intelligence, which would eventually become the standard IQ test we know today.

There is no question that a standardised measure of assessment holds great value in terms of being able to compare students and schools with on a national level and to flag up underachieving groups but there is a clear sense at the moment that standardised testing has become something else, that the tail is now wagging the dog and that the model may be in need of reform, something brought to national attention in the US not only by mass boycotting of tests but also by John Oliver on national TV:

For me there are several issues that focus not so much on testing itself but on the collateral damage caused by testing and how they’re being used far beyond their intended purpose:

1. Tests are no longer part of a judgement, they now are the judgement.

One of the proposed benefits of testing in the 19th century was that they would provide a diagnostic indicator of student progress and school efficacy that would inform a more wider, more balanced and measured judgement. However what we have now is a system where test scores are the judgement – it’s not so much that the tail is wagging the dog as the tail now is the dog. Referring to Horace Mann’s statewide roll-out of standardised tests in the mid 1800s, a US congress report from 1992 notes that:

It is important to point out what “standardization” meant in those days. It did not mean “norm-referenced” but rather that “. . . the tests were published, that directions were given for administration, that the exam could be answered in consistent and easily graded ways, and that there would be instructions on the interpretation of results. ’

The other issue is that the current model of testing risks exceeding its own mandate by being used mainly for school reform and teacher evaluation as opposed to pupil evaluation. Many opponents of standardised testing claim that they may not be fit for purpose on their own terms. According to James Popham standardised tests are like trying to “measure temperature with a tablespoon”

“Tablespoons have a different measurement mission than indicating how hot or cold something is. Standardized achievement tests have a different measurement mission than indicating how good or bad a school is. Standardized achievement tests should be used to make the comparative interpretations that they were intended to provide. They should not be used to judge educational quality.”

Combine this with an inspectorate that is data-obsessed and you have a perfect storm as this cautionary tale from Geoff Barton illustrates.

2. Standardised tests are really just a measure of who has access to specific resources

Does standardised testing evaluate a pupil’s aptitude or general knowledge or does it simply register their access to a particular set of resources? Resources here can mean a wide range of things such as the school, a private tutor, parental support but when a particular set of textbooks define success then there is an issue. This is well illustrated by the case of the Pennsylvania System of School Assessment (PSSA) where Meredith Broussard discovered that success in her daughter’s 3rd grade test was inextricably linked to specific textbooks:

“Standardized tests are not based on general knowledge. As I learned in the course of my investigation, they are based on specific knowledge contained in specific sets of books: the textbooks created by the test makers.”

Regardless of all the other inequities, the cost of these textbooks are prohibitive in many districts so the inevitable outcome is that schools in deprived areas simply cannot ‘win’ in standardised texts. Interestingly, in the 1960s the Civil Rights movement protested against standardised testing as it inevitably punished those from a certain social strata. The Coleman Report, found that a student’s home environment was the deciding factor in determining achievement. (Rumberger & Palardy, 2005) In a culture of high stakes accountability the losers will inevitably be the ones without access to the best resources.

2. Linking teacher accountability to student test scores raises difficult ethical questions

The logical end point of a high stakes system of accountability where teachers are judged on their students’ scores will be the occurrence of dubious ethical practices somewhere along the line, whether that be on a small scale with teachers ‘correcting’ a student’s coursework or on the more extreme end with institutional malpractice. Earlier this year in Atlanta eight educators were found guilty of organising a “criminal enterprise” under the Racketeer Influenced and Corrupt Organization (RICO) act for manipulating student scores on Georgia’s state standardized tests. Now I’m not for a second suggesting that they did this as a result of standardised testing, but the case raises some important issues around the kinds of pressure being put on teachers at the moment. What kind of a system makes people risk years in jail to improve student test scores?

Across the US right now there is a fairly widespread movement of civil disobedience with parents intervening in their children’s education and opting out of standardised tests. Whilst the moral and ethical dimensions of these decisions are unclear it is evident that the current fetish for testing has engendered a string of unintended consequences.

3. “But will this be in the test?”

Probably the most dispiriting thing a teacher can hear. We all want students to ‘achieve’ academically but should that be at the expense of intellectual curiosity and the ephemeral joy of learning that is often immeasurable? Now they are not mutually exclusive of course but when the outcomes for one far outweigh the other then something has to give and often that is the autonomy of the teacher to be able to go ‘off piste’ and follow a particular conversation or idea perhaps not directly related to the test. Teachers want their students to score well in tests but what about another measure of ‘success’ – what about the English teacher who has engendered a lifelong love of reading in a pupil? or the Physics teacher who has sparked a student’s curiosity about cosmic universe? or the languages teacher who has opened a student’s eyes to the values and customs of another country they now want to visit? Many of these things are not testable and are being subsumed by a focus on what is prescribed by an exam board and the techniques needed to be ‘successful’ in them. Harvard Professor Daniel Koretz notes that “If you impose a simplistic numerical measure and lose sight of the other important goals of the institution, then the other goals get short shrift.”

4. College dropout rates suggest that something is wrong 

In the UK, figures published by the Higher Education Statistics Agency indicated that over 32,000 students dropped out of university after a year of study in 2012/13. Of those 7,420 transferred to another university, while 24,745 dropped out of higher education altogether. In the US things the picture is even more bleak where there is the “lowest college completion rate in the developed world” (Organization for Economic Cooperation and Development.)

There seems to be a lot of blame attached to the universities themselves, but what if pre-university education is simply not preparing students for the intellectual rigour, criticality and independence needed at that level?  Are students effectively being herded through a set of tests to provide data that benefits policy makers and Ofsted rather than the actual pupils themselves? Is focusing on the ability to use a broad base of knowledge to think critically being sidelined for the narrow measure of how to pass an exam?

The knock-on effect of a high-stakes testing system with increased accountability will be the limiting of both teacher and pupil agency. Apart from the impact on student mental health and stress levels, teachers are increasingly being asked to teach how to pass an exam as opposed to impart knowledge and elicit dialogue. Schools are systems of deep uncertainty and flux in which teachers are often held accountable for the unaccountable.

Part of the solution has to be to move from a high stakes, Russian Roulette, sudden-death style system to one where pupils can be evaluated on their progress in a series of low stakes, non-threatening tests that foster not only an appreciation of knowledge for knowledge’s sake, but also the ability to think critically and to be able to embrace uncertainty. All of which will prepare them for the ‘tests’ they have ahead of them.


Why Poor Schools Can’t Win at Standardized Testing

Testing in American Schools: Asking the Right Questions. [Full Report.] 

17 comments

  1. teachwell

    The other half is a lecturer and it is clear that such an intense testing regime also goes hand in hand with other ill effects on students ability to be independent and take responsibility for their learning. The shift is real, when he (the other half that is) has to do extra tutorials for scared 1st years who are frightened of the kind of exams they have to take at university. This is a marked shift from my time at school – I don’t remember there being any massive difference except for the level of the content and course obviously.

    At his university lecturers have been told to make sure that everyone who deserves a 2:1 is a getting a 2:1 – very much like the graders in the video – they need to be able to ‘see’ 1st’s and 2:1’s more clearly. The number of firsts and 2:1’s has risen as universities are now ‘responsible’ for the grade that the students (their consumers) get. In addition, the pandering to students and worrying about how they will ‘evaluate’ them has infected universities too – so just like primary and secondary teachers, they too can be expected to dance like a monkey and entertain their students.

    The way that ‘accountability’ now means ‘responsibility’ for teachers and educators means that real learning will inevitably take a back seat to the tests that children sit. Take primary education. The way Year 2’s, but especially, Year 6’s are hothoused to do their tests is hardly the way forward. It also skews the system of support in schools so that Year 3, 4, 5 and to a lesser extent, Year 1 classes have less TA’s , fewer interventions, less support in general to close the gap. This means a perpetual state of crisis management in the SATs year groups which only a school which meets floor targets or has extra cash for support can get around.

    I honestly do think that people from different parts of the education structure need to get together more on some issues (the College of Teachers?). However, any time I mentioned this to colleagues at primary there was a tendency to assume that secondary and other teachers would come in and tell them what to do…. an entirely imagined scenario as primary and secondary have never really collaborated in this way.

    If we are to move education away from tests then we still need to make sure that there is a shared understanding of expectations but this is best done with all types of teachers together rather than splintering off into separate camps. In many ways this exacerbates the problems as the comprehensive solutions only come from the government as they, not us, have the overview of the education system.

    Like

  2. Leah K Stewart

    Hi Carl, here is the background to mass testing from Seth Godin’s work called ‘Stop Stealing Dreams (What is School for?)’… “In 1914 Frederick J. Kelly a professor in Kansas invented the multiple-choice test. There was an emergency on. World War I was ramping up, hundreds of thousands of new immigrants needed to be processed and educated, and factories were hungry for workers. The government had just made two years of high school mandatory, and we needed a temporary, high-efficiency way to sort students and quickly assign them to appropriate slots. A few years later, as President of the University of Idaho, Kelly disowned the idea, pointing out that it was an appropriate method to test only a tiny portion of what is actually taught and should be abandoned. The industrialists and the mass educators revolted and he was fired. [why do we still do mass testing?] The reason is simple. Not because it works. No, we do it because it’s the easy and efficient way to keep the mass production of students moving forward.” From a personal perspective as a student who learnt to be very good at school tests, I did it all in hope that there was some sense in it that I was too young/ignorant/stupid to understand. Teachers must shrink themselves to serve these tests, to get us these qualifications. It’s all strange. I’m happy you’re bringing this topic further into the light with your blog.

    Liked by 1 person

  3. julietgreen

    “Part of the solution has to be to move from a high stakes, Russian Roulette, sudden-death style system to one where pupils can be evaluated on their progress in a series of low stakes, non-threatening tests that foster not only an appreciation of knowledge for knowledge’s sake, but also the ability to think critically and to be able to embrace uncertainty. All of which will prepare them for the ‘tests’ they have ahead of them.” Indeed. And I’m convinced there is a strong role for technology in this.

    Like

  4. julietgreen

    Did I have the wrong idea about tests in all my years of sitting them, though? I never knew that they were a way of holding teachers accountable and I always believed they were a way of checking how much I’d learned about a subject. We could never ask the question, ‘Will this be on the test?’ because the answer would always have been, ‘It might be.’ We had to put time into learning the whole subject, because the questions could be from any part of it. That was fine by me, even though it was hard graft. I recall studying for my end of year tests when I was 12. I remember it as the most dogged and intensive revision I’ve ever done. We made ourselves timetables, recorded things on tapes, made notes, questioned each other. I have to reiterate a point I made in another blog about project work. It was impossible to do well in a subject by relying on the teaching alone. Is it the case that pupils and students now believe the teachers are entirely responsible for how they achieve and might that be the crux of the matter?

    Like

    • ijstock

      >>Is it the case that pupils and students now believe the teachers are entirely responsible…

      YES! The most visible sign I have seen of this is the decline in pupils’ appreciation of the amount and type of revision they need to do for exams. They increasingly seem to be relying on teacher-led crammer sessions, with small amounts of independent top-up from themselves.

      Liked by 1 person

      • Leah K Stewart

        The reason I know, as a 21st Century student, that these tests were for the teachers is because it’s so clear that my individual teachers did not have full control of the content/exercises/timings etc of what they taught. The truth is, and student’s see this, our teacher’s knowledge moves faster than can possibly be standardized into a national-scale school course and our teachers’ interests (and therefore what we as students actually want to learn/discover/explore with them) extend way beyond the standardized remits… but there is no time for this. Heads down, do the work. The message is the same to teachers as what they have to say to us students. “Why do teachers do this?” we, the students, ask. “Ah, yes, they’ve got jobs they’re afraid to lose.” And then we wonder why students shut down. We’re not presenting a way of living life as an adult that students want any part of.

        Like

    • chrisanicholson

      “Is it the case that pupils and students now believe the teachers are entirely responsible for how they achieve and might that be the crux of the matter?”

      My daughter is currently doing her GCSE exams. She has said to me that she had no idea how to revise for subjects like history or geography, or how long it would take, because she has never had to do it. She has never had any of the kind of end of year exams you mention, and the school (which only introduced mocks last year) held mocks after February half term, and they weren’t full mocks, so there simply wasn’t enough time to revise thoroughly by starting then. The way in which the school sets only minimal homework, and the absence of textbooks or any expectation that students should read and study material for themselves also contribute to the sense of ‘all I need to do is pay attention in class’ to learn. Also, the whole message of the child-centred teaching that has permeated her school years seems to be that learning should be fun, involve lots of ‘active’ learning and chatting in groups, and require little in the way of hard work. After 10 years of exposure to this, why should we be surprised that students have internalized this message?

      Liked by 1 person

      • julietgreen

        I have begun to tell my pupils this home truth. To do really well they need to do more than sit in lessons. I would imagine at least some of the success of east asian schools is down to this expectation.

        Like

  5. ijstock

    Superb post. This sums up the problem to a tee – so why is nothing being done about it? I suspect it is because the link between those making policy decisions (government) and those creating the specific conditions in schools (management) is not as direct as those in government seem to think. In order to change teacher/school behaviour. legislating directly (for example to create specific initiatives or to cut workload) will not work – what is needed is to change the climate (i.e. narrow conceptions of accountability) which creates it. This, I suspect no politician will be prepared to do.

    Like

  6. Pingback: Problem: “Puzzle math” | High School Math and Chess
  7. Leah K Stewart

    Hi Juliet & Chris. What Juliet says is true, but I don’t think it’s OK for it to be this way. When we’ve 10 odd subjects and every one requires evening and weekend work this amounts to a lot of extra study necessary for ‘good’ results (trust me, I did it and could only do it because my family was stable and I had quiet space). In order to do the other things that ‘good students’ are supposed to do (sport, volunteering, activities etc) we’re spread thin. So thin. All before we’ve even had a chance to ask; what do I want? It’s actually a great way to make people think they’re fundamentally lazy when all they ever want to do, when they have a second, is be alone and do nothing (read: play computer or chat on-line).

    Chris: (1) Your daughter needs the full syllabus for the subjects and she can download this from the internet (2) If she teaches herself to understand every word on there and can re-frame every sentence/paragraph in her own words then she knows enough for a grade C and all this can take a day or two (3) To get higher than a C she’ll need a decent revision guide and she uses it to flesh out each syllabus section… all she needs to focus on is being able to rewrite sections in her own words (4) Teachers say these things are important and then we’re put into stress conditions so, if we believe them, we must also trick our minds into not caring or the pressure is too much. I did this by telling myself the evening before that my grade is “already decided” and I just need to “live though” the next 12-24 hours “succumbing to my fate” type of thing. This is an awful mindset that must be dropped as soon as the exams are done… or you’ll “succumb to fate” your whole life and that is no way to live! You’re just doing this to ensure you’re revision time is not wasted by exam-hall panic. Final note: I’m not sure how long these compulsory exams by age group will last because they are a strange way to direct the energy and minds of young people.

    Like

  8. ibscience

    I am questioning the correlation/causation of poor college level results when the cost of college/university has skyrocketed. It is no longer the case that students could work their way through college or even delay graduation by working part time. The economic picture is so different now that to lay the dropout rate at the feet of (personally) much hated standardized testing I think is a bit of a reach. It seems highly plausible that authentic learning ‘habits’ and skills to tackle university are not being developed when the focus is on standardized testing. Students show relatively little ability for original thinking based on anecdote. As much as I hate the standardized testing, arguments against them should have the same rigorous analysis as those that are for them.

    Like

  9. Anthony Radice

    E D Hirsch points out the value of objective testing using multiple choice questions in ‘The Knowledge Deficit’. Well designed tests that actually test knowledge are an excellent thing. They must be broad ranging enough to make it impossible to teach to the test, but this cannot be achieved within the current exam system in the UK.

    The GCSE is a very poor measure of knowledge, because of the way it is designed. Well designed, broad ranging multiple choice tests are a much better form of summative assessment, as they have been proven to correlate well with real life outcomes. There is no such proof for GCSEs. GCSEs are tricky and fiddly, but not actually demanding in terms of subject knowledge. Thus they lead to a lot of teaching to the test, but little transfer of knowledge. Daisy Christodoulou points this out in her essay in ‘Changing Schools’.

    We should not be attacking testing per se, but the particular type of testing which has become traditional in the UK, a type of testing that does not promote broad ranging teaching of subject knowledge. This type of testing is anti-educational.

    There is a separate issue about the share of responsibility, and the idea that teachers not pupils are responsible for grades. In reality, of course, it is not either / or. Teachers do bear responsibility for teaching effectively. Student apathy must be tackled primarily by individual schools building the right culture of hard work and self-discipline in their pupils.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s