three-cs1

Is collaboration always the best way of working?

I’ve long thought that if the guiding principle for any initiative is the fact that it’s alliterative or that it’s a ‘handy acronym’, then there’s probably good reason to be suspicious of it. One of the most pervasive of these is the three Cs – collaboration, creativity and communication or a variant thereof, with the emphasis firmly on collaboration.

Collaboration is now the sine qua non in any learning or professional development scenario. We live in an age where to work alone is somehow seen as a seditious act. There is now a near constant injunction to collaborate, to ‘connect’ socially and to be a “team player.” We now have open-plan offices, “break-out spaces” and increasingly classrooms that look more like Times Square than somewhere where you might be able to reflect and think hard about something.

I worry that for many of us, there is now no refuge from this kind of stuff:

The classic collaborative activity is of course ‘brainstorming’ devised by Alex Osborn in 1939. The commonly held wisdom here is that if you withhold criticism, reserve judgement and allow a critical mass of ideas to form then you will arrive at the most creative solution to any given problem. However this approach is not supported by evidence.

in 2003 Charlan Nemeth from the University of California divided 254 students into different groups with the same problem to solve ‘How can we limit the impact of the traffic problem in the San Francisco Bay area?’

The first group were instructed to solve the problem in the classic collaborative, brainstorming method where they did not criticise ideas but simply collated as many ideas as possible. The second group were instructed to work differently and to debate and criticise ideas and preconceptions and the third group could do as they pleased.

The results were overwhelmingly definitive – the debate team were by far the most creative. Not only that, but when the teams were broken up and individuals were asked to think about the problem further on their own, the brainstorm team came up with an average of three extra ideas but the debate team came up with an average of seven ideas.

What I think is so interesting about this study is firstly the clear evidence that critical dissent is a powerful driver of not just creativity but also of the volume of ideas, but perhaps more interestingly, deliberate and solitary reflection (combined with debate and rational argument) would seem to be far more productive than traditional collaborative brainstorming or uni-directional ‘teamwork’ activities. (This chimes with Nick Rose’s clarion call for professional scepticism.) 

Nemeth writes:

This line of research maintains that the benefits of dissent stem from the cognitive conflict it generates; the dissent compels those in the majority to search for possible explanations as to why the dissenter is willing to openly disagree and suffer the rejection that often accompanies such disagreement. This search for explanations then fosters thinking on all sides of the issue (Nemeth, 2003).

As someone who valued the fact that I could sit, read and think for hours as a child undisturbed by the injunction to agree with everyone and then march around the room and stick post-it notes onto sugar paper posted on walls, I worry that we are not affording the same opportunities to kids today in an age where collaboration is not only valorised but mandated.

Add to this the constant distraction of technology and there is a reasonable argument to be made that the most progressive and indeed liberating thing you can do in education today is make kids read in silence for an hour.

There are lots of other reasons why we might want to work collaboratively such as social and relational benefits which are certainly important, but we should perhaps think twice in doing so in the name of being more creative or productive.


DW Taylor, PC Berry and CH Block, “Does Group Participation When Using Brainstorming Facilitate or Inhibit Creative Thinking?”Administrative Science Quarterly 3, no 1 (1958): 23-47

Matthew Feinberg, Charlan Nemeth (2008) “The ‘Rules’ of Brainstorming: An Impediment to Creativity?”, Institute for Research on Labor and Employment Working Paper Series (University of California, Berkeley) Paper iirwps-167-08; http://escholarship.org/uc/item/69j9g0cg

Education Festival 2015: Hearts and Mindsets

This year’s Education Festival was an apopemptic affair for me. Not only is my boss Anthony Seldon leaving but also several members of my own department, including festival director David James, the inestimable Joanna Seldon and my good friend Jim Heal who I trained with at King’s College. It has been a strange few weeks saying goodbye to so many colleagues that I am genuinely sad to lose and which infused the timbre of this year’s festival, one that proved to be an emotional one in many ways.

In just a few years the festival has gone from a few hundred attendees to over 5000 on both days this week. It is an incredible thing to have some of the most important voices in education come to your school and speak and I feel hugely fortunate to have been part of it.

This year as head of research I managed to convince Harvard faculty to hold their Research Schools International Symposium here at Wellington featuring a range of schools working on school based research to share and discuss ideas. We had schools from the US, Ecuador, France and even Hawaii. We began the day with me welcoming delegates and speaking a little about school based research. Then Dr. Christina Hinton outlined the work of Research Schools International and the notion of Usable Knowledge. There is a good summary here. I was especially please to meet Andy Tharby and Shaun Allinson as I sat down. We then had a panel discussion on school based research and the value of student voice with Glenn Whitman and Ian Kelleher from the CTTL, Tom Callahan from the Merck-Horton centre and the always fascinating Al McConville from Bedales.

panel

We then had a series of roundtable discussions with all the schools and some honoured guests including the fantastic Jude Enright from Greenford High School and Jonnie Noakes from Eton’s new Research centre. This discussion was incredibly useful in terms of getting a sense of how we might translate research on Growth Mindsets, Grit and resilience into something that might have impact in schools. I was blown away by the whole thing and will sift through my notes and post findings at a later date.

WP_20150618_11_38_54_Pro

As I was focussing on the symposium I had little time to do much else before the now legendary speakers dinner that evening. If you are unaware of it, every year at the festival Anthony hosts a dinner for speakers where guests are called at random to speak about a topic of his choosing. It is a brutal affair as you do your best to enjoy the food with a perpetual sword of Damocles hanging over your head. This year’s guests called on to speak included Tom Bennett, Daisy Christodoulou, Martin Robinson, Angela Duckworth Rob Coe, Claire Fox, David Didau, Laura McInerney, Dominic Randolph and surprisingly, yours truly. It was a somewhat terrifying experience in which I mumbled through something about why teachers should engage with research. Luckily by that stage of the night everyone was reasonably drunk so I think I got away with it. David James made the final speech of the night which was as heartfelt as it was funny and for which he was applauded with great brio not just for his contribution to the festival and its prosperity, but because he has been instrumental in the successes of many in the room.

I was especially pleased to be seated next to the two finest human beings on the face of the planet, John Tomsett and Martin Robinson, however any joy quickly evaporated with the arrival of David Didau who arrived dressed as an adult leprechaun and whose attempt to pass off what can only be described as an old sock for a bow-tie was particularly risible. (What if everything you know about evening wear is wrong?) Later I speculated that his entire outfit might have been a thinly veiled attempt to mock my Irish heritage in which case he deserves great credit.

IMG_9736 (1)

Later that night a rogue team including Salon Stalwarts David Didau, Claire Fox and Daisy Christodolou broke away to Wellington’s on-site secret bar, Napoleon’s Retreat where I was lucky enough to meet data expert Jack Marwood and progressive traducer Nevile Gwynne.

The next morning I went to see one of the most powerful talks on education I’ve ever seen featuring Jarlath O Brien and Maria Ramsey from Carwarden house, our own chaplain Tim Novis and Ed Venables give a talk on the unique collaboration between ourselves and their special school featuring many of the students presenting also. This talk was the one that had most impact on me over the two days, particularly my colleague Tim’s beautifully honest and personal account of the birth of his daughter and how he became involved with Carwarden House special school. It was not easy for Tim to say those words but I am glad he shared it with us all.  I have an unfathomable respect for everyone involved in this project and felt genuinely moved/uplifted by it not least as my own sister was given fantastic care and support from a special school in Dublin.

IMG_9753 (1)

I then raced off to the theatre to see Dylan Wiliam gave a masterclass on principled assessment design mainly focusing on the many issues with assessment. He outlined a plethora of problems with current assessment models that are simply not fit for purpose, notably the problematic practice of using tests to group students by ability.

IMG_9764

Dylan is a hugely engaging speaker, his talks are peppered with a torrent of powerful edu-aphorisms such as this gem:

Other pearls included:

“We are drowning in data but not learning anything”

“Scores suffer from spurious precision. Grades suffer from spurious accuracy.”

The full presentation can be downloaded here. I was lucky enough to have lunch with Dylan after along with Daisy Christodolou where we spoke about a wide range of things from why baseball provides more insight for education than football and why the solution to the Semmelweis problem might be Hattie Jacques. (Ask Daisy)

Another emotional moment was Tom Sherrington seeing a picture of one of his students during his talk and getting all choked up. I missed that talk but I have known Tom to break into tears before when talking to me about one of his students. It is this capacity for empathy that marks him out as a truly exceptional head teacher.

I then went to see a Battle of Ideas debate ‘Is teaching an art or a science?’ featuring Rob Coe,Tom Bennett, Alka Sehgal Cuthbert, Alistair McConville and Daisy Christodoulou and hosted by Claire Fox (who should be in charge of everything.) This was an engaging debate featuring some powerful points on both sides and avoided the usually reductive, reactionary truculence that this debate often descends into. A particularly strong point was that there are certain ethical mplications around ignoring evidence and that research should be seen as something that informs professional judgement as opposed to something that subsumes it. Sadly, I was to be distracted by Tom Starkey live-tweeting abuse at me. 

IMG_9767

After this I went to meet Carol Dweck at the master’s lodge where I had a cunning plan to get some of our student research council to interview her. She agreed and was absolutely delightful with them and gave of her time generously. These students have read a lot of her work over this last year and had a fascinating conversation with here. The recording of that meeting can be listened to here. I cannot express the pride I felt at seeing my students interviewing one of the most important voices in education.

IMG_9771IMG_9773

Another great moment was seeing Carol Dweck meet with Tom Sherrington and John Tomsett who spoke enthusiastically about how they had adapted her ideas in their schools. She was suitably impressed and kept talking about them in glowing terms afterwards.

A key element of her talk was the notion of ‘false growth mindset’ written about here eloquently by David Didau. Another act of human greatness was Laura McInterney taking the time to speak to one of my students who has dreams of being a journalist.

I had to then leave early to attend the TES awards in London where a number of friends were up for the blogger of the year award. I was at a table with Tom Bennett (who was late again so I ate his starter) and Emma Ann Hardy who introduced me to the living legend that is Fred Jarvis. At last the blogger nominees were announced and again I felt a swell of pride to see good friends get some much deserved attention especially Nick Rose who I nominated and who is one of the voices I respect most in education. (I was also delighted to see motivational speaker Andrew Old.) 

In the end, Nancy Gedge won and no one could have any complaints. Her writing over this past year has been insightful, erudite and honest and has given us all a valuable insight into a world that I think is too often marginalised in UK education.

A whirlwind couple of days then and as is the case with #Educationfest every year, the core messages and central themes take days/weeks to sift through. On a personal level, this year’s festival marks the end of an era in which many good friends leave that I will hope to see return as visitors next year.

Screen Shot 2015-06-21 at 16.37.37

Podcast: Students Interview Carol Dweck about Growth Mindset

One of the things I was keen to do this year in setting up an in-house research centre at Wellington College was to have a small number of students partner with us on our project with Harvard faculty on Growth Mindsets and Grit. A key point for me was what does this research actually look like in the classroom and and at the level of the student?

Another goal was to have them help us in designing a survey by having them pilot test some of the more problematic questions so we could get as reliable data as possible.

We asked the students to read some of the literature and research in these areas and then had a series of group discussion with them where we discovered a huge range of things that was really helpful in helping us understand Growth Mindsets from multiple perspectives. 

IMG_9771                            IMG_9773

At the education Festival this week we were hugely fortunate to have Carol Dweck as a speaker so when I met with her I was really keen that our student research council interview her and put some of their own pressing questions about student motivation, assessment and Growth Mindset to her from their own perspective as students. She was incredibly generous with her time and was really eager to meet with them.

DBP_1956_244_Semmelweis

The Semmelweis Reflex: Why does Education Ignore Important Research?

 

In 1846 the general hospital in Vienna was experiencing a peculiar problem. There were two maternity wards at the hospital but at the first clinic, infant mortality rate was around 16% while at the second clinic the rate was much lower, often below 4%. Mysteriously there were no apparent differences between the two clinics to account for this.

Part of the mystery was that there was no mystery. Almost all of the deaths were due to puerperal (childbed) fever, a common cause of death in the 18th century. This fact was well known outside the hospital and many expectant mothers begged to be taken to the second clinic instead of the first. The stigma around the first clinic was so great that many mothers preferred to give birth in the street than be taken there.

Working at the hospital at the time was Ignaz Semmelweis, a young doctor who had risen to the ranks of assistant professor where his duties included the examining patients before the professor’s rounds. Perturbed by the seemingly unsolvable nature of this mystery, young Ignaz recorded that it made him “so miserable that life seemed worthless” and so dedicated himself to finding a solution.

Ignaz_Semmelweis_1857_with_signature

Ignaz Semmelweis

The breakthrough came when his close friend and colleague Jakob Kolletschka died of the same infection after cutting himself with a surgeons scalpel during an operation. Semmelweis then noticed that in the first clinic doctors were routinely conducting autopsies whilst in the second this practice did not occur. He also noticed that doctors were often delivering babies and treating patients with the same unwashed hands they were performing autopsies with and so proposed that doctors were contaminating patients with “cadaverous particles.” He then insisted doctors wash their hands in a chlorinated lime solution before dealing with patients resulting in a drop in deaths from puerperal fever to around 1%.

However despite this transformative discovery many in the medical community at the time were not only skeptical of Semmelweis’s findings but openly mocked them. Charles Meigs, a prominent American obstetrician (and teacher) derided the notion of bacterial infection and antiseptic policy noting that “Doctors are gentlemen, and gentlemen’s hands are clean” and that any that it was morally unacceptable to “contravene the operations of those natural and physiological forces that the Divinity has ordained us to enjoy or to suffer.” Even when Semmelweis published his findings, he still came up against intransigence and entrenched partisanship. August Breisky, an obstetrician in Prague, referred to his work as “the Koran of puerperal theology.”

SemmelweisUse

This reactionary short-sightedness gave rise to the the term The Semmelweis Reflex: “the reflex-like tendency to reject new evidence or new knowledge because it contradicts established norms, beliefs or paradigms.” Now education is clearly very different from medical practice, but there are things we are discovering about the process of learning that the profession can no longer afford to ignore. In the last 10 years we have witnessed an explosion in what we know about the essential dynamic of how the brain retrieves and stores information. The field of neuroscience in particular has afforded huge opportunities to the profession by challenging and clarifying erroneous beliefs through not just solid evidence, but important evidence yet still many many of these problematic ideas are still widespread.

We now know for example that left and right brain thinking is not supported by any evidence, we know that the belief in learning styles is not just wrong but dangerous, we know that the claim that we only use 10% of our brain is completely unfounded and that if you are using Neuro linguistic programming for education purposes then you might as well be practising astrology. Yet despite these important findings, may of these beliefs still persist in education settings today.

Against this clearing out period of debunking and myth-busting, we have also had a number of significant discoveries such as Sarah Jayne-Blakemore’s work on the teenage brain and the fact that “disorders like anxiety disorders, depression, addictions, eating disorders, almost all of them will have their onset some time during the teenage years.” Her work proposes to explore how “genes and the environment influence brain development, like for example, how adolescent brain development differs between cultures is something that no one has yet asked, and yet it’s bound to.” These issues and the potential findings from them seem to be not only integral to the practice of teaching children but also ethically imperative. Yet how much of these findings are properly embraced by the profession in a way that is commensurate with other professional fields?

Even where important ideas have been adopted, implementation of them has been problematic. Dylan Wiliam’s work on assessment for learning from 1998 is a seminal contribution to the field of education research but has used reductively to simply mean kids knowing what level they are working at or teachers sub-levelling individual pieces of work. These issues raise some important questions:

Whole School Cognitive Dissonance: What is the value in a school preaching Growth Mindsets in an assembly yet basing their entire school enterprise on the reductive and fixed mode of target grades and narrow assessment measures based on poor data? Why are kids explicitly told that their brain is malleable but implicitly told their target grades are not?

Insufficient training: Why are teacher training courses essentially a mad trolley dash around hugely complex ideas over 8 months, where trainees spend more time collating paperwork against often arbitrary targets than engaging with significant research? (I don’t blame PGCE providers for this – they are working in an increasingly difficult area with less and less resources but where did the idea come that you could learn all you need to know about teaching and be classroom-ready in less than a year?)

If not evidence then what? If educators are not interested in evidence then what are we actually talking about? Are we basing our entire professional practice solely on our own experience in the classroom 10 or 20 years ago? Or should our professional judgement be one that is informed by and that engages with a wider body of knowledge from different disciplines? Are we so arrogant to think we have nothing to learn from different fields?

Ethical considerations: If research shows us that certain practices lack any evidence or are ultimately a waste of time, is it ethically right that teachers continue to use these practices? As mentioned, learning styles has been fairly widely debunked now but why does it continue to linger in so many educational arenas? And what are the implications of new research for educational monoliths like group-work, differentiation, Bloom’s taxonomy and traditional marking?

Three of the most important aspects of teacher development are trust, professional judgement and autonomy but surely that judgement and autonomy is only enhanced and not hindered by new findings about how learning takes place. As with Semmelweis’s detractors we shouldn’t be rejecting new ideas and discoveries, especially ones that are so relevant to our field but rather incorporating them into our own practice. One of the challenges to education research is the perceived threat of “evidence” as an axiomatic truth to be delivered on high that will limit teacher autonomy and agency in the classroom but I for one am excited by education incorporating these new findings around memory, retention and performance and incorporating them into their own professional practice and then making them applicable in the classroom where they can have the most impact.

DBP_1956_244_Semmelweis


Neuroscience and Education: Issues and Opportunities – ESRC

Neuroscience: implications for education and lifelong learning

Why the Widespread Belief in ‘Learning Styles’ Is Not Just Wrong; It’s Also Dangerous

testing

We’ve lost Sight of the Point of Testing

Testing in its current form is a relatively new phenomenon. In the Ancient World, Socrates would ‘test’ his students through a dialogue where there were no ‘correct’ responses but simply more questions and answers. The Socratic tradition of dialogue was largely continued in Europe with students being tested through oral responses and then essays until around 150 years where we begin to see the notion of testing as something that can be standardised in a uniform fashion.

In the 19th century, the “father of American public education” Horace Mann advocated testing to provide “objective information about the quality of teaching and learning in urban schools, monitor the quality of instruction, and compare schools and teachers within each school.” At the beginning of the 20th century psychologist Alfred Binet developed a standardized test of intelligence, which would eventually become the standard IQ test we know today.

There is no question that a standardised measure of assessment holds great value in terms of being able to compare students and schools with on a national level and to flag up underachieving groups but there is a clear sense at the moment that standardised testing has become something else, that the tail is now wagging the dog and that the model may be in need of reform, something brought to national attention in the US not only by mass boycotting of tests but also by John Oliver on national TV:

For me there are several issues that focus not so much on testing itself but on the collateral damage caused by testing and how they’re being used far beyond their intended purpose:

1. Tests are no longer part of a judgement, they now are the judgement.

One of the proposed benefits of testing in the 19th century was that they would provide a diagnostic indicator of student progress and school efficacy that would inform a more wider, more balanced and measured judgement. However what we have now is a system where test scores are the judgement – it’s not so much that the tail is wagging the dog as the tail now is the dog. Referring to Horace Mann’s statewide roll-out of standardised tests in the mid 1800s, a US congress report from 1992 notes that:

It is important to point out what “standardization” meant in those days. It did not mean “norm-referenced” but rather that “. . . the tests were published, that directions were given for administration, that the exam could be answered in consistent and easily graded ways, and that there would be instructions on the interpretation of results. ’

The other issue is that the current model of testing risks exceeding its own mandate by being used mainly for school reform and teacher evaluation as opposed to pupil evaluation. Many opponents of standardised testing claim that they may not be fit for purpose on their own terms. According to James Popham standardised tests are like trying to “measure temperature with a tablespoon”

“Tablespoons have a different measurement mission than indicating how hot or cold something is. Standardized achievement tests have a different measurement mission than indicating how good or bad a school is. Standardized achievement tests should be used to make the comparative interpretations that they were intended to provide. They should not be used to judge educational quality.”

Combine this with an inspectorate that is data-obsessed and you have a perfect storm as this cautionary tale from Geoff Barton illustrates.

2. Standardised tests are really just a measure of who has access to specific resources

Does standardised testing evaluate a pupil’s aptitude or general knowledge or does it simply register their access to a particular set of resources? Resources here can mean a wide range of things such as the school, a private tutor, parental support but when a particular set of textbooks define success then there is an issue. This is well illustrated by the case of the Pennsylvania System of School Assessment (PSSA) where Meredith Broussard discovered that success in her daughter’s 3rd grade test was inextricably linked to specific textbooks:

“Standardized tests are not based on general knowledge. As I learned in the course of my investigation, they are based on specific knowledge contained in specific sets of books: the textbooks created by the test makers.”

Regardless of all the other inequities, the cost of these textbooks are prohibitive in many districts so the inevitable outcome is that schools in deprived areas simply cannot ‘win’ in standardised texts. Interestingly, in the 1960s the Civil Rights movement protested against standardised testing as it inevitably punished those from a certain social strata. The Coleman Report, found that a student’s home environment was the deciding factor in determining achievement. (Rumberger & Palardy, 2005) In a culture of high stakes accountability the losers will inevitably be the ones without access to the best resources.

2. Linking teacher accountability to student test scores raises difficult ethical questions

The logical end point of a high stakes system of accountability where teachers are judged on their students’ scores will be the occurrence of dubious ethical practices somewhere along the line, whether that be on a small scale with teachers ‘correcting’ a student’s coursework or on the more extreme end with institutional malpractice. Earlier this year in Atlanta eight educators were found guilty of organising a “criminal enterprise” under the Racketeer Influenced and Corrupt Organization (RICO) act for manipulating student scores on Georgia’s state standardized tests. Now I’m not for a second suggesting that they did this as a result of standardised testing, but the case raises some important issues around the kinds of pressure being put on teachers at the moment. What kind of a system makes people risk years in jail to improve student test scores?

Across the US right now there is a fairly widespread movement of civil disobedience with parents intervening in their children’s education and opting out of standardised tests. Whilst the moral and ethical dimensions of these decisions are unclear it is evident that the current fetish for testing has engendered a string of unintended consequences.

3. “But will this be in the test?”

Probably the most dispiriting thing a teacher can hear. We all want students to ‘achieve’ academically but should that be at the expense of intellectual curiosity and the ephemeral joy of learning that is often immeasurable? Now they are not mutually exclusive of course but when the outcomes for one far outweigh the other then something has to give and often that is the autonomy of the teacher to be able to go ‘off piste’ and follow a particular conversation or idea perhaps not directly related to the test. Teachers want their students to score well in tests but what about another measure of ‘success’ – what about the English teacher who has engendered a lifelong love of reading in a pupil? or the Physics teacher who has sparked a student’s curiosity about cosmic universe? or the languages teacher who has opened a student’s eyes to the values and customs of another country they now want to visit? Many of these things are not testable and are being subsumed by a focus on what is prescribed by an exam board and the techniques needed to be ‘successful’ in them. Harvard Professor Daniel Koretz notes that “If you impose a simplistic numerical measure and lose sight of the other important goals of the institution, then the other goals get short shrift.”

4. College dropout rates suggest that something is wrong 

In the UK, figures published by the Higher Education Statistics Agency indicated that over 32,000 students dropped out of university after a year of study in 2012/13. Of those 7,420 transferred to another university, while 24,745 dropped out of higher education altogether. In the US things the picture is even more bleak where there is the “lowest college completion rate in the developed world” (Organization for Economic Cooperation and Development.)

There seems to be a lot of blame attached to the universities themselves, but what if pre-university education is simply not preparing students for the intellectual rigour, criticality and independence needed at that level?  Are students effectively being herded through a set of tests to provide data that benefits policy makers and Ofsted rather than the actual pupils themselves? Is focusing on the ability to use a broad base of knowledge to think critically being sidelined for the narrow measure of how to pass an exam?

The knock-on effect of a high-stakes testing system with increased accountability will be the limiting of both teacher and pupil agency. Apart from the impact on student mental health and stress levels, teachers are increasingly being asked to teach how to pass an exam as opposed to impart knowledge and elicit dialogue. Schools are systems of deep uncertainty and flux in which teachers are often held accountable for the unaccountable.

Part of the solution has to be to move from a high stakes, Russian Roulette, sudden-death style system to one where pupils can be evaluated on their progress in a series of low stakes, non-threatening tests that foster not only an appreciation of knowledge for knowledge’s sake, but also the ability to think critically and to be able to embrace uncertainty. All of which will prepare them for the ‘tests’ they have ahead of them.


Why Poor Schools Can’t Win at Standardized Testing

Testing in American Schools: Asking the Right Questions. [Full Report.] 

Teacher Knowledge: From ‘Outside-In’ to ‘Inside-Out’

In their book ‘Inside Outside: Teacher Research and Knowledge,’ Marilyn Cochrane Smith and Susan L. Lytle outline a fundamental problem with our profession, namely that there has been an outside-in model of knowledge creation about what effective teaching is and a ‘top-down’ model of school improvement. Teachers they claim, have effectively been passive participants in the process of what constitutes good practice, and in research terms have been mere ‘objects of study.’

“The primary knowledge source for the improvement of practice is research on classroom phenomenon that can be observed. This research has a perspective that is “outside-in”; in other words, it has been conducted almost exclusively by university based researchers who are outside of the day-to-day practice of schooling.”

With all the millions invested in education research, it’s somewhat ironic (and symptomatic of the age) that some of the biggest agents of impact recently have been various grassroots movements, driven through self-organising, informal communities connecting through social media. The establishment of forums like ResearchED are now functioning as a sort of fourth estate to the traditional trifecta of School/Academia/Government, indeed major funding is now being invested to evaluate the impact of both school to school support models and also brokerage models of research engagement.

What is most significant about these initiatives, is that they are being driven by classroom teachers at a grassroots level working from the inside out, as opposed to traditional top-down model of ‘experts’ dictating from the outside-in.

This movement from ‘fringe to forefront’ is fuelled by a real desire from classroom teachers for not just knowledge, but practical, usable knowledge that speaks to them about their own experiences and has real, demonstrable impact on the pupils in their charge. For too long, the creation of knowledge about what makes effective teaching has been one-way traffic with researchers observing and codifying the phenomenon of the classroom, with very little input from teachers themselves.

Stenhouse’s argument was radical: He claimed that research was the route to teacher emancipation and that “researchers should justify themselves to practitioners, not practitioners to researchers.” *

For me the single most important element in this process is the autonomy for teachers to be able to ask their own questions and then carry out the process of collaborative, systematic enquiry to explore those questions. What we have had until this point has largely been characterised by teachers being given answers to questions they didn’t ask. It is vital that those questions come not just from outside-in but from the direct issues that teachers experience everyday in the classroom. As Cochran-Smith and Lytle remind us:

The unique feature of the questions that prompt teacher research is that they emanate from neither theory nor practice alone but from critical reflection on the intersection of the two.

However teachers can’t do it alone with the present workload level and lack of training/expertise in research methods. The experience and knowledge of experienced academics and education research departments can play a vital role in working alongside teachers helping them shape their focus and provide crucial support in terms of methodology, literature and the wider evidence base and hopefully helping to create a truly collaborative model of knowledge creation about effective classroom practice that is not solely outside-in.


Works cited: Cochran-Smith, M. and Lytle, S. (1993) Inside –Outside: teacher research and knowledge, Teachers College Press, New York *Stenhouse in Ruddock and Hopkins, 1985 p.19

272804773_b3577b4b7d

The McNamara Fallacy and the Problem with Numbers in Education

Robert McNamara was by any standards, a wildly successful man. Harvard graduate, president of Ford motors then rising to the heights of U.S. Secretary of Defense in the 1960s, McNamara epitomised American élan and brio. But he had one major flaw – he saw the world in numbers.

During the Vietnam War, McNamara employed a strategic method he had successfully used during his days at Ford where he created data points for every element of production and quantified everything in a ruthless fashion to improve efficiency and production. One of the main metrics he used to evaluate progress and inform strategy was body counts. “Things you can count, you ought to count,” claimed McNamara, “loss of life is one.”

272804773_b3577b4b7d

The problem with this method was that the Vietnam war was characterised by the unmeasurable chaos of human conflict not the definable production of parts on a factory assembly line. Things spun out of control as McNamara’s statistical method failed to take into account numerous unseen variables and the public turned against US involvement in the war through a cultural outcry that would change the country. Although on paper America was ‘winning’ the war, ultimately they lost it.

As the war became more and more untenable, McNamara had to increasingly justify his methods. Far from providing an objective clarity, his algorithmic approach gave a misleading picture of what was becoming an unfathomably complex situation. In a 1967 speech he said that:

“It is true enough that not every conceivable complex human situation can be fully reduced to the lines on a graph, or to percentage points on a chart, or to figures on a balance sheet, but all reality can be reasoned about. And not to quantify what can be quantified is only to be content with something less than the full range of reason.”

While there is some merit to this approach in certain situations,  there is a deeply hubristic arrogance in the reduction of complex human processes to statistics, an aberration which led the sociologist Daniel Yankelovitch coining the term the “McNamara fallacy”:

1. Measure whatever can be easily measured.

2. Disregard that which cannot be measured easily.

3. Presume that which cannot be measured easily is not important.

4. Presume that which cannot be measured easily does not exist.

Sadly, some of these tenets will be recognisable to many of us in education – certainly the first two are consistent with many aspects of standardised testing, inspections and graded lesson observations. This fiscal approach been allowed to embed itself in education with the justification given often to ‘use data to drive up standards.’ What we should be doing is using “standards to drive up data” as Keven Bartle reminds us.

The fallacy is based on the misguided notion that you can improve something by consistently measuring it. In the classroom, this is best illustrated by the conflation between learning and performance, which For Robert Bjork, are two very different things – the former is almost impossible to measure, the latter much simpler. It is very easy to transpose observable performance onto a spreadsheet and so that has become the metric used to measure pupil achievement and concomitantly, teacher performance. In tandem with that you’ve had the hugely problematic grading of lesson observations on a linear scale against often erroneous criteria as Greg Ashman has written about here.

RobertMcNamara55

Two years after the Vietnam war ended, Douglas Kinnard, published a significant study called The War Managers  in which almost every US general interviewed said that the metric of body counts were a totally misguided way of measuring progress. One noted that they were “grossly exaggerated by many units primarily because of the incredible interest shown by people like McNamara.”

In education, the ‘incredible interest’ of the few over the many is having a disastrous impact in many areas. One inevitable endpoint of a system that audits itself in terms of numbers and then makes high-stakes decisions based on that narrow measurement is the wilful manipulation of those numbers. A culture that sees pupils as numbers and reduces the complex relational process of teaching to data points on a spreadsheet will ultimately become untethered from the moral and ethical principles that are at the heart of the profession, as the recent Atlanta cheating scandal suggests.

Even in the field of education research, there is a dangerous view in some quarters that the only game in town is a randomised controlled trial (its inherent problems have been flagged up by people like Dylan Wiliam.) If the only ‘evidence’ in evidence based practice is that which can be measured through this dollars and cents approach then we are again risking the kind of blind spots associated with the McNamara fallacy.

Teaching is often an unfathomable enterprise that is relational in essence, and resists the crude measures often imposed upon it. There should be more emphasis on phronesis or discretionary practitioner judgement that is informed by a deep subject knowledge, a set of ethical and philosophical principles and quality research/sustained inquiry into complex problems.

In my experience, the most important factors in great teaching are almost unmeasurable in numbers. The best teachers I know have a set of common characteristics:

1. They are not only very knowledgable about their subject but they are almost unreasonably passionate about it – something which is infectious for kids.

2. They create healthy relationships with those students in a million subtle ways, which are not only unmeasurable but often invisible to those involved.

3. They view teaching as an emancipatory enterprise which informs/guides everything they do. They see it as the most important job in the world and feel it’s a privilege to stand in a room with kids talking about their passion.

Are these things measurable in numbers and is it even appropriate to do so? Are these things helped or hindered by the current league table culture?

5Jsgx8bQ5430kshplPqhX7CYqmc

Robert McNamara died an old man and had opportunity to reflect on his long life, most notably in the Academy award winning documentary The Fog of War. His obituary in the Economist records that:

“He was haunted by the thought that amid all the objective-setting and evaluating, the careful counting and the cost-benefit analysis, stood ordinary human beings. They behaved unpredictably.”

Measuring progress is important. We need to know what we are doing is having impact against another approach that might yield better outcomes, but the current fetish of crude numerical quantification in education is misleading and fundamentally inappropriate for the unpredictable nature of the classroom. We need better ways of recording the phenomenon of the classroom that captures more than simply test scores and arbitrary judgements on teachers, and seeks to impose an order where often there is none.

Continue reading

The ‘Outstanding’ school fallacy (It’s not one where the staff work twice as hard as the kids)

This week an insurance firm specialising in coverage against teachers being off work published some revealing figures showing that stress causes twice the amount of time off as common ailments with the firm’s director Harry Cramer noting that “among men it is the single biggest reason.” In another (admittedly less rigorous) study, a preliminary online survey of 3,500 members of the NASUWT showed that 67% of teachers felt that the job was adversely affecting their mental health, with 76% saying they are “seriously considering” leaving the profession.

The usual suspects rolled out are Ofsted, consistent curriculum changes, poor behaviour and increased bureaucracy but one thing that’s rarely mentioned is that in many schools it would appear that teachers are working significantly harder than the pupils in their charge, and not so much because the kids are lazy but rather because of an institutionalised miasma that is obsessed with measuring everything (usually poorly) that privileges the spreadsheet over the individual and which has infantilised the process of learning to such a degree that actually knowing stuff is deemed less important than merely appearing to know stuff.

What are the contributing factors here?

1. Ownership of results 

 For whatever reason the ownership of student achievement has somehow transferred from pupil to teacher. We now talk about *our* results, not *their* results. Many teachers start off in September with a well intentioned focus on ‘independent learning’ only to end up in Feb-May doing a series of lessons that look more like the Gettysburg address. If results now *belong* to the teachers, why should the students work as hard?

 2. A culture of Spoon-feeding.

In a culture that audits itself purely in terms of readily quantifiable measures against often arbitrary targets (with very real consequences for the teacher as opposed to the student) the inevitable outcome will be for teachers to do ‘whatever it takes’ to hit those targets, and this has led to some of the most unethical practices ever seen and yet those same schools are deemed ‘outstanding.’

3. The “shrinking of intellectual aspiration.” 

Can’t claim credit for that phrase, I heard Tony Little say it last week and it struck a chord. Something I find more alarming than the proportion of kids who lack basic foundational knowledge (in terms of culture/history/politics) is the amount of teachers who think that’s ok. Too many schools now are bastions of anti-intellectualism that exist only to hit targets and where being clever and culturally aware comes second to passing an exam.

In a school culture that seems to think purely in financial terms, it will view individuals as expendable and the inevitable outcome is that canopy of meaningless bureaucracy and stress where teachers are more skilled at data entry that knowing their subject. And the saddest thing is that the kids then begin to think in this fiscal way, and demand ‘painting by numbers’ style teaching in order to pass exams and one day inevitably utter the most depressing sentence you can hear as a teacher: ‘but will this be in the exam?’

An outstanding school is not one where the teachers are working twice as hard as the kids. It’s a quivering house of cards that is constantly on the verge of collapse.

Originally posted here on Staffrm as part of #digimeet

Engagement: Just because they’re busy, doesn’t mean they’re learning anything.

I’ve long thought that one of the weakest proxy indicators of effective learning is engagement, and yet it’s a term persistently used by school leaders (and some researchers) as one of the most important measures of quality. In fact many of the things we’ve traditionally associated with effective teachers may not be indicative of students actually learning anything at all.

At the #ascl2015 conference last Friday, the always engaging Professor Rob Coe gave a talk entitled ‘From Evidence to Great Teaching’ and reiterated this claim. Take the following slide – How many ‘outstanding’ lessons have been awarded so based on this checklist?

Screen Shot 2015-03-21 at 21.15.21

Prof. Rob Coe From Evidence to Great Teaching ASCL 20 Mar 2015

Now these all seem like key elements of a successful classroom, so what’s the problem? and more specifically, why is engagement is such a poor proxy indicator – surely the busier they are, the more they are learning?

This paradox is explored by Graham Nuthall in his book ‘The Hidden Lives of Learners,’ (2007) in which he writes:

“Our research shows that students can be busiest and most involved with material they already know. In most of the classrooms we have studied, each student already knows about 40-50% of what the teacher is teaching.” p.24

Nuthall’s work shows that students are far more likely to get stuck into tasks they’re comfortable with and already know how to do as opposed to the more uncomfortable enterprise of grappling with uncertainty and indeterminate tasks. A good example of this as Alex Quigley has pointed out is that engagement in the form of the seemingly visible activity of highlighting is often “little more than colouring in.” Furthermore, teachers are more than happy to sanction that kind of stuff in the name of fulfilling that all important ‘engagement’ proxy indicator so prevalent in lesson observation forms.

The other difficulty is the now constant exhortation for students to be ‘motivated’ (often at the expense of subject knowledge and depth) but motivation in itself is not enough. Nuthall writes that:

“Learning requires motivation, but motivation does not necessarily lead to learning.”p.35

Motivation and engagement and vital elements in learning but it seems to be what they are used in conjunction with that determines impact. It is right to be motivating students but motivated to do what? If they are being motivated to do the types of tasks they already know how to do or focus on the mere performing of superficial tasks at the expense of the assimilation of complex knowledge then the whole enterprise may be a waste of time.

Learning is in many cases invisible as outlined many times by David Didau and is certainly not linear but rather more nebulous in actuality. As Prof. Coe reminds us, ‘learning happens when people have to think hard’ but unfortunately there is no easy way of measuring this, so what does he suggest is effective in terms of evidencing quality?

Ultimately he argues that it comes down to a more nuanced set of practitioner/student skills, habits and conditions that are very difficult to observe, never mind measure. Things like “selecting, integrating, orchestrating, adapting, monitoring, responding” and which are contingent on context, history, personalities, relationships” and which all work together to create impact and initiate effective learning. So while engagement and motivation are important elements in learning they should be seen as part of a far more complex conglomerate of factors that traditional lesson observations have little hope of finding in a 20 min drive-by.

This is where a more robust climate of research and reflective practice can inform judgements. It’s true that more time for teachers to be critically reflective will improve judgements but we also need to be more explicit in precisely what it is we are looking for and accept that often the most apparent classroom element may also be the most misleading.

Slides: Prof. Rob Coe:  From Evidence to Great Teaching ASCL 20 Mar 2015

Nuthall, Graham (2007). The Hidden Lives of Learners. Wellington: New Zealand Council for Educational Research Press