The Treasure Hunter

Today’s post, as promised, is another reference to research that shows the importance of  human feelings about themselves in any situation where people are being asked to do a particular task.  My argument, as it has been in my posts over the past week–and earlier in “Let Teachers Reinvent the Wheel”–is that you can’t formally “teach” students grit, perseverance, or resilience, but you can motivate them to develop those behaviors on their own by consistently treating them as important, powerful people.  

When the National Reading Panel began its work in 1998, one of its goals was to bring reading research up to the “Gold Standard” of medical research. Unfortunately, reaching this goal was –and still is–impossible because reading researchers, unlike medical researchers, cannot separate treatment effects from effects caused by other factors, such as classroom distractions, student hunger or illness, home culture, work habits, and personal feelings.

Medical researchers have the advantage of being able to remove or minimize outside factors by assembling very large groups of potential subjects and then forming randomly selected treatment and control groups. Just as important, they are able to use a “double blind” procedure in which neither the subjects nor the people applying the treatments know which doses are the experimental ones and which are placebos.

Rather than trying to explain how all the factors present in reading research may skew results, I have chosen to focus on the one I think is the most troublesome: the “Hawthorne Effect.” This term derives from a series of studies done in the Hawthorne Works Plant of Western Electric near Chicago from 1924 to 1932. There, researchers manipulated various physical conditions, such as lighting and the number of rest breaks, to see which ones most affected worker output. What they found was that almost any change increased production, and what they concluded was that being a part of a scientific experiment created feelings of importance and belonging that were a more powerful determinant of productivity than any change in conditions.

It is hard to believe that the outcomes of much reading research are not similarly affected. In many studies both the teachers and their students know when they are part of a research plan and whether they are in the treatment group or the control group. There is no way to “double-blind” such a study when the teachers are the people applying the treatment, only one group of students has new materials, and the type of instruction given to that group is different from what is going on in the rest of the school.

Under such conditions, the feelings of the experimental teachers and students are pretty certain to be positive. After all, being chosen to implement a scientific experiment is an honor. It implies that the researchers and the school’s administrators think you are intelligent, competent, and trustworthy enough to do it well. Add to this the special training for teachers and the frequent classroom visits of researchers monitoring implementation.

In contrast, neither the teachers nor the students in a control group get any extra attention, new materials, or special training. They may also feel dishonored by not being the chosen ones. Their feelings could be called a “Negative Hawthorne Effect”

Just as production increased in the Hawthorne studies, we can expect that the results will be better for the experimental group in the type of reading study just described, whether or not the new program or strategy is any better than the old ones. This explains why so many studies report positive results for the treatments applied, and it should make us cautious about accepting those conclusions.

But, hold on! There is a way to do reading research that eradicates the influence of the Hawthorne Effect: have two or more treatment groups, with each one trying out a different program and receiving the benefits of being a part of an experiment. Ideally, there should be a control group also, using the old program. Actially, a few research studies do follow this two-treatment format, but not enough of them. It is more expensive and more complicated to carry out, and most researchers—being human—believe ahead of time that one type of program is superior to all others and want to prove that when they design their studies.

The lesson for us as followers of education  research is to consider not only results but also how carefully those studies were carried out, especially how likely it was that the “Hawthorne Effect” came into play.

Today and on Sunday I will post essays I wrote a few years ago about research on human motivation that were originally published in Education Week.  I have chosen to post them again here because they are related to the pieces I posted on Sunday and Wednesday.  Today’s essay was originally titled, “Rediscovering the ‘Pygmalion Effect’ in American Schools.”

“High expectations” is the mantra of today’s school reformers, who are convinced that the trouble with public education is that students have been allowed to slide by with little effort. Their version of high expectations is requiring college-preparatory courses, advanced subject matter, more-difficult assignments, and a longer school day and year for all students. They believe that research and the records of selected schools show that demanding more of students brings the desired results.

But do they understand the research, or know what successful schools really do?

The original research on teacher expectations tells a far different story from what today’s reformers are calling for. More than 40 years ago, Robert Rosenthal and Lenore Jacobson conducted an experiment in a California elementary school that produced what they called “The Pygmalion Effect”, in a reference to a Greek myth and George Bernard Shaw’s famous play,“Pygmalion.” All three highlight the amazing transformation of an ordinary person into someone special. In their book Pygmalion in the Classroom, they described the study in detail and interpreted its lessons for education and other human interactions.

 The experiment consisted of giving false information to teachers about their students and then sitting back to see what happened. On the pretext of testing the reliability of a newly developed test to predict future student achievement, the researchers administered a traditional IQ test to all students at the beginning of the school year. Afterward, they reported to teachers, based supposedly on the tests, the names of students who were about to have a spurt in academic performance.

In reality, these students were a randomly selected percentage of the student body, and their scores showed nothing but their current IQs. At the end of the year, and again two years later, all students were retested, and the results showed that a significant number of the identified “spurters” had in fact made unusual intellectual and performance gains and maintained them over time. Teachers’ grades and written reports also recorded marked improvements in learning and behavior for most of those students.

Although the researchers did not examine what happened in classrooms that year, teachers’ written reports were clear about what did not happen: no extra time, no advanced curriculum, no individual tutoring, no differentiated instruction or assignments.

Rosenthal and Jacobson speculated that what teachers gave their spurters—but not their other students—were unmistakable signals of their faith in them: smiles, nods of approval, more opportunities to ask and answer questions, and a kindly tone of voice. Teachers’ expectations of student success, and their unconscious communication of those expectations, made all the difference.

In its time, this study, along with its replications in three other schools and similar behavioral studies, garnered widespread and authoritative attention. Although there was some criticism of methodology and score interpretation, critics did not contest the researchers’ conclusion that the expectations in teachers’ minds were the determining factor in the success of the identified children.

Now, 40 years later, the reality of the “Pygmalion effect” stands unrefuted by further research, while it is supported by considerable evidence from classrooms where poor and minority children have made great strides in their learning because their teachers believed they would. It is also supported by countless stories of successful people who were struggling in school and life until some adult—a teacher, a boss, a family friend—saw something special in them and encouraged them to make the most of it.

The discrepancy between the Pygmalion researchers’ concept of high expectations and that of today’s reformers stems from the multiple meanings of the word “expectation.” To the researchers, it meant the power of belief to influence the behavior of others. To the reformers, it means the power of authority to exact compliance from underlings.

As a lifelong educator, I am not so starry-eyed as to think that believing in students is all that teachers and schools have to do to enable them to succeed. Every school needs a strong curriculum, high-quality materials, well-planned instruction, extra-help options, and meaningful assessments. But all those components should be calibrated to the ages, interests, prior learning, and physical and emotional capacity of the students at hand, not to the illusions held by so many pundits, business leaders, and politicians.

Quality education and student self-esteem are not mutually exclusive. We can have both, but not until we understand the essential nature of human behavior and learning, recognizing that schools must appeal to and support the strengths of students, not play on their fears and weaknesses.

Schools are meant to be wellsprings of vigor, interest, exploration, growth, and illumination. Rigor, the word so often used by reformers to describe what schools should emphasize, is more properly the companion of harshness, inflexibility, and oppression. It is time to change the current conception of high expectations back to its original meaning.

P.S. If you wish to respond to this post or others with descriptions of your own experience that you’d like me to post, send your piece to <jyatvin@comcast.net.>

Yesterday both Diane Ravitch and I read an article on the front page of the New York Times,“Testing for Grit?  Schools Push on Social Skill.”  Neither of us liked the article, but our reasons were somewhat different.  She thinks the idea of testing those skills is ridiculous and I think that that trying to teach them directly is the wrong way to go.

The Times article tells about a project in California schools aimed at improving students’ behavior, attitudes and work habits through direct teaching and testing.  A few years ago, motivated by the writing of some respected educators, schools in various parts of the country began to adopt programs that teach such things as self control, perseverance, and resilliance.  Afterward, researchers determined that teaching those skills improved student academic achievement by several percentile points. From then on the enthusiasm for such teaching has increased considerably.

At the same time, key promoters of social-emotional teaching, such as Angela Duckworth, strongly disapprove of testing, claiming that the desired behaviors cannot be  precisely defined or measured by any test.  The tests used so far, surveys asking students to evaluate their behavior in typical school situations, are highly susceptible to falsehoods, exaggerations, and mis-judgments.  Nevertheless, for the time being school officials in many states believe that teaching social skills is crucial to improving the performance of American students, especially those mired in family poverty, and for determing the effectiveness of schools. Their goals are to increase the teaching of social-emotional skills and to find ways of evaluating the results accurately and fairly.

Ravitch’s first statement sums up her judgment on this new school emphasis quite clearly: “Just when you think American education can’t get any nuttier, along comes another crazy idea.”  Later she says, “Think of the data! Parents will soon have data points about their children’s grit. Think of the data-based decision making!”  And she closes  by declaring that “Pearson and the other giants of the testing industry must be delighted. More tests!”

Although I agree with Ravitch  about this new trend in testing, I believe there is a bigger problem in the idea of direct teaching of the skills in question.  In my opinion, none of those attitudes and behaviors mentioned are actually “skills” that can be taught the way  reading, math, or science are taught.  Instead, they are students’ emotional responses developed over time through interactions with others who have some control over them in one way or another: parents, teachers, older siblings, school bullies, police officers, etc.  For kids in school their behavior depends so much on how they are treated by those in power.  When an older kid on the playground takes the trouble to teach someone how to hit a baseball or when a teacher invites a student to eat lunch with her so they can get to know each other better, the recipients of this positive attention are likely to do anything asked of them later on with a smile and considerable effort and persistance.  On the other hand, when a teacher treats a kid as lazy or disruptive, his or her reaction is almost always the determination to be worse than before.

I could go on by giving examples of students’ reactions to positive treatment from my own experience and observations of teachers in my schools and the schools where I later was an independent researcher.  In fact, I plan to do just that in some up-coming posts.  But for now I will stop, hoping that readers will think about my argument against the practice of  directly teaching social-emotional skills in terms of their own experience.