When I first read the news story in the NY Times that makes up most of today’s post, I loved it. It makes good points about the problems of using data to judge the effectiveness of doctors and teachers. However, when I read it again today, I realized that it does not go far enough–especially in reference to teachers. The last paragraph really disappointed me in calling for “more targeted measures” rather than advocating for abandoning measurement altogether. So, I decided to write a new concluding paragraph explaining what would be a better way to evaluate doctors and teachers, who–by the way– are not machines in “industries” but complicated human beings.
Two of our most vital industries, health care and education, have become increasingly subjected to metrics and measurements. Of course, we need to hold professionals accountable. But the focus on numbers has gone too far. We’re hitting the targets, but missing the point.
Through the 20th century, we adopted a hands-off approach, assuming that the pros knew best. Most experts believed that the ideal “products” — healthy patients and well-educated kids — were too strongly influenced by uncontrollable variables (the sickness of the patient, the intellectual capacity of the student) and were too complex to be judged by the measures we use for other industries.
By the early 2000s, as evidence mounted that both fields were producing mediocre outcomes at unsustainable costs, the pressure for measurement became irresistible. In health care, we saw hundreds of thousands of deaths from medical errors, poor coordination of care and backbreaking costs. In education, it became clear that our schools were lagging behind those in other countries.
So in came the consultants and out came the yardsticks. In health care, we applied metrics to outcomes and processes. Did the doctor document that she gave the patient a flu shot? That she counseled the patient about smoking? In education, of course, the preoccupation became student test scores.
All of this began innocently enough. But the measurement fad has spun out of control. There are so many different hospital ratings that more thatn 1,600 medicalceners can now lay claim to being included on a “top 100,” “honor roll,” grade “A” or “best” hospitals list. Burnout rates for doctors top 50 percent, far higher than other professions. A 2013 study found that the electronic health record was a dominant culprit. Another 2013 study found that emergency room doctors clicked a mouse 4,000 times during a 10-hour shift. The computer systems have become the dark force behind quality measures.
Education is experiencing its own version of measurement fatigue. Educators complain that the focus on student test performance comes at the expense of learning. Art, music and physical education have withered, because, really, why bother if they’re not on the test?
At first, the pushback from doctors and teachers was dismissed as whining from entitled and entrenched guilds spoiled by generations of unfettered autonomy. It was natural, went the thinking, that these professionals would resist the scrutiny and discipline of performance assessment. Of course, this interpretation was partly right.
But the objections became harder to dismiss as evidence mounted that even superb and motivated professionals had come to believe that the boatloads of measures, and the incentives to “look good,” had led them to turn away from the essence of their work. In medicine, doctors no longer made eye contact with patients as they clicked away. In education, even parents who favored more testing around Common Core standards worried about the damaging influence of all the exams.
Even some of the measurement behemoths are now voicing second thoughts. Last fall, the Joint Commission, the major accreditor of American hospitals, announced that it was suspending its annual rating of hospitals. At the same time, alarmed by the amount of time that testing robbed from instruction, the Obama administration called for new limits on student testing. Last week, Andy Slavitt, Medicare’s acting administrator, announced the end of a program that tied Medicare payments to a long list of measures related to the use of electronic health records. “We have to get the hearts and minds of physicians back,” said Mr. Slavitt. “I think we’ve lost them.”
Thoughtful and limited assessment can be effective in motivating improvements and innovations, and in weeding out the rare but disproportionately destructive bad apples.
But in creating a measurement and accountability system, we need to tone down the fervor and think harder about the unanticipated consequences.
Measurement cannot go away, but it needs to be scaled back and allowed to mature. We need more targeted measures, ones that have been vetted to ensure that they really matter. In medicine, for example, measuring the rates of certain hospital-acquired infections has led to a greater emphasis on prevention and has most likely saved lives. On the other hand, measuring whether doctors documented that they provided discharge instructions to heart failure or asthma patients at the end of their hospital stay sounds good, but turns out to be an exercise in futile box-checking, and should be jettisoned.
************************************************
Measurement is not the proper tool for judging any form of human behavior. Doctors and teachers are not automatons. They have good and bad days, work under continually changing conditions, make errors and have lucky guesses; all the while trying to have a positve effect on other human beings who are as complex and vulnerable as they are. The best we can do in assessing the competence of such professionals is to examine the realities of each situation, then choose the best overseers and clients from within and trust their judgments. In medicine those people are the department heads and patients; in education they are the school principals and students. The results will not be perfect, but they will be better than trying to measure the immeasurable.
The Treasure Hunter 
Leave a Reply