èßäÉçÇø

What if a surgeon’s medical license could be taken away based on an error-prone statistical formula that ranked his abilities on a scale of 1 to 5, based on the success (or failure) of a small number of the operations he performed? Or imagine if a lawyer could lose her membership to the bar because a statistical estimate of her success predicted that she would lose the majority of her cases next year.

When teachers realized that these erratic scores could put their careers on the line, they started getting worried.

Last year, public school teachers in Tennessee faced precisely that situation, but they didn’t take it lying down. Instead, they started a year of creative actions that led to a decisive change in policy—despite a governor determined to keep an unreliable statistical formula as a key method of evaluating teachers.

Their campaign ended successfully on April 24, when Governor Bill Haslam signed a bill rolling back the use of a statistical instrument known as TVAAS in teacher licensing decisions—and hitting the pause button on an important facet of the testing trend in Tennessee, at least for the moment.

Education experts are divided as to what this development will mean for America’s schoolkids. But many believe that it could spark similar campaigns nationwide.

“The change in Tennessee sends a message about politics,” said Dan Goldhaber, director of the Center for Education Data and Research at the University of Washington. “It will embolden people in other states who think that tests ought not to be used for teacher evaluations to continue the pushback.”

According to Bob Peterson, president of the Milwaukee teachers’ union, the development in Tennessee is just one piece of the puzzle. “The success of the pushback in Tennessee is one part of the larger growing movement for testing reform, against the use of standardized tests to pigeonhole and sort our students, and to scapegoat our public schools and teachers,” Peterson said. “New York, California, Oregon—there’s growing grassroots activity.”

From “chickens and cows” to public education

The story begins in the early 1990s, when the state of Tennessee hired Dr. Bill Sanders, then a statistician at the University of Tennessee’s program in agriculture, to develop a statistical tool that could measure how well teachers were doing their jobs. So Sanders created the Tennessee Value Added Assessment System.

Also known as TVAAS, the state adopted the system in 1993, and school district administrators and board members used it as a diagnostic of how schools and teachers were performing.

While Sanders was a perfectly good statistician, some have cracked wise about his background. “His initial forays into statistical modeling were based on livestock—chickens and cows,” said Jim Wrye, manager of government relations for the Tennessee Education Association, the largest professional association of teachers in the state.

More than 400 teachers  went to Nashville during spring break to personally talk to legislators.

In the context of increasing federal pressure to emphasize testing in public schools, TVAAS was seen as a limited tool. It gathered information about how a given teacher’s students were doing on the state’s standardized test and spat out a number from 1 to 5, with 5 being best. It showed in a general way how well teachers were preparing students for the test. Soon other states began adopting systems based on TVAAS.

But then, beginning in 2010, Tennessee made several policy decisions that changed the role TVAAS played in teacher’s lives.

The reason for the change was a federal initiative called Race to the Top, introduced in 2010, which put states in competition for education funding. The states that came up with the best education reform plans—defined by a set of goals laid out by the federal government—would win a large public education grant. Among the goals of Race to the Top was teacher accountability. To meet that goal, United States Secretary of Education Arne Duncan encouraged school districts to use student test scores to decide whether teachers could receive or renew their teaching licenses, among other things.

In order to win Race to the Top funding—the state was awarded more than $501 million—Tennessee turned TVAAS scores into a major component of teacher evaluations. While 51 percent of a teacher’s total score still came from an observation—in which a school district official would visit a school and watch a teacher in action—TVAAS scores were now worth 35 percent.

In August 2013, the state Board of Education voted to adopt a policy, introduced by State Education Commissioner Kevin Huffman, which placed even more weight on TVAAS scores. Under this new policy, if a teacher scored lower than 2 out of 5 points for three years in a row, she would not be eligible to renew her license.

Before that decision, TVAAS scores didn’t cause much trouble because they were only a diagnostic, with no real consequences. But when teachers realized that these erratic scores could put their careers on the line, they started getting worried.

“The most valuable possession of a teacher is her license,” Wrye said. “A license allows her to do what she loves. It’s worth more than her house, it’s worth more than her car. And when you decide to take it away, you better have a rock-solid reason.”

Teachers saw TVAAS as anything but rock-solid. Like other statistical measurements, it has what’s known as a “standard error.” In other words, a teacher could receive a score of 5 for a given school year but actually deserve only a 3, according to other components of her evaluation. By the same token, a teacher could receive 3 points for a given year, but actually deserve a 5. Errors of this nature are built into statistics, but the likelihood that this would occur was relatively high for TVAAS.

The system was so unpredictable, said Wrye, “It was sometimes viewed as the crazy aunt in the basement.”

Wrye also said the system unfairly punished teachers who work in the neediest communities. “Places with high poverty, places with low resources, places that have a high influx of English language learners—there are places where, no matter what the teacher does, they will not have very good test scores or TVAAS scores.”

Last year, complaints about TVAAS began trickling in to the TEA. As the association investigated individual cases, they discovered how the system’s standard error, among other kinks, was affecting teachers, according to TEA President Gera Summerford. In one instance, eighth-grade teacher Mark Taylor was denied a bonus because of poor scores, even though the scores only reflected the work of about 16 percent of his students.

These kinds of complaints made it clear that the August 2013 decision—set to take effect in 2015—could cost a teacher her license and perhaps her whole career because of an erratic and often inaccurate statistical calculation. So the TEA launched a campaign to urge the Tennessee legislature to reverse the policy and ensure fairness in teacher evaluations.

How they did it

The TEA started by doing what teachers do best: educating. “What we ended up doing is, first and foremost, explaining to the legislature what exactly TVAAS was,” said Jim Wrye, who was among the TEA leaders who spearheaded the campaign.

On January 23, two TEA allies, Senator Mike Bell and House Rep. Mathew Hill, presented an initial bill, the Educator Respect and Accountability Act of 2014, that would completely remove student standardized test scores from teacher evaluations.

The campaign highlights the need for testing data to be used more carefully.

But because of pressure from a minority of legislators who supported the existing policy, the bill was . It amended the law so that the state department of education could not revoke a teacher’s license based on her TVAAS scores. It also took away the state Board of Education’s authority to adopt policies on teacher licensing.

Meanwhile, on January 31, the state Board of Education met to finalize new rules for teachers’ licenses. During that meeting, they rescinded the policy they had approved in August 2013.

But that reversal was only the beginning. The modified bill still had to survive a gauntlet of committees, subcommittees, and hearings.

On February 4, Jim Wrye and TEA General Counsel Rick Colbert gave a before the state’s House Education Committee, laying out the major problems with TVAAS. Other TEA leaders gave a similar presentation for the Senate Education Committee.

The TEA also created a petition asking the governor to treat teachers as professionals.

Once members of the legislature understood that TVAAS was not a state standardized test but rather an unreliable statistical estimate derived from standardized test scores, the bill gained widespread support. Within two weeks, 88 out of 99 members of the Tennessee legislature, including both House representatives and senators, signed on as co-sponsors.

Despite the growing support for a policy change in the legislature and in the Board of Education, the governor was determined to keep TVAAS as a core component of teacher evaluations. Tennessee Education Commissioner Kevin Huffman was the leading proponent of this policy, and Governor Haslam personally lobbied senators, urging them to support the commissioner’s position.

“One senator said he had spent over 10 years in the general assembly and [had] never been called to the governor’s office on a particular bill, before this one,” Wrye said. “He still voted for us in committee.”

The TEA leaders didn’t do all the talking. The association organized more than 400 teachers who went to Nashville during spring break to personally talk to legislators about TVAAS and other public education issues like charter schools and vouchers.

The TEA also created a petition asking the governor to treat teachers as professionals. After nearly , the TEA delivered it to Governor Haslam’s office with comments outlining the key reasons that the TEA’s licensure bill had gained such widespread support in the legislature.

The final twist of the screw came in the form of two lawsuits that the TEA filed against Governor Haslam and Commissioner Huffman. The lawsuits were based on separate complaints that two teachers—one of whom was eighth-grade teacher Mark Taylor, mentioned above—had brought to the association about TVAAS. In both cases, the teachers lost bonuses they believed they deserved, due to poor TVAAS scores that did not represent the full extent of their work.

TEA President Summerford stressed that the lawsuits did not address teacher licensure rules, the issue at the heart of the association’s licensure bill.

However, they “bring into sharp focus … the fundamental flaws of TVAAS in making high stakes decisions for teachers,” Wrye said.

In the end, there were only six “no” votes on the TEA’s licensure bill in the Tennessee legislature, out of a total of 132 voting members. And when Governor Haslam signed the bill on April 24, teachers across the state celebrated the development as a major policy victory.

“It showed tremendous care and support by the rank and file senators and House members for their teachers,” said Wrye.

Federal pressure to emphasize testing in public schools will continue to affect education policy in Tennessee and in the rest of the country. But the TEA’s campaign highlights the need for testing data to be used more carefully and demonstrates the power of teachers to act effectively when it is not.