Value Added, Value Lost?

Value-Added testing is enjoying increased popularity. But will this new approach help children learn better?

Educators often have difficulty specifying what is "good." Theyhave much less trouble with the concept of "better." To get "better,"they assess the current state of affairs, take that as a baselineand try to improve on it. This likely explains the great interestin the "value-added" model of teacher effectiveness, constructedby William Sanders at the University of Tennessee, which has beenin place in that state since 1992.

Sanders claims to have developed a technique for identifying thoseteachers who make kids "better" - they add value to the childrenby increasing the children's test scores. Some of the resultsare impressive: Children who have three consecutive years of whatSanders calls effective teachers have sharply rising test scores;students stuck with three years of ineffective teachers have plummetingtest scores.

Behind this apparently simple and precise outcome, though, aredifficulties and uncertainties. To begin with, since teachersare defined as effective on the basis of their ability to producetest-score changes, it should not surprise us that children whohave such "effective" teachers sequentially would have risingtest scores. It's circular.

More importantly, the entire model depends on the acceptance ofmultiple-choice tests as adequate and appropriate measures ofeducational outcomes and acceptance of changes in test scoresas the sole indicators of effectiveness. A much more appropriatelabel for the teachers who change test scores would be "test-effective."In practice, Sanders works only with norm-referenced standardizedtests because they produce the extended scale he needs for hisanalysis, namely, percentile ranks that run from 1 to 99. A scoringsystem like the 5-point Advanced Placement test scale or manystates' scales for writing assessment would be too broad, thoughin theory, performance tests could yield percentile ranks.

Sanders and his colleagues have argued that multiple-choice questionscan measure higher-order thinking and complex knowledge. Theycan, but only in rare settings. Usually they don't, and norm-referencedcommercial tests used by Tennessee (the California AchievementTest) and being considered elsewhere definitely do not assesshigher-order thinking.

Not only does the model rest on multiple-choice questions, itrequires that every child be tested every year in every subject.It is thus a budget drain on most school systems, but a boon fortestmakers.

What is badly lacking from the Sanders model is any research onwhat the test-effective teachers actually do to improve test scores.For the kinds of skills tested in the elementary years, the teachersmight well be relying on "drill and kill" work sheet activities.Such activities will indeed raise test scores, but few educatorswould call their use good pedagogy. Indeed, as summarized by psychometricianRobert Linn, the knowledge shown by gains in test scores doesnot generalize. That is, the skills are specific to the test.They do not transfer and do not represent increases in generalachievement.

NO INDEPENDENT EVIDENCE

Also missing from the model is any independent evidence that thetest-effective teachers are perceived as generally effective byparents, administrators and other teachers. Everything hangs ontest scores.

Where such tests are important in accountability schemes, teachingto the test will be even more prevalent. Important learning thatis not and often cannot be measured with multiple-choice testswill not be counted in the determination of value gained. Forschools that focus on more expansive and richer areas of learning,"value added" cannot represent what the school is trying to accomplish.In short, while ostensibly a means to assess progress in learning,"value added" reinforces the most narrowing aspect of testing- thereby reducing, not increasing, real value.

There are also possible problems with the model itself. We mustsay "possible" because Sanders has refused to tell anyone howit works. Indeed, he has contracted with a private firm to providehis analysis to school systems for a fee. This has greatly angeredassessment experts who would like to know how the model worksin order to improve on it, debunk it, or simply explore its possibilitiesand limits as is customary in the open world of research.

For instance, Sanders claims that because his model rests on priortest scores, it removes the impact of socio-economic status. Thatis, because it calculates changes in scores from, say, grade threeto grade four, he argues it is measuring changes independent ofwhere the child started from. This is debatable - some would arguethat the effects of socio-economic status are ongoing. Withoutaccess to the model, though, no debate is possible.

Suspicions about the utility of this particular value-added modelare increased by reports that crucial teacher quality statisticsare unstable. A teacher who is very effective one year, mightnot be the next year. This raises fundamental and vexing questionsabout the model's accuracy. Again, in the absence of open andscholarly debate, these questions cannot be addressed.

Finally, there are educational implications of the process thatgo beyond Sanders' specific model. As former U.S. Commissionerof Education Harold Howe put it in a letter written to the WashingtonPost, "In my view, what is really happening in our schools isthat the worship of accountability so dominates every aspect oflearning that it is narrowly defined into what can be measuredconveniently by standardized tests. Sanders appears to be givingit an effective but tragic boost."

Gerald Bracey is an educational consultant and writer based inAlexandria, Va. The above is reprinted with permission from theAugust issue of the FairTest Examiner.

Fall 2000