Since U.S. News & World Report began ranking colleges 30 years ago, that magazine has discovered a niche in the ratings industry. USN&WR ranks careers, hospitals, businesses and world leaders, but the magazine's biggest market has been its rankings of everything having to do with education -- undergraduate institutions, doctoral and masters programs, and public high schools.

Of course, given the commercial success of USN&WR's annual college rankings edition, competitors were bound to spring up. Fifteen years ago, Newsweek Magazine jumped into the fray by introducing the first national rankings of high schools.

Two weeks ago USN&WR released its best high school list and this week Newsweek and the Daily Beast published their annual high school rankings.

While it may be comforting to think that it's possible to qualitatively compare educational institutions, the reality is that the major rankings are subjective evaluations based upon largely arbitrary classification schemes. The problem is that people often ignore the evaluators' suspect methodologies and accept the ratings at face value.

For many years, Newsweek ranked high schools according to an index developed by Washington Post education reporter Jay Mathews. The Post continues to use the Mathews metric, in which schools are ranked based upon how many college-level tests the schools give. There is no attempt to evaluate how students perform on the tests, and many of the schools on Mathews' most recent list received poor ratings from their own state boards of education. Many of the schools failed to make Adequate Yearly Progress under the No Child Left Behind federal mandates.


Advertisement

Mathews nevertheless defends his rankings because they encourage schools to challenge students -- often beyond their capabilities -- and because his rating formula is easily understood.

Without a clear performance measure, however, ranking a school by how many tests it gives is comparable to judging school quality by the number of colors of paint on its buildings or varieties of fruit offered in its cafeteria.

At face value, the USN&WR and current Newsweek high school rankings appear to do a better job than the Post.

The U.S. News rankings purport to measure the schools that are best at serving all their students and were derived in three steps. First, schools are evaluated as to whether their students perform better than statistically expected based on the economic mix of the school when compared with other schools in the state.

Second, schools are evaluated as to how their non-Asian minority populations perform relative to state averages. Finally, schools are judged based on how well they prepare students for college using results from the International Baccalaureate (IB.) and Advanced Placement (AP) exams.

Newsweek has modified Mathews' formula, emphasizing what it calls the best indicators of college readiness: graduation rates, participation in college-level classes via AP and IB. programs, and acceptance into a two- or four-year college program.

The top-rated school nationally, according to the USN&WR rankings, is the School for the Talented and Gifted (TAG) in Dallas. TAG ranks fifth according to Newsweek and third on the Washington Post list. TAG is a magnet school -- as are many of USN&WR's, Newsweek's and the Post's top high schools -- generally accepting only the smartest applicants. So, TAG and many of the "nation's best public high schools" accept an economic and racial cross-section of academic all-stars.

Does that mean that TAG is a great high school or that others on the various lists are great? Not at all. None of the rankings attempt to measure students' progress, so it's impossible to know how the supposedly best high schools contribute to students' growth.

Paraphrasing the famous aphorism about the first President Bush, that he was born on third and thinks he hit a triple, the high schools on the various lists begin miles ahead of the game and are then judged as if they started along side all other public high schools. Without a value-added measurement though, the rankings tell us nothing about how well the schools serve the students they've picked.

School rankings may sell magazines and newspapers and popularize websites, but no one should be fooled into thinking that they are good measures of school quality.

Patrick Mattimore taught high school in the Bay Area for many years and now lives in Thailand.