College rankings

You may have heard that the US News college rankings came out today.  I used to love and be fascinated by these things.  I think Duke may have even made it as high as #4 back in my day (surely because they were a bit better at gaming the rankings back then).  Then I learned better.  The rankings do have some value (Harvard really is better than East Carolina), but they are ultimately an incredibly blunt instrument.  And they don’t really tell us what the “best” colleges are (presumably, best would mean provide the best education), but rather which colleges are the most prestigious (surely correlates with quality education,but I’d guess r=.4 at best) and the richest.

Meanwhile, at the Atlantic John Tierney provides a handy (and lengthy) summary of what’s wrong with these ratings.  And there’s a lot.  Among those shortcomings I find most compelling:

• The rankings don’t take into account measures of the quality of education at each institution, nor is there any consideration of “outcomes” (for example, what do students at College X actually learn, and do its students get jobs upon graduation?). This weakness in the rankings earned a recent slam from Education Secretary Arne Duncan and comment from President Obama when he announced in August the Administration’s own proposalfor rating colleges.
• A very substantial chunk (22.5 to 25 percent) of an institution’s ranking comes not from any hard data but from a “reputational” measure, in whichU.S. News solicits “peer assessments” from college presidents, provosts, and admissions directors, as well as input from high-school counselors. U.S. News claims that by giving “significant weight to the opinions of those in a position to judge a school’s undergraduate academic excellence,” the rankings allow for the inclusion of “intangibles” such as “faculty dedication to teaching.” Critics say this component turns the rankings into a popularity or beauty contest, and that asking college officials to rate the relative merits of other schools about which they know nothing becomes a particularly empty exercise because a school’s reputation is driven in large part by – you guessed it – the U.S. News rankings. According to Malcolm Gladwell, this reputational measure is simply a collection of “prejudices” that turn the U.S. News rankings into a “self-fulfilling prophecy.
• The whole exercise implies a sort of authoritative precision and rigor that impart real meaning to the rankings, but that it is simply nonsense to say that, for example, Duke “ranks higher” than Johns Hopkins or that Middlebury should rank 13 spots higher than Wesleyan.
• Because the rankings have a popular audience, they encourage colleges and universities to game the system – i.e., to do what they can to raise their place in the rankings by, for example, spending lots of money on things theU.S. News formula deems important or by aggressively increasing the size of their applicant pool so they can turn away a higher percentage of their applicants, thus showing themselves to be “more selective” and thereby raising their rank.

That gaming is definitely a huge part of it.  If US News gives weight to class sizes under 20, many universities will literally cap classes at 19 in response.  If class sizes over 50 are penalized, look for lots of caps right at 50.  I certainly took these ratings a helluva lot more seriously before 1) I became an actual social scientist and learned about concepts such as measurement and external validity and 2) actually spending the last 23 years of my life in a variety of academic institutions.

One thing I will say is that, ceteris paribus, better students does equal a better education.  You really do benefit considerably as a college student from having brighter, more engaged peers.  To me, that’s the real advantage of going to a Harvard or Duke.  The fact that they are doing any better job educating you there, I’m largely dubious of.  Though, not entirely dubious.  Insofar as the students are brighter and more ambitious, the faculty can more comprehensively and consistently challenge their students without fear of leaving a huge chunk of the class behind.  Like, I said, there is some there there in these rankings, but they need to be taken with a shaker’s worth of salt.

When it comes time for my kids to make college decisions, I’ll be far more interested in the Washington Monthly rankings.