Did Columbia University Cheat Its Way to the Top Of the US News Rankings?

In February, a mathematician named Michael Thaddeus posted a long and detailed essay on his website. It attracted almost no attention. The major news media missed it. I think I can safely describe myself as squarely in Thaddeus’s target audience, but I only heard about it because I saw a link to the essay deep in the message boards of the running website LetsRun.com. (I happen to love LetsRun. But it is not usually the place where I learn about important scholarly investigations.)

So I read it, belatedly, and the goal of this newsletter is to get you to drop everything and read it too. (Warning. It’s long.)

Here’s the paper: “An Investigation of the Facts Behind Columbia’s U.S. News Ranking.”

And if you need further convincing, let me give Thaddeus’s argument a brief introduction.

Thaddeus is a tenured professor at Columbia University, so the first thing to know is that he’s writing about his own school. And he wants to answer a puzzle: Why is it that Columbia has risen so dramatically in the U.S. News rankings—the benchmark that everyone in higher education is irrationally obsessed with? If you compare the U.S. News list from its inception in 1988 to today, there has been remarkably little movement at the top. Harvard: Check. Yale: Check. Stanford: Check.

Except for Columbia. Columbia has zoomed way up—from Number 18 to Number 2.

Now, I know many of you are thinking, “Why should we care?” The U.S. News rankings are a really dumb system, based on a silly set of criteria. They’ve done nothing but cause mischief and distort incentives in American higher education. But that distortion is exactly what Thaddeus is trying to test. Did his school get so carried away with the ranking game that they engaged in something fishy to get ahead?

And his conclusion is: yes, they did. Or rather, in the characteristically careful words of a statistician: “There is reason for concern that [Columbia’s] ascendancy may largely be founded, not on an authentic presentation of the university’s strengths, but on a web of illusions.”

Thaddeus’s methodology involves going through each of the metrics used by the U.S. News ranking algorithm and looking closely at Columbia’s numbers. Let’s start with one of his relatively minor observations, just to get in the spirit of the occasion.

And to back up his analysis, he presents six paragraphs of explanation.

Now this bit about faculty degrees is puzzling, isn’t it? Why would an institution like Columbia make a claim like this, that’s so easily disproven with a Google search? Are they deliberately cheating? Or did someone just screw up? I was prepared to give them the benefit of the doubt. But then I kept reading.

U.S. News really cares (for reasons that are largely mysterious) what percentage of a school’s faculty is full time. Columbia told U.S. News that 96.5 percent of its faculty is full time, putting it well ahead of its main rivals, Princeton, Harvard, and MIT.

But colleges have to give that same information to the National Center for Educational Statistics, a federal agency, and Columbia tells the government that their full time percentage is 74.1 percent—which would be the lowest of any school in U.S. News’ top 100. Is it possible that the National Center for Educational Statistics has a different definition of “full-time” than U.S. News? Maybe. But now we’re up to two puzzling discrepancies.

OK. Here’s another.

The U.S. News algorithm is obsessed with how small an institution's classes are—specifically, what percentage of undergraduate classes have 20 students or fewer. (Again, what exactly the size of a classroom says about the quality of the teaching eludes me. But whatever.) Columbia says 82.5 percent of its classes fall into that category, a statistic that Thaddeus calls “extraordinary.”

But where does Columbia’s number come from? Most colleges post what is called the “Common Data Set” on their websites, which contains the raw data from which this calculation is made. Columbia, weirdly, does not. So Thaddeus was forced to put on his mathematician's hat and do some calculations based on other available data. And what did he find?

It gets worse. Much worse. Ten percent of the U.S. News ranking algorithm depends on “financial resources per student.” The more money you spend on your students, U.S. News assumes, the better your school is. (Once again, to belabor the obvious, this is an idiotic assumption. But, once again, whatever.) That metric—“financial resources per student”—is a combination of a number of different subcategories, among them the amount spent per student on instruction. Columbia claims to spend $3.1 billion on student instruction. As Thaddeus points out:

How is that possible? The salaries of Columbia’s non-medical, full-time faculty came to $289 million, so it's not like Columbia is simply the most generous employer in the Ivy League.

But here Thaddeus notices something that might explain where this astronomical number is coming from. He compares the “instructional spending” data Columbia submits to U.S. News with what it claims on its own financial statements, and here’s what he finds. In its own statements, Columbia says it spent $2 billion on “instruction and educational administration”—high, but far below the number it gives to U.S. News. Another $1.2 billion, meanwhile, was reportedly spent on “patient care” in the large hospital attached to the school. In its communications with U.S. News, however, the “patient care” line-item disappears. What Columbia appears to have done is lumped in its hospital costs with its teaching costs and sent U.S. News that shiny, new, astonishingly large number for the benefit of the algorithm.

Here’s Thaddeus’s crucial passage:

With that last bit about how “we cannot judge the propriety of this decision,” I think Thaddeus was just being nice. After we’ve seen the games Columbia plays with full-time faculty, class size, and a ton of other things Thaddeus digs into, I think it's fair to call a spade a spade. The administration of Columbia University is making stuff up for the purposes of advancing in the U.S. News rankings.

As many of you know, I have devoted episode after episode of my podcast Revisionist History to the foolishness and venality of American elite education. “Food Fight” and “My Little Hundred Million,” in Season 1, are about how elite universities and donors spend extraordinary amounts of money to create the smallest marginal benefits, when their dollars could be going so much further. “Puzzle Rush” and “The Tortoise and the Hare,” in Season 4, are about the over-reliance on standardized testing and what it misses. Last year, in Season 6, I did two episodes on the most ludicrous of all metrics used by the U.S. News operation, the so-called reputation score: “Lord of the Rankings,” in which statisticians help me crack apart the U.S. News formula, and “Project Dillard,” about an excellent historically Black university in New Orleans that scores low every year on the reputation metric.

The whole business is hopelessly corrupt. So maybe we should just roll our eyes at Columbia’s shenanigans and get on with our lives. I am reminded of Oscar Wilde’s famous description of the British upper classes and their obsessive fox hunting: “the unspeakable in pursuit of the inedible.”

On the other hand, Columbia is one of the most reputable universities in the world. They should not be playing shady accounting games in order to gain temporary advantage in the rankings game, no matter how absurd that ranking game may be. For goodness’ sake, the FBI and Justice Department spent many years on the Varsity Blues investigation, going after parents who cheated to get their children into elite universities. People paid massive fines in that case. Went to jail. Had their reputations ruined. But those parents weren’t cheating to get their kids into Boise State. They were cheating to get their kids into schools that they’d been convinced—by systems like the U.S. News rankings—were worth cheating to get into. And now at least one of the schools that parents think are worth cheating to get into, is cheating in order to be on the list of schools worth cheating to get into.

My heart hurts.

In my next newsletter, I’m going to ask for a response from Columbia. And I’m also going to ask U.S. News whether they’ve read Thaddeus’s paper. I mean, do they care? Or did they long ago give up on the pretense that theirs is anything but a hopelessly cynical enterprise?

Stay tuned.

[Photo: Getty Images]