Columbia Fudged Its Numbers to Improve Its Ranking. What Should Happen Next?

My last newsletter was about the allegation by the mathematician Michael Thaddeus that the school where he teaches—Columbia University—had been engaged in shenanigans over the data it submits to U.S. News for the magazine’s annual college rankings. Columbia currently stands at Number 2 on the list, behind only Princeton, up from Number 18 when the rankings began. Thaddeus suggested that a good part of that remarkable rise was due to, essentially, making stuff up.

At the time I wrote my piece, Thaddeus’s critical study had been almost completely overlooked by the mainstream media for several weeks. That is no longer the case. The New York Times has now weighed in. So has the Chronicle of Higher Education. The Washington Post and Yahoo have covered it. Was Oh, MG the reason? A man can dream!

This week, I have a brief follow-up, followed by a new proposal—because, if you follow Revisionist History, you know I’m an anti-U.S. News zealot of long standing.

So for your edification, three quick takeaways.

1. How lame is U.S. News?

Very lame. This was true long before Columbia was accused of pulling strings, but Thaddeus’ analysis just makes the lameness all the more apparent. The whole ranking algorithm devised by U.S. News is absurd. What I hadn’t realized, though, was just how cynical its enterprise is.

I emailed Bob Morse, the rankings czar at U.S. News, to ask for his response to Thaddeus’s allegations. He replied:

In other words, U.S. News will happily profit off the rankings. Bask in the glory that they bring to their enterprise. Furrow their brows and pretend that they are providing an objective, sober assessment of the quality of America’s universities. But they can’t be bothered to check to see if any of the data being sent to them is accurate.

Late last year, the dean of the business school at Temple University, Moshe Porat, was found guilty of submitting false data to U.S. News to boost its ranking. You know what gave him the idea? In 2013, he sent three of his employees to Washington D.C., where they met with representatives from U.S. News to complain that their rankings were too low. And U.S. News told the Temple employees: We won’t adjust your ranking for you. But, just so you know, we don’t audit the data you send us. We don’t have the staff.

Porat and his colleagues learned, in other words, that all police officers had been removed from I-95, so they did what I'm guessing lots of us would do under those circumstances. They sped up.

In one of the depositions in the Porat case, a colleague of his describes what happened when the Temple brass gathered for a champagne party in 2018, toasting their dramatic rise to Number 1 in the online-MBA rankings. When one of Porat’s colleagues asked him if he felt the honor was unearned, he replied: “Well, if they haven’t caught it… what makes you think they will catch it now?”

Well yes, Dean Porat. That does not make your behavior any less reprehensible.

Still, if everyone is driving 100 mph on I-95 because all the traffic cops are on holiday, I think our problem has more than one culprit.

2. Did Columbia University commit a crime?

The clear point of comparison here is the Temple University case. There, Moshe Porat appears to have brazenly orchestrated a scheme to game the rankings, by submitting data that he just made up. For example, when it comes to online MBA programs, the more students you have who took the GMAT, the more points you get from U.S. News. Temple didn’t have a particularly high number of students who’d taken the GMAT. So Porat decided to arbitrarily raise the number to… 100 percent. He rounded up!

So how does Columbia’s behavior compare?

I wrote in my last newsletter about another one of Thaddeus’s brilliant observations, which has to do with faculty credentials. Columbia claims that 100 percent of its faculty hold the terminal degree in their field. (This percentage is another tidbit U.S. News claims to care a lot about.) The other elite schools like Harvard and Princeton report a “terminal-degree rate” in the range of 90 percent. But Columbia? 100 percent! Thaddeus was completely baffled as to how Columbia came up with that number, since a quick glance at its faculty list revealed dozens upon dozens of teachers without Ph.Ds.

But in the New York Times’ recent article on the case, we have an answer to the puzzle:

Let’s pause here. Columbia seems to be giving us two key insights. First: U.S. News claims to have strict criteria for judging academic merit, and rewards colleges according to the percentage of professors who hold the highest degree in their field. But whatever “highest degree possible” means, it turns out, is up to the college to decide. It’s not like U.S. News is measuring educational quality based on a set of actual standards. You get to make things up.

Second—and this is the best part—after coming up with an arbitrary answer to U.S. News’ arbitrary criterion, Columbia arbitrarily decided to… round its answer up to 100 percent!

Moshe Porat rounded up—a lot—and for this, he’s going to jail. Is Columbia in the same league? I think this is for the lawyers to answer. But from where I’m standing, it’s a difference in degree, not in kind.

Here’s a better example. The real smoking gun in Thaddeus’s analysis is his description of how Columbia gamed the “financial resources per student” category, which counts for a lot in the U.S. News algorithm. (U.S. News likes it when you spend lots of money on your students.)

As I wrote in my last newsletter, Thaddeus points out that the amount Columbia claims to devote to instruction is absurdly, crazily high. And he figured out why: Several years ago, the school abruptly changed the way it categorized its expenditures in annual reports to the U.S. Department of Education. As a result, spending in the “instruction” category seemed to shoot way up. The crucial chart in Thaddeus’s work is this:

As you can see, the spending data breaks down a little differently in two expense reports: one sent to the Department of Education—which is the dataset U.S. News uses to record instructional spending—and one filed in Columbia’s own records. There’s nothing automatically wrong with that. Different auditors sometimes want to track data in different ways.

But look at the individual line items on the right side, in Columbia’s consolidated financial statements. One is for “instruction and educational administration.” It comes to $2,061,981,000. Just over $2 billion. Then they have a line item for “patient care expense,” which comes to $1,215,438,000. Add those two numbers together and you get $3,277,419,000. OK. Keep that number in mind.

Now look at the figures on the left side. These are numbers that Columbia supplies to the DOE. The DOE doesn’t designate a separate category for “patient care expense.” There’s just a single figure for “Instruction,” which comes to… $3,126,101,000. Huh.

So Thaddeus says: Columbia’s clearly padding their instruction numbers. They’re saying that, at Columbia, patient care counts as instruction. And as a result, they should be able to add that $1.2 billion to the amount they report spending on teaching history and economics to undergraduates. This puts their per-student spending numbers far above those of any other school.

Is that as bad as what Porat did at Temple? Again, I’m not a lawyer! But this might be a good time to resuscitate that old cliche about ducks quacking and walking.

3. What should happen next?

This is actually a much more complicated question than it seems. The conventional answer is that U.S. News should audit the data it collects from schools, in order to catch the cheaters and the line-crossers. If U.S. News wants to maintain its lofty position as arbiter of higher education, they should at least have to lift a finger to protect the integrity of their creation.

Except… the U.S. News system has no integrity! How can we ask them to protect something that they don’t possess in the first place?

Surely the lesson of the Columbia debacle is that the U.S. News rankings are so irredeemably ridiculous, we should be looking for a way to kill them—not save them.

How do we do that? Everyone should cheat!

First of all, a ranking system only works if it measures a variable on which there is variation. You can’t rank college applicants based on whether they can read, because they all read. You need a discriminator that discriminates. The reverse is also true. The best way to kill a ranking system is to turn discriminators into non-discriminators. Meaning: Everyone should round up to 100 percent, whenever possible.

Here’s another way to do it. The stunt that Columbia pulls where it pretends that the medical resident taking out Johnny’s appendix is actually a “teacher” instructing a “student”—whether or not there are any students in the room—is a game that appears to be played by only a small number of colleges. New York University, for example, which has a very large medical facility of its own, doesn’t pull this same accounting trick, according to Thaddeus. Memo to NYU: Get on it! U.S. News isn’t checking. And Columbia has gotten away with this for years. What are you waiting for?

In retrospect, the fact that the prosecutors in Philadelphia went after the Temple University dean last year was a mistake. They should have done exactly what Thaddeus has done: expose to the rest of the world the shady strategies the cheaters are using, and then walk away. The only way this most stupid of American institutions will ever go away is if all of the animals in the barnyard jump in the slop with the pigs and we all get dirty together.

[Header Photo: Barry Winiker via Getty Images]