Last week’s revelation by Columbia University that it would not be participating in US News and World ReportThe upcoming ranking of colleges due to unanswered questions about the accuracy of its data is reverberating through the halls of higher education. One of the questions raised by this decision is whether other institutions might reconsider their own participation in the annual ranking system.
Here’s what happened at Columbia and why other colleges would be wise not to just review the reliability of the data they submit to American News but perhaps reconsider if they will continue to participate in the classification.
In February of this year, Michael Thaddeus, professor of mathematics at Columbia, published a long critique of the US News and many data that Columbia had submitted for the ranking, which placed it second (tied with Harvard) among the best universities in the country for 2022.
Thaddeus compared the institutional data to the numbers Columbia had sent and questioned the accuracy of several of the self-reported numbers, including the number of classes of different sizes, the percentage of faculty with a terminal degree, the percentage of courses taught by full-time students. faculty, student:faculty ratio, and how much Columbia has spent on teaching.
Initially, Columbia defended its data. But on Thursday last week, just before the July 1 deadline for submitting new data to american news, Columbia changed its tune and, in a statement from Provost Mary Boyce, said it would not be submitting data this year. The statement said the university had “begun a review of our data collection and submission process.”
The statement continued, “Columbia has long conducted what we believed to be a thorough process of collecting and reporting institutional data, but we are now closely reviewing our processes in light of the issues raised. Continuous review is a matter of integrity. We won’t take any shortcuts to get it right.
The university has pledged to release a common dataset this fall that would include much of the same information. American News blankets. That’s an interesting development on its own, given that Columbia — unlike most universities — hasn’t made its common dataset public. Columbia’s statement also did not say if or when it would participate in the American News process again.
Too bad for Colombia. What about other colleges and universities? Will Columbia’s decision not to participate cause more schools to question the veracity of their data and perhaps cause them to walk away – at least temporarily – from the American News rankings.
It could – and it should – for at least two reasons. First, you can be sure that there are more faculty or employees like Michael Thaddeus who are now inspecting their own colleges’ data with a keen eye for errors, discrepancies, and fraud. And there’s a good chance that at least some of these sleuths will uncover suspicious patterns, casting doubt that their institutions have tampered with the data to make it look better and rank higher than it deserves.
The damage done to a college’s reputation when it is revealed that it has deliberately or even unintentionally submitted inaccurate data is considerable, and as many university officials will recognize – at least in private – the process by which the data institutional assessments are internally reviewed/verified is far from robust. Columbia’s situation illustrates the dilemma – the risk of being charged or admitting to providing fraudulent data likely outweighs the risk of not participating in the ranking.
Second, the problem of universities submitting false data has been uncovered often enough to raise serious questions about the value of rankings. For example, in the last decade alone,
- The University of Southern California removed the Rossier School of Education from US News and World ReportThis is the 2022 ranking after discovering “a history of inaccuracies” in research and student enrollment data that he had reported for at least five years. In response, American News said the university would need to provide a letter certifying the accuracy of Rossier’s data submissions for the next three ranking cycles in order to be included.
- Moshe Porat, the former dean of Temple University‘s Fox School of Business, was convicted in 2021 of wire fraud and conspiracy to commit wire fraud for his role in falsifying data to improve his school’s rankings.
- In 2019, American News said they moved five universities to its “unranked” category because they had faulty data used to calculate their ranking. The schools were the University of California at Berkeley, Scripps College, Mars Hill University, University of North Carolina at Pembroke and Johnson & Wales University.
- Other recent and important “false journalists” cited by American News include the Oklahoma University, the University of Missourithe University of Virginia, and Washington University (Saint Louis).
- Emory University admitted in 2012 that its administrators had knowingly misreported information about SAT and ACT scores, class rankings, and grade point averages of incoming students for a number of years.
- That same year Claremont McKenna College revealed that one of its vice presidents had been misrepresenting admissions data for several years.
No one believes that the universities that got caught are the only culprits. But how big is the problem of questionable or doctored data? It’s unheard of, and most institutions aren’t motivated to dig too deep into their reporting methodology to find out if they might be cheating.
American News claims that only a “very small proportion” of ranked universities have submitted false data, but in February this year it listed a total of nearly 50 institutions that it had not ranked in various categories since 2018 due to erroneous data. At some point, a few more university leaders might conclude that this is a club their schools don’t need to belong to.
The college ranking industry has been criticized for years, often on the grounds that “gambling” or cheating the system is far too easy and common. For example, Colin DiverBreaking Ranks: How the Rankings Industry Rules Higher Education and What to Do About It is a fiery and often witty critique of the college ranking industry, particularly the neighborhood “gorilla”, American News.
Driver is the former president of Reed College, which made news in 1995 when he elected, under the guidance of a former president, to no longer participate in American News‘ rankings. This spurred hope among ratings critics that others would follow. A few schools have done so, but the vast majority of American institutions continue to play what many consider a pernicious game.
Perhaps Columbia University’s decision to skip next year’s rankings will encourage other university leaders to take a second look at their future participation. They know there are more Michael Thaddeus wannabes, maybe even on their own campuses.