Make Your Own College Rankings with this DIY Guide

College rankings are controversial, and no list can claim to be objective, authoritative, or right for every student. Check out Noodle Expert Mat Cusick’s guide to creating a list that suits your needs and desires — academic and otherwise.

Every September since 1983, U.S. News & World Report has published its famous college rankings.

They have been a source of controversy since the very beginning. You can refer to the Atlantic’s “Your Annual Reminder to Ignore the U.S. News & World Report College Rankings” for a summary of major objections — which range from the annual list’s role in driving up college costs to its failure to take into account what, if anything, students actually learn at the nation’s “best” colleges. Leaders in higher education have long complained that the rankings offer an arbitrary if not outright misleading assessment for potential students, and a perverse incentive for colleges to conform educational policies to meet the metrics — effectively gaming the system.

Even President Obama has stated that the U.S. News rankings list "encourages a lot of colleges to focus on ways to . . . game the numbers, and it actually rewards them . . . for raising costs," while announcing his administration's plan to develop its own college rankings based on affordability and value.

“Criticism of the rankings is nearly unanimous,” writes Colin Diver, president of Reed College and former dean of the University of Pennsylvania Law School, “but so is compliance with them.” A college’s position in the U.S. News rankings may have such an outsize impact on its enrollment and funding that schools participate despite serious objections from administrators and professors. “The rankings are merely intolerable,” said one college president to Diver, but “unilateral disarmament is suicide.”

Don't Believe the Hype

The stakes are so high that schools are compelled not only to conform to the report’s criteria, but also to manipulate the variables that they can control so as to rise in the rankings. Some colleges even award administrative bonuses if their schools move up on the list.

Baylor University went one step further and paid admitted students to retake the SAT in a transparent ploy to increase the school’s average score and influence its rank. A senior administrator at Claremont McKenna College simply inflated the SAT scores he sent to the U.S. News & World Report survey, admitting to having falsified the reports for seven years. Even when schools go test-optional, they may (at least in part) be motivated by the bump in U.S. News rankings that is likely to follow.

Since every school administrator understands how a few changes in some arbitrary indicators can affect a school’s ranking, altering or withholding information is not uncommon. As Colin Diver observes:

Such practices have included failing to report low SAT scores from foreign students, ‘legacies,’ recruited athletes, or members of other ‘special admission’ categories; exaggerating per-capita instructional expenditures by misclassifying expenses for athletics, faculty research, and auxiliary enterprises; artificially driving up the number of applicants by counting as a completed application the first step of a ‘two-part’ application process; and inflating the yield rate by rejecting or waitlisting the highest achievers in the applicant pool (who are least likely to come if admitted).

News reports over the years have recorded a litany of even more nefarious tactics for manipulating the rankings.

After reading a 1995 Wall Street Journal report about the rampant manipulation of rankings by colleges, Reed’s former president Steven Koblik chose to stop submitting data to U.S. News, asking that the magazine omit the school from consideration. “Instead, the editors arbitrarily assigned the lowest possible value to each of Reed’s missing variables,” recounts Diver, “with the result that Reed dropped in one year from the second quartile to the bottom quartile.” (You can read more on this from the school itself.) Reed carried on as usual in spite of the rankings retaliation, and has thrived without the meddlesome influence of the U.S. News survey, Colin Diver attests.

In 2007, an association of liberal arts colleges known as the Annapolis Group announced that a majority of the 80 school presidents in attendance had “expressed their intent not to participate in the annual U.S. News survey,” joining Reed in resisting the rankings. As Diver explains, the fundamental opposition comes from an understanding that “higher education is not a mass-produced commodity but an artisan-produced, interactive, and individually tailored service of remarkable complexity. Trying to rank institutions of higher education is a little like trying to rank religions or philosophies. The entire enterprise is flawed, not only in detail but also in conception.”

The Devil in the Details

The real issue, according to Ellen Hazelkorn, Dean of the Graduate Research School at the Dublin Institute of Technology and author of “Rankings and the Reshaping of Higher Education: The Battle for World-Class Excellence,” is that “there is no such thing as an objective ranking.” An obvious problem with attempting to rank colleges objectively is that not everything that matters to the ranking committee will matter to you, and vice versa. Even quantitative criteria that seem to reflect something important to you might not get to the heart of the matter qualitatively.

For instance, the average SAT and ACT scores for a school’s student body — which receive significant rankings weight — may seem an important indicator of whether or not you’ll be surrounding yourself with smart, engaging colleagues. But there are a lot of problems with such a proxy criterion: We know that there are many kinds of intelligence, of which these tests assess just a narrow range; that not all smart people score well on these tests; and conversely, that not all people who score well on the tests are always engaging to be around. SAT scores correlate more closely with family income than anything else. Moreover, if what you truly seek is a social group of high-achieving students, you might be better served by an honors program at a school with comparatively low average student SAT scores than at a school that tops the U.S. News rankings.

In fact, an economics professor at Princeton University and a researcher at Mathematica Policy Research found that students who chose a school with lower admissions standards over a more selective school earned just as much income as those who attended elite schools — although alumni earnings are obviously not the only outcome worth evaluating when choosing a school. Paradoxically, these researchers discovered that the most competitive school that rejects you “is a better predictor of your future income than the school you actually attended.” Simply put, you should be wary of what any standardized indicators purport to indicate. “Not everything that can be counted counts,” writes sociologist William Bruce Cameron, “and not everything that counts can be counted.”

Figuring Out What Really Matters to You

There are many criteria to consider when evaluating whether a school will provide the learning experience that you want. First, there are the obvious elements — school size, location, cost, average class size, academic programs offered, and so on. The Noodle college search engine is a great resource for quickly and easily collecting information for your own DIY rankings.

You should also reflect deeply on what you want to be doing during your time at school: Is there a specific academic program or special degree that only certain schools provide? Is there a particular activity or club that you want to participate in? Do you want to study abroad? Do you want to take music lessons through your school? Do you want to play sports, competitively or on an intramural team? Do you want a school with a strong focus on internships and job placement assistance? You don’t need to know everything you want to do, but the more you reflect on your desired experience, the more informed your search will be — and the more satisfying the outcome.

Finally, try to think beyond the standard rankings criteria to imagine what unique elements schools offer to make your heart sing and your spirit soar. There are schools with really offbeat majors that you might not have ever considered, and there are hundreds of schools that allow you to create your own major. There are schools that don’t necessarily distribute letter-grade assessments, like Reed College and Brown University. There are schools with campuses around the world, like New York University, and schools that travel around the world, like the Semester at Sea. There are also great alternatives to college — schools and other programs that help students develop real-world skills and expose them to employment opportunities, like General Assembly and the Experience Institute.

To aid in the process of reflecting, prioritizing, and ranking, I’ve created a three-page “DIY College Ranking Guide” that asks some basic questions and provides dozens of potential criteria for evaluating schools. You can choose the ones that really matter to you, add your own, and weight them according to their relative importance to you — and in this way, you’ll develop your own numerical rankings. While these rankings admittedly will be somewhat arbitrary, they’ll certainly be more applicable to you than any whose criteria you did not choose. Ultimately, it is the process and not the precise results of your rankings that should prove most helpful in your college search.

A World of Information

And you need not feel so all alone in making your DIY college rankings. Just because there are serious flaws in the U.S. News rankings doesn’t mean you have to renounce these lists altogether. Taken with a healthy dose of skepticism, the various approaches to rankings and lists can be thought-provoking and informative. Although President Obama abandoned his 2013 proposal to create college rankings “that parents and students can use to compare schools based on ... where you can get the most bang for your educational buck," opting instead to champion a massive online database of searchable college information called The College Scorecard, there are still plenty of alternatives to the U.S. News rankings. The Chronicle of Higher Education even satirized the abundance of superficial ratings lists with a “Make Your Own College Rankings” site, which compiles Top-20 lists by aggregating the information from other rankings, using tongue-in-cheek headers like “Fat Paychecks,” and “Prestige.”

“In response to the success of U.S. News, as well as the backlash against it, there has been a massive proliferation of new approaches to college rankings and scorecards,” observed The Atlantic last September. The Princeton Review now has 62 rankings lists based on student survey responses, “with categories like ‘Best Classroom Experience’ and less serious ones such as ‘Best Stone-Cold Sober Schools,’” the article noted. Washington Monthly’s various rankings include a Peace Corps rank, ROTC rank, and Bachelor’s to Ph.D. rank. Money took up President Obama's abandoned project of ranking the best returns on investment with a list based on "educational quality, affordability, and alumni earnings." And the nonprofit news organization ProPublica followed Obama's call "to make sure that we've got a better bargain for the middle class and everybody's who's working hard to get into the middle class." Its list evaluates schools based on “how they support their poorest students financially," ranking different types of colleges based on the percentage of students receiving federal Pell Grants, as well as the average cost, average discount, and average federal debt for these students.

While the president’s College Scorecard doesn’t provide school rankings of any kind, the information it offers can be used to support the creation of your own DIY College Rankings. Obama administration officials have said that the data powering the College Scorecard are being freely shared with companies and organizations that offer online college search tools, and ScholarMatch, StartClass and College Abacus are reportedly already using the data to improve their offerings.

A Final Word of Advice

If the whole college search and application process seems overwhelming and unseemly, here are a few final words of advice to keep in mind:

In higher education, as in all things, what you get out of your experience will depend largely upon what you put into it. Give it everything you’ve got.

Find the right school for you with the Noodle college search tool. Or discover more resources on college rankings on our site. And don't forget to use Mat Cusick's DIY College Ranking Guide, which is a supplement to this article.

Article Topics: