The College Rankings Series (1 of 4)
Although diminished some following several high-profile scandals, the College Rankings industry has become a very significant factor in the college application process for students and parents. More profoundly, though, is how significant a factor they have become in the marketing strategies and strategic planning of colleges and universities, such that many schools are enticed to make misguided decisions that boost rankings but are not in the best interest of student learning and success. And ultimately, the effects of the more popular rankings have been to drive costs up, increase racial inequities, and reinforce an unhealthy mindset when it comes to college education.
For those and other reasons, it is extremely important for parents and students to understand the limitations and problems associated with rankings and how to identify and utilize the better ones. In this blog, we're going to look rankings in 5 parts.
Ragbag Rankings - Faulty criteria and methodology
Regressive Rankings - The negative effects of college rankings
Rankings Rancor - How rankings play into misguided and harmful college practices
Rankle of Rankings - How rankings feed into an unhealthy mindset
Re-envisioning Rankings - How to identify rankings that can be useful
Part 1: Ragbag Rankings
Faulty Criteria and Methodology:
The bulk of most College rankings are based on inputs, including selectivity (GPA, test scores, rejections, etc.), faculty credentials and research status based publications, grants, etc., endowments, expenditures on research, instruction, and supports, faculty/student ratio, etc. Some include a small number of outputs, such as graduation rates, retention rates, graduate salaries (at a single point in time), or graduates employed right after college, etc.
All of these are indirect, meaning that we have to infer meaning from them, which always carries the potential for error by the person making the inference. We believe a small faculty to student ratio results in good outcomes, but does it always and for every student? Graduation rates are more direct for sure, but graduation rates are built upon the quality of faculty instruction, test design, grading, giving feedback, etc., making it much more indirect. A faculty pressured to increase graduation rates could easily relax grading standards overtime in either overt or covert ways.
Additionally, each input or output is weighted, meaning it's value is increased or decreased compared to other variables. For example, One ranking may hold that faculty research publications are the most important factor and make it 50% of the ranking, while graduation rate (ostensibly the primary mission of colleges) may only count for 5%. Another might say graduation rates are more important and weight them as 20% of the final score, but endowments are worth 40%.
To make matters worse, nearly all rankings treat every school the same, whether it is small private liberal arts school, a large research institution focused mostly on graduate education, a medium sized regional school that blends local workforce development roles (like a Community College) with a more traditional Undergraduate Degree focus, or a small for-profit Art school.
In this way, most rankings are simply a ragbag of arbitrary inferences that may or may not align with the perspective, goals, etc. of the individual parent or student relying on them or the mission and function of the school in question. And from a practical perspective, the more arbitrary and subjective they are, the more meaningless they are.
Many articles discussing the problems with college rankings will note that most rankings ignore campus culture, extracurricular activities, and support services, but even those require inferences and in many ways are even more subjective! Can you really sum up, in a short descriptive way, the whole culture of a campus of 15,000 students, 9,000 staff and faculty, 100 different academic or support units, an alumni community of over 100,000, and a massive sports program? If one college offers 200 extracurricular activities and another offers 50, does that say anything at all absent more context? Chances are, it says something different to different people and means different things across different campus contexts.
In reality, most rankings fail parents and students dramatically. The majority of criteria that are most useful to students and parents are outputs that are often overlooked or minimized, and there are some inputs that matter, too. Unfortunately, many of these are not provided by colleges and universities, although some 3rd parties have been able to approximate some of them. If colleges and universities really want to level the rankings and make them useful, they would provide all or most of the following:
Retention rate - the percentage of first-year students who return to the school in their sophomore year
Graduation, Transfer, and Drop/Stop Out rates
Graduation rates - the percentage of students who graduate on time, 150% of the time, and 200% of the time, because most students do not graduate on time, and every extra year adds to costs and student debt
Community Colleges - 2 year, 3 year, and 4 year graduation rates
Undergraduate Colleges - 4 year, 6 year, and 8 year graduation rates
And broken down by
Major/Academic Program
Family income level
Student GPA and Test Score ranges
Transfer rates - Average percentage of the previous 4 student cohorts that transfer out before graduation
Drop/Stop Out rates - Average percentage of the previous 4 student cohorts that either drop out or stop out prior to graduation
Earnings and Social Mobility metrics
Median salary and range by employment sector at multiple years post-graduation
Median salary and range by family income at multiple years post-graduation
Life/Career Satisfaction Metrics at 5 years, 10 years, and 20 years (or more) post-graduation
Broken down by major or employment sector and family income level (at time of matriculation)
Debt and Financial Aid metrics
Median debt and range by employment sector
Median debt and range by family income levels
Debt to Earnings ratio by employment sector and income levels
Debt to Costs ratio by employment sector and income levels
Average proportion of Cost of Attendance covered by Institutional Aid, Federal Loans, Private Loans, and Self-Pay, broken down by
Family income levels
Student GPA and Test Score ranges
Majors/academic programs
Loan Default rates at 2 or 3 years, 5 years, 10 years, "All Time" post-graduation
Teaching & Learning Metrics
% of students who participate in 2 or more High-Impact Learning Experiences, ideally broken down by experience
Capstone course
Undergraduate research
Intensive Student Leadership roles (SGA, Resident Assistant, Peer Mentor roles, etc. that involve significant training, time commitment, responsibility, and staff/faculty supervision and mentoring)
Intensive/Immersive Service Learning experiences (that involve meaningful exposure to issues of academic/cultural interest, significant self-reflection, and professionally guided discourse)
Study Abroad
Internships (with significant time commitment and professional/faculty evaluation and mentorship)
% of full-time, tenure-track faculty that
Teach undergraduate courses
Teach undergraduate introductory courses
Teach in First Year Experience seminars
Advise undergraduate students
% of Undergraduate introductory courses with failure rates over 30%, 40%, and 50%
Average class size broken down by academic program
Faculty to Student ratio broken down by academic program
Academic major rejection rate (for those programs that require application to a major)
In absence of these metrics, though, parents and students are left with relatively little data to make informed decisions on. Check out this article about using College Outcomes and Performance Metrics to guide your college search.
Next up: Part 2 - Regressive Rankings - The negative effects of college rankings (Coming Soon!)