How Reliable Are University Rankings?(arxiv.org)
arxiv.org
How Reliable Are University Rankings?
https://arxiv.org/abs/2004.09006
89 comments
Rankings are irrelevant for top 20 but for the rest it matters a lot. Vast majority of population ends up somewhere outside of top 20. They have to chose between A, B and C and rankings plays its part. Reliable ranking is important because it rewards universities with better enrollment (and vice versa).
> Standford
https://stamford.uconn.edu/
^ ;-)
https://stamford.uconn.edu/
> The fact that it's easily game-able also makes it more attractive to universities because they can do something proactive if they're dissatisfied with their ranking
Are you saying that there is nothing lower ranked universities could do to improve their ranking in a better, more objective ranking of universities? Because it smacks of elitism.
Are you saying that there is nothing lower ranked universities could do to improve their ranking in a better, more objective ranking of universities? Because it smacks of elitism.
Well, in a hypothetical ideal ranking system that is stable and not easily gamed, a lower ranked university would need to actually improve the quality of its education (and whatever other characteristics the ranking system measures) in order to move up the ranks. But that would likely to take multiple years or decades of committed effort and investment. It’s sadly rare to see such long term action in our world of instant gratification, whether we’re talking about education, business, government, entertainment, relationships, individual behavior...
It's very similar to sports teams.
The tops teams don't vary all that much from year to year. They might change positions a bit, but it's rare for a new team to show up who hasn't shown up before.
They essentially have to get lucky and get someone who is super talented into a long term contract, who then attracts additional talent.
Or they have to invent a whole new way of doing things, like how the Oakland A's invented a whole new way of recruiting using math (which incidentally no longer helps them because all the top teams adopted it and are back on top again).
It's all about network effects.
The tops teams don't vary all that much from year to year. They might change positions a bit, but it's rare for a new team to show up who hasn't shown up before.
They essentially have to get lucky and get someone who is super talented into a long term contract, who then attracts additional talent.
Or they have to invent a whole new way of doing things, like how the Oakland A's invented a whole new way of recruiting using math (which incidentally no longer helps them because all the top teams adopted it and are back on top again).
It's all about network effects.
I would add that the effect is even more pronounced in college sports (in the US) compared to pro sports, especially those that have a draft system (so the NBA but not Formula 1, e.g.). The persistent institutional/reputational advantages enjoyed by some of the top college sports programs really are perfect illustrations of the network effects you mentioned. I mean, just look at the rankings enjoyed by Notre Dame in college football year in and year out, even when they frequently get killed by lowered ranked opponents in bowl games.
You are describing exactly why I long stopped watching college sports.
The ranking system for teams is opaque, and I think it is intentionally so to keep the "top teams" top, and all others down. There are plenty of instances where a "lower" ranked team in football have an undefeated season, but have no chance to go the "playoffs" or even play in a more prestigeous bowl game. This is on top of other teams NOT having an undefeated season, but still get the playoffs or a better bowl game.
Try to actually bring that up and all you get back is "well you must have come from a college with a bad sports team".
The ranking system for teams is opaque, and I think it is intentionally so to keep the "top teams" top, and all others down. There are plenty of instances where a "lower" ranked team in football have an undefeated season, but have no chance to go the "playoffs" or even play in a more prestigeous bowl game. This is on top of other teams NOT having an undefeated season, but still get the playoffs or a better bowl game.
Try to actually bring that up and all you get back is "well you must have come from a college with a bad sports team".
Unsure of the exact metric you're referring to, and I'm about to be just as informal, but sports teams swing much more wildly and often than universities. In the NBA, most recently GSW, going from 5 straight finals with 3 wins to last in western conference. See also CHI before and after MJ era. LAL most recently after Kobe's retirement and pre-Lebron. In the NFL, SF post-Young pre-Harbaugh. I agree, there are often dynasties and teams can sit in the tops ranks for quite a while, however it is not at all uncommon to see once top ranked teams vying for lottery draft picks. Maybe the over-all mix does not churn that much, but I can't recall the last time a top 10 school dropped out of the top 100 without a scandal involved.
Generally speaking, it's just how networking/reputation works.
Talent goes somewhere -> successful things happen there -> reputation is spread -> some of that success gets associated with the place -> which attracts more talent there. Like a self-feeding loop.
Happens to pretty much every social organization - neighborhoods even. Just look at top tech companies. They attract a lot of talented tech people just because in the past talented people made those companies very successful. Conversely, if they were failures, not many people would be attracted to go there...
Anyhow, that's not to say networks can decline (as reputation also attracts less talented people). Obviously careers often span longer than sports team dynasties so it might take longer to see shuffles happen in school rankings.
Talent goes somewhere -> successful things happen there -> reputation is spread -> some of that success gets associated with the place -> which attracts more talent there. Like a self-feeding loop.
Happens to pretty much every social organization - neighborhoods even. Just look at top tech companies. They attract a lot of talented tech people just because in the past talented people made those companies very successful. Conversely, if they were failures, not many people would be attracted to go there...
Anyhow, that's not to say networks can decline (as reputation also attracts less talented people). Obviously careers often span longer than sports team dynasties so it might take longer to see shuffles happen in school rankings.
I would say sports just moves faster than academics. The average sports career is only a few years with a few exceptions, the average academic career is much longer.
That's true and, furthermore, a big part of why a university is well-ranked in year N boils down to it being well-ranked in year N-1 and N-2. With a sports team, if you have some critical injuries or otherwise lose important personnel from the prior year or two, no one cares that you won the championship a couple years back if you have a mediocre record this year. There's just a lot more inertia in university rankings both on objective factors and on things that basically depend on being a good university because they're a good university.
The same could be said of students. Universities are made or broken by their recruitment, coaching/professors, administrators, and facilities. Sports may be slightly more volatile, but I doubt it’s nearly as much as people assume.
Nothing easy or quick, no.
I'm mostly on board with the critique of university rankings, but I really don't buy this specific critique. It's a strawman.
Here's what they did: they took the top research universities and the top liberal arts colleges. They then observed that the rankings can't distinguish between the top institutions within each class, and that you can also flip which of those two classes is preferred.
But, in what sense is it even meaningful to compare Amherst College and Harvard [1]? Those are just enormously different types of institutions. It'd be like creating a ranking of "best cars" that includes the top 5 sedans and the top 5 trucks, and then observing that you can jiggle the rankings to get anyone on top. Is a Toyota Corolla better than an F-150? IDK. Stupid question. US News and World Report, for all its problems, does at least get this much right! they break down institutions by "type" and then rank within type.
Additionally, just because rankings are noisy and easy to game locally doesn't mean they are inaccurate or easy to game globally. Two institutions within 10-20 slots of one another are probably pretty similar and rankings aren't particularly helpful / are easy to game. But the #100 liberal arts college is probably not as good as Amherst, and the #78 National University is probably not as good as Harvard, and no amount of gaming is going to change that.
Rankings are indeed noisy and inaccurate and easy to game. But this particular article is not a compelling demonstration of that fact.
[1] For non-US readers: Amherst college (not to be confused with UMass Amherst) is in a class of peculiar institutions that are fairly unique to the USA as described here: https://en.wikipedia.org/wiki/Liberal_arts_college TL;DR: they're basically the diametric opposite of super-charged research universities like Harvard.
Here's what they did: they took the top research universities and the top liberal arts colleges. They then observed that the rankings can't distinguish between the top institutions within each class, and that you can also flip which of those two classes is preferred.
But, in what sense is it even meaningful to compare Amherst College and Harvard [1]? Those are just enormously different types of institutions. It'd be like creating a ranking of "best cars" that includes the top 5 sedans and the top 5 trucks, and then observing that you can jiggle the rankings to get anyone on top. Is a Toyota Corolla better than an F-150? IDK. Stupid question. US News and World Report, for all its problems, does at least get this much right! they break down institutions by "type" and then rank within type.
Additionally, just because rankings are noisy and easy to game locally doesn't mean they are inaccurate or easy to game globally. Two institutions within 10-20 slots of one another are probably pretty similar and rankings aren't particularly helpful / are easy to game. But the #100 liberal arts college is probably not as good as Amherst, and the #78 National University is probably not as good as Harvard, and no amount of gaming is going to change that.
Rankings are indeed noisy and inaccurate and easy to game. But this particular article is not a compelling demonstration of that fact.
[1] For non-US readers: Amherst college (not to be confused with UMass Amherst) is in a class of peculiar institutions that are fairly unique to the USA as described here: https://en.wikipedia.org/wiki/Liberal_arts_college TL;DR: they're basically the diametric opposite of super-charged research universities like Harvard.
Thank you for your feedback. I am one of the authors of this paper.
I don't think the paper gives any hints on us dividing universities into classes and comparing. Would love to know how you reached that conclusion.
At the same time, yes, we could have divided universities into different classes: research vs liberal arts, this vs. that state, big vs. small size, etc. These are all trivial groupings but none of these would change the conclusions of this paper. For all practical purposes, we could easily have replaced the university names with labels like U1, U2, ... and still the conclusions would not change. What matters is how a weight-based composite index can be gamed and the paper does show that in multiple ways. Pls review the ILP formulations yourself and run them on the dataset of your choice.
I don't think the paper gives any hints on us dividing universities into classes and comparing. Would love to know how you reached that conclusion.
At the same time, yes, we could have divided universities into different classes: research vs liberal arts, this vs. that state, big vs. small size, etc. These are all trivial groupings but none of these would change the conclusions of this paper. For all practical purposes, we could easily have replaced the university names with labels like U1, U2, ... and still the conclusions would not change. What matters is how a weight-based composite index can be gamed and the paper does show that in multiple ways. Pls review the ILP formulations yourself and run them on the dataset of your choice.
Why is there such a fixation on university rankings in the United States? Postsecondary rankings don't seem to matter as much in other Western countries.
Not true of the UK, France, Korea or Japan. I don’t know if they care about what are basically league tables but any educated citizen would find it no more difficult to reel off five to ten of their best universities than an American would to list Harvard, Yale, Princeton, Stanford, MIT etc.
I can see how it would look weird from a German, or possibly a Dutch or Nordic perspective but a deliberate effort to bring up the bottom and limited attempts to raise the top is extremely different from the US system, where there are large and growing returns to excellence.
Also the US elite just have smaller enrollments as a share of population. Oxbridge, the grandes écoles or Korea’s top three enrol 1-2% of yearly students. The top ten US universities maybe a tenth of that.
I can see how it would look weird from a German, or possibly a Dutch or Nordic perspective but a deliberate effort to bring up the bottom and limited attempts to raise the top is extremely different from the US system, where there are large and growing returns to excellence.
Also the US elite just have smaller enrollments as a share of population. Oxbridge, the grandes écoles or Korea’s top three enrol 1-2% of yearly students. The top ten US universities maybe a tenth of that.
From within Dutch universities, rankings are considered very important (more by the non-academic staff than by the academics). The general public doesn't really care all that much yet, but they do more and more each year. It's especially for international (student) recruiting that the rankings (national and international) are important, and that's seeping through into everything else for last 10 years or so now.
(not disagreeing with you, just adding some nuance about the direction of things)
(not disagreeing with you, just adding some nuance about the direction of things)
As a Dutch PhD student, I am not sure I agree. There is some hype around the rankings of the universities and specifically the bachelor and master tracks at a given university, but not many students really care about them. I certainly did not choose my university based on their ranking, and don't know many that did. Choice is based much more on travel distance and other factors, not on perceived quality of a university. I also would not be able to point out the best universities in the Netherlands, which says something as well.
Also, I think as a student you're quite shielded from it. Ask your professor (not the postdocs or the assocs, the real ones) some day, I'm sure they have an opinion on it.
Yes for sure, in academia the perceived ranking is quite important. But, at least in my field, ranking is decoupled from the ranking of a university and is more on a department basis. I.e., the physics department of a specific university might be well regarded internationally, even though the university itself not so much. This is of course helped by the fact that Dutch universities are all quite highly ranked.
Fair enough, although I'm not sure how what you say is at odds with what I said. As you indicate, Dutch students don't care much if any (yet). It probably depends on the university as well. The highly international ones (Wageningen, Maastricht, some institutes in Enschede, maybe Delft, at least the TU?) certainly do.
I think in the UK it is easy - both universities are ranked as excellent!
[deleted]
I have worked at universities in the UK and Norway, and believe me, this stuff matters. A lot.
Its great that students can make an informed choice about where to start, particularly if they do not come from an academic household. Its not a given that people have heard of places like MIT, and these lists are a somewhat neutral way of seeing which universities are the most "legit". You have to remember that a lot of low-quality institutions are marketing themselves pretty hard to students, and if it weren't for rankings it would be easy to make bad choices when choosing a place to study.
Its great that students can make an informed choice about where to start, particularly if they do not come from an academic household. Its not a given that people have heard of places like MIT, and these lists are a somewhat neutral way of seeing which universities are the most "legit". You have to remember that a lot of low-quality institutions are marketing themselves pretty hard to students, and if it weren't for rankings it would be easy to make bad choices when choosing a place to study.
Is there actually a big fixation on university rankings? I had no idea what my school's ranking was (apparently it's now #40) and I'm not sure I've ever had conversation where university rankings came up outside of specifically talking about where a high school student is thinking about applying. Even then it's not like it's common for people to pick which school to go to out of the ones that accepted them based solely on rankings.
People who care about Ivies and the such don't care because the Ivies have a high ranking. Someone who went to Harvard who wants to use that fact to their advantage isn't going to say that they went to a "top 3" school.
People who care about Ivies and the such don't care because the Ivies have a high ranking. Someone who went to Harvard who wants to use that fact to their advantage isn't going to say that they went to a "top 3" school.
> Is there actually a big fixation on university rankings?
I've sat in department wide meetings where deans of engineering compared their schools to 'the competition,' both in terms of US news rankings and their own preferred metrics. Usually to bemoan how unfair it is that Texas is a petrostate with 10x the population that can fund top tier engineering schools.
But by and large, the tiers are static and a uni in the #47 slot isn't making to the top 10 any time soon. Might not even make top 40. For undergraduate education I think you're right, nobody really cares. There's really three categories of schools: private schools / ivies, flagship state schools, and the rest. I'm guessing if you asked employers to rank schools, they would largely match selectivity -- how hard it is to get into the school. (or more depressingly, how good the athletics programs are doing lately)
I've sat in department wide meetings where deans of engineering compared their schools to 'the competition,' both in terms of US news rankings and their own preferred metrics. Usually to bemoan how unfair it is that Texas is a petrostate with 10x the population that can fund top tier engineering schools.
But by and large, the tiers are static and a uni in the #47 slot isn't making to the top 10 any time soon. Might not even make top 40. For undergraduate education I think you're right, nobody really cares. There's really three categories of schools: private schools / ivies, flagship state schools, and the rest. I'm guessing if you asked employers to rank schools, they would largely match selectivity -- how hard it is to get into the school. (or more depressingly, how good the athletics programs are doing lately)
If you dropped 200k-300k on an education, you'll be invested in either the continued value or and improvement in your investment's reputation, which as the years go on is really its only value.
200k is about what I spent on my education and I can't say I've really had any reason to care about my school's reputation past my first job after graduation.
Tech jobs generally don't care, but traditional management and those tracks really care about that.
Because getting a job in those areas is about abstract abilities that can't be (somewhat) measured with code tests, technology signifiers, and tricky questions like tech interviews.
So a certain school is a "class entry cue" for such tracks.
Because getting a job in those areas is about abstract abilities that can't be (somewhat) measured with code tests, technology signifiers, and tricky questions like tech interviews.
So a certain school is a "class entry cue" for such tracks.
Sure, but that first job is likely to determine the rest of your career, if your career path is somewhat traditional. So the school reputation would still have been quite important overall (especially if you wanted to aim high and get hired from a top employer right out of university).
> I'm not sure I've ever had conversation where university rankings came up outside of specifically talking about where a high school student is thinking about applying.
Most high school students/parents/guidance counselors/etc check the rankings, reputation, average salary of graduates, etc. It even extends to majors and rankings within majors - which might matter more.
> Even then it's not like it's common for people to pick which school to go to out of the ones that accepted them based solely on rankings.
Who said solely? Of course there are many other factors ( location, tuition, scholarships, etc ). But rankings/reputation/etc are a big part of a high schooler's college decision.
Most high school students/parents/guidance counselors/etc check the rankings, reputation, average salary of graduates, etc. It even extends to majors and rankings within majors - which might matter more.
> Even then it's not like it's common for people to pick which school to go to out of the ones that accepted them based solely on rankings.
Who said solely? Of course there are many other factors ( location, tuition, scholarships, etc ). But rankings/reputation/etc are a big part of a high schooler's college decision.
The rankings mattered to me when applying to schools because it seemed like a useful metric to judge the quality of education I would get, and the usability of that education.
When you're making one of the biggest decisions of your life, rankings can help you know how good your choices are.
Not saying rankings are correct...
But for those applying, they likely appreciate having a ranking mechanism.
It's one of the few objective measures of universities available.
How do you know, otherwise, if this university that wants to charge you $40,000 per year is really going to deliver on their promise?
You can't just go back to school again and start over.
It's a life decision, so there is a desperate desire for the applying student to have an objective way of stack ranking the choices.
(Disclosure: I attended MIT. And I basically just applied to the top 5 or 10 computer science universities -- again, using rankings. And hoped to get into one of those.)
When you're making one of the biggest decisions of your life, rankings can help you know how good your choices are.
Not saying rankings are correct...
But for those applying, they likely appreciate having a ranking mechanism.
It's one of the few objective measures of universities available.
How do you know, otherwise, if this university that wants to charge you $40,000 per year is really going to deliver on their promise?
You can't just go back to school again and start over.
It's a life decision, so there is a desperate desire for the applying student to have an objective way of stack ranking the choices.
(Disclosure: I attended MIT. And I basically just applied to the top 5 or 10 computer science universities -- again, using rankings. And hoped to get into one of those.)
> Is there actually a big fixation on university rankings?
I've observed that, among people who care about university rankings in hiring, there are exactly two categories: the top five, and everything else. If you went to #200, you're the "same" as somebody who went to #25.
I've observed that, among people who care about university rankings in hiring, there are exactly two categories: the top five, and everything else. If you went to #200, you're the "same" as somebody who went to #25.
Canadian university board member here ...
Rankings drive enrolment, especially international enrolment, which is a major source of revenue. Also prestige helps to attract the best researchers, who are being wooed by many top schools.
Rankings drive enrolment, especially international enrolment, which is a major source of revenue. Also prestige helps to attract the best researchers, who are being wooed by many top schools.
Well, it did. We're going to see how unsustainable our secondary education system is without the children of rich foreign families to sustain it.
A mix of trying to get a sense of educational quality and prestige.
In some specific programs, you'd want to aim for the schools known to have the more robust curriculum but outside of that, it's mostly ego/prestige. "I got to a top N school, I went to harvard", etc.
That badge signals _something_ to your peers, parents, and future employers. Think of all those companies that explicitly filter for "target schools". It works pretty well too, you'll get a call-back to an interview because you have an Ivy on your resume even if you barely passed your classes.
In some specific programs, you'd want to aim for the schools known to have the more robust curriculum but outside of that, it's mostly ego/prestige. "I got to a top N school, I went to harvard", etc.
That badge signals _something_ to your peers, parents, and future employers. Think of all those companies that explicitly filter for "target schools". It works pretty well too, you'll get a call-back to an interview because you have an Ivy on your resume even if you barely passed your classes.
Australia here. Universities consistently market their national ranking here. As a staff member I've found management consistently pushes things that will raise their ranking. As a student initially at a high ranked university, I found I got a better education by switching to a much lower ranked university.
> As a student initially at a high ranked university, I found I got a better education by switching to a much lower ranked university.
That’s interesting. I’ve consistently heard the same sentiment from ex-students and alumni of certain highly ranked Canadian universities (undergrad) as well. They’ll generally complain about a lack of student support and restrictive policies at these institutions, making it needlessly difficult for students to succeed.
I suspect the higher ranked universities might be under pressure to artificially increase the difficulty of their programs, in order to distinguish their alumni from the alumni of other schools, who’s programs cover more or less the same subject matter.
That’s interesting. I’ve consistently heard the same sentiment from ex-students and alumni of certain highly ranked Canadian universities (undergrad) as well. They’ll generally complain about a lack of student support and restrictive policies at these institutions, making it needlessly difficult for students to succeed.
I suspect the higher ranked universities might be under pressure to artificially increase the difficulty of their programs, in order to distinguish their alumni from the alumni of other schools, who’s programs cover more or less the same subject matter.
In Australia, the rankings often seem to be based on outcomes unrelated to undergraduate studies (citation needed). I've found that makes for a situation where there is very little need to increase or maintain the quality of learning at the lower levels. While Uni Melbourne is consistently rated high[1], the quality of teaching and support was very poor[2] and opportunities within the university extremely limited.
In contrast, when I moved interstate and transferred institutes, despite being at a much lower-ranked university I received (subjectively) better tuition and support from low and high-level staff. I was even (enthusiastically) given the opportunity to contribute to active research in my vacations.
[1] https://en.wikipedia.org/wiki/University_of_Melbourne [2] 3yrs Mech Eng undergraduate student first-hand experience.
In contrast, when I moved interstate and transferred institutes, despite being at a much lower-ranked university I received (subjectively) better tuition and support from low and high-level staff. I was even (enthusiastically) given the opportunity to contribute to active research in my vacations.
[1] https://en.wikipedia.org/wiki/University_of_Melbourne [2] 3yrs Mech Eng undergraduate student first-hand experience.
My wife did a law degree at a very highly ranked university in the UK after doing a business degree at a comparatively lowly ranked institution (although still very well regarded) - coincidentally the latter being where I did a CS degree and thought the teaching was excellent.
She said the quality of teaching at the higher ranked university was almost comically bad compared to the lower ranked university.
The usual justification I have read for these things is that the top universities have already selected the best candidates, so they don't actually need too much teaching. Which seems to be missing the point.....
She said the quality of teaching at the higher ranked university was almost comically bad compared to the lower ranked university.
The usual justification I have read for these things is that the top universities have already selected the best candidates, so they don't actually need too much teaching. Which seems to be missing the point.....
Also, which rankings are meaningfully measuring student experience and give it weight? I know some send around questionaires, but for the things I looked at it seemed a minor concern.
I'm not 100% up to speed on this as, for the most part, it doesn't impact me as lower-tier research staff. Management is often pushing us for more papers in higher-ranked journals as that seems to be the biggest factor. Student questionnaires seem to be used only internally though even then I suspect they're mostly ignored [anecdotal at best].
The justification we're given for higher ranking is overseas students are more likely to chose a university based on these (domestic students are more likely here to go to whichever university is closest). Due to fee-capping on local students, international students have become a big source of income for universities here. Whether it is accurate to do so or not, the universities keep insisting on higher rankings to attract those students.
The justification we're given for higher ranking is overseas students are more likely to chose a university based on these (domestic students are more likely here to go to whichever university is closest). Due to fee-capping on local students, international students have become a big source of income for universities here. Whether it is accurate to do so or not, the universities keep insisting on higher rankings to attract those students.
Education is a market. To make money, universities need to attract customers. And they do that by being highly ranked.
And I disagree with your statement. Many other western countries have their "ivy league" equivalent. Not all schools are equal in the rest of the world either. But I do however agree that the "marketing" aspect is definitely more pronounced in the US.
And I disagree with your statement. Many other western countries have their "ivy league" equivalent. Not all schools are equal in the rest of the world either. But I do however agree that the "marketing" aspect is definitely more pronounced in the US.
As a general rule, you aren't allowed admittance into the Cathedral unless you've demonstrated proper indoctrination at the most "prestigious" universities.
I was talking to a few European friends (1 Dutch, 1 French) and what they described was that even the most bright students typically go to their regional universities which are all "equally good". Here in the US, the rankings are almost a proxy for admissions rate. Since "top schools" are exclusive, the student population tends to be at least one the following: talented, precocious, or well-connected/rich. I attended a "top 25" university, which is a ways from the Stanford, Harvard, Yales of the world but the environment was very special. Hard to describe, but you are surrounded by passionate, ambitious people, you can't help but feel energized. Is this a fair system? That's a whole another conversation.
The professional connections pay for the tuition and much more.
the importance of professional connections are understated in my opinion. You can make a career out of only knowing the right things and you can make a career out of only knowing the right people but you really excel when you know the right things and know the right people.
It’s what drives enrollment and tuition fees are a huge part of revenue. In other countries, there’s more state support and you don’t need to market universities
Regardless, you should think hard before accepting at the "top" university that will take you. Many times it's better to be a big fish in a small pond than vice versa.
Also, in terms of learning, it's really about the access you can get to the ten or so percent of the faculty that actually worth anything. Figure out who they are and go after them.
Also, network, network, network. Smart or not, everyone you meet in school might give you a reference/job/funding someday.
(source: hard experience)
Also, in terms of learning, it's really about the access you can get to the ten or so percent of the faculty that actually worth anything. Figure out who they are and go after them.
Also, network, network, network. Smart or not, everyone you meet in school might give you a reference/job/funding someday.
(source: hard experience)
I could consider the primary advantage of a "top" university to be its network and the access it gives you to jobs. Namely, companies that do new grad hiring have a specific list of colleges that they target and will source and review resumes from. Not to say that people from other colleges can't get in, but that you'd have to apply directly and have something standout to get the recruiter to even consider you.
> Many times it's better to be a big fish in a small pond than vice versa.
Yes, this is seldom considered. For example, if you plan to graduate with a BS in physics, would you rather be in the top quartile at Cal Poly, or the bottom quartile at Cal Tech?
Yes, this is seldom considered. For example, if you plan to graduate with a BS in physics, would you rather be in the top quartile at Cal Poly, or the bottom quartile at Cal Tech?
This advice applies to jobs, I'm not sure it does to universities, which are more about prestige.
I.e. getting a ticket to pick any pond you want after graduation.
I.e. getting a ticket to pick any pond you want after graduation.
Care to elaborate? Did you have any problems after graduating from the "top" university?
More "during" than "after". Not sure if mine counts as "top", but most would say it's easily in the top 25 for my discipline. It was plenty hard for me, though, and in hindsight, I think it would have been better to be the brilliant kid in a lower-ranked school than an also-ran in a higher-ranked school (relative to the range of schools that would admit me).
That's just a guess, but I've seen a lot of others suffer from shooting too high.
That's just a guess, but I've seen a lot of others suffer from shooting too high.
University rankings have two key issues:
1. They are significantly biased to English language (universities/research output) and bigger countries in general 2. They are totally irrelevant from the student perspective as they mostly focus on research output, which means that you'll get the richest universities and the best professors will be writing papers rather than teaching bachelor level courses.
There are some attempts to fix these issues (see eg the EU "Multirank" which allows you to choose your priorities), but it's still hit and miss.
1. They are significantly biased to English language (universities/research output) and bigger countries in general 2. They are totally irrelevant from the student perspective as they mostly focus on research output, which means that you'll get the richest universities and the best professors will be writing papers rather than teaching bachelor level courses.
There are some attempts to fix these issues (see eg the EU "Multirank" which allows you to choose your priorities), but it's still hit and miss.
Those are useful issues, to be sure, but there is a lot more incoherence than those two: https://www.newyorker.com/magazine/2011/02/14/the-order-of-t...
Gaming of rankings aside, the ratings organizations themselves offer various 'services' to colleges looking for a reputation boost.
https://www.insidehighered.com/news/2013/05/29/methodology-q...
For instance, QS sells a 'star-ranking' system for schools. Coincidentally the schools who paid for a '5-star' ranking, such as the Unviersity of Bristol or the Universiti Malaya are also placed higher on the actual QS ranking than those that did not, including Georgia Tech, the University of Washington, Ecole Polytechnique and UIUC.
https://www.insidehighered.com/news/2013/05/29/methodology-q...
For instance, QS sells a 'star-ranking' system for schools. Coincidentally the schools who paid for a '5-star' ranking, such as the Unviersity of Bristol or the Universiti Malaya are also placed higher on the actual QS ranking than those that did not, including Georgia Tech, the University of Washington, Ecole Polytechnique and UIUC.
Related to expenses..
"And there are several perverse incentives in the marketplace that make it hard for colleges to cut costs. The most basic one is that the U.S. News algorithm rewards them for spending a lot of money: Higher faculty salaries and more spending on student services lead directly to better rankings. If you reduce your expenses, your ranking will fall, which means that next year your applicant pool will probably shrink."
You'd be surprised at the burgeoning expenses of universities, and often meager profitability, despite record-high tuition rates. It seems, often like startups, a burn rate signals potential growth!
https://www.nytimes.com/interactive/2019/09/10/magazine/coll...
"And there are several perverse incentives in the marketplace that make it hard for colleges to cut costs. The most basic one is that the U.S. News algorithm rewards them for spending a lot of money: Higher faculty salaries and more spending on student services lead directly to better rankings. If you reduce your expenses, your ranking will fall, which means that next year your applicant pool will probably shrink."
You'd be surprised at the burgeoning expenses of universities, and often meager profitability, despite record-high tuition rates. It seems, often like startups, a burn rate signals potential growth!
https://www.nytimes.com/interactive/2019/09/10/magazine/coll...
I'm gonna go out on a limb and say that above-market faculty salaries are not a major issue.
I would argue its headcount growth (not salary per employee), but admittedly I could not immediately find data for this.
Edit: Also upon looking through the Trinity College financial statements, a huge portion was categorized as "Other", which included consulting fees, travel, interest expenses and various other expenses.
Edit: Also upon looking through the Trinity College financial statements, a huge portion was categorized as "Other", which included consulting fees, travel, interest expenses and various other expenses.
Isn't the point of an audit is that it tells you what problems to fix? It would be concerning if this audit happens only shortly before, or even after, the main ranking happens.
But if the increased ranking is done based on data from, say a half a year later, then that just seems like a normal audit effect? That it might just make the client better at gaming the rules is true for a normal audit too.
But if the increased ranking is done based on data from, say a half a year later, then that just seems like a normal audit effect? That it might just make the client better at gaming the rules is true for a normal audit too.
https://www.bostonmagazine.com/news/2014/08/26/how-northeast...
> Freeland swept into Northeastern with a brand-new mantra: recalibrate the school to climb up the ranks. “There’s no question that the system invites gaming,” Freeland tells me. “We made a systematic effort to influence [the outcome].” He directed university researchers to break the U.S. News code and replicate its formulas.
> Freeland swept into Northeastern with a brand-new mantra: recalibrate the school to climb up the ranks. “There’s no question that the system invites gaming,” Freeland tells me. “We made a systematic effort to influence [the outcome].” He directed university researchers to break the U.S. News code and replicate its formulas.
Yeah this kind of gaming is probably widespread. UChicago is known for "artificially" driving down their acceptance rate with lots of decision rounds and marketing. Cornell isn't releasing their numbers at all this year - I'd guess because they don't like them.
I've heard many allegations about established universities gaming their ranking by, e.g., soliciting applications widely -- not so much to have a larger and more diverse pool from which to choose -- but to maintain/lower their acceptance rate metric.
Two things I like about the Northeastern story are that the official demonstrated even more ambitious gaming, and it was publicized.
No criticism of Northeastern; they have many great people, doing great work. And I've heard many faculty objected to the rankings-climbing emphasis as it was happening.
Two things I like about the Northeastern story are that the official demonstrated even more ambitious gaming, and it was publicized.
No criticism of Northeastern; they have many great people, doing great work. And I've heard many faculty objected to the rankings-climbing emphasis as it was happening.
graduated from there. totally agree with what's in that article. at 18 I was swept away by the "allure" of their rapid ascent and aggressive re-development of campus (which has continued to this day) and went there, despite the insane cost.
faculty were generally pleasant to interact with, the education was probably average. job prospects were mediocre - alot of competition in boston and NU is in a lower tier, comparatively.
as a measure of quality, based on my experience, that ranking is worthless.
faculty were generally pleasant to interact with, the education was probably average. job prospects were mediocre - alot of competition in boston and NU is in a lower tier, comparatively.
as a measure of quality, based on my experience, that ranking is worthless.
NEU has some really great people in CS, who could hold their own with their counterparts at the better-known schools in the Boston area.
I've also heard some good things about NEU cybersecurity.
I'm not familiar with the other departments.
I've also heard some good things about NEU cybersecurity.
I'm not familiar with the other departments.
Sometime not too long ago there was a blog post or article covered on HN, about data showing student scores on some academic achievement measure over time. If I remember, all students generally improved on this measure throughout undergrad, and there was some average tendency for universities with higher average entering standardized test scores to have higher scores, but the effect seemed to be mostly driven by the students. That is, it wasn't that some schools accelerated scores more, it was that they just tended to select for higher-performing students. Also, the signal provided by school, if I remember, was relatively weak. The author was basically arguing that universities in general provide some educational value (or students just learn over time; there was no control of students who didn't go to college) but that the ranking was by and large just a signaling thing.
I can't find it now though because all the search terms I can think of just return a bunch of other stuff related to universities and students.
I can't find it now though because all the search terms I can think of just return a bunch of other stuff related to universities and students.
Interesting. I wonder how one would measure whether an institution changes student trajectories? Or if that data is available somewhere.
If you only read one sentence from this article, it should be this one:
> In this paper, we take a fresh look at this ranking scheme using the public College dataset; we both formally and experimentally show in multiple ways that this ranking scheme is not reliable and cannot be trusted as authoritative because it is too sensitive to weight changes and can easily be gamed.
> In this paper, we take a fresh look at this ranking scheme using the public College dataset; we both formally and experimentally show in multiple ways that this ranking scheme is not reliable and cannot be trusted as authoritative because it is too sensitive to weight changes and can easily be gamed.
There is a very current thing happening where individual professors have too much power. The whole "remote teaching" thing is VERY new for some universities and professors. My son recently survived a final where > 90% of the class failed. Mostly due to poor execution of remote learning. Including simple stuff like classes overlapping meeting times, study material uploaded too late, etc.
No pressure on the professor or university, though. All on the students. There's a rant letter, from the prof, about effort where effort wasn't a factor. My kid survived solely because he has a very good memory. No accountability. I AM PISSED.
No pressure on the professor or university, though. All on the students. There's a rant letter, from the prof, about effort where effort wasn't a factor. My kid survived solely because he has a very good memory. No accountability. I AM PISSED.
> where > 90% of the class failed.
I can't imagine this happening in a college class where the instructor get to keep his class next year.
I can't imagine this happening in a college class where the instructor get to keep his class next year.
U.S. State college, a fairly well respected one, happening now. Chemical Engineering. They have no idea how to operate "online/remote". And that matters for classes like Fluid Dynamics, Organic Chem, Differential Analysis, etc. Is there some realistic path for me to escalate?
> Is there some realistic path for me to escalate?
Is the instructor an ad junct? Those are cheap and replaceable. They make maybe ~1/10th of what your average FAANG engineer makes because the labor supply is extraordinarily over-saturated. Enough complaints will probably get them fired, if that's what you're looking for, but realize you're probably putting someone with no prospects and no savings out on the street and they will almost certainly be replaced with someone who gives about as many shits as you can expect from someone with a PhD who makes 30k a year. That tuition you're paying isn't flowing to the people teaching the courses.
If the instructor is an actual professor, then probably there's nothing you can do.
BTW, let your kid at least try their hand at navigating this on their own, even if you provide oversight. Barely competent people doing a poor job is not unique to pandemic times and is definitely not unique to higher ed. Knowing how to use a bureaucracy to route around an incompetent person not doing their job well is a valuable life skill. Plus, everyone involved is going to be more sympathetic to an articulate student than to a "helicopter parent".
Is the instructor an ad junct? Those are cheap and replaceable. They make maybe ~1/10th of what your average FAANG engineer makes because the labor supply is extraordinarily over-saturated. Enough complaints will probably get them fired, if that's what you're looking for, but realize you're probably putting someone with no prospects and no savings out on the street and they will almost certainly be replaced with someone who gives about as many shits as you can expect from someone with a PhD who makes 30k a year. That tuition you're paying isn't flowing to the people teaching the courses.
If the instructor is an actual professor, then probably there's nothing you can do.
BTW, let your kid at least try their hand at navigating this on their own, even if you provide oversight. Barely competent people doing a poor job is not unique to pandemic times and is definitely not unique to higher ed. Knowing how to use a bureaucracy to route around an incompetent person not doing their job well is a valuable life skill. Plus, everyone involved is going to be more sympathetic to an articulate student than to a "helicopter parent".
Not a big fan of the "helicopter parent" term. I'm paying for it, so I want to understand what I'm paying for. Why is that a "helicopter"?
Adjuncts aren't allowed to fail that many students. They will be pressured to keep the pass rate up to keep the pipeline filled with mediocre students.
Is this your degree or your sons?
[deleted]
I think they're useful-ish, but it's important not to get too focused on specifics.
For example I did a stint in policy and I'm reminded of how everyone insisted for a long time that "top" law schools are the "top 14" or T14 for short. This is historically the measure used since Georgetown was in DC, and the children of the elite must be "top tier" regardless of if their school ranked outside top ten on every official metric such as incoming LSATs, publications, etc :D
For example I did a stint in policy and I'm reminded of how everyone insisted for a long time that "top" law schools are the "top 14" or T14 for short. This is historically the measure used since Georgetown was in DC, and the children of the elite must be "top tier" regardless of if their school ranked outside top ten on every official metric such as incoming LSATs, publications, etc :D
That’s not where T14 comes from.
> There exists an informal category known as the "Top Fourteen" or "T14," which refers to the fourteen institutions that regularly claim the top spots in the yearly U.S. News & World Report ranking of American law schools.[8] Furthermore, only these fourteen schools have ever placed within the top ten spots in those rankings.[9] Although "T14" is not a designation used by U.S News itself, the term is "widely known in the legal community."[10] While these schools have seen their position within the top fourteen spots shift frequently, they have generally not placed outside of the top fourteen spots since the inception of the rankings.[11] There have been rare exceptions to this, however, such as UCLA School of Law appearing in the top fourteen instead of Cornell and Northwestern in 1987 and University of Texas School of Law displacing Georgetown in 2018, although the significance of these changes has been debated.
https://en.wikipedia.org/wiki/Law_school_rankings_in_the_Uni...
> There exists an informal category known as the "Top Fourteen" or "T14," which refers to the fourteen institutions that regularly claim the top spots in the yearly U.S. News & World Report ranking of American law schools.[8] Furthermore, only these fourteen schools have ever placed within the top ten spots in those rankings.[9] Although "T14" is not a designation used by U.S News itself, the term is "widely known in the legal community."[10] While these schools have seen their position within the top fourteen spots shift frequently, they have generally not placed outside of the top fourteen spots since the inception of the rankings.[11] There have been rare exceptions to this, however, such as UCLA School of Law appearing in the top fourteen instead of Cornell and Northwestern in 1987 and University of Texas School of Law displacing Georgetown in 2018, although the significance of these changes has been debated.
https://en.wikipedia.org/wiki/Law_school_rankings_in_the_Uni...
>> That’s not where T14 comes from.
>There have been rare exceptions to this, however, such as UCLA School of Law appearing in the top fourteen instead of Cornell and Northwestern in 1987 and University of Texas School of Law displacing Georgetown in 2018, although the significance of these changes has been debated.
People select measures for a number of reasons. It's perfectly possible the single and only reason T14 is used is for the reasons you lay out.
OTOH I have personally had conversations with people who argued that the T14 is a measure of merit when Georgetown was in the T14 then "debated the significance" of the measure when Georgetown dropped out of the top 14 in 2018.
It was not clever or endearing.
>There have been rare exceptions to this, however, such as UCLA School of Law appearing in the top fourteen instead of Cornell and Northwestern in 1987 and University of Texas School of Law displacing Georgetown in 2018, although the significance of these changes has been debated.
People select measures for a number of reasons. It's perfectly possible the single and only reason T14 is used is for the reasons you lay out.
OTOH I have personally had conversations with people who argued that the T14 is a measure of merit when Georgetown was in the T14 then "debated the significance" of the measure when Georgetown dropped out of the top 14 in 2018.
It was not clever or endearing.
I personally think Universities and colleges are a scam that was necessary before we had access to the bulk of human knowledge on the tip of our fingers. We should go back to an apprentice system, go through elementary and middle school and then find a profession you would want to learn OTJ, this would shift the responsibility of training to companies but the people coming out of there would be immediately productive.
Rankings are a bit dumb, especially because any non english university is going to get ranked terribly basically on every of these rankings. There's no way that the oldest university in central Europe, started in 1348 should be placed 400-500th. That's just a joke.
The only ranking that matters to the vast majority of people:
Median starting salary / median cost to attend.
Sure, there are other considerations such as. If it will get you into a better grad school or something, but for most it’s really this simple.
Median starting salary / median cost to attend.
Sure, there are other considerations such as. If it will get you into a better grad school or something, but for most it’s really this simple.
Here is the code used by the paper: https://github.com/alidasdan/university-rankings
Any idea what they are measuring against? For example avg. income or accolades of graduates.
[deleted]
*>we both formally and experimentally show in multiple ways that this ranking scheme is not reliable and cannot be trusted as authoritative because it is too sensitive to weight changes and can easily be gamed.
That's a feature, not a bug, as far as USNWR (the magazine doing the ranking) is concerned. A more reliable ranking would be more stable and therefore less interesting for the universities. Why pay attention to it if Harvard, Standford, MIT are always 1, 2, 3? It's much more "interesting" and certainly sells more copies/gets more clicks if the rankings have a certain amount of volatility. The fact that it's easily game-able also makes it more attractive to universities because they can do something proactive if they're dissatisfied with their ranking. All these factors combine to make the ranking itself seem more important and prestigious.