Ask HN: Is the world suffering from shallow knowledge?
65 comments
Statistically, the opposite is happening: human knowledge is becoming deeper. The arc of human knowledge throughout history bends towards specialization: never before in human history have so many people been educated in so many different subdomains, each different in methodology, discipline, and scope. This is evinced by rising numbers of postgraduate degrees, increasing diversity in academic specialization, etc.
This poses its own problems:
* Academically: extreme specialization increases academic friction, as closely related but distinct subdomains struggle to consume each others' results
* Culturally: we're slowly losing a very important kind of shallow knowledge: the ability to repair small things, to converse intelligently about things outside one's area of expertise, to understand more generally that we can't derive every fact about every field from a small set of first principles.
So overall, I don't think so. I think we (including the youths) have more and deeper knowledge than ever before, at the cost of important, shallow, general knowledge.
This poses its own problems:
* Academically: extreme specialization increases academic friction, as closely related but distinct subdomains struggle to consume each others' results
* Culturally: we're slowly losing a very important kind of shallow knowledge: the ability to repair small things, to converse intelligently about things outside one's area of expertise, to understand more generally that we can't derive every fact about every field from a small set of first principles.
So overall, I don't think so. I think we (including the youths) have more and deeper knowledge than ever before, at the cost of important, shallow, general knowledge.
Also statistically, every species in history that has ever over-specialised has gone extinct.
With specialisation comes interdependence. If you spend 99% of your free time studying computers it is almost certain that you don't know how to grow a potato. You depend on others to do things for you.
Over time specialisation results in critical and basic survival skills being lost from the mainstream.
Usually then, some event occurs that wipes out a critical group of specialists, and everything that depends on them comes crashing down, resulting in the species being wiped out... because the knowledge cannot be rediscovered, or the skills re-learnt, before the food runs out.
We are rushing towards a future where all of our food, water and energy is grown/harvested/generated/distributed by AI-driven machines. The number of people on the planet who will know how those AIs actually work will be minuscule.
Specialists already do things like attend conferences... so gather in a single location... breathe the same air while there... stay at the same hotels... eat at the same restaurants... fly the same planes...
If we care about the long-term survival of the Human species, then we should be very, very careful not to allow over-specialisation to occur in critical fields.
With specialisation comes interdependence. If you spend 99% of your free time studying computers it is almost certain that you don't know how to grow a potato. You depend on others to do things for you.
Over time specialisation results in critical and basic survival skills being lost from the mainstream.
Usually then, some event occurs that wipes out a critical group of specialists, and everything that depends on them comes crashing down, resulting in the species being wiped out... because the knowledge cannot be rediscovered, or the skills re-learnt, before the food runs out.
We are rushing towards a future where all of our food, water and energy is grown/harvested/generated/distributed by AI-driven machines. The number of people on the planet who will know how those AIs actually work will be minuscule.
Specialists already do things like attend conferences... so gather in a single location... breathe the same air while there... stay at the same hotels... eat at the same restaurants... fly the same planes...
If we care about the long-term survival of the Human species, then we should be very, very careful not to allow over-specialisation to occur in critical fields.
I think generalization is very important, but I also think this oversells it slightly: every other species in history has lacked the privilege of writing its progress down. A PhD in farm science doesn't need to know how to grow a potato today, because (in theory) they can open a book and learn how it's done.
My concern about specialization is less that we'll fundamentally lose basic skills, but rather that our academic and cultural trends away from general knowledge are socially harmful and ultimately harmful to specialization itself (which is ultimately a good and necessary aspect of scientific progress).
My concern about specialization is less that we'll fundamentally lose basic skills, but rather that our academic and cultural trends away from general knowledge are socially harmful and ultimately harmful to specialization itself (which is ultimately a good and necessary aspect of scientific progress).
A PhD in farm science with access to a book about growing potatoes will have died of starvation 50-100 days before their first crop is ready to be harvested. It's the lack of time available to actually grow the food that proves terminal. Having access to knowledge is not enough. Being a species that writes stuff down is not, in and of itself, a guarantee of our survival.
If a group of critical specialists gets wiped out there will a) not be enough time to learn how to keep their complex system running, or b) not be enough time to learn and implement a less complex system to the required scale, before the food runs out and everyone dies.
For survival to be assured, there needs to be diversity in critical areas. The alternatives need to be actively studied and practised alongside whatever is dominant. The problem is that competitive economic systems systematically drive out the 'less efficient' approaches, leaving a gaping hole in our fall-back options. Put another way: 'Capitalism eliminates Plan B'.
The prevailing economic system (which rewards and promotes specialisation, homogenisation and efficiency) is incompatible with long-term survival (which depends on generalisation, diversity and inefficiency).
If a group of critical specialists gets wiped out there will a) not be enough time to learn how to keep their complex system running, or b) not be enough time to learn and implement a less complex system to the required scale, before the food runs out and everyone dies.
For survival to be assured, there needs to be diversity in critical areas. The alternatives need to be actively studied and practised alongside whatever is dominant. The problem is that competitive economic systems systematically drive out the 'less efficient' approaches, leaving a gaping hole in our fall-back options. Put another way: 'Capitalism eliminates Plan B'.
The prevailing economic system (which rewards and promotes specialisation, homogenisation and efficiency) is incompatible with long-term survival (which depends on generalisation, diversity and inefficiency).
> We are rushing towards a future where all of our food, water and energy is grown/harvested/generated/distributed by AI-driven machines. The number of people on the planet who will know how those AIs actually work will be minuscule.
Horseshit in the press emanating from marketing departments aside, there is no such thing as AI, and nobody has any idea how to make an AI system. Don't believe the hype.
Horseshit in the press emanating from marketing departments aside, there is no such thing as AI, and nobody has any idea how to make an AI system. Don't believe the hype.
> Also statistically, every species in history that has ever over-specialised has gone extinct.
Every other species to over-specialize has done it physically, not mentally.
Every other species to over-specialize has done it physically, not mentally.
>> Statistically, the opposite is happening: human knowledge is becoming deeper.
Do you have a source to point to? (honestly wondering)
>> to converse intelligently about things outside one's area of expertise
Just had this conversation the other day. 90% of my old college teachers (engineering) could talk about and explain in depth many many topics out of their field (philosophy, history, geography and what not). That's not the case today.
Do you have a source to point to? (honestly wondering)
>> to converse intelligently about things outside one's area of expertise
Just had this conversation the other day. 90% of my old college teachers (engineering) could talk about and explain in depth many many topics out of their field (philosophy, history, geography and what not). That's not the case today.
> 90% of my old college teachers (engineering) could talk about and explain in depth many many topics out of their field (philosophy, history, geography and what not).
How do you know they could explain it in depth?
I've heard a lot of people who think they know history/philosophy/economics (especially on HN), but what they really know is pop history/philosophy/economics.
How do you know they could explain it in depth?
I've heard a lot of people who think they know history/philosophy/economics (especially on HN), but what they really know is pop history/philosophy/economics.
I've had a similar experience. I think this loss of generalization is (dishearteningly) most visible in CS departments: many of my CS peers were openly scornful or disdainful of their liberal arts requirements, which in turn were diluted to ensure that mediocrity or inattention would not filter them out of their degrees. The end result: thousands of yearly graduates qualified to program (and by virtue bring money and renown back to the department), and nothing else.
Edit: And to be clear, these were not stupid people, and there is an important sense in which they were right to be scornful: the classes were clearly not intended to challenge them. I don't believe that particular blame can be levied on them.
Edit: And to be clear, these were not stupid people, and there is an important sense in which they were right to be scornful: the classes were clearly not intended to challenge them. I don't believe that particular blame can be levied on them.
College professors are like the academically top 1% of CS graduates. Are you comparing apples to apples?
I didn't come from CS :)
I think it is important to remember not to value judge these sorts of widespread skills. Does it matter today that I can't shape a flint or ride a horse?
I think the error here is in judging the absence of specific skills, and not in judging the broad absence of non-specialized skills.
In other words: it's wrong (and ridiculous) to judge someone for not knowing how to shape a flint, but it's correct to note that we should all have some general set of skills that aren't tied directly to our academic backgrounds or professions.
In other words: it's wrong (and ridiculous) to judge someone for not knowing how to shape a flint, but it's correct to note that we should all have some general set of skills that aren't tied directly to our academic backgrounds or professions.
I think there’s a huge difference between being able to do something and being able to intelligently converse about it. The former I would not expect of people for tasks outside their specialization, but the latter seems like a more reasonable ask - especially if the demographic is college professors.
There's this common sense that creativity and great ideas come when you gather two "unrelated topics", maybe riding a horse can spark some new idea in a new field (ie. robotics)
Presumably you can use a lighter and drive a car though.
One of those two may no longer be the case for people soon...
One of those two may no longer be the case for people soon...
Except perhaps theoretical physics and maybe pure math. You have to pretty much know everything of equal depth to make progress in those domains. You cannot specialize, you have to know it all.
I've heard the opposite: Pure math conferences described as places where the attendees struggle to talk in depth about their work with other random attendees, because everyone's working in different, deeply specialised subfields of math.
People don't want to do their jobs, but that doesn't mean they're completely unmotivated. Many people with jobs they're indifferent to, also have hobbies they pursue relentlessly.
If you want people to take a deeper interest in their jobs, give them more money. Way more money. Way, way more money. If you want people to pursue deeper understanding of their interests, give them more free time.
To the extent that "shallow knowledge" is a problem, I think it's likely because most people spend a lot of their time doing boring work for not-enough money.
If you want people to take a deeper interest in their jobs, give them more money. Way more money. Way, way more money. If you want people to pursue deeper understanding of their interests, give them more free time.
To the extent that "shallow knowledge" is a problem, I think it's likely because most people spend a lot of their time doing boring work for not-enough money.
> Way, way more money
Yeah, it seems like we could solve a lot of social problems if we had a lot more nurses, school teachers, mental health pros, librarians and psychologists, and if they all had deep knowledge and high motivation.
It would be worth paying triple their current salaries to attract more people with greater commitment and enthusiasm.
We would all be richer, and could probably pay for the extra compensation through reduced crime and violence, lower healthcare and public benefits, fewer prison inmates to guard and house.
Not to mention the dividends that would compound over decades with each new generation from this modest investment.
Yeah, it seems like we could solve a lot of social problems if we had a lot more nurses, school teachers, mental health pros, librarians and psychologists, and if they all had deep knowledge and high motivation.
It would be worth paying triple their current salaries to attract more people with greater commitment and enthusiasm.
We would all be richer, and could probably pay for the extra compensation through reduced crime and violence, lower healthcare and public benefits, fewer prison inmates to guard and house.
Not to mention the dividends that would compound over decades with each new generation from this modest investment.
I wish I could go deep into something useful and interesting but the compensation is really an issue. I guess for now I'll continue to make sure ads appear in youre lists of text updates :(
Absolutely. Anything less than a salary that guarantees a single family home paid off, and a large amount of savings, within 10-20 years, is not enough to completely stop looking for side hustles that might lead to those things.
I'm a little confused by this statement. Of course people want to obtain value very quickly; we live in a society where people are explicitly told their economic success and therefore basic survival is a zero-sum game in competition with other human beings (we are not told this is untrue unless we explicitly go out and discover this), where the opportunity to do basic things like be housed and have a family is dependent on economic achievement in a rapidly more unequal society.
Of course people are trying to obtain value as swiftly as possible. Every moment they aren't doing so means they are now trying to gain value in a situation where there is less value possible to gain. Additionally, our society glorifies "hustle culture" and "gig economy", so yeah, if we raise our children like this they will follow the values we teach them.
Of course people are trying to obtain value as swiftly as possible. Every moment they aren't doing so means they are now trying to gain value in a situation where there is less value possible to gain. Additionally, our society glorifies "hustle culture" and "gig economy", so yeah, if we raise our children like this they will follow the values we teach them.
> I couldn't help but notice that many young people don't want to spend months pulling their hair off to get a deeper understanding of what they do... they want to obtain value very quickly.
I feel like ... that's how humans are in general? Our brains are wired to seek easy paths, which is how habits are formed.
> The existence of a big number of people who can do complicated jobs is crucial for the modern world, and I think this whole thing will fall down if it continued with the current mode.
I think we may have different definitions of "complicated jobs" here, but I believe the number of people who are getting higher education (masters+) have been increasing.
Yes I do believe that people's attention span (mine included) are getting shorter and shorter. I've found myself watching more "summary videos" that are shallow instead of sitting down and deeply engaging with the topic. But I don't think it's to a point where the whole thing will fall down.
I do believe that we are criminally underpaying some crucial professions that require deeper understanding though: I can't believe I get paid more than electrical /mechanical engineers.
I feel like ... that's how humans are in general? Our brains are wired to seek easy paths, which is how habits are formed.
> The existence of a big number of people who can do complicated jobs is crucial for the modern world, and I think this whole thing will fall down if it continued with the current mode.
I think we may have different definitions of "complicated jobs" here, but I believe the number of people who are getting higher education (masters+) have been increasing.
Yes I do believe that people's attention span (mine included) are getting shorter and shorter. I've found myself watching more "summary videos" that are shallow instead of sitting down and deeply engaging with the topic. But I don't think it's to a point where the whole thing will fall down.
I do believe that we are criminally underpaying some crucial professions that require deeper understanding though: I can't believe I get paid more than electrical /mechanical engineers.
I would argue that people who go deep are invisible for most of their early career to the general public, and only become famous (if ever) once they use this knowledge years later, at which point someone might ask about their past and they could explain how many thousands of hours they spent holed up learning.
Even within your extended local social graph this is likely to be the case, imagine if your e.g. neighbor's teen spends years learning about any topic, if you ask your neighbor then they're not likely to tell you (might not even know!) about this, instead would tell you the typical "oh you know he's a teenager and hangs out with his friends and is all day playing videogames" or might tell you on the surface "he's interested in science". You might only get a more accurate picture within a close social graph, but then it's statistically very unlikely that any of those are one of those dig-deep geniuses.
I've seen at least 3 examples in the Open Source/DIY communities of news about some popular-ish people who shocked everyone when they revealed their age and were teenagers. Some times they hid it intentionally to avoid being judged against, some times they just never mentioned.
BTW, the more generic "young people are lazy/don't want to put effort" is a meme that has been said for literally centuries by the respective older generations:
https://i.kym-cdn.com/photos/images/original/002/407/503/119...
https://nymag.com/intelligencer/2013/05/me-me-me-generation-...
Even within your extended local social graph this is likely to be the case, imagine if your e.g. neighbor's teen spends years learning about any topic, if you ask your neighbor then they're not likely to tell you (might not even know!) about this, instead would tell you the typical "oh you know he's a teenager and hangs out with his friends and is all day playing videogames" or might tell you on the surface "he's interested in science". You might only get a more accurate picture within a close social graph, but then it's statistically very unlikely that any of those are one of those dig-deep geniuses.
I've seen at least 3 examples in the Open Source/DIY communities of news about some popular-ish people who shocked everyone when they revealed their age and were teenagers. Some times they hid it intentionally to avoid being judged against, some times they just never mentioned.
BTW, the more generic "young people are lazy/don't want to put effort" is a meme that has been said for literally centuries by the respective older generations:
https://i.kym-cdn.com/photos/images/original/002/407/503/119...
https://nymag.com/intelligencer/2013/05/me-me-me-generation-...
Just standard "what is wrong the kids!" fare but HN-flavored.
There are plenty of shallow thinkers of all ages.
There are plenty of shallow thinkers of all ages.
Not sure i agree... I've been watching my kids find a subject on youtube and delve deeeep into it. My boy (14) has spend the last 2 years solid teaching himself to program almost exclusively through youtube videos to a point where he is way ahead of his schools ability to keep up with him.
He's not going to coder dojo and finding that he's so far ahead of those kids that the leaders are asking him to train some of the other kids.
My daughter is more art focused and spends hours watching art tutorials.
The kids are specializing in my corner of the world. Not sure what you're seeing.
He's not going to coder dojo and finding that he's so far ahead of those kids that the leaders are asking him to train some of the other kids.
My daughter is more art focused and spends hours watching art tutorials.
The kids are specializing in my corner of the world. Not sure what you're seeing.
OP, you will likely be interested in this talk by Bert Hubert:
How Tech Loses Out over at Companies, Countries and Continents: On how companies lose deep knowledge and stop innovating
https://berthub.eu/articles/posts/how-tech-loses-out/
As a video: https://www.youtube.com/watch?v=PQccNdwm8Tw
How Tech Loses Out over at Companies, Countries and Continents: On how companies lose deep knowledge and stop innovating
https://berthub.eu/articles/posts/how-tech-loses-out/
As a video: https://www.youtube.com/watch?v=PQccNdwm8Tw
Thanks
My day to day assumption is for nearly anything I want to do, someone else in the relevant domain has already done the deep dive and it's a variety of premature optimization for me to over invest.
If it's important, or my domain, or something interesting, sure, I'll dig in deeper. However, we live at a time where vast expertise is a few clicks away, so I find it's usually more valuable to know how to find and analyze someone else's solution than to create my own.
If it's important, or my domain, or something interesting, sure, I'll dig in deeper. However, we live at a time where vast expertise is a few clicks away, so I find it's usually more valuable to know how to find and analyze someone else's solution than to create my own.
I don't think most people want to pull their hair out for anything, period.
Yet, young people will pursue what fascinates them very deeply when given the chance.
A huge problem is that we don't teach things in such a way that it's interesting, at all, to the majority of people.
Another problem is, of course, the focus on the next quarter's profit.
But, overall, I agree, it's a problem. Just not one I'd pin on the kids.
Yet, young people will pursue what fascinates them very deeply when given the chance.
A huge problem is that we don't teach things in such a way that it's interesting, at all, to the majority of people.
Another problem is, of course, the focus on the next quarter's profit.
But, overall, I agree, it's a problem. Just not one I'd pin on the kids.
"I couldn't help but notice that many young people don't want to spend months pulling their hair off to get a deeper understanding of what they do... they want to obtain value very quickly."
It might be that we have to obtain value quickly.
I'd love to dive deep into my systems and tech. My company doesn't want that. They want someone who can just knock out half-decent code as dastardly as possible. There's no thought to actually allowing or helping employees become experts. They don't want experts because they can't afford them, so they create an environment where everyone is mediocre.
It might be that we have to obtain value quickly.
I'd love to dive deep into my systems and tech. My company doesn't want that. They want someone who can just knock out half-decent code as dastardly as possible. There's no thought to actually allowing or helping employees become experts. They don't want experts because they can't afford them, so they create an environment where everyone is mediocre.
Is the world suffering? Maybe. You'll get a variety of responses and incorrections.
If so, why is the world suffering? You'll get a much wider variety of responses. I doubt many of them will be general shallow thinking, though there will surely be fingers pointed at some people for that crime.
What is shallow knowledge? Who's to determine the correct depth? I hope it doesn't work like Mississippi river boats because Mark Twain is already dead.
Is asking a web forum for an answer to something as important as the world's suffering itself an example of shallow knowledge? Or an attempt to elicit it?
If so, why is the world suffering? You'll get a much wider variety of responses. I doubt many of them will be general shallow thinking, though there will surely be fingers pointed at some people for that crime.
What is shallow knowledge? Who's to determine the correct depth? I hope it doesn't work like Mississippi river boats because Mark Twain is already dead.
Is asking a web forum for an answer to something as important as the world's suffering itself an example of shallow knowledge? Or an attempt to elicit it?
[deleted]
Just my own speculation, but yes. The lack of deep focus is endemic in the culture. There is too much cheap distraction. I’ve gone through my phase of thinking it will collapse. I think that’s possible, but a more realistic base case is a consolidation of key technologies and methods that can get the job done, although standard of living may reduce drastically.
I think there is an optimistic way to look at this and try to help direct it on a better path, but I don't know it.
Distraction is another aspect too, yes, but this is for everyone. This problem is a little bit different.
And if some jobs got easier, you still need people to deal with the low-level stuff.
Distraction is another aspect too, yes, but this is for everyone. This problem is a little bit different.
And if some jobs got easier, you still need people to deal with the low-level stuff.
Yes, some people need deep focus to get specialized tasks done ... brain-surgery, rocket-science. But that focus may lead them to miss some of the downsides of what they are creating, or awareness of better alternatives. I may be an expert widget-builder, so much so that I overlook the trend away from widgets.
Like everything else, cultures change. So does the meaning of 'standard-of-living', very culture-driven.
So there's also a need for 'bigger-picture' generalists. EG we need architects to oversee the bridge-builders, and planners to oversee the collective results of the architects, and philosophers to ask where in the hell we're headed. Sometimes the best solutions require painting outside our expert boxes.
Like everything else, cultures change. So does the meaning of 'standard-of-living', very culture-driven.
So there's also a need for 'bigger-picture' generalists. EG we need architects to oversee the bridge-builders, and planners to oversee the collective results of the architects, and philosophers to ask where in the hell we're headed. Sometimes the best solutions require painting outside our expert boxes.
I couldn't help but notice that many young people don't want to spend months pulling their hair off to get a deeper understanding of what they do... they want to obtain value very quickly.
so how big of a sample size you're talking about. A lot of young people are producing cutting edge research in math, physics, computer science, coding etc.
so how big of a sample size you're talking about. A lot of young people are producing cutting edge research in math, physics, computer science, coding etc.
I think the problem is that the payoff for engaging deeply into a topic often isn’t better than the payoff for engaging in shallow learning of a wide variety of different topics.
For example, why would I want to spend time learning deeply how to write my own compiler, vs just spending that time to do tutorials for a couple different new frameworks?
For example, why would I want to spend time learning deeply how to write my own compiler, vs just spending that time to do tutorials for a couple different new frameworks?
> For example, why would I want to spend time learning deeply how to write my own compiler, vs just spending that time to do tutorials for a couple different new frameworks?
I feel I can answer that easily:
Writing a compiler is vastly more interesting and satisfying!
Learning new frameworks, how boring. I'll do it for pay or prospects.
Writing compilers, now that I'll do for free on the evenings and weekends. You cannot pry my fingers from the keyboard when I'm on a compiler roll! Important household tasks will go neglected! Someone should remind me to eat!
I feel I speak for many here on HN with this sentiment. (See also database engines and distributed systems.)
To put it back on topic: Some things are intrinsically rewarding. It feels good to go deep with them, especially to advance them in some way. That's the payoff. I think this is a significant driving factor in the expansion of human knowledge; it's not just driven by economic incentives to learn things, although economic limitations do prevent people from having capacity to go deep into things they'd like to
I feel I can answer that easily:
Writing a compiler is vastly more interesting and satisfying!
Learning new frameworks, how boring. I'll do it for pay or prospects.
Writing compilers, now that I'll do for free on the evenings and weekends. You cannot pry my fingers from the keyboard when I'm on a compiler roll! Important household tasks will go neglected! Someone should remind me to eat!
I feel I speak for many here on HN with this sentiment. (See also database engines and distributed systems.)
To put it back on topic: Some things are intrinsically rewarding. It feels good to go deep with them, especially to advance them in some way. That's the payoff. I think this is a significant driving factor in the expansion of human knowledge; it's not just driven by economic incentives to learn things, although economic limitations do prevent people from having capacity to go deep into things they'd like to
I think the obvious answer is "no the world isn't"
Do you have any evidence that young people are significantly different than people of the past that is not ancedotal?
I would suggest you probably just have nostalgia for an imaginary past which never existed. You remember the impressive people of your youth and forget all the mediocre ones.
Do you have any evidence that young people are significantly different than people of the past that is not ancedotal?
I would suggest you probably just have nostalgia for an imaginary past which never existed. You remember the impressive people of your youth and forget all the mediocre ones.
Yes if anything the evidence suggests that the younger are smarter, more literate, etc. on aggregate than past generations, esp. when controlling for other factors like family income, etc. When you expand to a global view, this is undoubtedly true.
The tech industry has "eaten its young."
By this, I mean that it has systematically destroyed the source of people who grow up with the passion to learn tech at the deep levels that make for the people the tech industry likes to hire for the core, foundational stuff that makes things run.
About a decade ago, I understand Google's SRE hiring pipeline was struggling, because Google had singlehandedly destroyed their best source for hiring - university student sysadmins. Google Apps for Education was great, and allowed shutting down a ton of annoyingly expensive server rooms at universities - but it also meant that those universities no longer had nearly the number of various student sysadmins who were entirely unfazed by "The mainframe is serial linked to the G3 Mac for ethernet access, with software written by someone long since graduated, and..." sort of systems. It was a wonderful filter for people with a diverse set of experience across a range of "You did what to the what now? In assembly?" sort of systems, and they could usually translate that experience to the sort of "haven't seen this before..." stuff that Google ran at the time.
The same seems to be true in the really foundational parts of computing. I know very, very few people under the age of about 40 in the firmware/osdev/hypervisor/etc spaces, and that age has more or less been tracking along with me. It's not a base age for people getting into the space, it's the age of the youngest of the last generation to know how to do that stuff.
There are exceptions, but they're few, far between, and seem to not go into those spaces terribly often either.
The modern tech industry is just focused on distracting people as much as possible, as long as possible, to look at as many ads as possible, and that's entirely at odds with the requirements to get good at the deep weeds, which is long, sustained focus, in challenging and arcane spaces.
But it's worse, because the complexity of modern tech is increasing at such a rapid rate that you have to run to stand still. All the microarchitectural vulnerabilities and such have been "patched" with... complexity. When they stem from complexity. And the whole stack at this point can be summed up as "Complexity created to hide the problems caused by the previous round of complexity." It's nasty, all the way down, and then you discover that the people who are designing the foundations have no idea what they're doing either! Intel clearly doesn't understand their chips, given how many times the stated guarantees of SGX (that malicious ring 0 code can neither observe nor modify the state of production SGX enclave computation) have been violated by a range of mechanisms. L1TF/Foreshadow cracked production enclaves wide open with demonstrations of how to single step them and pull full state from every step, and Plundervolt involved undervolting the chip to the edge of faulting (so multiply and AES operations would fault but simpler stuff would retire fine), and then launching an SGX enclave that ran faulting operations. Whoops.
I don't know how much longer this sort of thing can continue, because the last wave of people who do understand the deep weeds that make it work are starting to retire, with nobody to replace them. And they just want to go do "something that has nothing to do with computers" anymore.
A recent comic on Reddit (https://www.reddit.com/r/ProgrammerHumor/comments/vnmmuo/whe...) demonstrates just how strong this sentiment has become in recent years. The hypervisor hackers and firmware devs are going to retire and rapidly forget everything they've known, and have a grand old time doing "anything else." I know an awful lot of these people, myself included.
It's going to be a very real problem in the decades to come.
By this, I mean that it has systematically destroyed the source of people who grow up with the passion to learn tech at the deep levels that make for the people the tech industry likes to hire for the core, foundational stuff that makes things run.
About a decade ago, I understand Google's SRE hiring pipeline was struggling, because Google had singlehandedly destroyed their best source for hiring - university student sysadmins. Google Apps for Education was great, and allowed shutting down a ton of annoyingly expensive server rooms at universities - but it also meant that those universities no longer had nearly the number of various student sysadmins who were entirely unfazed by "The mainframe is serial linked to the G3 Mac for ethernet access, with software written by someone long since graduated, and..." sort of systems. It was a wonderful filter for people with a diverse set of experience across a range of "You did what to the what now? In assembly?" sort of systems, and they could usually translate that experience to the sort of "haven't seen this before..." stuff that Google ran at the time.
The same seems to be true in the really foundational parts of computing. I know very, very few people under the age of about 40 in the firmware/osdev/hypervisor/etc spaces, and that age has more or less been tracking along with me. It's not a base age for people getting into the space, it's the age of the youngest of the last generation to know how to do that stuff.
There are exceptions, but they're few, far between, and seem to not go into those spaces terribly often either.
The modern tech industry is just focused on distracting people as much as possible, as long as possible, to look at as many ads as possible, and that's entirely at odds with the requirements to get good at the deep weeds, which is long, sustained focus, in challenging and arcane spaces.
But it's worse, because the complexity of modern tech is increasing at such a rapid rate that you have to run to stand still. All the microarchitectural vulnerabilities and such have been "patched" with... complexity. When they stem from complexity. And the whole stack at this point can be summed up as "Complexity created to hide the problems caused by the previous round of complexity." It's nasty, all the way down, and then you discover that the people who are designing the foundations have no idea what they're doing either! Intel clearly doesn't understand their chips, given how many times the stated guarantees of SGX (that malicious ring 0 code can neither observe nor modify the state of production SGX enclave computation) have been violated by a range of mechanisms. L1TF/Foreshadow cracked production enclaves wide open with demonstrations of how to single step them and pull full state from every step, and Plundervolt involved undervolting the chip to the edge of faulting (so multiply and AES operations would fault but simpler stuff would retire fine), and then launching an SGX enclave that ran faulting operations. Whoops.
I don't know how much longer this sort of thing can continue, because the last wave of people who do understand the deep weeds that make it work are starting to retire, with nobody to replace them. And they just want to go do "something that has nothing to do with computers" anymore.
A recent comic on Reddit (https://www.reddit.com/r/ProgrammerHumor/comments/vnmmuo/whe...) demonstrates just how strong this sentiment has become in recent years. The hypervisor hackers and firmware devs are going to retire and rapidly forget everything they've known, and have a grand old time doing "anything else." I know an awful lot of these people, myself included.
It's going to be a very real problem in the decades to come.
I watched this happen in real-time, though not at a university. I graduated with a network admin degree (AAS) right around the time the market was collapsing in 2008/2009. I watched as all the in-house IT jobs vanished and were replaced with cloud offerings as I worked on my degree. They had to cut costs, I get it, but it meant I couldn't actually do anything with the degree. All that was left were outsourcing firms that wanted lots of experience if they were hiring at all.
Good thing a state scholarship paid for most of it. I did at least get a little better at math and a lot better at writing while getting an introduction to economics. Those non-tech prerequisites fixed a lot that K-12 broke. I lost all interest in a tech career as my confidence in other areas grew. Tech was just the obvious track after running the family's network at home.
Good thing a state scholarship paid for most of it. I did at least get a little better at math and a lot better at writing while getting an introduction to economics. Those non-tech prerequisites fixed a lot that K-12 broke. I lost all interest in a tech career as my confidence in other areas grew. Tech was just the obvious track after running the family's network at home.
That's probably just due to lack of demand. Competitive new operating systems are so expensive that nobody writes them anymore, they just use Linux, and then you don't need so many people working on that for anything except drivers.
In the unlikely event that there starts to be shortages of kernel and firmware devs then salary changes will be enough to sort it out. Fundamentally, that sort of work isn't that hard to learn. C is not a complex language, the design is normally fixed by the hardware and software environment that's provided, the APIs don't churn every five minutes and so on.
As for SGX. Security is always hard. If your metric for deciding if someone understands something is "they never make security bugs ever" then you'll be perennially disappointed in how nobody seems to understand any really existing system.
In the unlikely event that there starts to be shortages of kernel and firmware devs then salary changes will be enough to sort it out. Fundamentally, that sort of work isn't that hard to learn. C is not a complex language, the design is normally fixed by the hardware and software environment that's provided, the APIs don't churn every five minutes and so on.
As for SGX. Security is always hard. If your metric for deciding if someone understands something is "they never make security bugs ever" then you'll be perennially disappointed in how nobody seems to understand any really existing system.
I think it's been like this for thousands of years. The forefront of human knowledge has not been advanced by everybody all doing their bit. It was by a vanishingly tiny number of super-intellects, plus a much bigger (but still small) group of experts and deep thinkers who help carry and refine and expand those ideas.
I would say most people don't have the capability to usefully contribute to the complicated problems of today. The good thing is that those who do are inclined to share it with the rest of us because they can't do that in isolation either, and are dependent on the blood and sweat of others too.
I would say most people don't have the capability to usefully contribute to the complicated problems of today. The good thing is that those who do are inclined to share it with the rest of us because they can't do that in isolation either, and are dependent on the blood and sweat of others too.
I think the number of people that are deep down into some specific knowledge is always fairly low. Our world currently works in a way that big majority can benefit from the work of a few. There are only a few thousand people in the world who design and create new medication, nuclear plants,or some high end microprocessors.
There are a few industries that are examples of some deep expertise. Airlines maintenance is by no means shallow, not in well ran airlines. All airspace industries I believe need deep expertise of what they are doing.
there are more industries like that, but yes, the average young person is not interested in gaining deep knowledge, they want value quick, at some point shallow knowledge will have less value and we as a collective will have to dig deeper... i hope
there are more industries like that, but yes, the average young person is not interested in gaining deep knowledge, they want value quick, at some point shallow knowledge will have less value and we as a collective will have to dig deeper... i hope
Young people may not want to work hard (when was that ever not the case), but they also respond to incentives. When they understand that higly specialized jobs are the surest path to their desired standard of living and consumption, many of them get cracking.
Teenager here.
Not every adult knows anything deep about any topic. only 1% of adults can talk in detail about atleast
one topic. It's the same in the next generation too. Not all kids want to learn, but some do. It's just about our outlook towards it
There is no stopping of "current" mode. Yet.
When presenting my PD work for the young guns, I UX them to unconsciousness:)
Scratching the surface of a given topic and focusing on comfortable for them communication is the key. I always let the door be open for the deeper questions without selling my "extra" knowledge or experience. They have to be the stars in the room, knowledgeable professionals have a critical way of thinking which is "annoying" and "unproductive".
Be soft, forgiving, friendly, and willing to learn the "new trends" without asking questions. Just tell you deeper knowledge to take back seat.
My only hope is in the recession. Maybe if we are lucky things will reach a critical point and all the "unicorn magic", "empty words", "big ideas" will meet reality. And we know that reality has no sense of humor or "culture fit". :)
Scratching the surface of a given topic and focusing on comfortable for them communication is the key. I always let the door be open for the deeper questions without selling my "extra" knowledge or experience. They have to be the stars in the room, knowledgeable professionals have a critical way of thinking which is "annoying" and "unproductive".
Be soft, forgiving, friendly, and willing to learn the "new trends" without asking questions. Just tell you deeper knowledge to take back seat.
My only hope is in the recession. Maybe if we are lucky things will reach a critical point and all the "unicorn magic", "empty words", "big ideas" will meet reality. And we know that reality has no sense of humor or "culture fit". :)
Generalizing about the younger generation, a tale as old as time.
You are so much smarter and wiser than all of them, good job.
You are so much smarter and wiser than all of them, good job.
Actually, yes. The tech industry has a strong fetish for "young and capable" and a proven record of ageism and undervaluation of professionals.
I have taken my share on both sides of the spectrum.
And I understand your "reactive" reaction perfectly. And approve it wholeheartedly, but we are reaching the point where things are looking scary and someone must take "the burden" and share honestly. Self-censoring is a real thing.
With all this "political correctness" we forget that this friction between "young and old" when managed properly can make the world a better place.
By the way, I respect and work with a lot of young people and regularly listen carefully for inspiration and fresh points of view from them.
I have taken my share on both sides of the spectrum.
And I understand your "reactive" reaction perfectly. And approve it wholeheartedly, but we are reaching the point where things are looking scary and someone must take "the burden" and share honestly. Self-censoring is a real thing.
With all this "political correctness" we forget that this friction between "young and old" when managed properly can make the world a better place.
By the way, I respect and work with a lot of young people and regularly listen carefully for inspiration and fresh points of view from them.
Why would I want a deeper understanding of what I do (assuming you mean what I do in exchange for money)? I want to know as little as possible to get by.
If there is such a need, people will create incentive so somebody will digging into this knowledge. This is easy actually, the incentive could be money, honor etc.
I think incentives are there already, but more and more people start to say "we will leave this to the boring people who love books, and we get all the fun".
They are not built in a way that helps them read a large book for example. They don't appreciate attention to details. They don't know how to find joy in something like that, they don't know how to value someone like Dennis Ritchie for example. If you told them a few recommendations, they start to lecture you about the bad side of perfection :) ....I be like ...you and I didn't even smell perfection yet, but they have no idea what you are talking about.
I don't feel that incentives will change how they are built.
They are not built in a way that helps them read a large book for example. They don't appreciate attention to details. They don't know how to find joy in something like that, they don't know how to value someone like Dennis Ritchie for example. If you told them a few recommendations, they start to lecture you about the bad side of perfection :) ....I be like ...you and I didn't even smell perfection yet, but they have no idea what you are talking about.
I don't feel that incentives will change how they are built.
Anti-intellectualism was much more popular with kids of past generations than it is of "Gen Z."
The youngest are not the generation that came up with the "math isn't cool", "nerd", etc. archetypes.
The youngest are not the generation that came up with the "math isn't cool", "nerd", etc. archetypes.
> many young people don't want to spend months pulling their hair off to get a deeper understanding of what they do
Every generation says this about the next generation, and yet here we are with scientists in their 30s making huge advancements in things like mRNA vaccines.
If you think that laziness (what you're implying this is) will be the end of the world, there are many things that are far more urgent. No high-tech industry is collapsing because they can't find enough STEM graduates.
Every generation says this about the next generation, and yet here we are with scientists in their 30s making huge advancements in things like mRNA vaccines.
If you think that laziness (what you're implying this is) will be the end of the world, there are many things that are far more urgent. No high-tech industry is collapsing because they can't find enough STEM graduates.
>and yet here we are with scientists in their 30s making huge advancements in things like mRNA vaccines
mrna vaccines are several decades older than the people you are likely thinking of, and by many estimates the return on pharmacological research per dollar invested, as a rough metric of scientific productivity, has slowed by two magnitudes.
>No high-tech industry is collapsing because they can't find enough STEM graduates.
civil and nuclear engineering would like to disagree. Shortages are so severe that retirees are routinely brought back these days.
mrna vaccines are several decades older than the people you are likely thinking of, and by many estimates the return on pharmacological research per dollar invested, as a rough metric of scientific productivity, has slowed by two magnitudes.
>No high-tech industry is collapsing because they can't find enough STEM graduates.
civil and nuclear engineering would like to disagree. Shortages are so severe that retirees are routinely brought back these days.
> civil and nuclear engineering would like to disagree. Shortages are so severe that retirees are routinely brought back these days.
This has to do with funding shortfalls. Looking at the current political/economic landscape, I would practically be an idiot to invest time in trying to become a nuclear engineer rather than getting deep knowledge of a subject that capital holders have shown more interest in rewarding.
This has to do with funding shortfalls. Looking at the current political/economic landscape, I would practically be an idiot to invest time in trying to become a nuclear engineer rather than getting deep knowledge of a subject that capital holders have shown more interest in rewarding.
I have multiple friends who went into civil or nuclear engineering, and none of them stayed in public research. They get paid 3x by industry. One of them now builds United's algorithm for flight pricing.
It isn't an issue of laziness or lack of knowledge. We don't treat scientists well, so they leave.
> mrna vaccines are several decades older than the people you are likely thinking of
Old or dead scientists are the basis of all modern science.
The question is whether there are young scientists building on those foundations and making major breakthroughs, and there are[1].
1. https://leaps.org/moderna-covid-vaccine/the-scientist-who-so...
It isn't an issue of laziness or lack of knowledge. We don't treat scientists well, so they leave.
> mrna vaccines are several decades older than the people you are likely thinking of
Old or dead scientists are the basis of all modern science.
The question is whether there are young scientists building on those foundations and making major breakthroughs, and there are[1].
1. https://leaps.org/moderna-covid-vaccine/the-scientist-who-so...
[deleted]
You may just not see those people, but they are there
> Is the world suffering from shallow knowledge?
No. We are far better off now.
But yes, we are stuck.
It's like "HN now is worse than the old days" meme. It's not worse but it hasn't matured along with the exponential growth of the internet.
We are stuck at Wikipedia. TikTok is probably the "best" successor to Wikipedia. Teaching people how to use a saw over Wikipedia telling people saws exist. We can do better.
No. We are far better off now.
But yes, we are stuck.
It's like "HN now is worse than the old days" meme. It's not worse but it hasn't matured along with the exponential growth of the internet.
We are stuck at Wikipedia. TikTok is probably the "best" successor to Wikipedia. Teaching people how to use a saw over Wikipedia telling people saws exist. We can do better.
I learn everything I need from Ted talks.
Paul Graham once tweeted that he was explaining to his son that he can't be Einstein and Messi, and his son said so I will be Messi. But no Messi if there is no Stadium, Stream, Cameras, etc...
The existence of a big number of people who can do complicated jobs is crucial for the modern world, and I think this whole thing will fall down if it continued with the current mode.