The Lesson to Unlearn(paulgraham.com)
paulgraham.com
The Lesson to Unlearn
http://paulgraham.com/lesson.html
587 comments
Science education is hard. I was constantly frustrated in science classes, especially physics, because every 2 minutes I wanted to say "but then why ___?" Every new thing you learn seems to contradict either common sense (this rock clearly falls faster than this paper!), or something else you've been taught (you mean F=ma was never actually true? and which other equations might turn out to need correcting?).
I recognize it's not feasible for a class of students to constantly interrupt a lecture. The questions that other students had were never the same ones that I had, either. The problem is indeed scale, but I don't think motivation is a primary factor. Eventually you learn to be quiet and accept what the teacher and the textbook say.
I recognize it's not feasible for a class of students to constantly interrupt a lecture. The questions that other students had were never the same ones that I had, either. The problem is indeed scale, but I don't think motivation is a primary factor. Eventually you learn to be quiet and accept what the teacher and the textbook say.
> The problem is indeed scale
Hear, hear! When people say that with better technology we could automate away teachers, I say that with better technology we could automate most everything else and spend the time mentoring one another.
Hear, hear! When people say that with better technology we could automate away teachers, I say that with better technology we could automate most everything else and spend the time mentoring one another.
> any standardized measure will miss the mark
Goodhart's Law (Popular formulation): When a measure becomes a target, it ceases to be a good measure.
... via https://github.com/globalcitizen/taoup
Goodhart's Law (Popular formulation): When a measure becomes a target, it ceases to be a good measure.
... via https://github.com/globalcitizen/taoup
Yes, it's hard in both sides. I'm one of those people who got good grades in high school physics (I was able to solve problems), and still thought that satellites were kept in orbit because "there's not gravity" (or is too small). And I was genuinely interested in learning.
Part of the problem is the lack of time. Both teachers and students need to cover all the topics in one semester/year, and it takes time to assimilate and integrate the knowledge. Add to that that as student, you don't know what's important, even when you're told so, if you're told at all. That's why we can hold contradictory models in our minds. They're good enough to solve some problems, but conceptually wrong.
I think that a possible solution would be to keep the grades open. You get a B in physics-101, since you got a flawed model in your mind, you got to physics-201 where new experiences show the problems in your model and force you to correct it, and now you prove you achieved a knowledge that deserves an A.
I know, that proposal it's impractical, and time consuming, but I think that's how most of us really learn.
Part of the problem is the lack of time. Both teachers and students need to cover all the topics in one semester/year, and it takes time to assimilate and integrate the knowledge. Add to that that as student, you don't know what's important, even when you're told so, if you're told at all. That's why we can hold contradictory models in our minds. They're good enough to solve some problems, but conceptually wrong.
I think that a possible solution would be to keep the grades open. You get a B in physics-101, since you got a flawed model in your mind, you got to physics-201 where new experiences show the problems in your model and force you to correct it, and now you prove you achieved a knowledge that deserves an A.
I know, that proposal it's impractical, and time consuming, but I think that's how most of us really learn.
It is amusing how many of my peers still hold to the notion that heavier things fall faster. Not a higher terminal velocity, but the intuition that they accelerate faster. Hilariously tough for folks to shake.
Add air resistance into the mix, and denser materials do fall faster. Coins vs. napkins. Plums vs. crumpled balls of paper
I'd actually take the lesson the other way. It's hilariously tough for model-makers to acknowledge real-world noise that prevents their theoretically precise models from delivering reliable results.
I'd actually take the lesson the other way. It's hilariously tough for model-makers to acknowledge real-world noise that prevents their theoretically precise models from delivering reliable results.
I've done so many physics questions that explicitly instruct me to disregard air resistance... Reminds me of Feynman's speech:
https://v.cx/2010/04/feynman-brazil-education
> There are no experimental results mentioned anywhere in this book, except in one place where there is a ball, rolling down an inclined plane, in which it says how far the ball got after one second, two seconds, three seconds, and so on.
> The numbers have ‘errors’ in them – that is, if you look at them, you think you’re looking at experimental results, because the numbers are a little above, or a little below, the theoretical values. The book even talks about having to correct the experimental errors – very fine.
> The trouble is, when you calculate the value of the acceleration constant from these values, you get the right answer. But a ball rolling down an inclined plane, if it is actually done, has an inertia to get it to turn, and will, if you do the experiment, produce five-sevenths of the right answer, because of the extra energy needed to go into the rotation of the ball.
> Therefore this single example of experimental ‘results’ is obtained from a fake experiment. Nobody had rolled such a ball, or they would never have gotten those results!
https://v.cx/2010/04/feynman-brazil-education
> There are no experimental results mentioned anywhere in this book, except in one place where there is a ball, rolling down an inclined plane, in which it says how far the ball got after one second, two seconds, three seconds, and so on.
> The numbers have ‘errors’ in them – that is, if you look at them, you think you’re looking at experimental results, because the numbers are a little above, or a little below, the theoretical values. The book even talks about having to correct the experimental errors – very fine.
> The trouble is, when you calculate the value of the acceleration constant from these values, you get the right answer. But a ball rolling down an inclined plane, if it is actually done, has an inertia to get it to turn, and will, if you do the experiment, produce five-sevenths of the right answer, because of the extra energy needed to go into the rotation of the ball.
> Therefore this single example of experimental ‘results’ is obtained from a fake experiment. Nobody had rolled such a ball, or they would never have gotten those results!
Thanks for that Feynman link!
> “If I ask you a question during the lecture, afterwards everybody will be telling me, ‘What are you wasting our time for in the class? We’re trying to learn something. And you’re stopping him by asking a question’.”
In my experience there are almost just 2 types of motivated students: those who want to understand the content of the course and those who want a good grade and ultimately graduate.
There's often a conflict of interest between them. For example the first group wants to learn more, ask more questions, go deeper. The second group sees this as a threat because the additional content may now be on the exam as well. And they often misunderstand this behavior of the first group as just being teacher pleasers/pets.
Or from own experience: when you tell the prof that he accidentally started to cover the same topic that we already covered in the last lecture (because you're there to learn more), you will annoy a lot of people who hoped to get one fewer lectures to study in the course. Less content, less effort, same grade.
You hear the first group having passionate discussions on physics or programming over a beer, while the other group discusses all the latest info of which prof is how demanding, what the grading criteria are, what the deadlines are, which departments you should or should not take courses from, where they have mandatory presence, who does multiple choice tests and who does free-text exams.
I don't have much to conclude, just an observation.
> “If I ask you a question during the lecture, afterwards everybody will be telling me, ‘What are you wasting our time for in the class? We’re trying to learn something. And you’re stopping him by asking a question’.”
In my experience there are almost just 2 types of motivated students: those who want to understand the content of the course and those who want a good grade and ultimately graduate.
There's often a conflict of interest between them. For example the first group wants to learn more, ask more questions, go deeper. The second group sees this as a threat because the additional content may now be on the exam as well. And they often misunderstand this behavior of the first group as just being teacher pleasers/pets.
Or from own experience: when you tell the prof that he accidentally started to cover the same topic that we already covered in the last lecture (because you're there to learn more), you will annoy a lot of people who hoped to get one fewer lectures to study in the course. Less content, less effort, same grade.
You hear the first group having passionate discussions on physics or programming over a beer, while the other group discusses all the latest info of which prof is how demanding, what the grading criteria are, what the deadlines are, which departments you should or should not take courses from, where they have mandatory presence, who does multiple choice tests and who does free-text exams.
I don't have much to conclude, just an observation.
The reality I've learned is your intuition isn't necessarily wrong, it might just need more information. Arisotlian..."physics" isn't wrong neccessarily, it's just a model of motion under certain conditions (everyday conditions on Earth under an atmosphere or/and on friction-full surfaces) Newton's 1st can be thought of a generalization of Arisotle's (or anyone's) intuition...which to make things worse, physics' 101 students are not taught about.
One of the things I think was the best ideas I learned in physics is that physics is all about approximations. Theorists tend to forget this annoyingly, but it helps keep naive scientists from using words like "fundamental" without qualifications, or saying garbage statements like "Newton's law X is wrong."
One of the things I think was the best ideas I learned in physics is that physics is all about approximations. Theorists tend to forget this annoyingly, but it helps keep naive scientists from using words like "fundamental" without qualifications, or saying garbage statements like "Newton's law X is wrong."
This is exactly why making more FCI-style questions is so hard. They shouldn't be sensitive to things like air resistance -- a question whose answer is completely changed by realistic air resistance isn't a very good one.
I covered that with terminal velocity. And it isn't just falling, but going down a hill. Folks commonly think heavier bikes will go faster down a hill. That isn't how that works.
Faster bikes will go downhill faster. Both the acceleration of the bikes and their terminal velocity are determined by aerodynamics (on very steep hills) and rolling friction (on less steep ones), and those do not change much with bike weight.
[deleted]
Heavier things really do fall faster in the sense that they also pull the earth towards them which increases the relative acceleration and thereby decreases the time until they hit each other.
F = G * m1 * m2 / r²
This tells you that the acceleration of mass 1 is independent of the mass 1 and only depends on mass 2 and the acceleration of mass 2 is independent of mass 2 and only depends on mass 1. But the relative velocities will increase when either m1 or m2 is increased.
E.g. a black hole with the mass of the earth will hit the ground of a (perfectly rigid) earth in almost half the time as a feather would.
(I won't go into relativistic effects here.)
F = G * m1 * m2 / r²
This tells you that the acceleration of mass 1 is independent of the mass 1 and only depends on mass 2 and the acceleration of mass 2 is independent of mass 2 and only depends on mass 1. But the relative velocities will increase when either m1 or m2 is increased.
E.g. a black hole with the mass of the earth will hit the ground of a (perfectly rigid) earth in almost half the time as a feather would.
(I won't go into relativistic effects here.)
My argument is predicated on measurable differences of realistic weights of bikes and riders.
That is, would you expect adding ten pounds to have a meaningful difference in how fast a bike will accelerate down a hill? Even fifty pounds?
It will impact brake force needed to stop. But the speeds are going to be near identical.
That is, would you expect adding ten pounds to have a meaningful difference in how fast a bike will accelerate down a hill? Even fifty pounds?
It will impact brake force needed to stop. But the speeds are going to be near identical.
Heavier things do gain kinetic energy faster when falling, it's easy enough to confuse "being hard to stop" with "moving fast." Intuitively, the heavier object has "more" of something but in casual conversation "kinetic energy" isn't a common or familiar term. If you haven't studied enough physics to reach for that concept, I'm not surprised a lot of people just settle on "faster" when attempting to describe it.
Yeah, and we have words for that. Momentum, is a good one.
I should clarify the last time I had this conversation, I'm talking about 40+ year olds building bikes. They were thinking a heavier bike would accelerate faster down a hill. That just is not how that works.
I should clarify the last time I had this conversation, I'm talking about 40+ year olds building bikes. They were thinking a heavier bike would accelerate faster down a hill. That just is not how that works.
It actually is though! The force due to air resistance is the primary thing slowing down the bike and rider, and the force, which doesn't change with mass, slows down a heavier bike less, and so a heavy bike will be rolling faster than a lighter one at the bottom of a hill, all other things being equal.
Right, you can hit a higher velocity. You will not get extra acceleration. That is my entire point.
That is to say, the force from the air does not actively slow you down. It prevents you from speeding up, after a point. (Aerodynamics are, of course, more complicated. But this general model is pretty solid.)
Such that if I want to accelerate faster down the hill, I have to pedal to cause that to happen. Or add a motor.
That is to say, the force from the air does not actively slow you down. It prevents you from speeding up, after a point. (Aerodynamics are, of course, more complicated. But this general model is pretty solid.)
Such that if I want to accelerate faster down the hill, I have to pedal to cause that to happen. Or add a motor.
The heavier bike will reach the terminal velocity of the lighter bike sooner than the lighter bike will. If that's not extra acceleration then I don't think you're looking at the problem from a practical perspective.
Yes, acceleration due to gravity is the same, but air resistance has less effect, at all speeds, on the heavier bike. The rate at which the heavier bike gains speed is higher. It accelerates faster.
Yes, acceleration due to gravity is the same, but air resistance has less effect, at all speeds, on the heavier bike. The rate at which the heavier bike gains speed is higher. It accelerates faster.
This is just disagreement over magnitude. And what it means to hit higher acceleration.
Take my statements to be, it won't hit a meaningful higher acceleration. Not that it won't necessarily accelerate for longer.
In general, you will accelerate down the hill at 9.8 m/s^2 modified by the incline. No matter how heavy the bike is. (Within the realm of realistic weights.).
Yes, the points you are raising are true. But within the realm of the biker and realistic bikes, not really relevant.
Take my statements to be, it won't hit a meaningful higher acceleration. Not that it won't necessarily accelerate for longer.
In general, you will accelerate down the hill at 9.8 m/s^2 modified by the incline. No matter how heavy the bike is. (Within the realm of realistic weights.).
Yes, the points you are raising are true. But within the realm of the biker and realistic bikes, not really relevant.
The velocity of the bike+rider is proportional to their mass. Realistically this can range over two orders of magnitude, which can be particularly relevant in a racing context.
I'm intrigued. What are you saying here?
It sounds like there could be a 100x difference between a light guy on a light bike and a heavy guy on a heavy bike. Sounds hard to believe, so I'm intrigued too :).
Could be 4x if you use mini order of magnitudes, and if you include kids and adult body builders... there’s your 4x!
There is a larger gravitational force on heavier things because they're heavier. Trouble is, they also have more inertia, which exactly counteracts the larger force (because of the principle of equivalence) and so they fall at the same acceleration as lighter objects.
But they also exert more force onto the earth. If you drop the sun onto the earth, for example...
That feels wrong.
You can say that two larger things attract each other more than smaller things. Especially as they get closer to each other. Catch, of course, is all things on the earth are in the noise range compared to, you know, the earth. If you are talking about gravity and acceleration on the earth, you are talking about 9.8 m/s^2. Pretty much period.
And yes, there are terminal velocities that impact maximum speed. Acceleration, though, is not velocity.
You can say that two larger things attract each other more than smaller things. Especially as they get closer to each other. Catch, of course, is all things on the earth are in the noise range compared to, you know, the earth. If you are talking about gravity and acceleration on the earth, you are talking about 9.8 m/s^2. Pretty much period.
And yes, there are terminal velocities that impact maximum speed. Acceleration, though, is not velocity.
It's correct. Terminal velocity is irrelevant here; it's true outside the atmosphere too. I really am talking about force and acceleration.
It's clear that there's more force on a heavier object. How much force do you have to apply to counteract a bucket full of rocks from falling to the ground, vs. an empty bucket? Your arm is certainly applying more force to the former.
So then the question becomes Why doesn't the full bucket fall faster? Because it has more inertia, and thus it's more difficult to accelerate. But there's also more force, which overcomes this difficulty. The amount of greater force is exactly identical to the greater inertia, so the acceleration remains the same. The "exactly identical" here is no coincidence.
https://en.wikipedia.org/wiki/Equivalence_principle
It's clear that there's more force on a heavier object. How much force do you have to apply to counteract a bucket full of rocks from falling to the ground, vs. an empty bucket? Your arm is certainly applying more force to the former.
So then the question becomes Why doesn't the full bucket fall faster? Because it has more inertia, and thus it's more difficult to accelerate. But there's also more force, which overcomes this difficulty. The amount of greater force is exactly identical to the greater inertia, so the acceleration remains the same. The "exactly identical" here is no coincidence.
https://en.wikipedia.org/wiki/Equivalence_principle
Ah, I see what I screwed up. I was sloppy and saying force, but really only talking about acceleration.
So yeah, that makes sense.
Similarly, I only mentioned terminal velocity because I said faster. Which could apply there. But my point was focused on acceleration of falling objects. I get that there are a lot of ways to muddy that discussion. As exemplified by my postings. :).
So yeah, that makes sense.
Similarly, I only mentioned terminal velocity because I said faster. Which could apply there. But my point was focused on acceleration of falling objects. I get that there are a lot of ways to muddy that discussion. As exemplified by my postings. :).
If your model can't be used to predict things in reality, then what good is it? If I throw a feather down from a rooftop and then I let a brick fall down, the brick will hit the earth first. The heavier thing falls "faster".
You can use a bunch of words to explain what's really going on there, but it won't change that the brick hits the earth first, hence it falls faster. Unless you can predict the future correctly and communicate this simply to your peers, what's the point of being correct in the terminology?
You can use a bunch of words to explain what's really going on there, but it won't change that the brick hits the earth first, hence it falls faster. Unless you can predict the future correctly and communicate this simply to your peers, what's the point of being correct in the terminology?
I distinguished between velocity and acceleration.
My point was strictly that many people will falsely think gravity somehow meaningfully pulls lighter objects at a lower rate. This despite days off lessons that gravity on Earth is basically a constant.
That is to say, if your model of speed of things falling is hinged solely on gravity, it will not be predictive in the extremes.
Of course, most folks aren't playing in the extremes, so the simple model will be fairly predictive of everything they do actually toss around the room. Can easily explain why the stack of books fell at the same speed as the basketball when the dresser was knocked over. :)
My point was strictly that many people will falsely think gravity somehow meaningfully pulls lighter objects at a lower rate. This despite days off lessons that gravity on Earth is basically a constant.
That is to say, if your model of speed of things falling is hinged solely on gravity, it will not be predictive in the extremes.
Of course, most folks aren't playing in the extremes, so the simple model will be fairly predictive of everything they do actually toss around the room. Can easily explain why the stack of books fell at the same speed as the basketball when the dresser was knocked over. :)
Doesn't explain why the paper plane glides, though. I was always frustrated in physics class. There just wasn't enough precision to describe reality and everytime a test said "ignore friction" I would mentally shout out "but there is friggin' friction". Math made a lot more sense and it was only with advanced math knowledge that I would finally be able to make sense of some of the entry physics stuff.
If only they had told me everything (including friction), then maybe phyics would have made a lot more sense to me. But then again, school math never even mentioned imaginary numbers. School is very incomplete.
If only they had told me everything (including friction), then maybe phyics would have made a lot more sense to me. But then again, school math never even mentioned imaginary numbers. School is very incomplete.
And yet friction alone wouldn't do it, either. Consider balloons.
Same goes for everything underwater. Which follows from viewing air as a medium you are traveling through.
Are there bad teachers that need to more honestly cover some of the assumptions? Yes. They are a useful tool, though.
Same goes for everything underwater. Which follows from viewing air as a medium you are traveling through.
Are there bad teachers that need to more honestly cover some of the assumptions? Yes. They are a useful tool, though.
Because science aims to discover what is really going on under the hood. And that matters. By breaking the problem into vacuum and non vacuum you get a better understanding.
One of the best physics tests that I took was one where each question has a sense of progression. So the first part laid out a fundamental law of physics and the question was to prove something using it. Then using the result from that the problem was tweaked in a few ways for a few more questions until you were far removed from the original concept but could still explain why and how something worked.
To take your gravity example, the first question might be something about calculating gravitational force on the ISS. Then explain that it's rotating in space and therefore has centripetal force that accounts for this gravity. Then compute how fast it must be rotating. And from there you can switch to a completely different topic in the same context. Maybe something about EM radiation from the Sun that falls on the space station.
To take your gravity example, the first question might be something about calculating gravitational force on the ISS. Then explain that it's rotating in space and therefore has centripetal force that accounts for this gravity. Then compute how fast it must be rotating. And from there you can switch to a completely different topic in the same context. Maybe something about EM radiation from the Sun that falls on the space station.
> If the people aren't motivated, any standardized measure will miss the mark
Shouldn't that be a sign that trying to come up with standardized measures is a fool's errand?
Short of blatant social engineering, I don't see how we can have a system that could improve people's motivation and drive for education. And if we don't have a way to actually change human nature, why do we keep trying to design this system that would magically work for so many different people?
Shouldn't that be a sign that trying to come up with standardized measures is a fool's errand?
Short of blatant social engineering, I don't see how we can have a system that could improve people's motivation and drive for education. And if we don't have a way to actually change human nature, why do we keep trying to design this system that would magically work for so many different people?
It's a trust issue. It comes from grades being used to evaluate people outside the particular schools they attend.
We could give more autonomy to individual schools or teachers in terms of designing tests, but then we'd be facing two problems. One, the grades given by different schools/teachers would not be directly comparable. An A in one school could be awarded only for exceptional circumstances, and in another just for above-local-average. Two, once you start using those grades to judge people in a competitive situation like the job market, you'll have schools selling grades, teachers inflating grades to help their pupils compete against those from schools that sell grades, etc. and the whole system will collapse to uselessness.
Standardized testing mitigates these problems by forcing everyone to grade by the same criteria, but in order to do that, the grading must be done on things that are extremely easy to measure - which removes both individual judgement of the teachers and any relation between testing and understanding of the material.
The more I think about it, the more I feel it all boils down to scale. The grading system needs to be scaled up with the size of the market people compete on, otherwise it'll be either trivially gamed, or deeply unfair to unlucky people.
We could give more autonomy to individual schools or teachers in terms of designing tests, but then we'd be facing two problems. One, the grades given by different schools/teachers would not be directly comparable. An A in one school could be awarded only for exceptional circumstances, and in another just for above-local-average. Two, once you start using those grades to judge people in a competitive situation like the job market, you'll have schools selling grades, teachers inflating grades to help their pupils compete against those from schools that sell grades, etc. and the whole system will collapse to uselessness.
Standardized testing mitigates these problems by forcing everyone to grade by the same criteria, but in order to do that, the grading must be done on things that are extremely easy to measure - which removes both individual judgement of the teachers and any relation between testing and understanding of the material.
The more I think about it, the more I feel it all boils down to scale. The grading system needs to be scaled up with the size of the market people compete on, otherwise it'll be either trivially gamed, or deeply unfair to unlucky people.
The Big Four accounting firms pay huge attention to college grades and it works well for them. Ditto law firms. And in those fields, the mechanical pursuit of grades -- independent of actual knowledge -- models the actual demands of the job pretty well.
Accounting and law firms are looking for people with the stamina and single-minded goal focus to get document-creation tasks done, one after another, to a high degree of "skill." Getting an A on the test and preparing an SEC-compliant bond prospectus are close cousins of each other.
In fields where real-world mastery doesn't have much to do with the orderly routines of getting a 97 on the midterm, employers are less likely to hunt for people who got 97s on the midterm. (John Steinbeck stumbled around Stanford for six years and never graduated. But he got more accomplished as a writer than a whole decade worth of creative-writing majors with high GPAs.)
Accounting and law firms are looking for people with the stamina and single-minded goal focus to get document-creation tasks done, one after another, to a high degree of "skill." Getting an A on the test and preparing an SEC-compliant bond prospectus are close cousins of each other.
In fields where real-world mastery doesn't have much to do with the orderly routines of getting a 97 on the midterm, employers are less likely to hunt for people who got 97s on the midterm. (John Steinbeck stumbled around Stanford for six years and never graduated. But he got more accomplished as a writer than a whole decade worth of creative-writing majors with high GPAs.)
> It comes from grades being used to evaluate people outside the particular schools they attend.
Are they really, or is just this elaborate illusion supported by most decision makers to give some rationalization to their own selection processes? When you have Harvard being sued for discriminating against Asians, how can we keep a straight face and tell the kids that they need to study hard because the grades matter?
I think this is part of what PG wants to defend in the essay: why do we actually care about grades at all?
Are they really, or is just this elaborate illusion supported by most decision makers to give some rationalization to their own selection processes? When you have Harvard being sued for discriminating against Asians, how can we keep a straight face and tell the kids that they need to study hard because the grades matter?
I think this is part of what PG wants to defend in the essay: why do we actually care about grades at all?
> why do we actually care about grades at all?
Because jobs are at a premium and evaluating prospective hires is hard, so employers look for anything they could use to filter and sort the stream of candidates. Education - both the what and the where is somewhat correlated, so that's what's being used.
Because jobs are at a premium and evaluating prospective hires is hard, so employers look for anything they could use to filter and sort the stream of candidates. Education - both the what and the where is somewhat correlated, so that's what's being used.
The "what" and the "where" do not require the "how many points" to work as a filter, do they?
I think I spent too much on this discussion already, but the point I am trying to make is that standard tests are bad both as a filter for selection and as predictor of performance.
While not necessarily agreeing, I do understand if someone says "I will hire someone good from a top-tier school vs a top student from a not-so-good university". The "what" and "where" are definitely useful data points to filter out candidates.
What I don't understand is any employer that says they will hire based on SAT, GPA or any variation of an IQ test. It seems to be so unreliable measures to the point of actually hindering the process.
I think I spent too much on this discussion already, but the point I am trying to make is that standard tests are bad both as a filter for selection and as predictor of performance.
While not necessarily agreeing, I do understand if someone says "I will hire someone good from a top-tier school vs a top student from a not-so-good university". The "what" and "where" are definitely useful data points to filter out candidates.
What I don't understand is any employer that says they will hire based on SAT, GPA or any variation of an IQ test. It seems to be so unreliable measures to the point of actually hindering the process.
We have to try, because the alternative to standardized testing isn't some egalitarian utopia, it's nepotism.
Yeah. I love HN and many of the thoughts expressed on it, but if there's one thing that always jumps out at me as being an incredibly widely held cognitive bias in the tech community, it's a tendency to view certain things in a very black and white, binary fashion, and to treat certain very complicated, messy problems with lots of social nuances as if they were simple problems, if only everyone could be rational about it. Grades and hiring are obvious examples, but you can also see plenty of it in discussions about open source business models and codes of conduct and copyright and so on.
Throughout my life and career I've seen so many statements of this nature from my peers: "the status quo will only ever be a 90% solution at best, therefore we should throw all of that away and just do everything on a case-by-case basis in isolation from one another, then we'd all live in an ideal meritocracy (or something)".
See also any discussion about flat management. "Explicit management hierarchies obviously aren't perfect, therefore let's get rid of them altogether and assume that everyone is awesome and will magically self-organize and do the right thing."
I guess the common theme is, someone is greatly overestimating their own rationality, and projects it onto other people, and ends up with an almost religious faith in the self-organizing competencies of unmanaged people, or the natural inclination of unregulated schools or industries to always do the right thing even when it's not in their own self-interest.
IME there are lots of fundamentally messy or imperfect things in life, where any choice you make has negative consequences. There's also a personality type that is repelled by this sort of messiness and will always be drawn towards throwing everything away and starting from scratch, often with a childlike belief that things they don't understand aren't important, and why can't everyone see how obvious and elegant their solution is? IME they're usually wrong about those situations, but I guess I'm still glad people less pessimistic than I am are tilting at those windmills.
Throughout my life and career I've seen so many statements of this nature from my peers: "the status quo will only ever be a 90% solution at best, therefore we should throw all of that away and just do everything on a case-by-case basis in isolation from one another, then we'd all live in an ideal meritocracy (or something)".
See also any discussion about flat management. "Explicit management hierarchies obviously aren't perfect, therefore let's get rid of them altogether and assume that everyone is awesome and will magically self-organize and do the right thing."
I guess the common theme is, someone is greatly overestimating their own rationality, and projects it onto other people, and ends up with an almost religious faith in the self-organizing competencies of unmanaged people, or the natural inclination of unregulated schools or industries to always do the right thing even when it's not in their own self-interest.
IME there are lots of fundamentally messy or imperfect things in life, where any choice you make has negative consequences. There's also a personality type that is repelled by this sort of messiness and will always be drawn towards throwing everything away and starting from scratch, often with a childlike belief that things they don't understand aren't important, and why can't everyone see how obvious and elegant their solution is? IME they're usually wrong about those situations, but I guess I'm still glad people less pessimistic than I am are tilting at those windmills.
This is an excellent observation.
The problem isn't hyperrationality per se; it's a lack of intellectual humility and self-awareness. The behavior you're talking about arises when someone looks at a problem, immediately wants to fix it and makes snap judgements to figure out how to do so. Instead, they should dispassionately consider 1) whether the problem is fixable in any meaningful sense, and 2) whether they have the requisite experience, skill and insight to solve it in five minutes of thought.
You could further distill this idea with the observation that a lot of people consider things too systematically, rather than holistically. "First principles" thinking is a powerful drug, which appears to crack open the world and solve every problem. But no problem-solving paradigm can deconstruct away unknown unknowns or inexperience.
The problem isn't hyperrationality per se; it's a lack of intellectual humility and self-awareness. The behavior you're talking about arises when someone looks at a problem, immediately wants to fix it and makes snap judgements to figure out how to do so. Instead, they should dispassionately consider 1) whether the problem is fixable in any meaningful sense, and 2) whether they have the requisite experience, skill and insight to solve it in five minutes of thought.
You could further distill this idea with the observation that a lot of people consider things too systematically, rather than holistically. "First principles" thinking is a powerful drug, which appears to crack open the world and solve every problem. But no problem-solving paradigm can deconstruct away unknown unknowns or inexperience.
Exercising power over other people is a serious thing. It's extraordinarily presumptuous to believe you have designed a rule that will do more good than harm. I'm much more worried about the cognitive bias that says that because someone is currently the decision-maker, they must be competent at it.
Sorry, it seems that you are jumping into conclusions that were never warranted. Chesterton's fence is a principle that I learned a while ago and I am not defending that we need to throw everything away.
What I am saying is that what you regard as a "90% solution at best" is actually not a solution at all. Standardized tests are terrible predictors of performance on actual jobs. If standardized tests (or IQ tests or anything that is supposed to be a proxy for measuring performance) shows time and again that it only manages to measure people that are good to "study for the test", then why do we keep trying to use them to measure performance? Is it because it "the only thing we have"? This seems like saying "I am lost in Rome but all I have is a map of Paris, let me use that anyway."
And no, I am not proposing some other solution to replace it.
What I am saying is that what you regard as a "90% solution at best" is actually not a solution at all. Standardized tests are terrible predictors of performance on actual jobs. If standardized tests (or IQ tests or anything that is supposed to be a proxy for measuring performance) shows time and again that it only manages to measure people that are good to "study for the test", then why do we keep trying to use them to measure performance? Is it because it "the only thing we have"? This seems like saying "I am lost in Rome but all I have is a map of Paris, let me use that anyway."
And no, I am not proposing some other solution to replace it.
What about that comment made you believe it was referring to something you said specifically?
Wait, what?! Why?!
I am not asking for any utopia. What I am asking is "Instead of setting ourselves for failure every time coming up with some scheme for testing performance, why can't we just use actual performance on the activity that we want to evaluate candidates"?
It works for people in sports. It works for trade professions. It works for traders. It works for performers, entertainers, even entrepreneurs.
Why can't we have an educational system that does not put people on tracks to mold them and instead just coach them to follow whatever path they seem fit?
I am not asking for any utopia. What I am asking is "Instead of setting ourselves for failure every time coming up with some scheme for testing performance, why can't we just use actual performance on the activity that we want to evaluate candidates"?
It works for people in sports. It works for trade professions. It works for traders. It works for performers, entertainers, even entrepreneurs.
Why can't we have an educational system that does not put people on tracks to mold them and instead just coach them to follow whatever path they seem fit?
Because proper evaluation takes time and a bad choice carries a huge cost. In the absence of any objective measures, an employer gets around this by just hiring the son of the guy they golf with. Connections.
Sports works, but only because sports are extremely popular, with tens of millions of people enthusiastically participating on a regular basis, and accessible to anybody no matter their age and income. This is not true for, say, working at a law firm. You can’t take the best clerk from a pool of a thousand kids who have been clerking since elementary school. It’s either who you know, or who got good grades and test scores.
Sports works, but only because sports are extremely popular, with tens of millions of people enthusiastically participating on a regular basis, and accessible to anybody no matter their age and income. This is not true for, say, working at a law firm. You can’t take the best clerk from a pool of a thousand kids who have been clerking since elementary school. It’s either who you know, or who got good grades and test scores.
> In the absence of any objective measures
One would argue that this is the current status.
How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
How my grades in Computer Engineering can be used to predict my performance as an Engineer in a country where 85% of the graduates would work in non-technical and more profitable fields?
If a company looking for a Quant gets someone with a Master in Math and a good tracking record knocking on their door, would they stop and ask to see if they can get someone with a PhD instead?
You are right that "proper evaluation takes time". What I am saying is that standardized tests should not count as proper evaluation.
One would argue that this is the current status.
How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
How my grades in Computer Engineering can be used to predict my performance as an Engineer in a country where 85% of the graduates would work in non-technical and more profitable fields?
If a company looking for a Quant gets someone with a Master in Math and a good tracking record knocking on their door, would they stop and ask to see if they can get someone with a PhD instead?
You are right that "proper evaluation takes time". What I am saying is that standardized tests should not count as proper evaluation.
Again, the alternative to standardized tests and credentials is hiring based on connections, ethnicity, pedigree, physical appearance, and superficial confidence. These problems are exactly why meritocracy was invented in the first place. The alternative is not everybody getting the jobs they want, it's you being turned away without any consideration at all because the boss already hired their cousin.
> How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
You can't take two proper WAIS-IV tests in two hours. In fact, unless you have been part of an unusual psychology study, you've probably never taken a real IQ test in your life. There are a lot of gimmicky free things on the internet that call themselves IQ tests but aren't predictive of anything.
> How objective are, e.g, IQ tests when I can take two different tests in a span of two hours and get from 128 to 146?
You can't take two proper WAIS-IV tests in two hours. In fact, unless you have been part of an unusual psychology study, you've probably never taken a real IQ test in your life. There are a lot of gimmicky free things on the internet that call themselves IQ tests but aren't predictive of anything.
You are moving the goal posts. I was talking only about standardized tests, and now you are also including credentials on the mix.
As for the IQ test - it wasn't online and it wasn't a study, as far as I know - but I can totally concede that it was not a "real" IQ test.
Anyway, the story happened over 20 years ago. Part of the driving exam in Brazil included a "neuro-psychological evaluation" and the doctor flagged me for some extra tests. From what I remember I went to the clinic the next week, did one test, did the "neuro-technical" exam and was told to wait for an IQ test. When I got the form, it was the same as I got before originally (different order of questions) Long story short I asked why I was re-taking the test and I thought I "failed" somehow. They actually said they were not aware of the first one and later on found it.
Now that I think about it, perhaps it was some weird psychology test? :)
As for the IQ test - it wasn't online and it wasn't a study, as far as I know - but I can totally concede that it was not a "real" IQ test.
Anyway, the story happened over 20 years ago. Part of the driving exam in Brazil included a "neuro-psychological evaluation" and the doctor flagged me for some extra tests. From what I remember I went to the clinic the next week, did one test, did the "neuro-technical" exam and was told to wait for an IQ test. When I got the form, it was the same as I got before originally (different order of questions) Long story short I asked why I was re-taking the test and I thought I "failed" somehow. They actually said they were not aware of the first one and later on found it.
Now that I think about it, perhaps it was some weird psychology test? :)
It's pretty straightforward to motivate people. You just have to make that what you teach them has value that they can use immediately. For example, to motivate someone to train in self-defence, just beat them up every week. Now learning self-defence will have value in their life. If they don't get beat up weekly, what's the point of learning self-defence? If it's only once a year, then who cares. Spending time every day to counter something that only happens once a year isn't worth it. It's less trouble to get beat up once a year than it is to spend learning self-defence daily.
There just has to be an immediate value in what you teach. If there isn't, there's no point in teaching it. Students will just forget it anyway after the test.
There just has to be an immediate value in what you teach. If there isn't, there's no point in teaching it. Students will just forget it anyway after the test.
I do work in a related field and give your answer really sad. The answer is simply that standardised tests are not a suitable measure of learning. They miss most of the important insights and even efforts to open up, like the attempt to measure 'problem solving'in PISA, are damned to stick to very narrow concepts.
Standardized testing is good for:
- measuring system performance against set standards - giving basic insights into student's average level of knowledge of a subject (if you use a thorough theoretical framework to conceptualise learning outcomes and link test items to this) - maybe measuring school performance against standards - maybe measuring teacher performance against standards
But for students this individual measure of an exam is simply not a good measure of learning in any modern education system. Yes it is a useful measure in really bad systems where teachers are bad and teach wrong things, or where eg attendance and exam fraud are huge issues.
But modern pedagogy is not about teaching book knowledge which is easily measured. In any Western education system we are mostly past that point. It's about helping students discover own interests and abilities, develop competence and confidence, experiment, fail and try again. This learning is simply too individual and hard to measure for a standardised test.
The only option is formative assessment (essentially teachers grading students repeatedly throughout the year on things like effort, achievement of general learning aims, additional development, results produced, presentations, support to other students, etc - a bit how you would measure a good office worker). Or maybe let's do like Finland and Estonia and let's mostly eliminate the testing burden.
Standardized testing is good for:
- measuring system performance against set standards - giving basic insights into student's average level of knowledge of a subject (if you use a thorough theoretical framework to conceptualise learning outcomes and link test items to this) - maybe measuring school performance against standards - maybe measuring teacher performance against standards
But for students this individual measure of an exam is simply not a good measure of learning in any modern education system. Yes it is a useful measure in really bad systems where teachers are bad and teach wrong things, or where eg attendance and exam fraud are huge issues.
But modern pedagogy is not about teaching book knowledge which is easily measured. In any Western education system we are mostly past that point. It's about helping students discover own interests and abilities, develop competence and confidence, experiment, fail and try again. This learning is simply too individual and hard to measure for a standardised test.
The only option is formative assessment (essentially teachers grading students repeatedly throughout the year on things like effort, achievement of general learning aims, additional development, results produced, presentations, support to other students, etc - a bit how you would measure a good office worker). Or maybe let's do like Finland and Estonia and let's mostly eliminate the testing burden.
Ever look at Keller Plan stuff?
I did Physics 2 that way and really liked it.
https://en.wikipedia.org/wiki/Keller_Plan
(also, wow, I just got that Principle Skinner in the Simpsons doesn't accidentally have that name. Face to palm.)
I did Physics 2 that way and really liked it.
https://en.wikipedia.org/wiki/Keller_Plan
(also, wow, I just got that Principle Skinner in the Simpsons doesn't accidentally have that name. Face to palm.)
You may be interested in Simon Singh's The Simpsons and their Mathematical Secrets which details the show's long tradition of hiring math geeks as writers and the plethora of math-based in-jokes that has resulted.
The problem is that so much of physics is quite counter-intuitive (that is one reason it took so long to discover).
The basic principle of education is to start where the student is and then lead them step-by-step to what you want them to understand. That being the case, I would say physics should start with student's intuitive understanding and then explain, like through examples they can understand, why some of their intuitions are mistaken. Maybe this could even be turned into Socratic questioning.
The basic principle of education is to start where the student is and then lead them step-by-step to what you want them to understand. That being the case, I would say physics should start with student's intuitive understanding and then explain, like through examples they can understand, why some of their intuitions are mistaken. Maybe this could even be turned into Socratic questioning.
Are there any resources good for finding out more about this and such studies? Myself, I went through this process in physics, and in many cases it was years afterwards that I properly understood the implications and context of the early courses, and only through repeat exposure could grok the actual connections in the problems. It seems especially hard to test and evaluate something that is planting the seeds for a more complete understanding, maybe years down the line.
I liked Randall Knight's Five Easy Lessons, which summarizes findings in the field and also comes with sample exercises and activities for the whole intro curriculum. There's also Mazur's Peer Instruction, which is a little older.
To illustrate just how hard: much of my work in academia was assisting in designing experiments or analyzing data for other people working on PhDs in "how to design decent tests" (i.e., Instructional Psychology). There was an entire department of the university dedicated to studying just that problem.
I'm glad he made the leap from school tests to funding tests, as it seemed to be a thinly veiled analogy from the start. I'm disappointed, though, that he didn't take the final step and admit that money itself is also a poor, hackable test.
Funding, growth, usage -- these are all still one level removed from something worthwhile or beneficial. Cigarettes have amazing usage numbers even in 2019.
I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing.
Funding, growth, usage -- these are all still one level removed from something worthwhile or beneficial. Cigarettes have amazing usage numbers even in 2019.
I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing.
> I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing.
This is literally all of open source software. The problem here is that you're waiting for an entrepreneur to say this, which is the wrong kind of person to expect this from.
This is literally all of open source software. The problem here is that you're waiting for an entrepreneur to say this, which is the wrong kind of person to expect this from.
Possibly. But for a little while, I did hear about "lifestyle businesses", i.e., the idea that you could be financially sustainable in tech without needing massive growth.
And these days, I think most of the open source software I use was actually contributed by a corporation.
And these days, I think most of the open source software I use was actually contributed by a corporation.
Be careful of believing what you hear. There are still plenty of lifestyle businesses, they just don't seek out or receive much publicity.
The media buzz is dominated by mostly-not-very-good VCs and mostly-not-very-good startups trying to find and impress each other. The VC market is designed to seek out the "unicorn" companies whose value will grow 1000x, and so the companies that might be that and try to look like that get the most buzz. There's nothing wrong with starting or funding that kind of company, but it's not for everyone, and it certainly isn't all that's out there.
It's still viable to start a smallish lifestyle business that produces enough money for yourself and doesn't take a ton of work to keep going, but will never grow to be 1000x the size. It's just fine if that's what you want to do too, but know that such businesses usually don't market too much, either for customers or investors. They don't want VCs, and VCs don't want them either.
The media buzz is dominated by mostly-not-very-good VCs and mostly-not-very-good startups trying to find and impress each other. The VC market is designed to seek out the "unicorn" companies whose value will grow 1000x, and so the companies that might be that and try to look like that get the most buzz. There's nothing wrong with starting or funding that kind of company, but it's not for everyone, and it certainly isn't all that's out there.
It's still viable to start a smallish lifestyle business that produces enough money for yourself and doesn't take a ton of work to keep going, but will never grow to be 1000x the size. It's just fine if that's what you want to do too, but know that such businesses usually don't market too much, either for customers or investors. They don't want VCs, and VCs don't want them either.
> The problem here is that you're waiting for an entrepreneur to say this, which is the wrong kind of person to expect this from.
I agree and can’t help but wonder if this is why we’re seeing more and more developers trying to make money from their Open Source projects.
I agree and can’t help but wonder if this is why we’re seeing more and more developers trying to make money from their Open Source projects.
Is it not because rent keeps going up while "entrepreneurs" use their work with minimal attribution and certainly no financial contribution to get rich? The same "entrepreneurs" who then act entitled to free support from the same developer struggling to pay the rent?
I think they're really just trying to solve the problem of "how do I afford to eat while still supporting software used by Facebook to make billions of dollars"?
I think they're really just trying to solve the problem of "how do I afford to eat while still supporting software used by Facebook to make billions of dollars"?
I think this is an excellent point. I really agree with pg's thesis, but I think it would be worthwhile for him to go more in depth into the "edge cases" he talks about in passing, mainly because I think these edge cases actually bolster his thesis, not detract from it.
Companies like Theranos and WeWork succeeded for a short while because they tried to "hack the test". E.g. Theranos hacked the metrics of "buzz", "investment amount", "notable people on your board", but they didn't actually have a working product. Adam Neumann certainly "hacked the test" to the tune of a billion plus dollars, but WeWork certainly didn't succeed (I'd note there is still a critical difference between Theranos and WeWork in that customers actually really like the product WeWork provides, the question is just can they provide it profitably).
I think hacking the test can get you so far in the startup world, but eventually reality rears its head and what matters is whether you can provide a product people want, profitability.
One quick note on your cigarettes example. The problem here is that companies are giving people what they want, they've just figured out how to "hack the test" so to speak with respect to what the human body wants. This is basically the case with all addictive substances - there is essentially a hackable part of the human "desire" subsystem that people have taken advantage of.
Companies like Theranos and WeWork succeeded for a short while because they tried to "hack the test". E.g. Theranos hacked the metrics of "buzz", "investment amount", "notable people on your board", but they didn't actually have a working product. Adam Neumann certainly "hacked the test" to the tune of a billion plus dollars, but WeWork certainly didn't succeed (I'd note there is still a critical difference between Theranos and WeWork in that customers actually really like the product WeWork provides, the question is just can they provide it profitably).
I think hacking the test can get you so far in the startup world, but eventually reality rears its head and what matters is whether you can provide a product people want, profitability.
One quick note on your cigarettes example. The problem here is that companies are giving people what they want, they've just figured out how to "hack the test" so to speak with respect to what the human body wants. This is basically the case with all addictive substances - there is essentially a hackable part of the human "desire" subsystem that people have taken advantage of.
Theranos issue was not lack of product. Their problem was massive fraud going effectively demanded from top levels.
Framing it as lack of product is unfair to all startups who lacked good product, hacked buzz, got investment, got notable people and then did not committed fraud.
Plus it suggests that Theranos fraud is result of not having product. The same people would be engaging in unethical conduct with product too.
Framing it as lack of product is unfair to all startups who lacked good product, hacked buzz, got investment, got notable people and then did not committed fraud.
Plus it suggests that Theranos fraud is result of not having product. The same people would be engaging in unethical conduct with product too.
Disagree. "Not having product", when you say you DO have product, and are very specific about the capabilities of said non-existent product, is the actual definition of fraud.
It is possible to not have product and be vague and get investment and keep going before crashing.
Plus, Theranos fraud was waaay more bigger then that.
Plus, Theranos fraud was waaay more bigger then that.
What's about homeopathy? It is advertised to have medical benefits, presumably has none, and is a multi-billion business, with many companies which exist for decades and decades.
For any non-trivial product there's information asymmetry problem, so pretty much everything is hackable, but to different extents.
For any non-trivial product there's information asymmetry problem, so pretty much everything is hackable, but to different extents.
You are implying that PG thinks money is all important - but he doesn’t think that.
PG: if you had to boil it down to one quality to look for, authenticity would be the most important one. “You’re looking for people who are real friends,” he said to Chang. “Not just for people who got together for the purposes of this startup. You don’t want people who were in it just for the money.
He also talks about how that startup founders that are more interested in the product or the challenge of building a business, make buttloads of cash as a side-effect.
And YC funds nonprofit startups too?
Also from another 2004 PG essay:
“””Money Is Not Wealth
If you want to create wealth, it will help to understand what it is. Wealth is not the same thing as money. [3] Wealth is as old as human history. Far older, in fact; ants have wealth. Money is a comparatively recent invention.
Wealth is the fundamental thing. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that could on command make you a car or cook you dinner or do your laundry, or do anything else you wanted, you wouldn't need money. Whereas if you were in the middle of Antarctica, where there is nothing to buy, it wouldn't matter how much money you had.
Wealth is what you want, not money.”””
PG: if you had to boil it down to one quality to look for, authenticity would be the most important one. “You’re looking for people who are real friends,” he said to Chang. “Not just for people who got together for the purposes of this startup. You don’t want people who were in it just for the money.
He also talks about how that startup founders that are more interested in the product or the challenge of building a business, make buttloads of cash as a side-effect.
And YC funds nonprofit startups too?
Also from another 2004 PG essay:
“””Money Is Not Wealth
If you want to create wealth, it will help to understand what it is. Wealth is not the same thing as money. [3] Wealth is as old as human history. Far older, in fact; ants have wealth. Money is a comparatively recent invention.
Wealth is the fundamental thing. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that could on command make you a car or cook you dinner or do your laundry, or do anything else you wanted, you wouldn't need money. Whereas if you were in the middle of Antarctica, where there is nothing to buy, it wouldn't matter how much money you had.
Wealth is what you want, not money.”””
I didn't mean to imply that he thinks that. Sorry for the apparent ambiguity.
"Wealth" is still problematic, as per the cigarette example.
"Wealth" is still problematic, as per the cigarette example.
This.
I work towards wanting less stuff. Living a simple life, I do feel richer for sure. Wealthier, perhaps not, I don't even care actually.
I work towards wanting less stuff. Living a simple life, I do feel richer for sure. Wealthier, perhaps not, I don't even care actually.
> I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing.
Closest to this today might be Elon Musk, maybe...but there are other complexities there?
Closest to this today might be Elon Musk, maybe...but there are other complexities there?
> I'm waiting for the entrepreneur who will say he doesn't care how much money he makes or how many customers he has, only that it's worthwhile and needed doing.
I believe this is the role of the nonprofit sector.
I believe this is the role of the nonprofit sector.
Not necessarily.
I think it's PG who mentioned it in some other essays, but "non-profit" and "company" can be seen as different ways to make things happen, a different models of getting society's help with bringing your visions into the real world. Non-profits tend to depend on other people's generosity; for-profits have to earn their money directly. Both have to care about money to some extent, as money is the lifeblood of all large-scale endeavors in this world.
All of that is orthogonal to doing something worthwhile. There's plenty of non-profits that exist primarily to make money for the founders. Conversely, there's e.g. Elon Musk and SpaceX, which is a successful for-profit with an explicit, unchanging, non-monetary terminal goal: making humanity multiplanetary, by bringing down the costs of space so that it's economical to venture there and colonize Mars (you can argue whether that's worthwhile, but I think it's well proven that Musk/SpaceX are honestly pursuing space, not money).
I think it's PG who mentioned it in some other essays, but "non-profit" and "company" can be seen as different ways to make things happen, a different models of getting society's help with bringing your visions into the real world. Non-profits tend to depend on other people's generosity; for-profits have to earn their money directly. Both have to care about money to some extent, as money is the lifeblood of all large-scale endeavors in this world.
All of that is orthogonal to doing something worthwhile. There's plenty of non-profits that exist primarily to make money for the founders. Conversely, there's e.g. Elon Musk and SpaceX, which is a successful for-profit with an explicit, unchanging, non-monetary terminal goal: making humanity multiplanetary, by bringing down the costs of space so that it's economical to venture there and colonize Mars (you can argue whether that's worthwhile, but I think it's well proven that Musk/SpaceX are honestly pursuing space, not money).
Every free (as in speech) program is written by somebody that said that. But they are generally not entrepreneurs.
The entrepreneur is by definition somebody that cares about money.
The entrepreneur is by definition somebody that cares about money.
An entrepreneur to my mind (en-gb native) is "a person who takes on the risk and reward of a new business venture, particularly a novel business (in the geographically area they are in)".
What matters is not whether one starts a company, but why. A company is a way to get money and manpower to help one realize their goals. The important question is, what are their goals? Making money? Or building a product? Unfortunately, most companies are about the former (despite what their marketing copy says).
Pretty hard for companies to not skew very heavily towards being about making money when people require money for food, shelter, and healthcare (at least in the states).
Unless the people involved are in a financial state where they are not dependent on revenue from the company, or we divorce survival from capital, I doubt it's really possible for "not making money" to disappear as a terrifying core motivator for every business and every employed person regardless of whether their marketing copies are disingenuous.
Unless the people involved are in a financial state where they are not dependent on revenue from the company, or we divorce survival from capital, I doubt it's really possible for "not making money" to disappear as a terrifying core motivator for every business and every employed person regardless of whether their marketing copies are disingenuous.
I'm not saying companies should not make money. I'm talking about instrumental vs. terminal values. Money can and should be viewed as means to an end, as instrumental for achieve the company's purpose - be it feeding people, making good tools or getting them to space. But very often it becomes terminal - the company's purpose becomes making money, and what they do becomes means to an end. The entire structure becomes reversed.
Yes, what I'm arguing is that it's very difficult for it to not become terminal because people die without income.
That's the root of the issue, not motivation or purpose or what the individuals value or the intentions behind the creation of the company.
That's the root of the issue, not motivation or purpose or what the individuals value or the intentions behind the creation of the company.
Like Wozniak? Twitter launched without a business plan, Jack was on the Colbert Report in the 2000s saying he'll figure out how to monetize it eventually. I really question how much Uber cares about money recently...
An entrepreneur is somebody looking to start a company in my mind. I didn’t look up a definition, though.
Regardless, Jack knew he needed to make money eventually is what that shows. Uber may be foolish in its attempts to make money, but it absolutely is trying to in the long term.
There are other people that intend to swindle people (make money for themselves, not the company).
There are other reasons to start a company, but then you have an expensive hobby. If all you ever do is blow money you didn’t earn (your inheritance) starting businesses that fail, at best you are going to be called a failed entrepreneur.
Regardless, Jack knew he needed to make money eventually is what that shows. Uber may be foolish in its attempts to make money, but it absolutely is trying to in the long term.
There are other people that intend to swindle people (make money for themselves, not the company).
There are other reasons to start a company, but then you have an expensive hobby. If all you ever do is blow money you didn’t earn (your inheritance) starting businesses that fail, at best you are going to be called a failed entrepreneur.
Or "Mr. President"
The real final step is admitting that the "worthwhile and needs doing" judgment in your head is also a test, which isn't always more reliable than the tests you face in university or the test of money.
[deleted]
Perhaps, Purism (Social Purpose Corporation) could fit: https://www.hostingadvice.com/blog/purism-respects-user-righ...
I agree.
Also, most small businesses do stuff people want. But it's mostly commodities. So the hard part in many small businesses is marketing.
And marketing is mostly hacking.
As for startups,marketing is very important for a startup.
But is it common to sucseed without something people want, or at least get addicted to ?
Also, most small businesses do stuff people want. But it's mostly commodities. So the hard part in many small businesses is marketing.
And marketing is mostly hacking.
As for startups,marketing is very important for a startup.
But is it common to sucseed without something people want, or at least get addicted to ?
Those metrics reveal something intrinsic though. Take cigarettes.
They're bad, not liked, etc etc. The usage in 2019 reflects something deeper: their effectiveness, perhaps a cohort effect and the difficulty of quitting, and a certain value proposition.
There's a term I'm perhaps misusing which is: revealed preferences.
This statement is perhaps a stretch but maybe what paul meant is that learning history for history's sake is helpful but learning potentially untrue test patterns (aka launching on a tuesday to try to maximize funding) is not.
This might not get a lot of traction but it seems your last sentence describes the Ayn Rand novel where the guy spends years toiling in anonymity versus his peer who is "fake" and successful, where the value is in the work itself.
They're bad, not liked, etc etc. The usage in 2019 reflects something deeper: their effectiveness, perhaps a cohort effect and the difficulty of quitting, and a certain value proposition.
There's a term I'm perhaps misusing which is: revealed preferences.
This statement is perhaps a stretch but maybe what paul meant is that learning history for history's sake is helpful but learning potentially untrue test patterns (aka launching on a tuesday to try to maximize funding) is not.
This might not get a lot of traction but it seems your last sentence describes the Ayn Rand novel where the guy spends years toiling in anonymity versus his peer who is "fake" and successful, where the value is in the work itself.
How about DHH? He is the closest to this I know.
I got into "computer science" before there was a CS degree. Our CS classes were taught in the math dept (fortran), business dept (cobol), or engineering (assembler).
The profs were literally 1 chapter ahead of the students.
I, however, was in love with the subject. I was the "student advisor" in our "machine room". We had 5 teletypes connected to a remote mainframe at Rutgers. I lived and breathed CS.
I wrote tests for the profs (even thought I was also in the class) and I answered questions for students about the tests when they came for computer help.
Pick a subject you love so deeply that you're always at the leading edge (which essentially means a new subject the school wants to teach). For example, CMU introduced an AI curiculum. Learn how to do NNs, DNNs, GANs, etc. Read the latest papers. Write working code. Chat with the profs. Hang around the dept.
Proof systems are a hot topic. Learn LEAN, COQ, AGDA. Learn to write proofs by machine. Read the papers. Get ahead of the curve. It is hard to "fail" a student who knows a lot more than you, especially when they help craft the tests.
Intel stuck an FPGA inside its CPU. Learn Verilog and learn to hack new CPU instructions (e.g. read Gustufson's book "The End of Error" that introduces a new kind of floating point arithmetic). Write code that outperforms BLAS code.
Learn BPF (Berkeley Packet Filter) so you can do impossible things about the kernel in User mode. Make your networking class look like it is stone-age.
I could go on but you either "get it" or you don't.
The profs were literally 1 chapter ahead of the students.
I, however, was in love with the subject. I was the "student advisor" in our "machine room". We had 5 teletypes connected to a remote mainframe at Rutgers. I lived and breathed CS.
I wrote tests for the profs (even thought I was also in the class) and I answered questions for students about the tests when they came for computer help.
Pick a subject you love so deeply that you're always at the leading edge (which essentially means a new subject the school wants to teach). For example, CMU introduced an AI curiculum. Learn how to do NNs, DNNs, GANs, etc. Read the latest papers. Write working code. Chat with the profs. Hang around the dept.
Proof systems are a hot topic. Learn LEAN, COQ, AGDA. Learn to write proofs by machine. Read the papers. Get ahead of the curve. It is hard to "fail" a student who knows a lot more than you, especially when they help craft the tests.
Intel stuck an FPGA inside its CPU. Learn Verilog and learn to hack new CPU instructions (e.g. read Gustufson's book "The End of Error" that introduces a new kind of floating point arithmetic). Write code that outperforms BLAS code.
Learn BPF (Berkeley Packet Filter) so you can do impossible things about the kernel in User mode. Make your networking class look like it is stone-age.
I could go on but you either "get it" or you don't.
I think I fall into the "don't get it" category. Could you explain what I should be taking away from this?
I understand that being at the leading edge has benefits, but it's not feasible to understand the state-of-the-art across an entire CS curriculum.
I understand that being at the leading edge has benefits, but it's not feasible to understand the state-of-the-art across an entire CS curriculum.
Of course, no one can understand everything, but he said, "pick a subject". Just one, not everything. I'm reminded of this description of a PhD:
http://matt.might.net/articles/phd-school-in-pictures/
http://matt.might.net/articles/phd-school-in-pictures/
I disagree with this cartoon in that getting to the edge is not easier by focusing on only one thing. Not knowing other things is just as likely to hold you back from some insight as not knowing your specialty well enough. It's so common that people unaware of progress in other fields spin their wheels solving problems with known solutions that I think the circle should have been more of a confounded sphere where knowing a bit of EE and ChemE makes your work on BioEng 10x closer to the edge. Knowing more than one field reinforces the strength of your innovation. Simple example, how much easier is it to make progress in applied physics if you know enough EE and machining to design and build the best test apparatus in the world?
Pick a focus and whenever you get to the point that you're a strong expert (or whatever level you feel is useful) and can do the job you want to do you should reach a bit and pick up something you're curious about to add more diversity to your skillset.
Being a generalist isn't bad, I basically have a PhD in being a generalist and tons of people want to work with me because someone who only understands optics will be lost when they need to use a photodiode and a lock in amplifier or someone who wants to test magnetic properties will be lost when they don't know how to actually construct a custom magnet. Among many obvious examples.
Unless you're at a place so big that you can just glide on assuming other people will fill in the gaps in your knowledge, you need both general and specific skills. A PhD doesn't have to mean you are only capable of one thing unless you let it, even if that was the common meaning 20 years ago. There just aren't the jobs in academia to support that kind of singular focus anyway.
We're going to live long enough that it's a terrible career decision to pigeon hole yourself so badly 50 years before you're even theoretically going to retire. Especially with online classes it's only going to get more common for people to have multiple significant skills and major changes in their field over the course of their career. Required even. Very few people are going to go for 50 years with a single technical expertise, certainly jobs for such experts are far harder to find than jobs for people who can do a few things at a more modest level of expertise.
I know a ton about materials science and physics, have genetically engineered viruses, have built chemical reactors, have designed catalysts, have done machine learning, have designed lots and lots of electronics, have programmed as required (not my favorite kind of work, but I've done 8051 assembly up to ARM firmware, Matlab, Scheme, Python, and shudder LabView), am more than half decent at machining, teach karate, perform music, and make art... trying to be the best at just one thing to the exclusion of all else is unnecessary and limiting both professionally and personally.
I was terrible at all of these things when I started. The article echoes my philosophy as well -- you have to learn to learn because you want to and not because you want a grade. I got a C- in differential equations. Two years later in stat mech I was the only one of my friends who recognized LaGrangian multipliers being used (by name even) because when I took the course my goal was to learn differential equations regardless of what grade I got, so I didn't forget it all the second the final was done. If you are the kind of person who just won't do homework unless it's graded, you should work on that because it will hold you back and you're training yourself to rapidly forget information that isn't required to pass a test.
Don't be afraid to try new things that interest you. Be okay with skilled people saying you're bad at something you did your best at, grow a thicker skin since those people are the ones likely to be able to give you advice to become good at it. Professors want you to come to office hours, especially if you're struggling.
You have to be okay with getting that C-, who cares what grade they assign you as long as ten years later you still remember convolutions or whatever. Be solid at a bunch of stuff rather than extremely good at, I don't know, something super specific like the spin states in cobalt platinum alloys or matlab modeling of alloy surface energy same as you shouldn't refuse to learn anything but Python if you want to still be a developer in 2069.
Pick a focus and whenever you get to the point that you're a strong expert (or whatever level you feel is useful) and can do the job you want to do you should reach a bit and pick up something you're curious about to add more diversity to your skillset.
Being a generalist isn't bad, I basically have a PhD in being a generalist and tons of people want to work with me because someone who only understands optics will be lost when they need to use a photodiode and a lock in amplifier or someone who wants to test magnetic properties will be lost when they don't know how to actually construct a custom magnet. Among many obvious examples.
Unless you're at a place so big that you can just glide on assuming other people will fill in the gaps in your knowledge, you need both general and specific skills. A PhD doesn't have to mean you are only capable of one thing unless you let it, even if that was the common meaning 20 years ago. There just aren't the jobs in academia to support that kind of singular focus anyway.
We're going to live long enough that it's a terrible career decision to pigeon hole yourself so badly 50 years before you're even theoretically going to retire. Especially with online classes it's only going to get more common for people to have multiple significant skills and major changes in their field over the course of their career. Required even. Very few people are going to go for 50 years with a single technical expertise, certainly jobs for such experts are far harder to find than jobs for people who can do a few things at a more modest level of expertise.
I know a ton about materials science and physics, have genetically engineered viruses, have built chemical reactors, have designed catalysts, have done machine learning, have designed lots and lots of electronics, have programmed as required (not my favorite kind of work, but I've done 8051 assembly up to ARM firmware, Matlab, Scheme, Python, and shudder LabView), am more than half decent at machining, teach karate, perform music, and make art... trying to be the best at just one thing to the exclusion of all else is unnecessary and limiting both professionally and personally.
I was terrible at all of these things when I started. The article echoes my philosophy as well -- you have to learn to learn because you want to and not because you want a grade. I got a C- in differential equations. Two years later in stat mech I was the only one of my friends who recognized LaGrangian multipliers being used (by name even) because when I took the course my goal was to learn differential equations regardless of what grade I got, so I didn't forget it all the second the final was done. If you are the kind of person who just won't do homework unless it's graded, you should work on that because it will hold you back and you're training yourself to rapidly forget information that isn't required to pass a test.
Don't be afraid to try new things that interest you. Be okay with skilled people saying you're bad at something you did your best at, grow a thicker skin since those people are the ones likely to be able to give you advice to become good at it. Professors want you to come to office hours, especially if you're struggling.
You have to be okay with getting that C-, who cares what grade they assign you as long as ten years later you still remember convolutions or whatever. Be solid at a bunch of stuff rather than extremely good at, I don't know, something super specific like the spin states in cobalt platinum alloys or matlab modeling of alloy surface energy same as you shouldn't refuse to learn anything but Python if you want to still be a developer in 2069.
It appears that there is a sense in which he is correct:
E.g., early in my career I stumbled onto some of what he is saying but then didn't appreciate the situation. Basically, just stay with what is hot. Get into a hot job, learn what is hot, jump to another job that wants the hot stuff, keep going for the hot stuff.
This way get to focus: (1) Actually can get okay with the point of the focus at the hot stuff because that point is not very large or hard if only because what is "hot" is from a public that doesn't know very much and (2) do not have to learn computer science broadly. If take the money from learning the hot stuff, get a good Ph.D. in what is arguably the better stuff, go back to the job market, and no longer be with the then current hot stuff, then can find that your career is in trouble.
Then using your actual competence in some actually important stuff, do a startup where need to please the users but not some naive public fooled by mostly simplistic hot stuff.
E.g., early in my career I stumbled onto some of what he is saying but then didn't appreciate the situation. Basically, just stay with what is hot. Get into a hot job, learn what is hot, jump to another job that wants the hot stuff, keep going for the hot stuff.
This way get to focus: (1) Actually can get okay with the point of the focus at the hot stuff because that point is not very large or hard if only because what is "hot" is from a public that doesn't know very much and (2) do not have to learn computer science broadly. If take the money from learning the hot stuff, get a good Ph.D. in what is arguably the better stuff, go back to the job market, and no longer be with the then current hot stuff, then can find that your career is in trouble.
Then using your actual competence in some actually important stuff, do a startup where need to please the users but not some naive public fooled by mostly simplistic hot stuff.
It's feasible to know the shape of the current field, so you know where to drill deeper for more information if you want it. For example, you have a file in one format and you want it in another one... if that doesn't make the word 'compiler' or at least 'parser' pop up, you need some more education. You probably don't know everything about compilers, but you can search for information about compilers and find out enough to get by.
The thing to avoid is becoming a person of one book, or a person of one tool, where you know one thing and only one thing, and you experience deep anxiety when you're out of your (one, singular) comfort zone. Being an expert in everything is impossible, but being an expert in one thing with no knowledge of anything else is definitely something to avoid.
The thing to avoid is becoming a person of one book, or a person of one tool, where you know one thing and only one thing, and you experience deep anxiety when you're out of your (one, singular) comfort zone. Being an expert in everything is impossible, but being an expert in one thing with no knowledge of anything else is definitely something to avoid.
It may not be feasible for the entire curriculum, but it may be feasible for a student to roll their own curriculum that is.
It sounds like what you’re saying is: the best way to hack a test is to actually learn the subject.
Thus, the best way to learn is to be in love with the subject.
Thus, the best way to learn is to be in love with the subject.
[deleted]
[deleted]
My wife and I are contemplating unschooling our kids. The longer our son is in primary school, the more we see the deterioration of his willingness to learn anything school-related. He hates homework (as do most kids), and this is even more painful to see given that it has been shown that homework is essentially useless for learning performance. He still likes to write and do math, but as long as he is in school, we have to take care that his interest doesn’t go south.
During school holidays, he actually starts to do school exercises for fun; those are the very same exercises he would fight against doing for hours on a regular school day. Go figure.
Btw, Gates, Page, Brin - all were not in a conventional school. The capacity to just build things without fear of being judged is invaluable.
During school holidays, he actually starts to do school exercises for fun; those are the very same exercises he would fight against doing for hours on a regular school day. Go figure.
Btw, Gates, Page, Brin - all were not in a conventional school. The capacity to just build things without fear of being judged is invaluable.
I remember this history teacher who was a breath of fresh air. He told us in advance his test where very simple. And the reason was quite genius: if you just listened and took a few notes here and there, you’d pass the test. Which freed the mind to really absorb the knowledge and have fun with the subject matter. He was also very good in discerning knowledge and teaching why a certain topic was good to know. I think I remember more from those classes than any other.
I had a software security class that was taught in effectively the same way. The professor knew they had to give grades so they just set it up such that the midterm and final accounted for the bulk of the grade and made them really easy. This ensured that almost all students would get at least a B using conventional brute Force studying methods. This removed all of the traditional grade optimizing pressures. He then made the homeworks and labs quite difficult and made them require a lot of independent research so that you could be free to try hard and potentially fall short without fear of getting owned. You would basically get out of the class what you put in. It has so far been my favorite class in my program, the one I learned the most from, and one of the most empowering classes I've taken.
Sounds good, but I could easily see this backfire with students that have been taught to hack the system: "Great, we're getting an easy pass here, let's spend zero time on it and focus on those other subjects that require rote memorising"
Right, but that same thing would happen with more intensive testing anyways. You can lead a horse to water but not make him drink and all that.
> but as long as he is in school, we have to take care that his interest doesn’t go south. [...] During school holidays, he actually starts to do school exercises for fun; those are the very same exercises he would fight against doing for hours on a regular school day. Go figure.
That does not sound like exercises themselves would be bad or that he would cease to be interested in learning overall.
It more sounds like after many hours at school, he wants to do different things that day. But when school is not present, he seeks similar activities and his willingness to learn is not deteriorated. That kind of sounds healthy and not wrong at all. In school, kids spend a lot of time learning or doing focused activities, a kid wanted to just play after that should not be sign of something grave.
The only thing that is wrong or unusual is the hours long fight against homework. I dont think that is normal for majority of kids. Only some parents I know report that much daily fights, most have fight here and there once in a while only.
That does not sound like exercises themselves would be bad or that he would cease to be interested in learning overall.
It more sounds like after many hours at school, he wants to do different things that day. But when school is not present, he seeks similar activities and his willingness to learn is not deteriorated. That kind of sounds healthy and not wrong at all. In school, kids spend a lot of time learning or doing focused activities, a kid wanted to just play after that should not be sign of something grave.
The only thing that is wrong or unusual is the hours long fight against homework. I dont think that is normal for majority of kids. Only some parents I know report that much daily fights, most have fight here and there once in a while only.
My parents and teachers gave up on attempting to assign me homework and make me do it. I despised homework. Interesting assignments were great! But day to day homework? Never did it, which isn’t a good thing (and this isn’t me attempting to brag).
The only reason they stopped is I was getting top marks in every subject anyway, and the only reason I had that was I had a tutor who pushed me so far beyond high school level that I was finally interested in the topics again.
The neat part was: to learn those topics, she gave me a crash course of most of the year 11 and year 12 curriculum in 3 or 4 months or so, because I needed the base knowledge to learn the cool stuff
The only reason they stopped is I was getting top marks in every subject anyway, and the only reason I had that was I had a tutor who pushed me so far beyond high school level that I was finally interested in the topics again.
The neat part was: to learn those topics, she gave me a crash course of most of the year 11 and year 12 curriculum in 3 or 4 months or so, because I needed the base knowledge to learn the cool stuff
If the kid is doing the exactly same exercise for fun during holidays, then the issue is not the exercise is not interesting for that kid.
Most kids with good grades don't have tutor, unless they struggle a lot or it is before entrance tests for high school and college.
Most kids with good grades don't have tutor, unless they struggle a lot or it is before entrance tests for high school and college.
> it has been shown that homework is essentially useless for learning performance
I am curious, is this about some particular kind of homework? From my own experience, I would think that exercising a subject alone (which is what most homework was in my school system at least) is a very good way of understanding it better. I of course remember not wanting to do homework many times, and I remember excessive or idiotic homework assignments, but overall homework seemed to be one of the major modes of learning for me.
Also, related to Gates, Page, Brin - I don't generally think that it's a good idea to look at a small number of very successful people and try to emulate some part of their life experience - you are, în general, very likely to fall for some kind of survivorship bias, or accidentally home in on a less important detail (for example, Gates was probably helped much more by his extremely wealthy and well-connected family than any particular aspect of his schooling).
On the other hand, there are many voices saying that one of the important purposes of the public education system is to engender conformity, which is very rarely conducive to a truly out of the ordinary life. Noam Chomsky would be one of the most credible, and he has often credited his own career to his non-standard early schooling.
I am curious, is this about some particular kind of homework? From my own experience, I would think that exercising a subject alone (which is what most homework was in my school system at least) is a very good way of understanding it better. I of course remember not wanting to do homework many times, and I remember excessive or idiotic homework assignments, but overall homework seemed to be one of the major modes of learning for me.
Also, related to Gates, Page, Brin - I don't generally think that it's a good idea to look at a small number of very successful people and try to emulate some part of their life experience - you are, în general, very likely to fall for some kind of survivorship bias, or accidentally home in on a less important detail (for example, Gates was probably helped much more by his extremely wealthy and well-connected family than any particular aspect of his schooling).
On the other hand, there are many voices saying that one of the important purposes of the public education system is to engender conformity, which is very rarely conducive to a truly out of the ordinary life. Noam Chomsky would be one of the most credible, and he has often credited his own career to his non-standard early schooling.
The homework has to be just easy enough that every kid can do it, which means that for many kids, it's mainly a test of focus and motivation. This is all the more prevalent today, when homework has turned into drill work for the questions on the standardized tests.
I've got two kids, and so I can attest from limited anecdotal evidence, that it's completely unpredictable how each kid will relate to homework.
I've got two kids, and so I can attest from limited anecdotal evidence, that it's completely unpredictable how each kid will relate to homework.
Alternative option, tell the school he's not doing homework - they're providing a service to parents to aid with a child education, if that's not aiding then change the contract.
FWIW the UK Education Act actually enshrines this concept (parents are responsible for a child being educated), though this has been eroded considerably by Tory policy in the last decade or so. We did tell our primary school "don't expect homework from $child[0]" one year, they didn't complain. However we also wanted to do flexschooling and the head refused it, which I'm still smarting over and consider to have been a significant detriment.
That same primary school ditched regular style homework in favour of projects, which $child[1] wants to do, and we can decide easily to curtail the time used doing it if we want. It's a great way of doing homework IMO; very child-centric.
FWIW the UK Education Act actually enshrines this concept (parents are responsible for a child being educated), though this has been eroded considerably by Tory policy in the last decade or so. We did tell our primary school "don't expect homework from $child[0]" one year, they didn't complain. However we also wanted to do flexschooling and the head refused it, which I'm still smarting over and consider to have been a significant detriment.
That same primary school ditched regular style homework in favour of projects, which $child[1] wants to do, and we can decide easily to curtail the time used doing it if we want. It's a great way of doing homework IMO; very child-centric.
Worth a shot, I‘m not sure it would work legally that way in Austria, but we will check!
It's worth contemplating. We homeschooled our kids and it was a good choice for our family.
I'm sympathetic to unschooling, but I've also seen it go wrong. If you decide to unschool, don't be dogmatic about it. Feel free to be more or less unschooly if that seems to be needed. Don't get sucked into a community where unschooling "purity" is highly valued.
I'm sympathetic to unschooling, but I've also seen it go wrong. If you decide to unschool, don't be dogmatic about it. Feel free to be more or less unschooly if that seems to be needed. Don't get sucked into a community where unschooling "purity" is highly valued.
Your comment jumped out at me, my wife and I are homeschooling/unschooling and some of people we've met focus on "purity" and all I can think when I talk with them is, "The point isn't to completely avoid anything that looks like schoolwork, the point is to guide the children to learn more than they would have otherwise."
Yup. Unschooling has some tendency to become an ideology instead of a practical point of view, and people get stuck in it. When all your homeschooling friends are stuck in it too, it becomes hard to get out of it even when it's not working well.
"The capacity to just build things without fear of being judged is invaluable"
For me, I had to unlearn self judgement. This has helped so much over the last few years.
For me, I had to unlearn self judgement. This has helped so much over the last few years.
How did you do that? It seems to be a big stumbling block to many.
You can try a two-step exercise mentioned in this post: https://os.me/forgive-yourself/
> Btw, Gates, Page, Brin - all were not in a conventional school.
This is Anecdata^3, you probably shouldn't make rash decisions about your children based on 3 successful technical entrepreneurs.
This is Anecdata^3, you probably shouldn't make rash decisions about your children based on 3 successful technical entrepreneurs.
Plus, most "successful entrepreneurs" happen to come from wealthy and politically connected families.
https://fortune.com/2015/07/17/entrepreneurs-family-money/
https://fortune.com/2015/07/17/entrepreneurs-family-money/
Especially since going to any kind of private school tracks closely with “parents have lots of money” which probably has a lot more to do with it.
Not necessarily, there are lots of private schools which do things very normally.
And many of those turn out tons of successful people you’ve heard of, too. Bet the ones that do so at the highest rate tend to be among the most expensive.
Yes it has been shown that having successful friends will make yourself more successful. Putting your kid into a school for rich people will more likely lead to rich friends later on.
True, it‘s just worth bearing in mind that deviating from the ordinary school system can indeed work.. kids will not automatically fail miserably if they don’t do tests or homework. That was the belief my mother (who was a teacher) ingrained in me, and I actually have to convince myself on a regular basis that life without the standard school system could also work out.
> The longer our son is in primary school, the more we see the deterioration of his willingness to learn anything school-related
Been there myself.
Most school systems have been developed to produce workers, not scholars and researchers.
Hence, rota memorization in prioritized.
Getting people to hate learning is not seen as a serious failure of the school system.
Also, students are never encouraged to question the information they receive or the decision around what is important to learn and what is not.
Been there myself.
Most school systems have been developed to produce workers, not scholars and researchers.
Hence, rota memorization in prioritized.
Getting people to hate learning is not seen as a serious failure of the school system.
Also, students are never encouraged to question the information they receive or the decision around what is important to learn and what is not.
> learn anything school-related
non scholae sed vitae discimus.
non scholae sed vitae discimus.
Don't know about Page and Brin, but according to the Netflix documentary Gates won state-level math Olympiad at a level three grades higher than his own. So he wasn't a modal child. You can't draw any conclusions from that about education at scale.
You actually can: if he had a better level of education than everyone else, why wouldn't he do better than people taking traditional educational paths?
As someone who participated in many academic competitions as a child, the winners were consistently weighted 9:1 private:public.
As someone who participated in many academic competitions as a child, the winners were consistently weighted 9:1 private:public.
No, you cannot. IQ is highly correlated with wealth and is also highly heritable. So in all likelihood what you've observed is that smart kids win academic competitions, and also happen to be from rich families. If you take a bunch of kids with an IQ of 100 (or 115, or even 130) and put them into private schools they'll do a little better but they won't be winning any academic competitions.
The available data disagrees with you on this:
https://doi.org/10.1126/science.1238041
https://doi.org/10.1126/science.1238041
> The available data disagrees with you on this:
> https://doi.org/10.1126/science.1238041
That article doesn't seem relevant. 'Poverty impedes cognitive function' is entirely compatible with 'IQ is highly correlated with wealth and is also highly heritable', and with the claim that private school can't make a genius out of a kid who lacks an extremely high IQ.
edit: maybe I misunderstood, and you only were only pointing out that some potential-genius kids have that potential stifled by poverty. But that doesn't seem to be the original point of disagreement.
> https://doi.org/10.1126/science.1238041
That article doesn't seem relevant. 'Poverty impedes cognitive function' is entirely compatible with 'IQ is highly correlated with wealth and is also highly heritable', and with the claim that private school can't make a genius out of a kid who lacks an extremely high IQ.
edit: maybe I misunderstood, and you only were only pointing out that some potential-genius kids have that potential stifled by poverty. But that doesn't seem to be the original point of disagreement.
Some information about the flaws with that data:
https://www.researchgate.net/publication/259207757_Comment_o...
https://www.researchgate.net/publication/259207757_Comment_o...
Let's see: https://www.maa.org/sites/default/files/pdf/AMC/usamo/2019/2...
On the first page, the breakdown is 23 kids from public schools, 7 kids from private schools, 4 kids listing some sort of math circle / whatever Arateem is, 5 kids listing universities (home schooled/taking uni courses? No idea). Possibly a few of the private school kids are on scholarship. The 7 kids go to 4 schools.
On the first page, the breakdown is 23 kids from public schools, 7 kids from private schools, 4 kids listing some sort of math circle / whatever Arateem is, 5 kids listing universities (home schooled/taking uni courses? No idea). Possibly a few of the private school kids are on scholarship. The 7 kids go to 4 schools.
> Gates, ... - all were not in a conventional school
wikipedia said he went to a regular private school then went to Harvard which sounds pretty conventional. Yeah he didn't finish, but worry about that later.
wikipedia said he went to a regular private school then went to Harvard which sounds pretty conventional. Yeah he didn't finish, but worry about that later.
Theres truth to the OPs statement. Some online sources say gates attended Montessori for elementary but it's not conclusive. Brin and page for sure attended Montessori. Bezos as well. I went to a Montessori until 5th grade and honestly didnt think it was anything special. Probably because they mixed a lot of traditional school elements in as well.
Private schools cost lots of money, there is nothing regular about them.
> He hates homework (as do most kids)
I just didn't do my homework and got bad grades. Luckily in my country you can always get in via standardized tests instead of grades so I did well in life anyway. I'm not sure why homework is weighed so heavily in grades, it mostly just shows how much free time the students are willing to sacrifice. The biggest cost this had for me was the guilt that I hadn't done what I was supposed to do, which made me feel like a bad person, but in hindsight I did much better than most of my more studious peers so it really wasn't all that important.
I just didn't do my homework and got bad grades. Luckily in my country you can always get in via standardized tests instead of grades so I did well in life anyway. I'm not sure why homework is weighed so heavily in grades, it mostly just shows how much free time the students are willing to sacrifice. The biggest cost this had for me was the guilt that I hadn't done what I was supposed to do, which made me feel like a bad person, but in hindsight I did much better than most of my more studious peers so it really wasn't all that important.
[deleted]
All true known and well established. But homeschooling requires a parent to sacrifice anything else and be devoted for that. Not an easy decision. It's much easier to find a good school!
Not necessarily true. I admit it’s a lifestyle change, but it can work quite well if both parents are freelancers.
We thought the same way but it’s difficult to replace the social component. Still it’s hard to see how bad the school system is and force our kids to participate. It could be so much better.
Look for an unschooling school, strange as that may sound. Free democratic and Sudbury Valley model are some of the things to search for. There aren't many around but the numbers are increasing.
We send our son to one. It's a leap of faith; the kids essentially "do whatever". But it seems worth it compared to the damage done by middle and high school. And the kids that I've seen come out of this school seem to do well in society (and college) and generally seem much more mature than high school graduates.
We send our son to one. It's a leap of faith; the kids essentially "do whatever". But it seems worth it compared to the damage done by middle and high school. And the kids that I've seen come out of this school seem to do well in society (and college) and generally seem much more mature than high school graduates.
Agile Learning Centers and Liberated Learners are inspiring networks of small schools in this vein (and both less dogmatic than Sudbury - in a good way, IMO).
Montessori schools as well: "Follow the child"
Be careful with Montessori.
I went to a "traditional Montessori" school for a few months when I was very young. Long story short, their educational method insisted that we needed instruction in how to play with toys before we were allowed to do so, and we needed to do so on mats, which needed to be rolled up tidily at the end of the prescribed playtime.
The problem with that was that we already knew what Legos were. But we were supposed to pretend we didn't, in order to appease the teacher.
I think this was the same school that insisted to my parents that I couldn't read because they (the school) hadn't taught me to read. My parents taught me to read. So they had me demonstrate, and the teacher decided my parents must have... trained me to pretend to read.
I went to a "traditional Montessori" school for a few months when I was very young. Long story short, their educational method insisted that we needed instruction in how to play with toys before we were allowed to do so, and we needed to do so on mats, which needed to be rolled up tidily at the end of the prescribed playtime.
The problem with that was that we already knew what Legos were. But we were supposed to pretend we didn't, in order to appease the teacher.
I think this was the same school that insisted to my parents that I couldn't read because they (the school) hadn't taught me to read. My parents taught me to read. So they had me demonstrate, and the teacher decided my parents must have... trained me to pretend to read.
Not these days. There are so many home school groups that it's very easy to replace the social component.
I'm curious about Page & Brin - I wasn't aware they didn't go through conventional schooling. What was their background?
They both attended Montessori schools. [0][1][2][3][4][5][6]
The person who mentioned waldorf must have been confused. While they’re both small private schools, they are two very different approaches to education.
[0] Wiki Brin: https://en.wikipedia.org/wiki/Sergey_Brin
[1] Post Article: https://www.washingtonpost.com/local/education/montessori-lo...
[2] ABC Interview: https://www.youtube.com/watch?v=0C_DQxpX-Kw
[3] Wiki Page: https://en.wikipedia.org/wiki/Larry_Page
[4] Guardian Article: https://www.theguardian.com/sustainable-business/2014/jul/23...
[5] BI Article: https://www.businessinsider.com/tech-innovators-who-went-to-...
[6] Forbes Article: https://www.businessinsider.com/tech-innovators-who-went-to-...
The person who mentioned waldorf must have been confused. While they’re both small private schools, they are two very different approaches to education.
[0] Wiki Brin: https://en.wikipedia.org/wiki/Sergey_Brin
[1] Post Article: https://www.washingtonpost.com/local/education/montessori-lo...
[2] ABC Interview: https://www.youtube.com/watch?v=0C_DQxpX-Kw
[3] Wiki Page: https://en.wikipedia.org/wiki/Larry_Page
[4] Guardian Article: https://www.theguardian.com/sustainable-business/2014/jul/23...
[5] BI Article: https://www.businessinsider.com/tech-innovators-who-went-to-...
[6] Forbes Article: https://www.businessinsider.com/tech-innovators-who-went-to-...
Montessori iirc
Waldorf/Steiner iirc
What about socializing?
That’s the big issue. We‘re thinking about Karate/swimming/chess etc courses, but ideally you would have access to a community of unschoolers. Some big cities have this, but currently not an option for us.
I think sports are pretty good at forcing kids outside of their comfort zone. I definently didnt like it as a kid, but it honestly made me grow so much.
Yes, most tests are bad, but there is a quick advice that improves them immensely: never ask questions that are not at least Level 3 in Bloom's taxonomy of knowledge: https://blog.testdome.com/blooms-taxonomy/
I cofounded a technical screening startup, and despite our efforts to educate our customers that enter their questions on our platform, 95% of their custom questions are bad questions. They tend to ask trivia questions, that can quickly be googled, instead of asking work-sample questions (which we at TestDome.com prefer). I think they learned that from years of paper tests in school, it is very hard to unlearn.
Just to give you an idea, for testing web programmers we suggest questions where you need to find bugs in HTML code (https://www.testdome.com/questions/html-css/inspector/17629), while our customers would ask questions like "What does CSS stand for?"
I cofounded a technical screening startup, and despite our efforts to educate our customers that enter their questions on our platform, 95% of their custom questions are bad questions. They tend to ask trivia questions, that can quickly be googled, instead of asking work-sample questions (which we at TestDome.com prefer). I think they learned that from years of paper tests in school, it is very hard to unlearn.
Just to give you an idea, for testing web programmers we suggest questions where you need to find bugs in HTML code (https://www.testdome.com/questions/html-css/inspector/17629), while our customers would ask questions like "What does CSS stand for?"
Thanks for the article.
The advise against level 5 questions is interesting. I found them to be very useful for answering if a candidate has really applied in the field before, especially comparing two big fields against each other. For example asking if they prefer sql or nosql. Anyone who doesn't answer some variant of "it depends" has probably not had enough hand on with either of them. But regardless if the candidate has a strong opinion or not, and regardless if it's the same opinion as yours, the answer is going to give you endless amount of openers for dialogue and follow-up questions on how they reached that conclusion and in which projects they used such techniques and how it worked out.
As the articles notes, the questions can easily be misperceived as checking if the candidate has the same opinions as yourself, or you becoming outsmarted by eloquence. Since you are yourself asking and evaluating the questions you have to ensure this doesn't become the case, by putting opinions aside and only asking about fields which you yourself deeply understand. Not sure if it might be off-putting for a candidate to get this type of question though, if they don't understand the deeper purpose of the question? Should you give a disclaimer that you are not actually not interested in their final opinion?
The advise against level 5 questions is interesting. I found them to be very useful for answering if a candidate has really applied in the field before, especially comparing two big fields against each other. For example asking if they prefer sql or nosql. Anyone who doesn't answer some variant of "it depends" has probably not had enough hand on with either of them. But regardless if the candidate has a strong opinion or not, and regardless if it's the same opinion as yours, the answer is going to give you endless amount of openers for dialogue and follow-up questions on how they reached that conclusion and in which projects they used such techniques and how it worked out.
As the articles notes, the questions can easily be misperceived as checking if the candidate has the same opinions as yourself, or you becoming outsmarted by eloquence. Since you are yourself asking and evaluating the questions you have to ensure this doesn't become the case, by putting opinions aside and only asking about fields which you yourself deeply understand. Not sure if it might be off-putting for a candidate to get this type of question though, if they don't understand the deeper purpose of the question? Should you give a disclaimer that you are not actually not interested in their final opinion?
I guess that "putting opinions aside and only asking about fields which you yourself deeply understand" is actually very very hard to consistently apply in practice ?
"Why is Pluto no longer classified as a planet?"
You've passed the "categorization" level, but might have actually failed the real "understanding" level, for which "We have discovered many large((r)) objects beyond it." might actually be a more correct answer!
After the discovery of dozens of new trans-neptunian "planets", categorizing Pluto as a "real planet", while it probably had more in common with them became problematic.
The best definition that we had to come up with to cleanly separate "real" planets from the others was therefore quite unlikely to include Pluto!
After the discovery of dozens of new trans-neptunian "planets", categorizing Pluto as a "real planet", while it probably had more in common with them became problematic.
The best definition that we had to come up with to cleanly separate "real" planets from the others was therefore quite unlikely to include Pluto!
can anybody comment on the difference between "understanding" and "applying" in that taxonomy. The examples provided don't seem very good to me.
is it the difference between understanding a language and not speaking it, perhaps? following a conversation, but not being able to participate in it?
is it the difference between understanding a language and not speaking it, perhaps? following a conversation, but not being able to participate in it?
I have learned Bloom's taxonomy with comprehension instead of understanding. So if in an algorithm's class, you were taught that algorithm X was exponential time, then the exam question at the comprehension level can be, "describe why does it take exponential time to run algorithm X".
An applying level question could be, "Find the running time of algorithm Y (which was never discussed in class)."
An applying level question could be, "Find the running time of algorithm Y (which was never discussed in class)."
Understanding is being able to read python code with no reference book/manual.
Applying is writing python code with no reference book/manual.
In both cases doing it properly means you are reasonably fast while being confident about the correctness (either that you correctly predicted the program's behavior or that you wrote bug-free software).
Applying is writing python code with no reference book/manual.
In both cases doing it properly means you are reasonably fast while being confident about the correctness (either that you correctly predicted the program's behavior or that you wrote bug-free software).
I usually find PG’s writing to be very illuminating, but I largely disagree with this piece.
I think the reason founders want introductions to influential people is not because they think that this is what you need to be successful. It’s because they have seen many cases where influential people talked up startups that seemed to be pretty mediocre, and it helped them succeed. And they think “well my startup is at least as good as those other ones, so publicity by famous people will accelerate our path to success.”
They’ve seen press coverage of lousy startups propel them into fundraising successes and growing revenues, even if they never became profitable. And they think “I’d succeed faster if my legit startup had that kind of exposure.”
It’s not that they think these things are a substitute for having a good product. They think (correctly) that having these things will accelerate their growth and somewhat lower the bar to success (particularly if there are network effects involved).
I think the reason founders want introductions to influential people is not because they think that this is what you need to be successful. It’s because they have seen many cases where influential people talked up startups that seemed to be pretty mediocre, and it helped them succeed. And they think “well my startup is at least as good as those other ones, so publicity by famous people will accelerate our path to success.”
They’ve seen press coverage of lousy startups propel them into fundraising successes and growing revenues, even if they never became profitable. And they think “I’d succeed faster if my legit startup had that kind of exposure.”
It’s not that they think these things are a substitute for having a good product. They think (correctly) that having these things will accelerate their growth and somewhat lower the bar to success (particularly if there are network effects involved).
> They think (correctly) that having these things will accelerate their growth and somewhat lower the bar to success
Incorrectly, unless you define success as a local maxima. You usually can’t influence your way to broad user adoption
Incorrectly, unless you define success as a local maxima. You usually can’t influence your way to broad user adoption
What makes you say that? For early stage startups, it seems especially likely that increased awareness can (though usually will not) create a material amount of additional success.
I know that a few fortunate tweets my startup received drove tons of traffic our way, and likely resulted in the press coverage we received soon after. And that press coverage brought more users, and also the proof points of having been covered by major media. That makes fundraising and partnerships easier, etc.
I know that a few fortunate tweets my startup received drove tons of traffic our way, and likely resulted in the press coverage we received soon after. And that press coverage brought more users, and also the proof points of having been covered by major media. That makes fundraising and partnerships easier, etc.
I know that learning how to hack systems is valuable because that's how I got into YCombinator. I set out specifically to hack the application process and succeeded. It wasn't even that difficult to do.
If pg believes what he's saying in this essay, I have two questions:
1. why did he design a testing system that was easily hacked?
2. why does YCombinator itself focus so much on hacking the venture capital system, as opposed to making great products? [1]
I think the answer to both is that the ideas presented in this essay don't scale. If you can't create a 200 person per year startup incubator that focuses on true learning instead of process hacking, then how could someone do it for a 20,000 person university? Things get boiled down to metrics like grades and valuations because they are the only way to measure/manage the productive output of a large number of people.
The insight that I've had is that as long as large civilizations and organizations exist, there will be easily hackable systems. There is a direct relationship between the number of humans involved in a thing, and the number of opportunities to hack that thing. Understanding and exploiting this seems like a valuable ability.
[1] I was personally struck by how much of YCombinator was (in 2008) oriented around learning what magic words to say to investors and exactly when to time techcrunch launch articles, etc. There seemed to be no time given to "how to make a great product", likely because this is something that can't be taught
If pg believes what he's saying in this essay, I have two questions:
1. why did he design a testing system that was easily hacked?
2. why does YCombinator itself focus so much on hacking the venture capital system, as opposed to making great products? [1]
I think the answer to both is that the ideas presented in this essay don't scale. If you can't create a 200 person per year startup incubator that focuses on true learning instead of process hacking, then how could someone do it for a 20,000 person university? Things get boiled down to metrics like grades and valuations because they are the only way to measure/manage the productive output of a large number of people.
The insight that I've had is that as long as large civilizations and organizations exist, there will be easily hackable systems. There is a direct relationship between the number of humans involved in a thing, and the number of opportunities to hack that thing. Understanding and exploiting this seems like a valuable ability.
[1] I was personally struck by how much of YCombinator was (in 2008) oriented around learning what magic words to say to investors and exactly when to time techcrunch launch articles, etc. There seemed to be no time given to "how to make a great product", likely because this is something that can't be taught
That's also my largest concern with this essay. PG describes the world they way he wants it to be, but not the world the way it is. And I haven't been inside of YC, but everything I've seen is that YC is essentially program that combines some level of "support" with essentially learning to hack VCs.
If it were simply build the best product then there wouldn't be a need for YC. If it were simply build the best product there wouldn't be a need for marketing or sales or business degrees.
The hacks involved in marking and sales became so important to any companies success that they've over time become entire fields of their own. Same thing with an MBA - essentially a degree to hack business. Regardless of what you think of it, it's clear based on the financial returns of it that it works.
One of the biggest lessons constantly mentioned of startups is how naive engineers are to the business side of things and there is more than building a great product. In this essay PG takes all of that and throws it out the window. Is that suddenly no longer true? My read of it is that he (rightfully) likes YC and as a result doesn't see how it has the same flaws as the systems he describes. Essentially like every parent who only sees their kid as beautiful.
If it were simply build the best product then there wouldn't be a need for YC. If it were simply build the best product there wouldn't be a need for marketing or sales or business degrees.
The hacks involved in marking and sales became so important to any companies success that they've over time become entire fields of their own. Same thing with an MBA - essentially a degree to hack business. Regardless of what you think of it, it's clear based on the financial returns of it that it works.
One of the biggest lessons constantly mentioned of startups is how naive engineers are to the business side of things and there is more than building a great product. In this essay PG takes all of that and throws it out the window. Is that suddenly no longer true? My read of it is that he (rightfully) likes YC and as a result doesn't see how it has the same flaws as the systems he describes. Essentially like every parent who only sees their kid as beautiful.
This going to be true of most self help writing by survivors.
> If you can't create a 200 person per year startup incubator that focuses on true learning instead of process hacking, then how could someone do it for a 20,000 person university?
As a university professor, I can at least try. Not at scale -- certainly not for my entire university -- but for my own classes as an individual.
There are several constraints, of course. I have to give grades, and students care about grades, a lot. These grades should measure (to some extent) how prepared students are to proceed to whatever is next. Students have a number of conflicting priorities, and it is natural to concentrate on whatever is most urgent. And students come with wildly differing backgrounds and interest levels (and amounts of available time), and I have to be consistent and fair to everyone.
But, when I grade, I have a lot of leeway to decide what I believe is worth measuring, and measure that. My aim -- which I hope I at least partially realize -- is to ensure that the best way to "hack" my classes is to learn what I hope the students will learn.
As a university professor, I can at least try. Not at scale -- certainly not for my entire university -- but for my own classes as an individual.
There are several constraints, of course. I have to give grades, and students care about grades, a lot. These grades should measure (to some extent) how prepared students are to proceed to whatever is next. Students have a number of conflicting priorities, and it is natural to concentrate on whatever is most urgent. And students come with wildly differing backgrounds and interest levels (and amounts of available time), and I have to be consistent and fair to everyone.
But, when I grade, I have a lot of leeway to decide what I believe is worth measuring, and measure that. My aim -- which I hope I at least partially realize -- is to ensure that the best way to "hack" my classes is to learn what I hope the students will learn.
Agree wholeheartedly. That's exactly the approach I take, but the problem of course is that 1) it's difficult to do successfully, and 2) students "freak out" because the exams become unpredictable and they end up getting more stressed out that they need to about succeeding at the exam. Of course, my regular high-performing students do well regardless, but I haven't found a way to convince them that they have the skills already to do well on the exam. Instead, they pull all-nighters and ultimately damage their results on exam day when, if I had given it as a surprise exam, they would have been just fine. So on some level, it's a need to teach self-confidence, self-diagnostics, and introspection. This (again) is a TON of work, since it's necessarily person-dependent, so only professors who have the time and energy can actually attempt to do it. The others will just say "screw it" and give an exam of sentences copy/pasted from the textbook, but with keywords, blanked out. Yet another extreme are the professors who endlessly allow exam re-submissions, so students just brute-force hack their way to a passing grade.
I want the obvious response to the above to be "okay, just raise the bar for teaching professors" but, unfortunately, those decisions are usually made by people who are completely ignorant to the process, and who rely on broken systems of teaching evaluation. Those who would be good at making those decisions are busy doing the teaching, and doing it well. Also, I don't personally think we want a system that so micro-manages our professors' pedagogical styles. I get fairly positive student feedback, but so do the professors who offer endless exam re-submissions and whose students have basically zero material retention after they pass the class. The optimal (but impossible, partially due to sheer laziness) way would be for students to self-select themselves into classes with professors that they believe will ultimately teach them more and better material, without having the extra pressure of the effect on their GPA.
(edited with some last-minute thoughts).
I want the obvious response to the above to be "okay, just raise the bar for teaching professors" but, unfortunately, those decisions are usually made by people who are completely ignorant to the process, and who rely on broken systems of teaching evaluation. Those who would be good at making those decisions are busy doing the teaching, and doing it well. Also, I don't personally think we want a system that so micro-manages our professors' pedagogical styles. I get fairly positive student feedback, but so do the professors who offer endless exam re-submissions and whose students have basically zero material retention after they pass the class. The optimal (but impossible, partially due to sheer laziness) way would be for students to self-select themselves into classes with professors that they believe will ultimately teach them more and better material, without having the extra pressure of the effect on their GPA.
(edited with some last-minute thoughts).
The problem in my eyes is while you are fighting the fight your department is fighting another one entirely within a university of departments competing in yet another fight.
Who is fighting at the organizational level for the students?
The problem is systemic. All of the solutions I see, here and elsewhere, describe a small piece of the problem. At best — a few courses in a curriculum.
Who is fighting at the organizational level for the students?
The problem is systemic. All of the solutions I see, here and elsewhere, describe a small piece of the problem. At best — a few courses in a curriculum.
> Who is fighting at the organizational level for the students?
Plenty of people. Sometimes at cross purposes, rarely with little effect.
Unlike Tolstoy's happy families, good teachers are not all alike. For the most part, I can't envision what a productive organizational-level fight would even look like.
Plenty of people. Sometimes at cross purposes, rarely with little effect.
Unlike Tolstoy's happy families, good teachers are not all alike. For the most part, I can't envision what a productive organizational-level fight would even look like.
> Yet another extreme are the professors who endlessly allow exam re-submissions, so students just brute-force hack their way to a passing grade.
I know professors who have changed the answers during the re-submission time, so students brute-force their way to a failing grade...
I know professors who have changed the answers during the re-submission time, so students brute-force their way to a failing grade...
Companies are this way too. The best companies are all run by OKRs and metrics. But if you peek under the hood, all of it is insanely hackable; you have to be naive to try and actually meet your explicitly stated objectives because that's not how you get promoted.
It reminds me of a quote from The Elephant in the Brain. I don't remember the wording exactly, but it's something like "if there is a behavior in a large group of people that doesn't make sense to you, not only is it by design, but it's also probably the whole point."
Incidentally, I was also struck by how much of YC was (in 2009) about hacking venture capital. But lots of people from that era now say this behavior is unethical disavow all knowledge that it was ever actively promoted. It's like we lived through different realities.
If I were to steelman The Lesson to Unlearn I think I'd say this. On an individual level there is enormous benefit to learning how to operate without hacking a test. That's how you do great work; that's what original thinking is. On a societal level this seems unshakable. But societies have made dramatic shifts in behavior and social organization before. From hunter gatherers to farmers to city dwellers, for example. So while this is hard to imagine it might just be that-- lack of imagination. The social return to abandoning this system might indeed be enormous.
It reminds me of a quote from The Elephant in the Brain. I don't remember the wording exactly, but it's something like "if there is a behavior in a large group of people that doesn't make sense to you, not only is it by design, but it's also probably the whole point."
Incidentally, I was also struck by how much of YC was (in 2009) about hacking venture capital. But lots of people from that era now say this behavior is unethical disavow all knowledge that it was ever actively promoted. It's like we lived through different realities.
If I were to steelman The Lesson to Unlearn I think I'd say this. On an individual level there is enormous benefit to learning how to operate without hacking a test. That's how you do great work; that's what original thinking is. On a societal level this seems unshakable. But societies have made dramatic shifts in behavior and social organization before. From hunter gatherers to farmers to city dwellers, for example. So while this is hard to imagine it might just be that-- lack of imagination. The social return to abandoning this system might indeed be enormous.
I really like the steelman interpretation. I suspect that often times, PG's trying to express some sort of core idea, but readers bring their own context with it, and apply it to something else, in this case, YC. And in a different thread, interviews.
But if one were to find oneself in a situation where there are no gatekeeper tests, then how would you go about doing great work? And by being able to see it as an idealism, at least we have a target to shoot for, even if we might fall short in practice.
But if one were to find oneself in a situation where there are no gatekeeper tests, then how would you go about doing great work? And by being able to see it as an idealism, at least we have a target to shoot for, even if we might fall short in practice.
Your last graph is very interesting. I think it resonates because tests are setup with the test-devisers biases and blind spots, but original thinking (tautologically) has to be outside the current consensus that built the environment that built the test makers.
There's a snake eating its tail situation here.
There's a snake eating its tail situation here.
It's because he doesn't believe getting into YC is the achievement. Creating a company with users / profitability / growth is. You can get into YC and still fail with your company. And it's desirable for founders and YC to be more interested in building a successful company.
When he says, "Make a good product", I think he's trying to convey a core and fundamental principle of business that gets obfuscated by all the things you have to do be successful in a startup. That's necessary when teaching beginners.
But he's traditionally had a hard time conveying this seemingly simple concept, and he's always been puzzled by it. You can see it in the previous essays he referenced in this one--the high school one, and how founders over-complicate things.
Though he's talking about this topic as if creating the company is the goal, I'm not sure to what degree he realizes how YC contributes to the effect he's seeing.
Even after getting into YC, I was moderately surprised by how you're still being judged, sometimes explicitly, and other time implicitly. I had batchmates that didn't want to go into office hours, for fear of appearing weak and hence wouldn't get a follow-on recommendation to investors from a YC partner.
Of course, I think PG would say that's stupid. If you go into office hours and fix your problem, then by virtue of being a better startup, you'll be a more attractive investment. Trying to do it the other way is putting the cart before the horse.
That said, the effect is real. Like the Observer Effect in quantum mechanics, by virtue of being a conduit to further funding, founders will turn their attention to the tests. Because if you don't, it's easy to be written off and not be able to get the help that you need.
When he says, "Make a good product", I think he's trying to convey a core and fundamental principle of business that gets obfuscated by all the things you have to do be successful in a startup. That's necessary when teaching beginners.
But he's traditionally had a hard time conveying this seemingly simple concept, and he's always been puzzled by it. You can see it in the previous essays he referenced in this one--the high school one, and how founders over-complicate things.
Though he's talking about this topic as if creating the company is the goal, I'm not sure to what degree he realizes how YC contributes to the effect he's seeing.
Even after getting into YC, I was moderately surprised by how you're still being judged, sometimes explicitly, and other time implicitly. I had batchmates that didn't want to go into office hours, for fear of appearing weak and hence wouldn't get a follow-on recommendation to investors from a YC partner.
Of course, I think PG would say that's stupid. If you go into office hours and fix your problem, then by virtue of being a better startup, you'll be a more attractive investment. Trying to do it the other way is putting the cart before the horse.
That said, the effect is real. Like the Observer Effect in quantum mechanics, by virtue of being a conduit to further funding, founders will turn their attention to the tests. Because if you don't, it's easy to be written off and not be able to get the help that you need.
> It's because he doesn't believe getting into YC is the achievement.
But so much of YC felt (and probably does more so now) like you're being judged all the time. Which makes total sense to me, because seeking prestige is so fundamental to human nature. That co-evolved with social cooperation in a virtuous cycle.
If pg really believed what he wrote in this essay, YC would be structured very differently. The moment pg allowed some companies/founders to have better access to him than others, or the moment he talked about how some founders are better than others, he implicitly set up a criteria for judging founders and their companies. And people will always naturally feel envy and try to hack that criteria. But a precondition for doing good original work is not feeling judged. So he should have gone on a crusade to completely eliminate all forms of favoritism if he wanted to maximize the odds of success for a modal company. But that's not what he actually ended up doing. At all.
Now the question is, is the essay wrong or is the YC structure/culture wrong. If I were a betting man, I'd put all my money on the former every single time, and I think I'd never come lose that bet once.
But so much of YC felt (and probably does more so now) like you're being judged all the time. Which makes total sense to me, because seeking prestige is so fundamental to human nature. That co-evolved with social cooperation in a virtuous cycle.
If pg really believed what he wrote in this essay, YC would be structured very differently. The moment pg allowed some companies/founders to have better access to him than others, or the moment he talked about how some founders are better than others, he implicitly set up a criteria for judging founders and their companies. And people will always naturally feel envy and try to hack that criteria. But a precondition for doing good original work is not feeling judged. So he should have gone on a crusade to completely eliminate all forms of favoritism if he wanted to maximize the odds of success for a modal company. But that's not what he actually ended up doing. At all.
Now the question is, is the essay wrong or is the YC structure/culture wrong. If I were a betting man, I'd put all my money on the former every single time, and I think I'd never come lose that bet once.
> If pg really believed what he wrote in this essay, YC would be structured very differently
Yes, I agree. I can't tell if he's ever thought about it. I would have thought Jessica, being the social radar, would have picked up on it. Maybe it just wasn't a high order bit on the list of priorities.
I've come to think of PG as the disapproving father I never had.
Yes, I agree. I can't tell if he's ever thought about it. I would have thought Jessica, being the social radar, would have picked up on it. Maybe it just wasn't a high order bit on the list of priorities.
I've come to think of PG as the disapproving father I never had.
> Now the question is, is the essay wrong or is the YC structure/culture wrong.
I think that's a false dichotomy.
> The moment pg allowed some companies/founders to have better access to him than others, or the moment he talked about how some founders are better than others, he implicitly set up a criteria for judging founders and their companies.
It's correct that this creates a hackable test, but the losses incurred via that might be less than "going on a crusade to completely eliminate all forms of favoritism". This does not imply that hackable tests aren't bad, meaning the the essay is wrong, just that the pendulum can swing too far in the other direction too. Which isn't that striking of a realization.
I think that's a false dichotomy.
> The moment pg allowed some companies/founders to have better access to him than others, or the moment he talked about how some founders are better than others, he implicitly set up a criteria for judging founders and their companies.
It's correct that this creates a hackable test, but the losses incurred via that might be less than "going on a crusade to completely eliminate all forms of favoritism". This does not imply that hackable tests aren't bad, meaning the the essay is wrong, just that the pendulum can swing too far in the other direction too. Which isn't that striking of a realization.
Spot on. If you're interested in this sort of stuff, I'd recommend the book Seeing Like A State. The author basically walks through most of history as viewed through the lens of authorities trying to measure things (usually to extract taxes or exercise control). But basically traces everything from the layout of cities to how we name ourselves down to authority trying to standardize things so they can measure them.
> There seemed to be no time given to "how to make a great product", likely because this is something that can't be taught
It can be taught, but the general principles are weak (it’s basically “how to be an effective futurist”), with most of the useful knowledge being domain-specific.
And the thing with the specific domains is, either they’re not understood yet—in which case everyone is flailing around—or they are well-understood—in which case the market has already consolidated around the companies who can best operationalize the set of techniques required to create great products for that market, and the best you’ll hope to do is to get bought out by one of them, not to compete or replace them.
Because, in the end, having a great product is a multiplier, but so is execution; and execution provides compound interest on its gains, such that corporations inevitably execute exponentially better as they age.
In other words, the only time where you can set out to win a market, and actually have a hope of doing so, is precisely when we don’t yet know how to operationalize the production of great products for that domain.
It can be taught, but the general principles are weak (it’s basically “how to be an effective futurist”), with most of the useful knowledge being domain-specific.
And the thing with the specific domains is, either they’re not understood yet—in which case everyone is flailing around—or they are well-understood—in which case the market has already consolidated around the companies who can best operationalize the set of techniques required to create great products for that market, and the best you’ll hope to do is to get bought out by one of them, not to compete or replace them.
Because, in the end, having a great product is a multiplier, but so is execution; and execution provides compound interest on its gains, such that corporations inevitably execute exponentially better as they age.
In other words, the only time where you can set out to win a market, and actually have a hope of doing so, is precisely when we don’t yet know how to operationalize the production of great products for that domain.
My take on this:
Having a great product, building things people want (i.e. the "non-hackable" parts of building a startup) are necessary but not sufficient requirements for building a successful startup. What YC tries to teach is all the other parts (how to deal with VCs, what metrics to focus on, how to manage your time, etc.) that are also necessary, but not the core of what makes a good company. These are the "hackable" parts if you will. But you're very likely to be unsuccessful if you only have those hackable parts right, but still don't have a product people find useful and want to buy.
Since you're not posting anonymously, I'd be curious if you could offer more information on what parts of YC you think you hacked, and how your startup turned out.
Having a great product, building things people want (i.e. the "non-hackable" parts of building a startup) are necessary but not sufficient requirements for building a successful startup. What YC tries to teach is all the other parts (how to deal with VCs, what metrics to focus on, how to manage your time, etc.) that are also necessary, but not the core of what makes a good company. These are the "hackable" parts if you will. But you're very likely to be unsuccessful if you only have those hackable parts right, but still don't have a product people find useful and want to buy.
Since you're not posting anonymously, I'd be curious if you could offer more information on what parts of YC you think you hacked, and how your startup turned out.
This was my immediate objection as well. School, as it stands, was actually -incredibly- useful to me. Because it taught me that the stated objectives of a person, organization, etc, are often lies, and that the trick is to learn the real objectives, and optimize accordingly. Between classes that actually taught me the material, classes that expected me to learn the material without teaching it to me (but still expected me to complete relevant projects that required understanding), and classes that required me to learn completely separate things (i.e., yes, physics class, where the taught material, homework material, and tested material, were all ENTIRELY DISTINCT FROM EACH OTHER, and so I had to learn to navigate that and reach out for help from TAs and past students and etc), I feel like I got a more complete education in how the world works.
That's not to say "hack the system", but it is to say look deeper. There's value in learning. There's value in the stated objectives; after all, there's a reason they're being called out. But there often are unstated, important systems and objectives to understand and address, too.
I'd love this dream world PG describes where everything aligns, the stated priorities are the only ones, and the systems and cultures and tests optimize only for them. But that's not the world we live in.
That's not to say "hack the system", but it is to say look deeper. There's value in learning. There's value in the stated objectives; after all, there's a reason they're being called out. But there often are unstated, important systems and objectives to understand and address, too.
I'd love this dream world PG describes where everything aligns, the stated priorities are the only ones, and the systems and cultures and tests optimize only for them. But that's not the world we live in.
I think what he's saying is that the approach is wrong. The highly industrialized school system is the way it is because of scale. His perspective is the initial point (paraphrasing: tests should be the same as taking a blood test).
To add to that point, take a urine test: people have been hacking it for years. The purpose of the urine test is to see what residual byproducts from certain drugs are in my system. I can hack it by using someone else's urine or imbibe on drugs that have a short lifetime in my bloodstream. Otherwise, to pass the test I have to not consume drugs.
His complaint is that people who pass the tests really shouldn't have. It's like a principal assuring parents that a strict urine exam is given to all teachers only to find out all the teachers who work for you are drug addicts (drug addicts who can hack the pee test).
His reference to private schools teaching students how to hack tests resonates with me as social workers teaching drug users how to hack their pee tests. I take that to mean that if teachers are doing their best but resort to teaching how to hack tests then their needs to be more work done outside of the teacher-student system.
[This is where disparities due to parent's wealth and income come in:] Students who have access to tutors can get the help they in a way having a teacher can't. A teacher can be a tutor, but cannot be a tutor for everyone (an issue with scale).
To add to that point, take a urine test: people have been hacking it for years. The purpose of the urine test is to see what residual byproducts from certain drugs are in my system. I can hack it by using someone else's urine or imbibe on drugs that have a short lifetime in my bloodstream. Otherwise, to pass the test I have to not consume drugs.
His complaint is that people who pass the tests really shouldn't have. It's like a principal assuring parents that a strict urine exam is given to all teachers only to find out all the teachers who work for you are drug addicts (drug addicts who can hack the pee test).
His reference to private schools teaching students how to hack tests resonates with me as social workers teaching drug users how to hack their pee tests. I take that to mean that if teachers are doing their best but resort to teaching how to hack tests then their needs to be more work done outside of the teacher-student system.
[This is where disparities due to parent's wealth and income come in:] Students who have access to tutors can get the help they in a way having a teacher can't. A teacher can be a tutor, but cannot be a tutor for everyone (an issue with scale).
If the emphasis is on teaching, rather than conformity or daycare, then I suspect teachers being tutors for everyone would actually be more cost-effective !
Observation: creators and maintainers of hackable systems will typically spend a lot of effort to convince you that these systems are not hackable and absolutely reward only "honest effort". Pg's essay meshes very well with this observation.
[deleted]
pg’s articles are all about how to build a great product (Startup=Growth, for example). Are these VC tips-and-tricks secret insider-only info? They don’t publish anything like that
> YCombinator was (in 2008) oriented around learning what magic words to say to investors and exactly when to time techcrunch launch articles, etc. There seemed to be no time given to "how to make a great product”
> YCombinator was (in 2008) oriented around learning what magic words to say to investors and exactly when to time techcrunch launch articles, etc. There seemed to be no time given to "how to make a great product”
Check out their advice on startup school, e.g. the "fundraising" section:
https://www.startupschool.org/library
I've watched many of the videos on YouTube. I would say that each speaker consistently emphasizes the need to "build something users want" before making a big press launch, before applying growth hacking techniques, before talking to investors, etc.
They always say that in the first 5 minutes of the talk, and then go onto more tactical stuff.
They do teach you about hacking investors, or maybe more charitably seeing it from their side, and seeing their motivations.
I think it's true that you can't teach the "build something users want" part, so a lot of the advice is lower level than that.
Advice about building things tends to sound very vague and general, and for every rule there's an exception. Probably the most memorable advice is "live in the future and build what's missing", which I think is true, but hard to implement if you're not the right person.
Whereas the hacking tactics are more specific and actionable (although more likely to be out of date).
https://www.startupschool.org/library
I've watched many of the videos on YouTube. I would say that each speaker consistently emphasizes the need to "build something users want" before making a big press launch, before applying growth hacking techniques, before talking to investors, etc.
They always say that in the first 5 minutes of the talk, and then go onto more tactical stuff.
They do teach you about hacking investors, or maybe more charitably seeing it from their side, and seeing their motivations.
I think it's true that you can't teach the "build something users want" part, so a lot of the advice is lower level than that.
Advice about building things tends to sound very vague and general, and for every rule there's an exception. Probably the most memorable advice is "live in the future and build what's missing", which I think is true, but hard to implement if you're not the right person.
Whereas the hacking tactics are more specific and actionable (although more likely to be out of date).
Thanks for the link looks great
“Startup = Growth” is pure actionable advice and probably the most useful document i’ve ever seen on starting companies (I say that as someone who has founded 3 venture funded startups and two bootstrapped before that)
“Startup = Growth” is pure actionable advice and probably the most useful document i’ve ever seen on starting companies (I say that as someone who has founded 3 venture funded startups and two bootstrapped before that)
The tech industry has unfortunately adopted the methodology of centralized hackable tests as the canonical gatekeeping method in the form of programming interviews.
Most big tech companies don't care about how good you have been at delivering some value through creating software: they want to see you deliver a very specific type of performance at a whiteboard. Interviewers are given specific math puzzle questions to ask. Interviewees are explicitly told by the same companies' hiring departments that they should aim to hack the system by studying books like "Cracking the Code Interview".
This is an industry that prides itself on supposedly making data-driven decisions through A/B testing. When it comes to hiring people to make those decisions, everybody just plays along to a decades-old script.
Most big tech companies don't care about how good you have been at delivering some value through creating software: they want to see you deliver a very specific type of performance at a whiteboard. Interviewers are given specific math puzzle questions to ask. Interviewees are explicitly told by the same companies' hiring departments that they should aim to hack the system by studying books like "Cracking the Code Interview".
This is an industry that prides itself on supposedly making data-driven decisions through A/B testing. When it comes to hiring people to make those decisions, everybody just plays along to a decades-old script.
Being a hiring manager at a big company, I can tell you this is just as frustrating for me as it is for candidates. I hate “leet code” and frankly find algorithmic interviews to be very low signal compared to more practical, open-ended, domain-specific problems.
I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room.
Suppose you give each team high autonomy to hire whoever they like using whatever “good” process they come up with. 90% of the time this results in good hires. But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with.
It becomes a real problem when relatively lower performing people end up concentrated on a team, and then start being the hiring gatekeepers for that team, thus multiplying the number of lower performing hires.
Later you start having institutional problems when everyone starts to perceive that the engineers in Department A are generally better than the engineers in Department B. Engineers in Department A are more likely to leave if they perceive the company is getting worse at engineering - it becomes a self-fulfilling prophecy.
Then you get enormous pressure to come up with standardized testing - aka algorithms on the whiteboard, or some other academic inspired exercise - imposed by higher level leadership that wants to address a genuine problem (skill disparity across the org) but does not know any better way to do it.
I think, as PG points out, there may be a real opportunity to innovate here, and probably a big financial opportunity if anyone can figure out how to productize and scale a solution.
I struggle to see an easy answer, though. In a utopian universe (for a hiring manager) I’d do something like pay candidates to come on site and work for a week, then make a hire/no-hire decision based on that. But I think that is far too onerous for candidates (and a big company) to have legs.
I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room.
Suppose you give each team high autonomy to hire whoever they like using whatever “good” process they come up with. 90% of the time this results in good hires. But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with.
It becomes a real problem when relatively lower performing people end up concentrated on a team, and then start being the hiring gatekeepers for that team, thus multiplying the number of lower performing hires.
Later you start having institutional problems when everyone starts to perceive that the engineers in Department A are generally better than the engineers in Department B. Engineers in Department A are more likely to leave if they perceive the company is getting worse at engineering - it becomes a self-fulfilling prophecy.
Then you get enormous pressure to come up with standardized testing - aka algorithms on the whiteboard, or some other academic inspired exercise - imposed by higher level leadership that wants to address a genuine problem (skill disparity across the org) but does not know any better way to do it.
I think, as PG points out, there may be a real opportunity to innovate here, and probably a big financial opportunity if anyone can figure out how to productize and scale a solution.
I struggle to see an easy answer, though. In a utopian universe (for a hiring manager) I’d do something like pay candidates to come on site and work for a week, then make a hire/no-hire decision based on that. But I think that is far too onerous for candidates (and a big company) to have legs.
I was a hiring manager at a big company for years. We never did coding tests, and I like to think that I made good choices every time. I kept a high-functioning team together, under fairly humble pay, and stressful, sometimes demoralizing, conditions, for decades.
I'm mediocre, at best, at these tests. I don't come from a traditional CS background (started as an EE). I tend to take unusual, hybrid approaches to solving problems; often incorporating elements of new-fangled tech with patterns that have been around for thirty years, and I've found that people get VERY uncomfortable with "thinking outside the box."
I also tend to take some time arriving at the final release, going through iterations of improvement. My first bash is usually fairly naive and clumsy. It works, but not so well. I make each iteration improve on it, maintaining that working software throughout; so there's always something working and testable.
Despite all the aspiration to "disruption," it seems that people don't like to step out of their comfort zones.
Making software that SHIPS is a learned and earned skill, and one that I believe, can be most effectively demonstrated with portfolios.
When a designer goes for a job, they bring with them a large black case. It's filled with their designs and working drafts. Much of the interview consists of the designer going through this portfolio with the hiring manager; discussing each example, and talking about why they did this, or why they didn't do that, etc.
No design manager in their right mind would ignore that portfolio; instead, throwing a matchbook on the interview table, and asking the applicant to "Draw Spunky."
I'm actually shocked that there is so little importance placed on software portfolios. A portfolio represents fairly substantial work product. That's why I find open-source contribution work to be so attractive.
I'm mediocre, at best, at these tests. I don't come from a traditional CS background (started as an EE). I tend to take unusual, hybrid approaches to solving problems; often incorporating elements of new-fangled tech with patterns that have been around for thirty years, and I've found that people get VERY uncomfortable with "thinking outside the box."
I also tend to take some time arriving at the final release, going through iterations of improvement. My first bash is usually fairly naive and clumsy. It works, but not so well. I make each iteration improve on it, maintaining that working software throughout; so there's always something working and testable.
Despite all the aspiration to "disruption," it seems that people don't like to step out of their comfort zones.
Making software that SHIPS is a learned and earned skill, and one that I believe, can be most effectively demonstrated with portfolios.
When a designer goes for a job, they bring with them a large black case. It's filled with their designs and working drafts. Much of the interview consists of the designer going through this portfolio with the hiring manager; discussing each example, and talking about why they did this, or why they didn't do that, etc.
No design manager in their right mind would ignore that portfolio; instead, throwing a matchbook on the interview table, and asking the applicant to "Draw Spunky."
I'm actually shocked that there is so little importance placed on software portfolios. A portfolio represents fairly substantial work product. That's why I find open-source contribution work to be so attractive.
I've often wondered as well why tech companies seem to go out of their way to ignore portfolios. First I thought it's a misguided but honest attempt to make the process look more like a blind meritocracy, to basically turn the interview into a modern version of the Imperial Examination of ancient China.
Then I read "Soul of a New Machine" by Tracy Kidder, a book that documents the design process of a minicomputer in the late '70s. The idolized manager of that project wants to hire young engineers fresh out of school so that he can mold them according to his wishes and have them working all-nighters. If that's your hiring principle, you're not going to get anyone with portfolios.
So I suspect it's an industry sickness that's been turned into a virtue over time.
Then I read "Soul of a New Machine" by Tracy Kidder, a book that documents the design process of a minicomputer in the late '70s. The idolized manager of that project wants to hire young engineers fresh out of school so that he can mold them according to his wishes and have them working all-nighters. If that's your hiring principle, you're not going to get anyone with portfolios.
So I suspect it's an industry sickness that's been turned into a virtue over time.
The benefit of a portfolio for picking a designer is that most often the designer was the only designer on a project, therefore the design can be said to theirs, whereas programmers are seldomly the only programmer on a team and the portfolio can be unclear as to what they contributed.
I don’t think that’s true for typical commercial design work. Clearly explaining what you did as part of the creative team is a major element of presenting the portfolio.
I suppose you're right, I made the mistake of thinking the role of designer in web design was the same all around.
Also there are a lot of good people who can't show you their portfolios beyond a few sentences on a resume. And most places are hiring for group projects, not solo projects where you could make portfolio projects in your spare time.
But everyone can do a whiteboard interview!
But everyone can do a whiteboard interview!
Not everybody. I can DO them, but I pretty much guarantee that I won't do them WELL.
After a couple of these fiascos, I just gave up, and decided to start my own gig. I'm fairly good at what I do, and it really is too bad that it doesn't show in half-hour theory tests.
I do have a pretty massive portfolio, though. Comes from doing open-source software for over twenty years. I was paid to manage; not develop, so I needed to keep my tech chops up in other projects.
I also realize that this is not a typical thing. It's just my journey.
I thought it was funny that I mentioned this once, and someone insisted that I could have faked the portfolio.
If someone can fake ten years' worth of commit history, in dozens of repos, and tens of thousands of lines of code, as well as hundreds of pages of documentation, and dozens of articles, you should hire them immediately, because they are a genius.
After a couple of these fiascos, I just gave up, and decided to start my own gig. I'm fairly good at what I do, and it really is too bad that it doesn't show in half-hour theory tests.
I do have a pretty massive portfolio, though. Comes from doing open-source software for over twenty years. I was paid to manage; not develop, so I needed to keep my tech chops up in other projects.
I also realize that this is not a typical thing. It's just my journey.
I thought it was funny that I mentioned this once, and someone insisted that I could have faked the portfolio.
If someone can fake ten years' worth of commit history, in dozens of repos, and tens of thousands of lines of code, as well as hundreds of pages of documentation, and dozens of articles, you should hire them immediately, because they are a genius.
Your username checks out here.
Maybe I won't become a specifically-software person.
Maybe I won't become a specifically-software person.
Unfortunately portfolios would make finding jobs very difficult for myself and my colleagues. We aren’t really allowed to talk about what we work on outside of what could easily fit in a vague, 1-sentence summary on LinkedIn.
This is actually frustrating me quite a bit because as I look for new jobs people like to ask, in detail, what I worked on, and I can’t really tell them.
This is actually frustrating me quite a bit because as I look for new jobs people like to ask, in detail, what I worked on, and I can’t really tell them.
I completely understand.
That was a big reason that I learned to draw people out when they interviewed. I asked them to "tell me stories" about projects they worked on, without getting detailed about specifics. In my experience, they were always able to demonstrate plenty of enthusiasm and creativity without revealing the family jewels. A half-hour whiteboard test would have been completely useless. I was usually able to establish a fairly comfortable level of tech knowledge fairly quickly, and the bulk of the interview was really about how well I thought they'd fit the team (NOTE TO SELF: Don't go for homogeneity).
However, this isn't a problem that's unique to our industry. My father was in the CIA, and I never found out until just a few years before his death. Lawyers have legal cases they can't discuss, doctors have medical cases they can't discuss in detail, without violation of HIPAA, etc.
Somehow or another, these folks are able to interview for jobs that often have a far greater risk than a line programmer on a major initiative.
True, some are screw-ups, and that doesn't become apparent until after they are hired, but the same goes for an engineer that can ace every problem on HackerRank, but goes all to pieces, when presented with 100 KLoC of spaghetti code, and told to fix a problem (exactly what happened to me, on one of my first programming jobs. 100 KLoC of 1970s-vintage FORTRAN -I fixed the bug, but had to hold my nose, while doing it. I later learned that this was what everyone did).
What bothers me most, is that a chief reason that I'm given for these tests, is that companies want to find people that can come up with innovative and unique solutions, yet they seem to actually have the opposite effect; filtering for people that only come up with standard solutions.
I remember once doing a "take home" test that asked me to apply a third-party library. I did what I always do with dependencies; I encapsulated it, and that seemed to totally freak out the interviewer. To this day, I have no idea why they lost their bottle so badly over it. My solution worked extremely well, and it also gave the problem tremendous "future-proofing." That's why I encapsulate dependencies.
That was a big reason that I learned to draw people out when they interviewed. I asked them to "tell me stories" about projects they worked on, without getting detailed about specifics. In my experience, they were always able to demonstrate plenty of enthusiasm and creativity without revealing the family jewels. A half-hour whiteboard test would have been completely useless. I was usually able to establish a fairly comfortable level of tech knowledge fairly quickly, and the bulk of the interview was really about how well I thought they'd fit the team (NOTE TO SELF: Don't go for homogeneity).
However, this isn't a problem that's unique to our industry. My father was in the CIA, and I never found out until just a few years before his death. Lawyers have legal cases they can't discuss, doctors have medical cases they can't discuss in detail, without violation of HIPAA, etc.
Somehow or another, these folks are able to interview for jobs that often have a far greater risk than a line programmer on a major initiative.
True, some are screw-ups, and that doesn't become apparent until after they are hired, but the same goes for an engineer that can ace every problem on HackerRank, but goes all to pieces, when presented with 100 KLoC of spaghetti code, and told to fix a problem (exactly what happened to me, on one of my first programming jobs. 100 KLoC of 1970s-vintage FORTRAN -I fixed the bug, but had to hold my nose, while doing it. I later learned that this was what everyone did).
What bothers me most, is that a chief reason that I'm given for these tests, is that companies want to find people that can come up with innovative and unique solutions, yet they seem to actually have the opposite effect; filtering for people that only come up with standard solutions.
I remember once doing a "take home" test that asked me to apply a third-party library. I did what I always do with dependencies; I encapsulated it, and that seemed to totally freak out the interviewer. To this day, I have no idea why they lost their bottle so badly over it. My solution worked extremely well, and it also gave the problem tremendous "future-proofing." That's why I encapsulate dependencies.
Re: your point about interviewers looking for their expected/standard answer vs an actually innovative one...
I take that as part of my job in interviewing the the company/interviewers. I will very deliberately give a more radical, non-standard but correct answer precisely to see if they fit into that broken/fixated mindset and see how they deal with being challenged. The dynamic of how they deal with that shows everything about not just how clever they are but how intellectually honest, curious, and emotionally mature they really are.
I take that as part of my job in interviewing the the company/interviewers. I will very deliberately give a more radical, non-standard but correct answer precisely to see if they fit into that broken/fixated mindset and see how they deal with being challenged. The dynamic of how they deal with that shows everything about not just how clever they are but how intellectually honest, curious, and emotionally mature they really are.
Sure you can. You can talk about the hard problems that have nothing to do with the fact that it was part of a missile guidance system or whatever. If you worked on anything non-trivial there are always some interesting problems you had to deal with be they algorithmic, engineering, human, or whatever. If people press for inappropriate details when you've given them plenty of other interesting bits then that's telling you more about them (and whether you want to work with them).
I consult some (maybe a lot). My work product belongs to the client. I am not legally allowed to share.
Some of it I would like to share (and some of it not!), but it really isn't my choice.
(Also - because I am fairly cross-disciplinary, the code for a controller to stand up a 100 ton rig, is not the same as the code to monitor the growth/feed/heat/light for algea in vertical tubes. And neither is likely to be asked of me for projects having to do with creating controllers for missions related to diabetic control or scooping up space debris. The ability to analygize only goes so far.)
Some of it I would like to share (and some of it not!), but it really isn't my choice.
(Also - because I am fairly cross-disciplinary, the code for a controller to stand up a 100 ton rig, is not the same as the code to monitor the growth/feed/heat/light for algea in vertical tubes. And neither is likely to be asked of me for projects having to do with creating controllers for missions related to diabetic control or scooping up space debris. The ability to analygize only goes so far.)
The big problem with using a software portfolio is that plenty of competent engineers don't have impressive work that they can share. If you don't (or can't, because of other obligations like parenting) code in your free time and aren't a recent graduate, you probably don't have much to show.
Of course, difficult whiteboard interviews that require studying to pass have the same problems probably and give less useful information. So, :shrug:.
Of course, difficult whiteboard interviews that require studying to pass have the same problems probably and give less useful information. So, :shrug:.
I know. I'm fortunate to have one. Like I said, I completely realize that it isn't usual for people to have portfolios; especially ones with the scope of mine.
I just feel that it's rather self-destructive to ignore those rare instances where they exist.
I just feel that it's rather self-destructive to ignore those rare instances where they exist.
> I was a hiring manager at a big company for years. We never did coding tests, and I like to think that I made good choices every time.
I don't doubt that you did, but I doubt that everyone at your company places the same effort as you do. At my current company, 90% of interviewers half ass their interviews. They don't have much incentive to care, so you want interview questions with consistent standards and produce some signal with minimal effort. Algo questions fit this bill perfectly.
I don't doubt that you did, but I doubt that everyone at your company places the same effort as you do. At my current company, 90% of interviewers half ass their interviews. They don't have much incentive to care, so you want interview questions with consistent standards and produce some signal with minimal effort. Algo questions fit this bill perfectly.
> 90% of interviewers half ass their interviews
Good interviewing is difficult, it may not necessarily be because they don’t care.
Good interviewing is difficult, it may not necessarily be because they don’t care.
> But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with.
I feel that beyond hiring, our entire society has a problem with tails of distributions. There is this strong movement behind standardization of everything everywhere to minimize the variance, but that not only eliminates both tails, it also lowers the mean. So e.g. here, in order to avoid exceptionally bad hires, companies not only forgo the chance of getting exceptionally good hires, they also lower the quality of a typical hire.
I feel that beyond hiring, our entire society has a problem with tails of distributions. There is this strong movement behind standardization of everything everywhere to minimize the variance, but that not only eliminates both tails, it also lowers the mean. So e.g. here, in order to avoid exceptionally bad hires, companies not only forgo the chance of getting exceptionally good hires, they also lower the quality of a typical hire.
> But as you grow, that ten percent of underperforming people becomes large in absolute numbers, and is very painful to deal with.
I think you could reduce the 10% to 1% simply by actively firing bad employees, and the easiest way to do that is with a probationary hiring period (let's say 6 months, maybe a year).
I am fully aware of how systemic rot can be, and how hard it can be to remove. The real problem is when you hire a bad manager, because they will not only let the rot fester but encourage it to grow.
I think you could wildly relax hiring standards for ICs and be incredibly successful. What we need are better practices around hiring managers more than anything.
I think you could reduce the 10% to 1% simply by actively firing bad employees, and the easiest way to do that is with a probationary hiring period (let's say 6 months, maybe a year).
I am fully aware of how systemic rot can be, and how hard it can be to remove. The real problem is when you hire a bad manager, because they will not only let the rot fester but encourage it to grow.
I think you could wildly relax hiring standards for ICs and be incredibly successful. What we need are better practices around hiring managers more than anything.
Alas, for all of the focus on the interview processes to "prevent" false positive hires, those same companies suck as actually getting rid of the bad hires. Some is rational risk aversion to lawsuits and the like but that clearly bs when you see just how fast people that aren't liked by someone high enough up the food chain are shown the door.
The effect on morale and real productivity of getting rid of the toxic and worst performers is amazing.
The effect on morale and real productivity of getting rid of the toxic and worst performers is amazing.
The first job I had there was a probationary period - it was good and bad. It was good because you knew everybody there was extremely talented and you could rely on your coworkers. There was a bit of an understanding about the cut throat nature of the place though, and obviously that's not for everyone. But at the end of the day, when it came time to just get shit done, there was never any problem.
I can't name that place without doxing myself, but it sounds like Netflix has a similar culture. Some people love it. Some people hate it. But I don't think many people would say they have an issue with employing less-than-stellar employees.
Of course this is only "solving" (it's certainly not perfect) the problem of hiring for engineers, which most of us are on HN (at least I assume, but certainly more engineers than engineering managers). It does nothing to address the issue of hiring bad managers. I have no solution for that, other than to make sure the first manager is excellent. A players hire A players because they want to do the best work they can, and will go out of their way to hire people better than themselves. B players hire C players because they just want to make themselves look good. Once you have a bad manager, the assumption is everyone beneath him will be as bad or worse (obviously there are exceptions).
I can't name that place without doxing myself, but it sounds like Netflix has a similar culture. Some people love it. Some people hate it. But I don't think many people would say they have an issue with employing less-than-stellar employees.
Of course this is only "solving" (it's certainly not perfect) the problem of hiring for engineers, which most of us are on HN (at least I assume, but certainly more engineers than engineering managers). It does nothing to address the issue of hiring bad managers. I have no solution for that, other than to make sure the first manager is excellent. A players hire A players because they want to do the best work they can, and will go out of their way to hire people better than themselves. B players hire C players because they just want to make themselves look good. Once you have a bad manager, the assumption is everyone beneath him will be as bad or worse (obviously there are exceptions).
As I mentioned in a sibling comment, the toxic (and incompetent) manager problem is fundamentally a leadership problem. The single biggest responsibility of the founders is the culture of the organization -- everything stems from that, one way or another.
Leadership, from the top down, is absolutely responsible for the mentoring, training, and environment created by the management. None of the systemic problems with & induced by management is new. It comes down to: does leadership actually care enough to do anything about it? Companies institute all sorts of formal stuff like internal surveys and NPS scoring and so forth but it always comes down to trust & communication and then follow through. And the follow through is the clearest way to create trust. All of the things that even well intentioned leaders spout, if not followed through, quickly erodes and ultimately destroys trust. So, just like with kids, don't make promises you won't keep.
Leadership, from the top down, is absolutely responsible for the mentoring, training, and environment created by the management. None of the systemic problems with & induced by management is new. It comes down to: does leadership actually care enough to do anything about it? Companies institute all sorts of formal stuff like internal surveys and NPS scoring and so forth but it always comes down to trust & communication and then follow through. And the follow through is the clearest way to create trust. All of the things that even well intentioned leaders spout, if not followed through, quickly erodes and ultimately destroys trust. So, just like with kids, don't make promises you won't keep.
Just out of curiosity, why do you make multiple replies instead of one larger reply?
Because you made a variety of specific points and it's easier to address them specifically.
Note that I'm also not talking about companies doing forced rankings and nuking the bottom X%. That's a different form of systemic toxicity and, alas, plays into the technical industry's huge problem with toxic "geek machismo" culture.
Unless you work in a crazy place like Germany with the extremes driven by the work councils, probationary periods are an HR gimmick. If people aren't working out, they can be let go. Have simple, clear, legal policies and simple documentation, WIPs, and most importantly managers who actually care, communicate, and help their team members succeed. The fact is that most companies and managers don't fit those criteria and so create their own hells.
And let's not forget how scant real mentorship and training of managers is in this industry. Nor, how we've setup the power/money hierarchy that many people feel the only career growth path is to become a people manager (even if they aren't any good at it). :-(
Those "bad" managers are correctly following their incentives:
- More headcount is always better for the manager's prestige and career prospects, as long as those people contribute to project success at least a little bit (and most low performers do, just less than you'd hope). Managing infinitely many people each producing epsilon above breakeven is much better than having 1 rockstar.
- Headcount availability ebbs and flows with the business cycle. It's important to take everything you can while the money is flowing in case you might need it during a contraction. It's important to have nonessential staff on hand during a contraction so you can satisfy your X% cut requirement without compromising effectiveness too badly.
- As a manager of a high-performing division at a growing company, you will be given an integer multiple of your current headcount, and expected to deliver proportional results with it. If your strategy was "employ only high performers" you will fail, someone will be brought in immediately above or immediately below you to "help out," and within six months you'll be "spending more time with your family."
- The hiring process is hard to mess with. Recruiters put your ICs onto the panels directly; systematically leaning on them to change their votes will probably get the ethics hotline involved. But the decision to retain year over year is entirely yours.
- More headcount is always better for the manager's prestige and career prospects, as long as those people contribute to project success at least a little bit (and most low performers do, just less than you'd hope). Managing infinitely many people each producing epsilon above breakeven is much better than having 1 rockstar.
- Headcount availability ebbs and flows with the business cycle. It's important to take everything you can while the money is flowing in case you might need it during a contraction. It's important to have nonessential staff on hand during a contraction so you can satisfy your X% cut requirement without compromising effectiveness too badly.
- As a manager of a high-performing division at a growing company, you will be given an integer multiple of your current headcount, and expected to deliver proportional results with it. If your strategy was "employ only high performers" you will fail, someone will be brought in immediately above or immediately below you to "help out," and within six months you'll be "spending more time with your family."
- The hiring process is hard to mess with. Recruiters put your ICs onto the panels directly; systematically leaning on them to change their votes will probably get the ethics hotline involved. But the decision to retain year over year is entirely yours.
Stack ranking was invented for this exact case, cutting the rot out when it's already metastasized.
It has a bad reputation as it's often used year on year - it's meant to be done just once or twice to get back on track after years of over-hiring and reluctant firing.
It has a bad reputation as it's often used year on year - it's meant to be done just once or twice to get back on track after years of over-hiring and reluctant firing.
I’m a pretty good engineer, and I’m never going to join any job with a probationary period. It could turn out to be a bad manager or something and I would end up with a messed up resume. Why would I accept some kind of second-class status when Google etc. are willing to hire me with normal employment terms? I think you’d have a serious adverse selection problem if you only hire people who are willing to take such a job.
I can honestly say the place I worked that had a probationary period had the most talented team of anywhere I have ever been, and I have worked with FAANG before, as well as unicorns.
Like I said, it's not for everyone. A bad manager can ruin your experience anywhere and fire you regardless. That place was just very forthcoming about the fact that you need to demonstrate value if you want to keep your job.
Like I said, it's not for everyone. A bad manager can ruin your experience anywhere and fire you regardless. That place was just very forthcoming about the fact that you need to demonstrate value if you want to keep your job.
On the manager side, I can't upvote that enough.
Toxic systems are always the responsibility of the managers and that's from the top down. The systemically perverse issues created by people "managing up" is one of the most critical problems that people higher up the food chain need to correct for.
Toxic systems are always the responsibility of the managers and that's from the top down. The systemically perverse issues created by people "managing up" is one of the most critical problems that people higher up the food chain need to correct for.
> the easiest way to do that is with a probationary hiring period
How often do you actually boot someone though? For juniors we have pretty low expectations so they never fail this.
How often do you actually boot someone though? For juniors we have pretty low expectations so they never fail this.
People got booted from my first place of work a lot - probably something like 1 in 5 didn't survive the probation period. The idea is to make the probation period long enough that the employee can demonstrate value. If they can't demonstrate value in that time period, they are let go.
This was not the practice where I worked, but you could have different probation periods for different roles. So someone who is primarily focused on technical debt might have a longer probation period than someone focused on rapid feature development.
This was not the practice where I worked, but you could have different probation periods for different roles. So someone who is primarily focused on technical debt might have a longer probation period than someone focused on rapid feature development.
"do something like pay candidates to come on site and work for a week" by doing this you've essentially cut your hiring pool down to 3 categories: new graduates; mediocre engineers who were fired from their last job; or terrible engineers who can't find a job.
Or freelancers who probably aren't looking for full time work anyways! But yeah this is exactly why a weeklong trial won't work
The other option is taking leave. In many countries people might have a week saved up, or maybe they work for one of those fancy companies with Unlimited Leave*
* conditions apply.
* conditions apply.
I would like to see data on if these interview systems actually achieved their goal of rejecting bad candidates.
I worked at a FAANG company and saw several people get let go for underperforming. I have no idea if they could have been productive doing something else but they were clearly not productive at the tasks assigned them or that they chose to do. I also saw several other people I thought were a candidate for going down the same path before I left. Whether they got moved to something they were more productive on or whether they were showing value in other ways I have no idea.
The point being the hiring system didn't reject these people. My guess is they either got lucky like me, easy questions, or they are good at answering the questions but not good in actual production.
For example one person that was let go was clearly smart and could make the code but they would refactor forever looking for perfection and end up taking 4x - 8x longer than others around them. Maybe in some place that's a plus but for our team we needed to ship and this person was not able to prioritize shipping and after several attempts they were let go.
Another had a task that they were taking a long time on. Others on the team gave them the benefit of the doubt that it was harder than it looked but when they were finally let go it turned out it was not harder than it looked and was finished rather quickly.
Another wrote bloated obfusticated code that seemed to be 6x the number of lines it needed to be. Maybe they could have been helped via code review.
Yet another was one I interviewed and they passed the "can write code test" but in the entire time I was there I don't think I saw them submit a single PR. I have no idea what was filling their time.
I don't know if that ratio of unproductive candidates is better or worse than anywhere else I've worked on average. Pretty much every place I worked before FAANG was 100 people or less and my team 30 people or less and those teams were 10 engineers or less. There were always 1 or 2 engineers in the company known for being slow or writing bad designs. Sounds about the same but I didn't measure.
I worked at a FAANG company and saw several people get let go for underperforming. I have no idea if they could have been productive doing something else but they were clearly not productive at the tasks assigned them or that they chose to do. I also saw several other people I thought were a candidate for going down the same path before I left. Whether they got moved to something they were more productive on or whether they were showing value in other ways I have no idea.
The point being the hiring system didn't reject these people. My guess is they either got lucky like me, easy questions, or they are good at answering the questions but not good in actual production.
For example one person that was let go was clearly smart and could make the code but they would refactor forever looking for perfection and end up taking 4x - 8x longer than others around them. Maybe in some place that's a plus but for our team we needed to ship and this person was not able to prioritize shipping and after several attempts they were let go.
Another had a task that they were taking a long time on. Others on the team gave them the benefit of the doubt that it was harder than it looked but when they were finally let go it turned out it was not harder than it looked and was finished rather quickly.
Another wrote bloated obfusticated code that seemed to be 6x the number of lines it needed to be. Maybe they could have been helped via code review.
Yet another was one I interviewed and they passed the "can write code test" but in the entire time I was there I don't think I saw them submit a single PR. I have no idea what was filling their time.
I don't know if that ratio of unproductive candidates is better or worse than anywhere else I've worked on average. Pretty much every place I worked before FAANG was 100 people or less and my team 30 people or less and those teams were 10 engineers or less. There were always 1 or 2 engineers in the company known for being slow or writing bad designs. Sounds about the same but I didn't measure.
> I will say though, the problem is one of “standardization” across an organization where it’s too big for everyone to fit in a room.
I think you've got a lot of this right (disclaimer: we've built the product I think you're describing)
The don't think the most important problem is standardisation though, it's observability/instrumentation ie. if you don't measure what's working, you can't improve things.
The very best tech companies measure quite a lot, and often look back at their hiring processes in the event of a mis-hire to figure out what went wrong and how they can avoid the same happening in future... but even then they only do that in exceptional cases because it's done fairly manually. That means they have low statistical significance and a stuttering cycle of learning.
I believe they should be constantly looking at what's working well, for every hire. So that's what we built.
Once your hiring pipeline is trivially visible, a lot of these questions go away. You can see what's working well and try new things in safety, you can optimise with your eyes wide open.
One thing we did straight away was to deprioritise CVs and replace them with written scenario-based questions relevant to the job. If managed properly that takes your sift stage from a predictive power around r=0.3 to a performance we find typically above r=0.6. Far fewer early false negatives makes your hiring funnel (a) less leaky, (b) more open to pools of talent previously ruled out by clumsy CV sifting, and (c) potentially shorter as the improved sift accuracy allows companies to consider dropping their phone interview stage(s)
Our NPS rating for HR teams is currently running at 85, and MRR churn is under 1% so there's clearly some value to the approach.
I think you've got a lot of this right (disclaimer: we've built the product I think you're describing)
The don't think the most important problem is standardisation though, it's observability/instrumentation ie. if you don't measure what's working, you can't improve things.
The very best tech companies measure quite a lot, and often look back at their hiring processes in the event of a mis-hire to figure out what went wrong and how they can avoid the same happening in future... but even then they only do that in exceptional cases because it's done fairly manually. That means they have low statistical significance and a stuttering cycle of learning.
I believe they should be constantly looking at what's working well, for every hire. So that's what we built.
Once your hiring pipeline is trivially visible, a lot of these questions go away. You can see what's working well and try new things in safety, you can optimise with your eyes wide open.
One thing we did straight away was to deprioritise CVs and replace them with written scenario-based questions relevant to the job. If managed properly that takes your sift stage from a predictive power around r=0.3 to a performance we find typically above r=0.6. Far fewer early false negatives makes your hiring funnel (a) less leaky, (b) more open to pools of talent previously ruled out by clumsy CV sifting, and (c) potentially shorter as the improved sift accuracy allows companies to consider dropping their phone interview stage(s)
Our NPS rating for HR teams is currently running at 85, and MRR churn is under 1% so there's clearly some value to the approach.
If you can't stop bad people from accumulating, you will have a problem no matter how selective is your hiring process.
Can't you just unhire them?
Yes but there's all sorts of silliness that people and companies need to unlearn so they can do this effectively and efficiently.
Firing is too hard for humans, and too easy for bureaucracies.
Hesitation to fire is real.
>But I think that is far too onerous for candidates (and a big company) to have legs.
Actually I did exactly this as a candidate at one place for an afternoon. I left after 2.5 months. :-) In the end I felt a bit sorry for them since they put a lot of effort into the process, being a really small company. On the other hand I really didn't want to stay there. It was technically really interesting and I could contribute but the business was run really badly IMHO
Actually I did exactly this as a candidate at one place for an afternoon. I left after 2.5 months. :-) In the end I felt a bit sorry for them since they put a lot of effort into the process, being a really small company. On the other hand I really didn't want to stay there. It was technically really interesting and I could contribute but the business was run really badly IMHO
it's a really simple problem honestly. Make a standardized test, but don't put an hour time limit on it. If you gave the exact same algorithm test with a 24-hour time limit vs. a 1-hour time limit then had a 1-hour interview explaining the solution to the problem you would be testing for something closer to programming acumen than memorization. I have no idea why tech companies find this so challenging.
The key bit is getting people to explain the answer, not the answer itself.
I have had interviews where the company has done that. I would rate all of those interviews to be very high quality. No nonsense trick/leading questions. Just simple: can this person actually program? What does he understand? What doesn't he understand? I felt like it was obvious what I could and could not do...there was nowhere to hide.
Btw, in terms of investment, this seemed far cheaper for the companies too. What is cheaper? Arranging an hour-long interview where you go through a simple problem that is focused and will clearly identify knowledge. Against an hour-long interview where you probe someone randomly about their life and projects that the interviewer has no idea about (funnily enough, no matter how good you are communicating ideas...interviewers almost always get it wrong).
I have had interviews where the company has done that. I would rate all of those interviews to be very high quality. No nonsense trick/leading questions. Just simple: can this person actually program? What does he understand? What doesn't he understand? I felt like it was obvious what I could and could not do...there was nowhere to hide.
Btw, in terms of investment, this seemed far cheaper for the companies too. What is cheaper? Arranging an hour-long interview where you go through a simple problem that is focused and will clearly identify knowledge. Against an hour-long interview where you probe someone randomly about their life and projects that the interviewer has no idea about (funnily enough, no matter how good you are communicating ideas...interviewers almost always get it wrong).
Cheating? Even with phone screens, some candidates still cheat at these interviews (have some engineer with them answering questions, or looking up similar questions online).
I agree a full blown project over 24 hours would be better, but it's more costly to create questions and score them (if you want to do it in a way where cheating is hard/impossible). I've seen startups do this, and it works well for them, but might not be scalable for companies that hire thousands of employees each year.
I agree a full blown project over 24 hours would be better, but it's more costly to create questions and score them (if you want to do it in a way where cheating is hard/impossible). I've seen startups do this, and it works well for them, but might not be scalable for companies that hire thousands of employees each year.
what even is cheating on an algorithm problem? looking up an answer online? asking someone in your social network to help you work through the problem? because that's what you actually do in the real world. as long as you can explain / defend your solution what does cheating even mean.
That's an idealistic way of looking at things.
People would look-up solutions to that exact problem; it's very hard to come up with a unique problem that was never asked before, and questions leak pretty quickly. (especially at larger companies)
The goal of a test is to evaluate whether you'd be a good employee; I agree algorithmic questions are not representative of day-to-day work, but evaluating your friend's ability to help you is out of scope. Companies want to hire someone that has decent programming skills; you can't rely on other people to solve all your problems, you need to have a minimum level of skill.
[again, playing devil's advocate here; we all agree the process is suboptimal, but let's not ignore the negatives of some of the alternatives suggested here]
People would look-up solutions to that exact problem; it's very hard to come up with a unique problem that was never asked before, and questions leak pretty quickly. (especially at larger companies)
The goal of a test is to evaluate whether you'd be a good employee; I agree algorithmic questions are not representative of day-to-day work, but evaluating your friend's ability to help you is out of scope. Companies want to hire someone that has decent programming skills; you can't rely on other people to solve all your problems, you need to have a minimum level of skill.
[again, playing devil's advocate here; we all agree the process is suboptimal, but let's not ignore the negatives of some of the alternatives suggested here]
how many things did you learn by seeing / being taught the solution and how many did you re-invent as you're basically asked to do at an interview? To give maybe a bad example, I don't know how to do a quicksort. I've written it maybe twice in my life by either being taught by a teacher or by looking it up on wikipedia. I didn't have to think of the solution on the spot. I'm not saying I can't think of solutions but still, the majority of my knowledge comes from having been taught and then using that taught knowledge over and over, not from re-invention.
Note I agree we have the same goals to hire people that can do the work not people whose friends can do the work but I'm not convinced that the typical interview puzzles (find the longest segment of an array that contain 3 values) has anything to do with actual work skills. Maybe it would be better with more typical tasks? If the person claims to be back end engineer ask them to write a query to select all people between the ages of 30 and 35 in Nebraska? Maybe if they are front end ask them to implement a select/option UI?
Note I agree we have the same goals to hire people that can do the work not people whose friends can do the work but I'm not convinced that the typical interview puzzles (find the longest segment of an array that contain 3 values) has anything to do with actual work skills. Maybe it would be better with more typical tasks? If the person claims to be back end engineer ask them to write a query to select all people between the ages of 30 and 35 in Nebraska? Maybe if they are front end ask them to implement a select/option UI?
> Cheating? Even with phone screens, some candidates still cheat at these interviews (have some engineer with them answering questions, or looking up similar questions online).
The only difference between these people and those who pass normally is that the latter group looked up the answers to the problem before you even asked them. Nobody comes up with things like mods of dfs or n-pointer problems on the spot. These problems were the subjects of doctoral dissertations the first times they were solved. We pass them because we've seen problems just like it. The system has been gamed and doesn't work.
The only difference between these people and those who pass normally is that the latter group looked up the answers to the problem before you even asked them. Nobody comes up with things like mods of dfs or n-pointer problems on the spot. These problems were the subjects of doctoral dissertations the first times they were solved. We pass them because we've seen problems just like it. The system has been gamed and doesn't work.
My solution is smaller companies - that way the underperforming ones can be eliminated more easily.
At Amazon, perhaps one of the largest and most successful employers, the solution to this is that when interviewing, everyone's main goal is to hire people better than the current average of the team.
To ensure this, someone experienced with interviewing from outside the team/organization, so called bar raiser, is always added to the interviewing team. This person is there to ensure high quality hires and has hiring veto power.
The rest of the interviewers are from within the team, likely under pressure to compromise on quality when they're desperate enough to hire someone, but they can't do it unless the bar raiser agreed.
To ensure this, someone experienced with interviewing from outside the team/organization, so called bar raiser, is always added to the interviewing team. This person is there to ensure high quality hires and has hiring veto power.
The rest of the interviewers are from within the team, likely under pressure to compromise on quality when they're desperate enough to hire someone, but they can't do it unless the bar raiser agreed.
> At Amazon, perhaps one of the largest and most successful employers, the solution to this is that when interviewing, everyone's main goal is to hire people better than the current average of the team.
At this point it's Kool-aid. In practice bar raisers have so much more interview experience that you can very easily pick them out, but so much of their behavior results in simply weeding out the bad.
I have received LOTS of offers from Amazon over more than a decade (only one accepted at one point), and my friend used to be a bar raiser. In his first interview at a new company he was both rude to an interviewee and flat-out wrong technically on multiple points along with it.
You can't raise the bar if you can't pick the bar raisers.
At this point it's Kool-aid. In practice bar raisers have so much more interview experience that you can very easily pick them out, but so much of their behavior results in simply weeding out the bad.
I have received LOTS of offers from Amazon over more than a decade (only one accepted at one point), and my friend used to be a bar raiser. In his first interview at a new company he was both rude to an interviewee and flat-out wrong technically on multiple points along with it.
You can't raise the bar if you can't pick the bar raisers.
FWIW this wasn't my experience when interviewing with Amazon; I felt the bar was much lower than other companies I interviewed with, even when I wasn't that good at algorithmic problems.
That's just one data point, and maybe what you're describing is for more senior hires?
That's just one data point, and maybe what you're describing is for more senior hires?
I've found a generally higher bar than places like Microsoft but definitely a much lower bar than other FAANG companies, unicorns, and even tiny-but-potentially-elite startups run by people who know what they're doing.
Agreed, Microsoft was an even lower bar... but in a good way: I felt they just wanted someone with basic CS knowledge, whereas the Amazon interviewers seemed to think they were some sort of elite group of engineers.
A federation.
To be honest, if a large company has multiple departments that essentially govern themselves, perhaps they could just give them proper names, as if they were subsidiary companies? Because it seems to me that the mechanism behind the "best performers leave when they feel the average performance of workers in the company goes down" is really bounded by the company name. I.e. people in department of frobnification leave because of the people in department of foobing bars. But people in Frobnificator perhaps wouldn't leave because of people in FooBars.io, even if legally/administratively the two situations were equivalent.
To be honest, if a large company has multiple departments that essentially govern themselves, perhaps they could just give them proper names, as if they were subsidiary companies? Because it seems to me that the mechanism behind the "best performers leave when they feel the average performance of workers in the company goes down" is really bounded by the company name. I.e. people in department of frobnification leave because of the people in department of foobing bars. But people in Frobnificator perhaps wouldn't leave because of people in FooBars.io, even if legally/administratively the two situations were equivalent.
No, it's not that. It's that often, to make a new product you have to have multiple parts of the business work together. Then you will find yourself working with the other department and they'll frustrate you.
Some places attempt to solve this by saying that you can use whatever vendors you need to solve the problem but often the best place to solve it is the other department since the solution is infeasible without them.
The problem is that often the solution is infeasible with them because the guys who made them good have left and now you only have the lameos.
As to why they can't operate as independent business unit? Well, there's the Theory of the Firm to tell you why that doesn't work often.
Some places attempt to solve this by saying that you can use whatever vendors you need to solve the problem but often the best place to solve it is the other department since the solution is infeasible without them.
The problem is that often the solution is infeasible with them because the guys who made them good have left and now you only have the lameos.
As to why they can't operate as independent business unit? Well, there's the Theory of the Firm to tell you why that doesn't work often.
The tech industry has unfortunately adopted the methodology of centralized hackable tests
I think the opposite is the case. Every industry has some sort of test to get into it. But the other industries are the most centralized and hackable ones. The tech industry is doing the best job of any of the top industries of having an application process that is open to skilled people who don't match the "centralized" standard.
Your grades in college, the experience on your resume, those are "centralized" tests. You have one resume and you send it out to everyone.
When companies give programming tests to applicants, each company is free to measure the applicants' skill in any way they see fit. One might ask math puzzle questions, but another company might give take-home Rails projects. And companies are free to just do a "soft" interview and look at your resume if they want.
Facebook hires far more software engineers with no software engineering education than Chevron hires chemical engineers with no chemical engineering education.
Plus, starting a company is open to anyone with a credit card to open an AWS account. No interview required. Just make something people want.
I think the opposite is the case. Every industry has some sort of test to get into it. But the other industries are the most centralized and hackable ones. The tech industry is doing the best job of any of the top industries of having an application process that is open to skilled people who don't match the "centralized" standard.
Your grades in college, the experience on your resume, those are "centralized" tests. You have one resume and you send it out to everyone.
When companies give programming tests to applicants, each company is free to measure the applicants' skill in any way they see fit. One might ask math puzzle questions, but another company might give take-home Rails projects. And companies are free to just do a "soft" interview and look at your resume if they want.
Facebook hires far more software engineers with no software engineering education than Chevron hires chemical engineers with no chemical engineering education.
Plus, starting a company is open to anyone with a credit card to open an AWS account. No interview required. Just make something people want.
Haha here is the email i have from my facebook interview thats coming up.
"Your initial Facebook interview is coming up and we want you to ace it!
Here are some tips and resources so you know what to expect and can prepare adequately. Preparing is key.
Our initial interview determines whether to continue with a full series of onsite interviews. This initial interview is primarily a coding interview that will take place between you and a Facebook engineer.
It's important for any engineer to brush up on their interview skills, coding skills and algorithms. Practice coding on a whiteboard or with pen and paper, and time yourself. Preparing increases your odds significantly! Below are two helpful links to a Facebook Interview Prep Course led by Gayle Laakmann McDowell, author of “Cracking the Coding Interview”. Use password FB_IPS to access the videos. Cracking the Facebook Coding Interview - The Approach Cracking the Facebook Coding Interview - Problem Walk-Through This article provides much more advice about how to prepare: Preparing for your Software Engineering Interview at Facebook
Here's a sample problem to get started.
Sample Problem Write a function to return if two words are exactly "one edit" away, where an edit is: Inserting one character anywhere in the word (including at the beginning and end) Removing one character Replacing exactly one character Most importantly, you can view this and other Facebook sample interview questions and solutions here.
Want more sample questions? Try HackerRank, LeetCode, and CodeLab. Be sure to practice questions in a variety of subjects and difficulty levels."
"Your initial Facebook interview is coming up and we want you to ace it!
Here are some tips and resources so you know what to expect and can prepare adequately. Preparing is key.
Our initial interview determines whether to continue with a full series of onsite interviews. This initial interview is primarily a coding interview that will take place between you and a Facebook engineer.
It's important for any engineer to brush up on their interview skills, coding skills and algorithms. Practice coding on a whiteboard or with pen and paper, and time yourself. Preparing increases your odds significantly! Below are two helpful links to a Facebook Interview Prep Course led by Gayle Laakmann McDowell, author of “Cracking the Coding Interview”. Use password FB_IPS to access the videos. Cracking the Facebook Coding Interview - The Approach Cracking the Facebook Coding Interview - Problem Walk-Through This article provides much more advice about how to prepare: Preparing for your Software Engineering Interview at Facebook
Here's a sample problem to get started.
Sample Problem Write a function to return if two words are exactly "one edit" away, where an edit is: Inserting one character anywhere in the word (including at the beginning and end) Removing one character Replacing exactly one character Most importantly, you can view this and other Facebook sample interview questions and solutions here.
Want more sample questions? Try HackerRank, LeetCode, and CodeLab. Be sure to practice questions in a variety of subjects and difficulty levels."
> Write a function to return if two words are exactly "one edit" away, where an edit is: Inserting one character anywhere in the word (including at the beginning and end) Removing one character Replacing exactly one character
I've been writing code my entire life, and I've been employed as a software developer for longer than I'd care to admit. I don't have a quick answer to that problem. I know there's an algorithm for computing the "distance" between two strings, so this is probably about that. I have no idea how to write it, and it would never come up in my job, and if it did I'm sure I'd import a library to do it.
After searching it, this is what they're talking about https://en.m.wikipedia.org/wiki/Levenshtein_distance
That means my only chance of getting this problem correct was either:
1) I prepared for it by studying the bank of questions and memorized the formula for Levenshtein distance.
or
2) I considered this problem for the first time during the interview, was immediately struck by the same insight that mathematician Vladimir Levenshtein had in 1965, and developed the algorithm on a white board in real-time.
At least they're transparent that if I do lots of HackerRank, LeetCode, and CodeLab then I'll be able to pass their test. I suppose that's better than only those "in the know" realizing how to be employed at FB.
I really hope I never have to do another job interview again.
I've been writing code my entire life, and I've been employed as a software developer for longer than I'd care to admit. I don't have a quick answer to that problem. I know there's an algorithm for computing the "distance" between two strings, so this is probably about that. I have no idea how to write it, and it would never come up in my job, and if it did I'm sure I'd import a library to do it.
After searching it, this is what they're talking about https://en.m.wikipedia.org/wiki/Levenshtein_distance
That means my only chance of getting this problem correct was either:
1) I prepared for it by studying the bank of questions and memorized the formula for Levenshtein distance.
or
2) I considered this problem for the first time during the interview, was immediately struck by the same insight that mathematician Vladimir Levenshtein had in 1965, and developed the algorithm on a white board in real-time.
At least they're transparent that if I do lots of HackerRank, LeetCode, and CodeLab then I'll be able to pass their test. I suppose that's better than only those "in the know" realizing how to be employed at FB.
I really hope I never have to do another job interview again.
You don’t need edit distance. The solution is literally a for loop and three if statements. The fact that you can’t stop and think for a few seconds about how you might solve this means you’ve been on autopilot for many years now. This is a proxy test to weed out people who work on autopilot and never think.
Are you proposing brute forcing it? Depending on requirements that might be fine, but I doubt FB would like that kind of solution. If they did, they'd have a different kind of question.
This isn't "brute force." It's solving the specific problem posed rather than a generalization, which in this case is both easier and more efficient (since there are cases in which you could terminate the loop early).
Thanks! I completely misunderstood what the commentator I replied to was saying. Not understanding how to solve it well I assumed they were referring the first option with loops and tests that popped into my head: make every one-edit possible and see if it's one of them.
1st for loop with first word with charecter counts map and second for loop subtracting counts from frist map. See whats remaining in the end, either a map with 1 char left or 1 char left in second. so O(m + n).
abc , adc
a - 1
b - 1
c - 1
after second word loop
b - 1, d
abc , adc
a - 1
b - 1
c - 1
after second word loop
b - 1, d
I might be misunderstanding, but doesn't that treat 'abc' and 'cba' as distance 0?
Regardless, I was also thinking Levenshtein is sub-optimal, and that you can probably solve it in O(n+m).
Even if you go for Levenshtein first, you should identify a simple modification to stop calculating the matrix early if the distance is necessarily greater than 1. Get a bit fancier and you can just 'go down the diagonal' and greatly optimize the algorithm.
Regardless, I was also thinking Levenshtein is sub-optimal, and that you can probably solve it in O(n+m).
Even if you go for Levenshtein first, you should identify a simple modification to stop calculating the matrix early if the distance is necessarily greater than 1. Get a bit fancier and you can just 'go down the diagonal' and greatly optimize the algorithm.
ah yea, sorry i misread the question.
That only works up to permutations.
Many of these sorts of problems seem intimidating at first, but if you’ve had any training / exposure to implementing recursive solutions (i.e. a bit of SICP), the solutions come much easier because you are trained to think in terms of “what is the tiniest amount of work I can do to advance the problem by one step?”
In this case, all we need at each step is to look at one character plus one lookahead character from each string. We’ll call them a and b for string 1 and c and d for string 2.
If a and c are equal, advance.
If a and d are equal, delete c and advance.
If b and c are equal, delete a and advance.
If b and d are equal, change one of a or c so they match, then advance.
Once you use up an edit, the algo changes to “a and c must match” for the rest of the string.
Edit: terminate the algo when one of a or c is a NULL char. If they are both NULL, the string passes. If only one is NULL, then your one edit allowance was not enough to make the strings equal.
Also, rather than “deleting” a char, it is sufficient to simply advance that pointer an extra step.
In this case, all we need at each step is to look at one character plus one lookahead character from each string. We’ll call them a and b for string 1 and c and d for string 2.
If a and c are equal, advance.
If a and d are equal, delete c and advance.
If b and c are equal, delete a and advance.
If b and d are equal, change one of a or c so they match, then advance.
Once you use up an edit, the algo changes to “a and c must match” for the rest of the string.
Edit: terminate the algo when one of a or c is a NULL char. If they are both NULL, the string passes. If only one is NULL, then your one edit allowance was not enough to make the strings equal.
Also, rather than “deleting” a char, it is sufficient to simply advance that pointer an extra step.
[deleted]
Look up "edit distance"... it is one of the first problems an Algorithms course will discuss that can be solved by dynamic programming. But ya, no way anyone is coming up with that unless they have seen it before.
I 100% would be able to solve this problem without seeing it before. Yes, this is because I've practiced a lot of algorithm problems (and I enjoy them, frankly).
Yes, this is hackable, but relying on resumes like every other industry (which is implicitly relying on hoop-jumping and college prestige and GPA) is a much worse form of hackable. This levels the playing field and is a signal for (1) how smart someone is and (2) how much they prepared for the interview, both of which are weakly related to the job. Further, people who excel at algorithm are very unlikely to be bad at the kind of work at big tech.
It's not perfect, but finding something better at scale is hard.
Yes, this is hackable, but relying on resumes like every other industry (which is implicitly relying on hoop-jumping and college prestige and GPA) is a much worse form of hackable. This levels the playing field and is a signal for (1) how smart someone is and (2) how much they prepared for the interview, both of which are weakly related to the job. Further, people who excel at algorithm are very unlikely to be bad at the kind of work at big tech.
It's not perfect, but finding something better at scale is hard.
I don't think it is a proxy to how smart someone is. I will take point (2) tho.
An algorithm for "do these strings differ by an edit distance of at most 1?" is much simpler, and should be easy to figure out. No dynamic programming required.
This is something I would expect a freshman CS major to be able to solve.
It only requires knowledge of looping, conditionals, and how to work with strings in your language (e.g, indexing, finding length, maybe substrings, etc.).
This is like a less insulting version of fizzbuzz.
People are way overcomplicating this. Any professional programmer could solve this.
It only requires knowledge of looping, conditionals, and how to work with strings in your language (e.g, indexing, finding length, maybe substrings, etc.).
This is like a less insulting version of fizzbuzz.
People are way overcomplicating this. Any professional programmer could solve this.
This is an initial phone screen and in this context this a valuable approach. This is one level past fizzbuzz. 'Can you code something past "hello world"?' is what they're looking for. There's nothing worse than an onsite with somebody who just cannot code at all.
The particular example is bad, though, because there’s room for a massive algorithmic insight. There’s an obvious-enough but terribly inefficient way to do it, and a fancy efficient algorithm that either you know or you don’t. If given this question I would middle through the inefficient way, aware that there must be a better algorithm out there that’s eluding me, and I’d probably get a “vaguely pass” mark, but the kid fresh out of college who has spent the last three months doing leetcode study will regurgitate the correct solution and pass with flying colours.
(Once in a while maybe a complete genius takes the test and reinvents the good algorithm on the spot, but only gets the same grade as the diligent student, for his effort.)
Too many basic coding screens seem to rely on some algorithmic insight. I would rather give a candidate a complicated-but-straightforward problem (like a super-fizzbuzz that takes half a page to describe,with a bunch of complicated rules and exceptions to those rules) because that’s closer to what a real world programming task looks like. No room for a clever insight, just translate the problem statement, carefully, into code, and test whether it runs correctly.
(Once in a while maybe a complete genius takes the test and reinvents the good algorithm on the spot, but only gets the same grade as the diligent student, for his effort.)
Too many basic coding screens seem to rely on some algorithmic insight. I would rather give a candidate a complicated-but-straightforward problem (like a super-fizzbuzz that takes half a page to describe,with a bunch of complicated rules and exceptions to those rules) because that’s closer to what a real world programming task looks like. No room for a clever insight, just translate the problem statement, carefully, into code, and test whether it runs correctly.
> because that’s closer to what a real world programming task looks like
To make it even closer, you'd have to have people coming in at various points with contradictory updates to the task, impossible requests, an overbearing managerial diktat, etc.
To make it even closer, you'd have to have people coming in at various points with contradictory updates to the task, impossible requests, an overbearing managerial diktat, etc.
Actual questions in phonescreen i got were these
https://leetcode.com/problems/powx-n/
https://leetcode.com/problems/product-of-array-except-self/
https://leetcode.com/problems/powx-n/
https://leetcode.com/problems/product-of-array-except-self/
For the second one:
> Please solve it without division
This is just an absurd constraint…
> Please solve it without division
This is just an absurd constraint…
That's good, they're trying to put everyone in a level play field given the system they use.
I think I blame Google, because it was started by a couple of graduate students.
I don't know how it was at Stanford, but at Berkeley in Physics there was a series of big honking tests ("Qualifying Exams") that dominated our attention the first two years. You were expected to know all of undergraduate Physics. Fail it, and you're out.
Grad students don't know any better. "How do we hire the best people? I know - they'll have to pass a big hard test, just like we did."
Then everyone else goes, "How do we hire the best people? I know - we'll do it like Google does, they seem pretty successful."
Ugh.
No, I don't know that it really went like that. I just suspect it.
I don't know how it was at Stanford, but at Berkeley in Physics there was a series of big honking tests ("Qualifying Exams") that dominated our attention the first two years. You were expected to know all of undergraduate Physics. Fail it, and you're out.
Grad students don't know any better. "How do we hire the best people? I know - they'll have to pass a big hard test, just like we did."
Then everyone else goes, "How do we hire the best people? I know - we'll do it like Google does, they seem pretty successful."
Ugh.
No, I don't know that it really went like that. I just suspect it.
Stanford physics PhD student here. We actually don't have qualifying exams, but my unpopular opinion is that we should. Physics is a lot of fun to study. The undergraduate and core graduate curriculum show off all its greatest hits. Why not learn that by heart? I spent two years doing that, it pays off every day in research, and I'd have been happy even if it didn't.
There are alternatives but they are not necessarily better. E.g. academia seems to rely on references and publications for hiring. This seems closer to your wish of caring about "how good you have been". But consider the downsides - it is inherently more insular (to work for a famous professor you need a reference from someone, preferably also a famous professor) and also your thesis advisor can easily ruin your whole career.
Contrast that with our industry where a bright kid out of nowhere can study for the (imperfect, hackable, dehumanizing - I agree!) interview and have a realistic shot at that job at FAANG. At least I wish it still works this way, although of course it helps to be a Stanford graduate!
Contrast that with our industry where a bright kid out of nowhere can study for the (imperfect, hackable, dehumanizing - I agree!) interview and have a realistic shot at that job at FAANG. At least I wish it still works this way, although of course it helps to be a Stanford graduate!
When I started preparing for interviewing, I spent most of my times studying general data structures & algorithms and practicing applying them to different problems. I don't think this is so bad and can lead to a lot of learning you wouldn't do otherwise.
The frustrating thing, is how easy it is to hack the test. After one particular interview, I remember talking to a friend about how hard the technical interview was. He told me that he had already seen the problem and knew it would come up because he bought leetcode premium for the interview. Kind of frustrating to spend hours and hours learning data structures and algorithms when the real key to success is getting lucky and memorizing the problem before hand
The frustrating thing, is how easy it is to hack the test. After one particular interview, I remember talking to a friend about how hard the technical interview was. He told me that he had already seen the problem and knew it would come up because he bought leetcode premium for the interview. Kind of frustrating to spend hours and hours learning data structures and algorithms when the real key to success is getting lucky and memorizing the problem before hand
Leetcode premium doesn't mean you'll see the problem. I suspect the probability of that is low, given these problems are constantly changing (there's a banned problem list at Google) and the interviewer pool is as well.
This is exactly what I said in my retweet to PG's essay. Having hackable bad tests in the very tech industry proves the point that artificial tests are still the way of thinking for many and something we collectively need to unlearn. The question is how.
When I see a company conducting these kinds of tests during interviews, it's a signal to me that they are probably not a very forward-thinking company, and probably not a good fit for someone like me who tends to think outside the box. If it's another cog in their machine that they want, these kinds of interview tests are probably pretty good. But if they're an innovative company trying to change the world, they'd be much better off with a different, more pragmatic approach designed to attract the independent thinkers.
Beyond that, in my opinion, nearly anyone with average programming experience can hack a typical programming/whiteboard interview, so these kinds of interviews are definitely not the best way to find the best fit for particular roles. It's a good way to filter out the inexperienced, for sure, but it's a pretty low bar to set, especially if it's a company looking for top tier engineers, people with the ability to see the bigger picture.
It's also likely that a great engineer would pass the test with ease, but because their skillset and abilities have been inaccurately assessed due to the poor interview process, the company could have initially assigned them a much more effective role where both the company and the employee could have benefited to a much greater extent, but didn't. Overall, relatively speaking, the "hackable test" approach wastes time and hurts everyone in the long run, both employers and employees. A little investment up front with more personalized, specific interviews can go a long way.
I think we should start refusing these impractical interview processes, or at the very least, from an interviewee standpoint, turn down jobs (if you can) and let the potential employer know that their interview process is the reason.
Beyond that, in my opinion, nearly anyone with average programming experience can hack a typical programming/whiteboard interview, so these kinds of interviews are definitely not the best way to find the best fit for particular roles. It's a good way to filter out the inexperienced, for sure, but it's a pretty low bar to set, especially if it's a company looking for top tier engineers, people with the ability to see the bigger picture.
It's also likely that a great engineer would pass the test with ease, but because their skillset and abilities have been inaccurately assessed due to the poor interview process, the company could have initially assigned them a much more effective role where both the company and the employee could have benefited to a much greater extent, but didn't. Overall, relatively speaking, the "hackable test" approach wastes time and hurts everyone in the long run, both employers and employees. A little investment up front with more personalized, specific interviews can go a long way.
I think we should start refusing these impractical interview processes, or at the very least, from an interviewee standpoint, turn down jobs (if you can) and let the potential employer know that their interview process is the reason.
> Beyond that, in my opinion, nearly anyone with average programming experience can hack a typical programming/whiteboard interview
I agree with your conclusion that all it does is filter out inexperienced/incompetent candidates. This is valuable, but to me it's the "first question" - can they code?
I don't think there's really much value between an "Amazing coder" and a "can get the job done" coder. Very rapidly you're trying to answer much softer questions like "does this person work well others?" or "how fast can they learn new systems?" and there isn't a good way to do that.
The interview question I usually give is algorithmically very simple. There's no linked lists, no graph theory, none of that. What I actually look for is mostly how they interact with me and how they go about solving it.
There's a pretty straightforward gotcha that more or less everyone hits. If the candidate hits the gotcha and then keeps piling on special cases without ever taking a step back, that's a big minus. If they keep doing it even after I suggest there's a simpler solution, that's worse still. I much prefer somebody who takes a step back but can't quite figure it out without a little more help than one who just remains confident in their solution no matter what.
Yeah it's inherently shitty that I have 45 minutes to figure out if this person will benefit the org for years to come, but there's a lot more signal there than just "Did they open their CS 201 algorithms textbook in the last few week?"
I agree with your conclusion that all it does is filter out inexperienced/incompetent candidates. This is valuable, but to me it's the "first question" - can they code?
I don't think there's really much value between an "Amazing coder" and a "can get the job done" coder. Very rapidly you're trying to answer much softer questions like "does this person work well others?" or "how fast can they learn new systems?" and there isn't a good way to do that.
The interview question I usually give is algorithmically very simple. There's no linked lists, no graph theory, none of that. What I actually look for is mostly how they interact with me and how they go about solving it.
There's a pretty straightforward gotcha that more or less everyone hits. If the candidate hits the gotcha and then keeps piling on special cases without ever taking a step back, that's a big minus. If they keep doing it even after I suggest there's a simpler solution, that's worse still. I much prefer somebody who takes a step back but can't quite figure it out without a little more help than one who just remains confident in their solution no matter what.
Yeah it's inherently shitty that I have 45 minutes to figure out if this person will benefit the org for years to come, but there's a lot more signal there than just "Did they open their CS 201 algorithms textbook in the last few week?"
> it's a signal to me that they are probably not a very forward-thinking company
Can you give me an example of a forward thinking company according to you?
Can you give me an example of a forward thinking company according to you?
>When I see a company conducting these kinds of tests during interviews, it's a signal to me that they are probably not a very forward-thinking company,
Can you list all 'forward thinking' companies for the benefit of all of us?
Can you list all 'forward thinking' companies for the benefit of all of us?
I agree with all you said. However, 99% of major tech companies employ exactly the same process of hiring software engineers. I've been on hundreds of interview loops and unfortunately just one bad interview in a loop of 5-6 could make the candidate look like a "bad" fit and thus receive no offer. That particular candidate might be in fact great fit, but the way he was tested could not reveal that.
I don't know the solution to this problem. Eliminating tests whatsoever and just talking with the candidate about his experience and probing his knowledge on different topics is not efficient either. There are a lot of talking heads around who when given a simple task fail miserably.
We as a tech community need to come up with better ways to assess other people's competency while also making sure those people fit within our company's culture, work efficiently with others and after all create value. This is a hard problem. So, we try to simplify the problem by imposing the "proven" way of finding such people -- give them arbitrary tests and hope they pass them.
I don't know the solution to this problem. Eliminating tests whatsoever and just talking with the candidate about his experience and probing his knowledge on different topics is not efficient either. There are a lot of talking heads around who when given a simple task fail miserably.
We as a tech community need to come up with better ways to assess other people's competency while also making sure those people fit within our company's culture, work efficiently with others and after all create value. This is a hard problem. So, we try to simplify the problem by imposing the "proven" way of finding such people -- give them arbitrary tests and hope they pass them.
loh, Can you list all 'forward thinking' companies that you have dealt in the past? I'm sure there will also be other people interested.
I work at a very large company (IBM) and I refuse to interview people that way. My track record for great hires (both full-timers and interns) is arguably impeccable. No whiteboard or brain teaser quizzes involved. I wrote a little about my interviewing approach in my latest "hiring" post: https://programmingzen.com/new-ibm-internship-positions-in-m...
Software Engineering interviews are the worst offenders of this “test for grades approach to hire candidates”. On that other hand, programmers are spending huge amount of time, money and energy into hacking these coding interviews just to get a job. Situation is horrible and needs immediate disruption.
As PG implies, the work environment at big companies consists of more hackable tests - so (ironically) that type of interview might be a proper test.
I loathe all types of coding or design challenges when interviewing! I once designed and coded a five page website.. it took awhile, yet never heard back from the company. Thus, Im going to seek out all opportunities that do not force me to waste my time, especially design as it's subject. Overall you liked my portfolio well enough to consider me then lets chat and see if we jive/im a good fit for you/your team/company!
So far I've been very fortunate that recruiters reach out on LinkedIn and I rarely have to do such time wasting activities as they vouch for me.
Recently dealt with a company who pays parity (everyone makes the same .. no woman, man, etc can negotiate their worth), had five to 6 interviews and 2 to 3 coding/design challenges. WoW I guess they are looking only for a subset of talent who would do all that. People might do that for a FAANg company or when jobs in the field are scarce, but this was no FAANg company and thankfully there is still a good amount of demand!
So far I've been very fortunate that recruiters reach out on LinkedIn and I rarely have to do such time wasting activities as they vouch for me.
Recently dealt with a company who pays parity (everyone makes the same .. no woman, man, etc can negotiate their worth), had five to 6 interviews and 2 to 3 coding/design challenges. WoW I guess they are looking only for a subset of talent who would do all that. People might do that for a FAANg company or when jobs in the field are scarce, but this was no FAANg company and thankfully there is still a good amount of demand!
There was a time where computers weren't as fast and libraries were not as high-level/user friendly, and you required to know these things in order to get things done.
Today, unless you are doing things at scale (tiny fraction of startups), you don't need to know how to make things run in the most optimized way possible.
Today, unless you are doing things at scale (tiny fraction of startups), you don't need to know how to make things run in the most optimized way possible.
I think testing algorithmic questions is certainly evil, but it's a necessary evil because all other methods are either too time-consuming or unrealistic. You wouldn't believe in a big company how many clearly unqualified applicants there are, and there has to be some quick and dirty rule to run a filter through them. Perhaps it's the aura of FANG companies that make unqualified applicants try because they have nothing to lose. My experience is that just about 1% of interviewees can actually confidently write a depth first search and explain its time complexity. If they can't even do that (which is basically among the first things schools teach in a CS curriculum), how do you trust they that they have the skills to design and implement even more complicated systems?
> I think testing algorithmic questions is certainly evil, but it's a necessary evil because all other methods are either too time-consuming or unrealistic.
> write a depth first search and explain its time complexity.
Part of our test goes "here's a function, in plain English what does it do?", and then "there's a few bugs in this implementation, what are they and how can they be fixed?"
I think there's a lot of alternatives to traditional algorithmic questions that just don't usually get considered. This one I liked because it involves things you'd actually do day-to-day: Reading other people's code, figuring out intent, and finding/fixing bugs.
> write a depth first search and explain its time complexity.
Part of our test goes "here's a function, in plain English what does it do?", and then "there's a few bugs in this implementation, what are they and how can they be fixed?"
I think there's a lot of alternatives to traditional algorithmic questions that just don't usually get considered. This one I liked because it involves things you'd actually do day-to-day: Reading other people's code, figuring out intent, and finding/fixing bugs.
I've never done that (self-taught dev), but I've set up SSO, terraformed production systems, wrote ETLs for system critical data... I'm not an amazing developer, but I get things done. That test would filter me out simply because I've never tried it before.
I totally understand. Different job expectations. Whenever I tell people I actually need to write dynamic programming in my job people are always incredulous. To many, dynamic programming is a technique they learn only to pass interviews and not use in real life. It's really that if you don't know certain algorithmic stuff, you can't actually work on the lowest levels of the stack, but not all companies expect their developer to do that.
> My experience is that just about 1% of interviewees can actually confidently write a depth first search and explain its time complexity. If they can't even do that (which is basically among the first things schools teach in a CS curriculum), how do you trust they that they have the skills to design and implement even more complicated systems?
My experience with former BigCo engineers I've had to work with is that 1% of interviewees can actually confidently design a relational data model from scratch for a weekend MVP of reasonably any major web service you could think of. If they can't even do that (which is basically among the first things you do as an engineer standing up a service from scratch), how do you trust that they have the skills to design and implement even more complicated systems?
See, I just took what you said and replaced it with a couple of key phrases. But the truth is, when it comes to designing and implementing even more complicated systems, my passage (while reductive and still arguably wrong) makes a lot more sense than yours. I'm not saying this to pick on you, just to say that you probably want to revise what you say at the very end. Instead of "how do you trust they that they have the skills to design and implement even more complicated systems?" you should really be phrasing what you're looking for as "How do you trust that they have the skills to maintain and build efficient modules at Google scale?" because there are plenty of "even more complicated systems" The truth is that the architectures that consist of the grunt work of what you do scaling a company from $0B to $100M or $100M to $1B or $1B to $10B that are by and far more important than a depth first search. The fact that you leapt to do that rather than something more widely applicable shows exactly what's wrong with this mentality. It's not that you're incorrect; you're not even wrong. You never got to the point of knowing what the right question is.
My experience with former BigCo engineers I've had to work with is that 1% of interviewees can actually confidently design a relational data model from scratch for a weekend MVP of reasonably any major web service you could think of. If they can't even do that (which is basically among the first things you do as an engineer standing up a service from scratch), how do you trust that they have the skills to design and implement even more complicated systems?
See, I just took what you said and replaced it with a couple of key phrases. But the truth is, when it comes to designing and implementing even more complicated systems, my passage (while reductive and still arguably wrong) makes a lot more sense than yours. I'm not saying this to pick on you, just to say that you probably want to revise what you say at the very end. Instead of "how do you trust they that they have the skills to design and implement even more complicated systems?" you should really be phrasing what you're looking for as "How do you trust that they have the skills to maintain and build efficient modules at Google scale?" because there are plenty of "even more complicated systems" The truth is that the architectures that consist of the grunt work of what you do scaling a company from $0B to $100M or $100M to $1B or $1B to $10B that are by and far more important than a depth first search. The fact that you leapt to do that rather than something more widely applicable shows exactly what's wrong with this mentality. It's not that you're incorrect; you're not even wrong. You never got to the point of knowing what the right question is.
> which is basically among the first things schools teach in a CS curriculum
I'm pretty sure that didn't come up until at least my 2nd or 3rd year - which was 25+ years ago (and I've never had to implement it by hand since.) You seem to be biasing your selection criteria towards "new/very recent graduates"?
I'm pretty sure that didn't come up until at least my 2nd or 3rd year - which was 25+ years ago (and I've never had to implement it by hand since.) You seem to be biasing your selection criteria towards "new/very recent graduates"?
Sure, but what's the alternative?
I know big companies do a lot of AB testing with interviews and change their approach over time (Google stooped asking why are manholes round and looking at AST scores because data showed them those things had almost zero correlation with performance).
I know big companies do a lot of AB testing with interviews and change their approach over time (Google stooped asking why are manholes round and looking at AST scores because data showed them those things had almost zero correlation with performance).
> Most big tech companies don't care about how good you have been at delivering some value through creating software: they want to see you deliver a very specific type of performance at a whiteboard
PG specifically points out that success within a large tech company is predicated on bogus hackable tests. So it makes sense that their selection criteria should also be a hackable test. The better you do on that artificial selection criterion for admission, the better you're likely to do on the artificial selection criteria of internal career progression. So in this particular case, the hackable admission test is a good proxy.
PG specifically points out that success within a large tech company is predicated on bogus hackable tests. So it makes sense that their selection criteria should also be a hackable test. The better you do on that artificial selection criterion for admission, the better you're likely to do on the artificial selection criteria of internal career progression. So in this particular case, the hackable admission test is a good proxy.
I've been curious for a while about the inflection point between the more informal startup programming interview style and the bigco interviewing style, and while there's obviously variability between orgs, whether it comes with headcount, customer profile, fund raising milestones, etc
We (Screenhero, W13) may well have been one of the many YC companies PG’s referring to! But instead of hoping there was some test to hack, we were worried that there was some unknown hack out there that we didn’t know. It was quite a huge relief when PG told us something to the effect of “all the successful startups have found their initial grown through just one thing: delighting their users with a great product”.
That realization / confirmation freed us four engineering cofounders from worrying about some “growth hack” we never learnt, and helped us focus on prioritizing our product development through talking to users. We then used Sean Ellis’ “very disappointed” survey methodology (which Rahul @ Superhman recently wrote about) and used it as our North Star to find the few features to focus on polishing.
We got acquired by Slack in 2015, and built out Slack Calls. I left last year, and I’m now in the final stages of launching a new product around super-charged meetings for remote teams (picking up where Screenhero / Slack Calls / Zoom left off), and am using the same principles again. No hacking of tests, just building a product that people want, and using their feedback as the only valuable bit of data in determining priorities. Thanks PG and YC (and Sean Ellis!) for startup principles and methodologies that have stood the test of time.
That realization / confirmation freed us four engineering cofounders from worrying about some “growth hack” we never learnt, and helped us focus on prioritizing our product development through talking to users. We then used Sean Ellis’ “very disappointed” survey methodology (which Rahul @ Superhman recently wrote about) and used it as our North Star to find the few features to focus on polishing.
We got acquired by Slack in 2015, and built out Slack Calls. I left last year, and I’m now in the final stages of launching a new product around super-charged meetings for remote teams (picking up where Screenhero / Slack Calls / Zoom left off), and am using the same principles again. No hacking of tests, just building a product that people want, and using their feedback as the only valuable bit of data in determining priorities. Thanks PG and YC (and Sean Ellis!) for startup principles and methodologies that have stood the test of time.
This spends too many words justifying the stale "real hacker" tradition of dunking on education, and kindof ignores the problem of how to measure something so subjective as learning.
So people who are good at tests get ahead. We get that you're not interested in grades. We all know people who play the game and win, not sincerely engage with their job or education or whatever.
But I'd like to hear from pg / other people in YC how they think they are "hacked". What do their successful applicants actually optimise for when "delight" and even "growth" are just as subjective as "learning"? After all they fund lots and lots of companies and founders, many of whom make no returns. How do founders keep their funding "success" long past the point it was deserved?
So people who are good at tests get ahead. We get that you're not interested in grades. We all know people who play the game and win, not sincerely engage with their job or education or whatever.
But I'd like to hear from pg / other people in YC how they think they are "hacked". What do their successful applicants actually optimise for when "delight" and even "growth" are just as subjective as "learning"? After all they fund lots and lots of companies and founders, many of whom make no returns. How do founders keep their funding "success" long past the point it was deserved?
"Growth" in the YC definition ("revenue growth with positive unit economics", or the closest proxy to that) seems a pretty objective test to me.
It can take a lot of resources to get to that point, no?
So there's still a question what companies should do until they reach this growth possibility.
So there's still a question what companies should do until they reach this growth possibility.
They’re often selecting applicants before growth is a factor, no? Surely there are hacks to this process
This is a very interesting thing to think about, I thought about it a bunch of times already, and have a couple of thoughts about it too.
First of all, once you're 55, it's easy to say stuff like that, because you won't be tested anymore. And I think there is a good chunk of survivorship bias: "Hey, I made it without worrying about tests, so you can, too!" Although that is probably not true for a big chunk of the population, especially once we look at non CS people.
Then I think, he is fundamentally right. Grades shouldn't matter so much, it is about what you learn. And I like to approach things that way too, but in the end, I always have to study for the tests as well.
The problem is exactly how deeply it is ingrained in everything. If I just stop caring, it doesn't really help. I will just end up in a worse position, since everyone around me still cares about test results. The people that need to change their mind are the people that use tests as measures of qualities that they are not a good measure for. I think a lot of people are falling thought the cracks because they don't fit the expectation of a HR person close enough.
In the end, we will always have to have tests. A completely individualized assessment of peoples qualities or fit for certain roles just doesn't scale. It would be great though to come up with new ways of testing that are maybe more in line with actual learning. For example, it would be nice to have tests at university where I can google things, just like in the real world.
I am lucky enough to be able to work in the booming tech sector where there are so many jobs that it is fine if I don't work towards a test.
First of all, once you're 55, it's easy to say stuff like that, because you won't be tested anymore. And I think there is a good chunk of survivorship bias: "Hey, I made it without worrying about tests, so you can, too!" Although that is probably not true for a big chunk of the population, especially once we look at non CS people.
Then I think, he is fundamentally right. Grades shouldn't matter so much, it is about what you learn. And I like to approach things that way too, but in the end, I always have to study for the tests as well.
The problem is exactly how deeply it is ingrained in everything. If I just stop caring, it doesn't really help. I will just end up in a worse position, since everyone around me still cares about test results. The people that need to change their mind are the people that use tests as measures of qualities that they are not a good measure for. I think a lot of people are falling thought the cracks because they don't fit the expectation of a HR person close enough.
In the end, we will always have to have tests. A completely individualized assessment of peoples qualities or fit for certain roles just doesn't scale. It would be great though to come up with new ways of testing that are maybe more in line with actual learning. For example, it would be nice to have tests at university where I can google things, just like in the real world.
I am lucky enough to be able to work in the booming tech sector where there are so many jobs that it is fine if I don't work towards a test.
> For example, it would be nice to have tests at university where I can google things, just like in the real world.
One of the best professors I had at university for a Linux course always said: “If you don’t know something, google it!”
He backed up his words by allowing googling on the actual coding part of his exams. You had to understand the underlying concepts but you never had to memorize syntax. One time, he had us write a small program in C that involved threading during a proctored timed exam. Everyone ran out of time for that part.
His course and exams were _rigorous_. You could never cram for his class and expect to pass and indeed I know of one student repeatedly failing his course.
This was one of the courses in university where I gained the bulk of my useful knowledge and practical skills I use to this day. I can count the courses that were of similar quality I took at university on one(1) hand. The rest of my degree involved hackable tests, apathetic professors and were massive time-wasters. I learned nothing from them, obviously.
The good professors made me realize how bad the rest of my degree was and how much of a racket higher education could be.
I look back at my undergrad days and wonder how much more I would have learned if I had not focused so much on grades. I’m still actively trying to unschool myself.
One of the best professors I had at university for a Linux course always said: “If you don’t know something, google it!”
He backed up his words by allowing googling on the actual coding part of his exams. You had to understand the underlying concepts but you never had to memorize syntax. One time, he had us write a small program in C that involved threading during a proctored timed exam. Everyone ran out of time for that part.
His course and exams were _rigorous_. You could never cram for his class and expect to pass and indeed I know of one student repeatedly failing his course.
This was one of the courses in university where I gained the bulk of my useful knowledge and practical skills I use to this day. I can count the courses that were of similar quality I took at university on one(1) hand. The rest of my degree involved hackable tests, apathetic professors and were massive time-wasters. I learned nothing from them, obviously.
The good professors made me realize how bad the rest of my degree was and how much of a racket higher education could be.
I look back at my undergrad days and wonder how much more I would have learned if I had not focused so much on grades. I’m still actively trying to unschool myself.
> He backed up his words by allowing googling on the actual coding part of his exams. You had to understand the underlying concepts but you never had to memorize syntax. ... His course and exams were _rigorous_. You could never cram for his class and expect to pass and indeed I know of one student repeatedly failing his course.
When I studied law, all our exams were open book. Bring whatever you want. I brought casebooks and printouts of legislation and important cases. Others also brought headnotes and volumes of Halsbury's (a legal encyclopaedia).
None of this helped if you didn't know the material well enough to identify the legal issues in each question.
Which is why I failed Torts.
When I studied law, all our exams were open book. Bring whatever you want. I brought casebooks and printouts of legislation and important cases. Others also brought headnotes and volumes of Halsbury's (a legal encyclopaedia).
None of this helped if you didn't know the material well enough to identify the legal issues in each question.
Which is why I failed Torts.
> None of this helped if you didn’t know the material well enough to identify the legal issues in each question.
My machine learning professor who was from CMU would allow us to put anything we wanted on both sides of one piece of printer paper to bring to exams.
No amount of notes helped if you had no understanding of the algorithms and formulas.
I almost failed that course too, but it was also one of the cornerstones of my university career. It made me realize how much I had been optimizing for the wrong thing: grades instead of understanding.
My machine learning professor who was from CMU would allow us to put anything we wanted on both sides of one piece of printer paper to bring to exams.
No amount of notes helped if you had no understanding of the algorithms and formulas.
I almost failed that course too, but it was also one of the cornerstones of my university career. It made me realize how much I had been optimizing for the wrong thing: grades instead of understanding.
>"Hey, I made it without worrying about tests, so you can, too!"
To be fair he did say "For me, as for most students, the measurement of what I was learning completely dominated actual learning in college. I was fairly earnest; I was genuinely interested in most of the classes I took, and I worked hard. And yet I worked by far the hardest when I was studying for a test."
>I think a lot of people are falling thought the cracks because they don't fit the expectation of a HR person close enough.
In retrospect I'm sure this was the case for me. It's probably why I ended up starting my own business.
To be fair he did say "For me, as for most students, the measurement of what I was learning completely dominated actual learning in college. I was fairly earnest; I was genuinely interested in most of the classes I took, and I worked hard. And yet I worked by far the hardest when I was studying for a test."
>I think a lot of people are falling thought the cracks because they don't fit the expectation of a HR person close enough.
In retrospect I'm sure this was the case for me. It's probably why I ended up starting my own business.
> The problem is exactly how deeply it is ingrained in everything
You can you use "test" as a proxy for the impression you want to make on your boss and peers. Maybe even friends and family.
It's up to you to redefine that test (or create a complementary one) based on your own goals and your own timeline.
You can you use "test" as a proxy for the impression you want to make on your boss and peers. Maybe even friends and family.
It's up to you to redefine that test (or create a complementary one) based on your own goals and your own timeline.
I took a class from John White at UCONN. He gave us a couple dozen CS papers at the start of the year. Every class had 3 paper presentations, 20 minutes each. A random student was picked for each presentation.
So you had to read and understand each of the upcoming 3 papers because there was a random chance you would have to present one of the three.
The result is that you eventually read all of the papers well enough that you could give a 20 minute presentation.
There were no grades. It was pass/fail.
It was one of the best classes I ever took.
So you had to read and understand each of the upcoming 3 papers because there was a random chance you would have to present one of the three.
The result is that you eventually read all of the papers well enough that you could give a 20 minute presentation.
There were no grades. It was pass/fail.
It was one of the best classes I ever took.
I had a high school professor do a somewhat abridged version of that for our final. It was impossible to spend an adequate amount of time on all three so you really had to focus on two and hope you got lucky.
Easily the most stressful final I've ever taken.
Easily the most stressful final I've ever taken.
But the person who presented the very first paper won't have to care about the rest of the semester right? Or can you get randomly picked multiple times?
“But you can't blame teachers if their tests are hackable. Their job is to teach, not to create unhackable tests. The real problem is grades, or more precisely, that grades have been overloaded.”
Two important things here: * Grade inflation makes for happier students and counterbalances the value of test hacking. Today there are very few schools that don’t do grade inflation. Without considering grade inflation, much of this essay is simply Graham entertaining his own nostalgia. (I’m not necessarily a proponent of inflation but it’s a key phenomena missing from the essay).
* Good teachers hold lots of office hours and give good feedback through those offerings. Email support is also a lot more popular today. You can also play games like test corrections, rough drafts, etc. Students don’t learn well without feedback, and most teachers who don’t invest in it won’t succeed today. This focus towards feedback in modern teaching is at odds with the education community profiled in this essay.
If there’s one lesson to un-learn, it’s to forget about giving attention to non-constructive papers that lack evidence. While this essay gives a thorough criticism of tests, it offers no concrete alternative. And it offers no grounding for the claimed ‘good’ student who ‘focuses on valuable learning.’ Graham, like any other VC, seeks to upwell sentiment in the interest of controlling it (e.g. creating an investment asset out of it). Reading this essay doesn’t teach you anything about hacking unless you recognize that the author is trying to hack you himself.
Two important things here: * Grade inflation makes for happier students and counterbalances the value of test hacking. Today there are very few schools that don’t do grade inflation. Without considering grade inflation, much of this essay is simply Graham entertaining his own nostalgia. (I’m not necessarily a proponent of inflation but it’s a key phenomena missing from the essay).
* Good teachers hold lots of office hours and give good feedback through those offerings. Email support is also a lot more popular today. You can also play games like test corrections, rough drafts, etc. Students don’t learn well without feedback, and most teachers who don’t invest in it won’t succeed today. This focus towards feedback in modern teaching is at odds with the education community profiled in this essay.
If there’s one lesson to un-learn, it’s to forget about giving attention to non-constructive papers that lack evidence. While this essay gives a thorough criticism of tests, it offers no concrete alternative. And it offers no grounding for the claimed ‘good’ student who ‘focuses on valuable learning.’ Graham, like any other VC, seeks to upwell sentiment in the interest of controlling it (e.g. creating an investment asset out of it). Reading this essay doesn’t teach you anything about hacking unless you recognize that the author is trying to hack you himself.
The suggested alternative I got from the essay was "try to cause a real, specific outcome in the world. If the action you took caused the outcome, you passed the test."
Did Sam Altman pass the test with Loopt?
Just because you made waves somehow or filled your pockets with money doesn’t mean your time was ‘better spent’ than trying to make A’s in organized education. School can very well teach you how to make better outcomes.
But, yes, school steers youthful energy away from making change... err making money for VCs.
Just because you made waves somehow or filled your pockets with money doesn’t mean your time was ‘better spent’ than trying to make A’s in organized education. School can very well teach you how to make better outcomes.
But, yes, school steers youthful energy away from making change... err making money for VCs.
My son is 8, and yesterday he came home from school and said, "I'm not above grade level in math." He sounded a little disappointed, but not too much; he also seemed to be asking if that really means anything.
We had a really interesting conversation. I've been a middle and high school math teacher most of my life, so this is quite familiar ground for me. Over dinner I asked him how he would go about measuring whether a kid is "above grade level". He got really animated and started to tell me what he thought second graders should know and be able to do, and how third graders should expand on that, and what he thought he'd start to learn in fourth and fifth grade.
I told him I could start to teach him some of the operations and skills he'll learn in the coming years, but that it probably wouldn't mean a whole lot. Instead I asked him what ideas he's already familiar with that other kids his age don't know about, because we talk about math almost every day. "I know what phi is, and pi!" Yes, and we've also talked about limits, calculus, negative numbers, fractions, decimals, and he's looked at pages of math that's well beyond my understanding as well.
I told him that when he gets to middle and high school classes, he's going to see higher level math and it's going to sound familiar because we've already been talking about the ideas for years. He's going to watch other kids fail and drop out of those classes because the ideas and the skills will be too new and too difficult for them to take in. I said we're probably much better off just continuing to talk about fun and interesting areas of math for the time being instead of worrying about "grade level".
We had a really interesting conversation. I've been a middle and high school math teacher most of my life, so this is quite familiar ground for me. Over dinner I asked him how he would go about measuring whether a kid is "above grade level". He got really animated and started to tell me what he thought second graders should know and be able to do, and how third graders should expand on that, and what he thought he'd start to learn in fourth and fifth grade.
I told him I could start to teach him some of the operations and skills he'll learn in the coming years, but that it probably wouldn't mean a whole lot. Instead I asked him what ideas he's already familiar with that other kids his age don't know about, because we talk about math almost every day. "I know what phi is, and pi!" Yes, and we've also talked about limits, calculus, negative numbers, fractions, decimals, and he's looked at pages of math that's well beyond my understanding as well.
I told him that when he gets to middle and high school classes, he's going to see higher level math and it's going to sound familiar because we've already been talking about the ideas for years. He's going to watch other kids fail and drop out of those classes because the ideas and the skills will be too new and too difficult for them to take in. I said we're probably much better off just continuing to talk about fun and interesting areas of math for the time being instead of worrying about "grade level".
> The way you get lots of users is to make the product really great.
That is wrong – or, at the very least, incomplete. "Build it and they will come" is the dream and misconception of every programmer who's talented at software development but not at sales. Hell, this is the number one advice of every business how-to book ever: You can't rely on people finding out about your great product by chance, you need to put in the work and sell it!
> Then people will not only use it but recommend it to their friends [...]
That's not how it works for B2B software, and only rarely for B2C. Usually, if a product is successful, then as a result of good marketing (a good product ist somewhat necessary, but not sufficient).
That is wrong – or, at the very least, incomplete. "Build it and they will come" is the dream and misconception of every programmer who's talented at software development but not at sales. Hell, this is the number one advice of every business how-to book ever: You can't rely on people finding out about your great product by chance, you need to put in the work and sell it!
> Then people will not only use it but recommend it to their friends [...]
That's not how it works for B2B software, and only rarely for B2C. Usually, if a product is successful, then as a result of good marketing (a good product ist somewhat necessary, but not sufficient).
100% right, and the corollary of this is: you don't actually need to make a great product, if your sales&marketing game is good enough.
Or, in other words, a lot of successful business - including startups - is exactly the "test hacking" PG urges founders to unlearn.
I'm honestly surprised by this essay arguing that "hacking the test" is the wrong approach in startups/business. "Hacking the test" is essentially what your marketing is supposed to do. These "non-authoritarian" tests like selling things are just as hackable as school tests; you just have to discover how the system really operates.
The whole authoritarian/non-authoritarian split doesn't carve reality at the joints, IMO. Football match is really an authoritarian test - it tests who wins under the game rules, which are given from the top. It may be hard to hack, but that's because passing that test is usually synonymous with the terminal goals of the test takers - i.e. they want to win the game. You'll note however that, once money gets involved, people are sometimes made to deliberately lose games; in these scenarios, there are usually other people involved who are absolutely hacking the test.
The best way I've found to carve the reality at the joints is to talk about terminal and instrumental goals. Learning useful things isn't the terminal goal for most students, getting a good career is (and/or not pissing off parents by getting bad grades).
So I'm ultimately surprised that PG argues that building good things, not hacking the test, is how you win the startup game - I would think most of the startups have products as instrumental goals; the exit is the terminal goal. And building a great product isn't the best way to achieve that goal.
Or, in other words, a lot of successful business - including startups - is exactly the "test hacking" PG urges founders to unlearn.
I'm honestly surprised by this essay arguing that "hacking the test" is the wrong approach in startups/business. "Hacking the test" is essentially what your marketing is supposed to do. These "non-authoritarian" tests like selling things are just as hackable as school tests; you just have to discover how the system really operates.
The whole authoritarian/non-authoritarian split doesn't carve reality at the joints, IMO. Football match is really an authoritarian test - it tests who wins under the game rules, which are given from the top. It may be hard to hack, but that's because passing that test is usually synonymous with the terminal goals of the test takers - i.e. they want to win the game. You'll note however that, once money gets involved, people are sometimes made to deliberately lose games; in these scenarios, there are usually other people involved who are absolutely hacking the test.
The best way I've found to carve the reality at the joints is to talk about terminal and instrumental goals. Learning useful things isn't the terminal goal for most students, getting a good career is (and/or not pissing off parents by getting bad grades).
So I'm ultimately surprised that PG argues that building good things, not hacking the test, is how you win the startup game - I would think most of the startups have products as instrumental goals; the exit is the terminal goal. And building a great product isn't the best way to achieve that goal.
> I would think most of the startups have products as instrumental goals; the exit is the terminal goal.
Depressing, if true.
I would hope that most people have a product that they want to build, and more money is an instrumental goal that lets them build it better. For example, I'd expect someone who loves to code to have software they want to write, and want enough money to allow them to do so—as opposed the executive who loves money and views writing software as the best way to get it. I'm not convinced that any great work was ever crafted by someone who was primarily crafting for instrumental reasons.
But you're probably right, and that's probably a big part of what's so toxic about startup culture these days.
Depressing, if true.
I would hope that most people have a product that they want to build, and more money is an instrumental goal that lets them build it better. For example, I'd expect someone who loves to code to have software they want to write, and want enough money to allow them to do so—as opposed the executive who loves money and views writing software as the best way to get it. I'm not convinced that any great work was ever crafted by someone who was primarily crafting for instrumental reasons.
But you're probably right, and that's probably a big part of what's so toxic about startup culture these days.
[deleted]
> you just have to discover how the system really operates.
One might say that, by analogy, this is like saying you cheated on a test by studying. “Learning how the system really operates” isn’t hacking; it’s the thing you’re formally setting out to do as a business!
One might say that, by analogy, this is like saying you cheated on a test by studying. “Learning how the system really operates” isn’t hacking; it’s the thing you’re formally setting out to do as a business!
The hacking starts once you learn that building great products isn't the only, or even the best, way to create a successful business.
What about marketing churn? If people try your product and have a bad experience, maybe they won't try anything else you offer either? It seems like the product at least needs to be somewhat decent?
A lot of companies seem fine with churn, as long as they can acquire new customers faster, or if they form an oligopoly (banks and telcos come to mind).
In general yes, the product has to meet some minimum bar of decency. How high? Depends on a lot of factors. Note the popularity of business models involving some kind of lock-in - e.g. network effect, holding data hostage. Those are all attempts at hacking around the need for a decent product.
In general yes, the product has to meet some minimum bar of decency. How high? Depends on a lot of factors. Note the popularity of business models involving some kind of lock-in - e.g. network effect, holding data hostage. Those are all attempts at hacking around the need for a decent product.
"Make people want something that is actually junk." doesn't have quite the same ring to it, nor does it sound remotely more plausible.
Presumably the 10% weekly growth that YC and PG advocate as the default growth metric isn't expected to be in the absence of sales but the contrary, sales plus anything that works that is ethical and not some ponzi scheme.
He provided examples via the link to http://paulgraham.com/ds.html where he describes stripe founders engaging in zealous activities many would call sales despite being the type of startup that could have leaned back and had ample demand.
Presumably the 10% weekly growth that YC and PG advocate as the default growth metric isn't expected to be in the absence of sales but the contrary, sales plus anything that works that is ethical and not some ponzi scheme.
He provided examples via the link to http://paulgraham.com/ds.html where he describes stripe founders engaging in zealous activities many would call sales despite being the type of startup that could have leaned back and had ample demand.
> "Make people want something that is actually junk." doesn't have quite the same ring to it, nor does it sound remotely more plausible.
It certainly doesn't have the same ring to it, but it's an extremely successful business model.
It certainly doesn't have the same ring to it, but it's an extremely successful business model.
I skimmed that essay. Zealous activities to make existing users happy seemed to be the emphasis. It's possible that there were zealous activities for acquiring users mentioned somewhere, but I missed them because they were less emphasized.
> "Build it and they will come" is the dream and misconception of every programmer (...)
Well, the amount of marketing required strongly depends on the quality of and demand for your product.
I built a software product and launched it with very very minimal marketing (I emailed a blogger and published it on a mailing list).
People liked my app so much that I had a 1000 downloads within a few weeks (which I consider decent for a somewhat niche app).
Working on that app has since become my full time job, and I do very minimal marketing.
Maybe I could make more revenue if I focussed more on marketing, but to be honest I'm pretty amazed how well word of mouth advertising works.
Well, the amount of marketing required strongly depends on the quality of and demand for your product.
I built a software product and launched it with very very minimal marketing (I emailed a blogger and published it on a mailing list).
People liked my app so much that I had a 1000 downloads within a few weeks (which I consider decent for a somewhat niche app).
Working on that app has since become my full time job, and I do very minimal marketing.
Maybe I could make more revenue if I focussed more on marketing, but to be honest I'm pretty amazed how well word of mouth advertising works.
No. I consult with B2B startups on marketing and have seen the internal data for 20+ B2B tech companies. If anything, the best products with the lowest churn have less referrals, since people don't want their competitors to have the competitive advantage the product provides.
> people don't want their competitors to have the competitive advantage (...)
That sounds like a very cut-throat business.
In my target audience, most people are happy to share how they work and which tools they use. I even occasionally see random people recommending my product online (which makes me very happy)
That sounds like a very cut-throat business.
In my target audience, most people are happy to share how they work and which tools they use. I even occasionally see random people recommending my product online (which makes me very happy)
Generally, different VC firms give a lot of money to similar companies at the same time. In one vertical, I worked with the same company for three years. CPC for the same keywords doubled every year for all 3 years, when the unit economics of the industry didn't change. It's fiercely competitive.
As an example, can you name a b2b or b2c product that you feel is truly great but overlooked?
B2B:
(1) The scripting language Rexx.
(2) The text editor KEdit with macro language KEXX much like Rexx.
(1) and (2) are far and away my two most important tools in computing, from IBM mainframes to PCs, DOS, OS/2, Windows, and my startup on Windows 7 64 bit Professional on an AMD FX-8350 64 bit 8 core processor at 4.0 GHz with Microsoft's .NET.
E.g., I typed in the 100,000 lines of text for the software for my startup using KEdit and never an IDE (integrated development environment) -- KEDIT worked FINE!
(3) LINPACK.
(4) Microsoft's PhotoDraw of about year 2000.
(5) Robocopy.
(6) Western Digital's Data Lifeguard Tools and their backup and recovery software.
(7) D. Knuth's TeX (not LaTeX, just TeX).
(1) The scripting language Rexx.
(2) The text editor KEdit with macro language KEXX much like Rexx.
(1) and (2) are far and away my two most important tools in computing, from IBM mainframes to PCs, DOS, OS/2, Windows, and my startup on Windows 7 64 bit Professional on an AMD FX-8350 64 bit 8 core processor at 4.0 GHz with Microsoft's .NET.
E.g., I typed in the 100,000 lines of text for the software for my startup using KEdit and never an IDE (integrated development environment) -- KEDIT worked FINE!
(3) LINPACK.
(4) Microsoft's PhotoDraw of about year 2000.
(5) Robocopy.
(6) Western Digital's Data Lifeguard Tools and their backup and recovery software.
(7) D. Knuth's TeX (not LaTeX, just TeX).
> (5) Robocopy.
Ha! We use robocopy at work because IT refuses to fix our internal network. We can reliably reproduce an NFS mount dropping off the network around the 12th GB of a large file and the only answer is "stop bugging us and use robocopy". All it does is turn a 30-40 minute process into a 3-4 hour process.
Ha! We use robocopy at work because IT refuses to fix our internal network. We can reliably reproduce an NFS mount dropping off the network around the 12th GB of a large file and the only answer is "stop bugging us and use robocopy". All it does is turn a 30-40 minute process into a 3-4 hour process.
What's the best Rexx interpreter for Linux?
I use Open Object Rexx on Windows (3.1, 2000, XP, now 7, 10).
I'm not sure what is available on Linux but maybe the same thing.
Rexx can be used as a macro language for nearly any software, but I use it just for command line scripting. I have 200+ Rexx scripts I regard as serious and write more for one-shot applications a few times a week.
E.g., I have several Rexx scripts for running Robocopy, and each script has VERY carefully worked out, tested, etc. options.
One Rexx script I leave running in each Windows command line window as a shell: It's useful but dirt simple, just reads the command line, does what it can do and otherwise passes to Windows. In particular with that shell I have the cutest little file system tree walking, jumping, etc. commands.
One of my favorite Rexx scripts I invoke with an icon in the UL corner of my screen. This script moves the windows, maintaining the current Z-order and X position order, so that the UL corners of the windows are on a line from the UR of the screen to the middle left. This way at least a little of each window is visible and can be made fully visible without moving any of the other windows and still leaving at least a part of all the other windows visible. To me THAT is how to arrange screen windows. So, yes, Rexx has a little API to some of the Windows API.
In any command line window, Rexx can do good things with the environment variables of that window. So, I initialize each command line window with environment variables like
mark.hn=C:\Users\user1\DATA05\TOPICS\MEDIA\HACKER_NEWS\
So, that environment variable has the file system directory I use for HN posts, etc. Then via my little shell, I can type
g hn
(go to hn), and my little shell looks up environment variable MARK.HN and makes that tree name the current drive/directory.
Apparently when using an hierarchical file system to implement a taxonomic hierarchy of subject matter, some of the leaves of the hierarchy are wanted much more often than the directories on the path to that leaf. For such leaves, say, x, my little environment variable MARK.x work nicely.
It's common for me to have a file system directory tree name in KEdit and then want to open a window with that tree name the current drive/directory. Okay. In KEdit I put the tree name on the Windows system clipboard, and then my little Rexx shell command
g
makes the tree name on the clipboard current. Dirt simple to implement and darned handy.
There's lots more in my little command shell, and it's easy to implement more.
There's a book on Rexx by Mike Cowlishaw, and it's nicely done, but Open Object Rexx comes with some nice PDF files that might be documentation enough.
Rexx is elegant. E.g., it's a total sweetheart for its string manipulation facilities. The only data type is a string. If the content of a string is the decimal digits of a number, then can do arithmetic on it. Yes, can ask if the content is a number.
Built-in quite nicely are stem variables, essentially collection classes or, if you will, AVL trees. So, for variable i can have
array.i
where array is the collection and i is the key. If have strings i, j, k then can have a three dimensional key
array.i.j.k
Powerful. Easy. Elegant!
Yes, there is a statement interpret where can enter a string and have it executed immediately. So, e.g., could have some code that constructed a string that was the source code for a do-loop, give that string to the interpret statement, and have the loop executed.
Yes there are some plenty good enough trace and debugging facilities.
Rexx is no toy: For some years Rexx on 3000+ IBM mainframes connected with VNET (at the user level much like the Internet) was the basis of administrative computing in IBM.
Can get add-ons for sorting, TCP/IP, manipulating windows on Windows, scientific functions, etc.
I suspect that on Windows PowerShell is more powerful but harder to use for when Rexx will work.
In time I'll have to get okay facility with PowerShell, but I'll likely never give up on Rexx.
Rexx used to be the macro language for KEdit but eventually KEdit wrote KEXX, their own version of Rexx. KEXX is nice, sometimes works better with KEdit than an external macro language could, and has built-in documentation that is likely good enough, especially if had a start with Rexx.
Rexx and KEdit let me have just simple text as the basis of nearly all my computing and essentially all my typing. Rexx and KEdit are for text like a chef's knife and cutting board are for a cook. AFAIK, all source code is still just simple text!
Long live the kings, text, TeX, Rexx, Kexx!!!
I'm not sure what is available on Linux but maybe the same thing.
Rexx can be used as a macro language for nearly any software, but I use it just for command line scripting. I have 200+ Rexx scripts I regard as serious and write more for one-shot applications a few times a week.
E.g., I have several Rexx scripts for running Robocopy, and each script has VERY carefully worked out, tested, etc. options.
One Rexx script I leave running in each Windows command line window as a shell: It's useful but dirt simple, just reads the command line, does what it can do and otherwise passes to Windows. In particular with that shell I have the cutest little file system tree walking, jumping, etc. commands.
One of my favorite Rexx scripts I invoke with an icon in the UL corner of my screen. This script moves the windows, maintaining the current Z-order and X position order, so that the UL corners of the windows are on a line from the UR of the screen to the middle left. This way at least a little of each window is visible and can be made fully visible without moving any of the other windows and still leaving at least a part of all the other windows visible. To me THAT is how to arrange screen windows. So, yes, Rexx has a little API to some of the Windows API.
In any command line window, Rexx can do good things with the environment variables of that window. So, I initialize each command line window with environment variables like
mark.hn=C:\Users\user1\DATA05\TOPICS\MEDIA\HACKER_NEWS\
So, that environment variable has the file system directory I use for HN posts, etc. Then via my little shell, I can type
g hn
(go to hn), and my little shell looks up environment variable MARK.HN and makes that tree name the current drive/directory.
Apparently when using an hierarchical file system to implement a taxonomic hierarchy of subject matter, some of the leaves of the hierarchy are wanted much more often than the directories on the path to that leaf. For such leaves, say, x, my little environment variable MARK.x work nicely.
It's common for me to have a file system directory tree name in KEdit and then want to open a window with that tree name the current drive/directory. Okay. In KEdit I put the tree name on the Windows system clipboard, and then my little Rexx shell command
g
makes the tree name on the clipboard current. Dirt simple to implement and darned handy.
There's lots more in my little command shell, and it's easy to implement more.
There's a book on Rexx by Mike Cowlishaw, and it's nicely done, but Open Object Rexx comes with some nice PDF files that might be documentation enough.
Rexx is elegant. E.g., it's a total sweetheart for its string manipulation facilities. The only data type is a string. If the content of a string is the decimal digits of a number, then can do arithmetic on it. Yes, can ask if the content is a number.
Built-in quite nicely are stem variables, essentially collection classes or, if you will, AVL trees. So, for variable i can have
array.i
where array is the collection and i is the key. If have strings i, j, k then can have a three dimensional key
array.i.j.k
Powerful. Easy. Elegant!
Yes, there is a statement interpret where can enter a string and have it executed immediately. So, e.g., could have some code that constructed a string that was the source code for a do-loop, give that string to the interpret statement, and have the loop executed.
Yes there are some plenty good enough trace and debugging facilities.
Rexx is no toy: For some years Rexx on 3000+ IBM mainframes connected with VNET (at the user level much like the Internet) was the basis of administrative computing in IBM.
Can get add-ons for sorting, TCP/IP, manipulating windows on Windows, scientific functions, etc.
I suspect that on Windows PowerShell is more powerful but harder to use for when Rexx will work.
In time I'll have to get okay facility with PowerShell, but I'll likely never give up on Rexx.
Rexx used to be the macro language for KEdit but eventually KEdit wrote KEXX, their own version of Rexx. KEXX is nice, sometimes works better with KEdit than an external macro language could, and has built-in documentation that is likely good enough, especially if had a start with Rexx.
Rexx and KEdit let me have just simple text as the basis of nearly all my computing and essentially all my typing. Rexx and KEdit are for text like a chef's knife and cutting board are for a cook. AFAIK, all source code is still just simple text!
Long live the kings, text, TeX, Rexx, Kexx!!!
Thanks for the enthusiastic pitch :) There was once an internal IBM editor called E3, with many Rexx extensions. It was amazing.
I don't recall E3. But there really WAS an IBM editor XEDIT, written on his own by an IBM guy in Paris. He understood elegance. Its macro language was Rexx. KEdit is a PC version of XEDIT.
> Robocopy
I don't get ms. There's a good kind of copy and a bad kind of copy. Why is the bad kind the default?
I don't get ms. There's a good kind of copy and a bad kind of copy. Why is the bad kind the default?
The bad kind keeps a legacy API that allows batch scripts from 1985 to work.
In every modern OS, the system packages—and most installable first-party packages in a package ecosystem—are there to make the (legacy) software you might install “just work”, not to provide the best experience for users using those packages directly.
Users, for their own needs, are expected to install their own tools and runtimes, or to consume collections of them through “toolchain environments” provided by software like IDEs.
In every modern OS, the system packages—and most installable first-party packages in a package ecosystem—are there to make the (legacy) software you might install “just work”, not to provide the best experience for users using those packages directly.
Users, for their own needs, are expected to install their own tools and runtimes, or to consume collections of them through “toolchain environments” provided by software like IDEs.
If the product is great(not only the software, but support, vision, market fit, etc), you need a minimum base of users until it takes off, doesn't matter is B2C or B2B, people will talk.
[deleted]
Yeah it’s a matter of emphasis. When you are starting up, not having a great product is your biggest problem by far. Also your attitude, if widely applied, would result in a world filled with useless crap with brilliant marketing.
What makes you think the world isn't? It hardly ever is the best service or product that wins, but the one that sells best. And that is, to a huge degree, marketing. Maybe crap is to harsh, but not the best products? I certainly would sign that sentiment.
You are perhaps too optimistic. Just heard a scooter start up called ‘Unicorn’ went bust without making a single scooter, losing all the money people gave them for pre-orders. All the money was spent on facebook ads. This is the purest application of marketing, where there is no product, let alone a crap one.
Not sure I agree with this.
The real world is very much about optimizing for and beating tests.
It might be producing a specific CV or preparing for an interview whiteboard session to land a job.
It might be socializing and networking in a specific way in order to land funding.
The real world does not commonly reward just being really good at arbitrary things. It's almost always focused on meeting a need that someone else has defined, much like a test.
The real world is very much about optimizing for and beating tests.
It might be producing a specific CV or preparing for an interview whiteboard session to land a job.
It might be socializing and networking in a specific way in order to land funding.
The real world does not commonly reward just being really good at arbitrary things. It's almost always focused on meeting a need that someone else has defined, much like a test.
> The real world
It's not the real world - its the human world. The real world is very different, and getting more so imho. Skills to survive the human world are usually useless in the real physical world. It's a distinction I read recently from a philosopher that I lost track of, maybe someone here knows. Her point was (I think) that when someone says the real world, with relation to education, it's not really - it's the world of human systems that are very much open to degradation and corruption, and they can be changed.
It's not the real world - its the human world. The real world is very different, and getting more so imho. Skills to survive the human world are usually useless in the real physical world. It's a distinction I read recently from a philosopher that I lost track of, maybe someone here knows. Her point was (I think) that when someone says the real world, with relation to education, it's not really - it's the world of human systems that are very much open to degradation and corruption, and they can be changed.
> and they can be changed
is that a hard fact or wishful thinking?
In the context of this discussion, the rewards do not come from "the physical world" but from other humans (money) , so generic mountain-survival skills are irrelevant
is that a hard fact or wishful thinking?
In the context of this discussion, the rewards do not come from "the physical world" but from other humans (money) , so generic mountain-survival skills are irrelevant
Of course they can be changed - how hard they are to change is another matter. The current system isn't set in stone, or handed down from above.
Found the article in case anyone is still browsing here http://www.theabsolute.net/minefield/humevas.html its interesting
That is only true if your life ambition is securing a job with that successful tech company.
If your ambition is being successful on your own, as an academic, an entrepreneur or as an artist, there are no standardized tests to beat.
If your ambition is being successful on your own, as an academic, an entrepreneur or as an artist, there are no standardized tests to beat.
Being successful academic is all about optimizing for various tests and metrics and beating tests. It would be hard to find more institutionalized occupation.
disagree about academics. it's a job that's so mindnumbingly bureaucratic these days. perhaps you mean "science"
There is a predictable component to the market. When we exchange articles about "how we did $something-entrepreneurish right and reaped rewards", we're essentially trading exam questions from previous years.
> The real world is very much about optimizing for and beating tests.
thank you.
it's awesome for someone like PG to bring the subject upfront. but what you are saying resonates with me.
genuine interest is missing in our field, and to some extent many others. "beating tests", as PG puts it, is at all time high. i am at a point where i don't know who or what is right.
the numbers are telling an important story! those who learned to hack the tests or the system are popular online or offline. and they are "succeeding" in life. they seem to have outnumbered those who put genuine interest first.
for instance, as many of my peers, i wanna learn machine learning and AI. but it is hard for me to find materials that resonate with me. materials that teach from first principles like those that got me hooked back in the days. they are lacking because we are so good at shortcuts and hacking. maybe we don't know how to do it anymore?
but who am i to blame anyone? there are so many, many, ways to hack and beat the tests and the system. and seemingly you can get ahead of many others in life by doing so. #AceYourFirstInterviewAfterBootcamp #TensorflowPyTorch #BeSureToLikeBelow #ThanksToMyPatreons #SubForChatAndEmojis #InstagramFacebookDown
i am very thankful for PG for starting the discussion. and for this comment.
thank you.
it's awesome for someone like PG to bring the subject upfront. but what you are saying resonates with me.
genuine interest is missing in our field, and to some extent many others. "beating tests", as PG puts it, is at all time high. i am at a point where i don't know who or what is right.
the numbers are telling an important story! those who learned to hack the tests or the system are popular online or offline. and they are "succeeding" in life. they seem to have outnumbered those who put genuine interest first.
for instance, as many of my peers, i wanna learn machine learning and AI. but it is hard for me to find materials that resonate with me. materials that teach from first principles like those that got me hooked back in the days. they are lacking because we are so good at shortcuts and hacking. maybe we don't know how to do it anymore?
but who am i to blame anyone? there are so many, many, ways to hack and beat the tests and the system. and seemingly you can get ahead of many others in life by doing so. #AceYourFirstInterviewAfterBootcamp #TensorflowPyTorch #BeSureToLikeBelow #ThanksToMyPatreons #SubForChatAndEmojis #InstagramFacebookDown
i am very thankful for PG for starting the discussion. and for this comment.
There is truth to this. It reflects the shift of mentality in tech entrepreneurs since the 2000s (experimentally entrepreneurial, introverted hackers, libertarian) to the 2010s (work for FANG or make a feature that FANG wants to buy, well-curated github and social media, social justice).
I could dare say that this mentality flowed from academia to the greater economy. Academic funding has been generally formalistic and test-driven for decades.
I could dare say that this mentality flowed from academia to the greater economy. Academic funding has been generally formalistic and test-driven for decades.
How many decades? I've heard that things were very different in the 60's/70's...
Agreed. Life is a series of tests. The big difference between life and school, is that it is usually much harder in life to anticipate what is going to be tested. That is where the real skill lies.
When a startup prioritizes feature A over feature B, they are making a decision on what is going to be tested by customers. The startup then optimizes for passing the anticipated test.
When a startup prioritizes feature A over feature B, they are making a decision on what is going to be tested by customers. The startup then optimizes for passing the anticipated test.
I think there are some good insights to be gleaned from this essay, as with a lot of pg's essays. The perspective of the guy who founded and ran Y Combinator is always an interesting one to hear from.
However for this essay in particular, had I been among the set of draft readers, I would have suggested toning down the “I hadn't realized X until now, so now I think other people haven't realized X either” message.
Back in college I remember noticing choices of “better learning vs better grade” and deliberately choosing the former with a proudly “screw the authorities” attitude. I'm sure plenty of other Tara Ploughmans could say the same.
However for this essay in particular, had I been among the set of draft readers, I would have suggested toning down the “I hadn't realized X until now, so now I think other people haven't realized X either” message.
Back in college I remember noticing choices of “better learning vs better grade” and deliberately choosing the former with a proudly “screw the authorities” attitude. I'm sure plenty of other Tara Ploughmans could say the same.
An important corollary is that school and tests teach you not to be wrong. They teach you that incorrect answers will be punished.
Yet, most interesting questions in life don’t yet have defined answers. Thus you need to have and test a hypothesis, which will very often be “incorrect” the first time around. But that doesn’t actually matter - the mere act of thinking about and defining what an answer could be sets you up to iterate and test. Eventually, you may find product market fit (a - not the - right answer).
You have to be willing to be wrong at first to learn...and win.
Yet, most interesting questions in life don’t yet have defined answers. Thus you need to have and test a hypothesis, which will very often be “incorrect” the first time around. But that doesn’t actually matter - the mere act of thinking about and defining what an answer could be sets you up to iterate and test. Eventually, you may find product market fit (a - not the - right answer).
You have to be willing to be wrong at first to learn...and win.
Even worse, most tests have a well defined question and a correct answer.
In real life you don’t know the question, there isn’t a single canonical answer, and there is no score-keeper to tell you if you are passing.
In real life you don’t know the question, there isn’t a single canonical answer, and there is no score-keeper to tell you if you are passing.
This is great seeing so many PG essays! I've really missed his writing.
> And at elite universities, that means nearly everyone, since someone who didn't care about getting good grades probably wouldn't be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades.
> When I started advising startup founders at Y Combinator, especially young ones, I was puzzled by the way they always seemed to make things overcomplicated.
I suspect part of the problem (of early founders being focused on hacking tests) was related to how heavily YC funded people from elite universities—which select people who do exactly that.
> And at elite universities, that means nearly everyone, since someone who didn't care about getting good grades probably wouldn't be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades.
> When I started advising startup founders at Y Combinator, especially young ones, I was puzzled by the way they always seemed to make things overcomplicated.
I suspect part of the problem (of early founders being focused on hacking tests) was related to how heavily YC funded people from elite universities—which select people who do exactly that.
> This is great seeing so many PG essays! I've really missed his writing.
I love reading his tweets! They're all so down to earth and illuminating. That said, I'm not a fan of his blog writing as much. I find it full of overly-sparse nuggets of wisdom. It needs to be more concise, to the point, with fewer clever metaphors. This article could've been compressed 70% and still have been human readable.
I love reading his tweets! They're all so down to earth and illuminating. That said, I'm not a fan of his blog writing as much. I find it full of overly-sparse nuggets of wisdom. It needs to be more concise, to the point, with fewer clever metaphors. This article could've been compressed 70% and still have been human readable.
I've read the article in full. It raises an important point, but it's indeed a laborious read. As I read it, the haunting paragraph from The Elements of Style came to mind:
"Omit needless words. Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all sentences short, or avoid all detail and treat subjects only in outline, but that every word tell."
https://www.harpercollins.com/9780060932237/essays-of-e-b-wh...
"Omit needless words. Vigorous writing is concise. A sentence should contain no unnecessary words, a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all sentences short, or avoid all detail and treat subjects only in outline, but that every word tell."
- - -
https://www.harpercollins.com/9780060932237/essays-of-e-b-wh...
pg seems to have a backlog of essay ideas that he's catching up on. This is all good, but also means that they get written a bit too quickly. It's the classic "had I had more time to spend on this text, it would have been shorter" syndrome.
Indeed, time is of the essence; playing tug-of-war with quality.
I'm a non degree student at an ivy league and I took a course this semester because I wanted to learn the basic fundamentals of electrical engineering. Because I'm learning because I want to, I put in effort into things I find interesting and difficult without the pressure of deadlines and grades. I love grades because it gives me the feedback I need (otherwise I would "learn" from youtube videos and pdfs) to assess how well I know the subject.
I can take breaks from truly hard problems until I have enough clarity of mind to approach the problem with a new perspective. I can't do that with a deadline.
I don't care if the answers "are out there" meaning the professor has returned graded tests and posted the answers; I do the work because I want to learn. That paradox of students wanting to review graded homeworks and students who want to learn by doing the homework is resolved practically by those that are more vocal about their wants.
I'm happy to take my time, doing my best (with the understanding that my best takes time) and observing my grade afterwards. Sometimes my learning takes me down paths that harder and more rewarding with the consequence that I take longer than others: I am a slow learner.
I can take breaks from truly hard problems until I have enough clarity of mind to approach the problem with a new perspective. I can't do that with a deadline.
I don't care if the answers "are out there" meaning the professor has returned graded tests and posted the answers; I do the work because I want to learn. That paradox of students wanting to review graded homeworks and students who want to learn by doing the homework is resolved practically by those that are more vocal about their wants.
I'm happy to take my time, doing my best (with the understanding that my best takes time) and observing my grade afterwards. Sometimes my learning takes me down paths that harder and more rewarding with the consequence that I take longer than others: I am a slow learner.
I don't think students are the blameless victim of this "system". I know of a continuous evaluation system adopted in a college, where the final tests had a less than 50% weightage for the grade. Thr students rebelled. It was far easier for a majority to burn the midnight oil for a few weeks than be diligent through the course. The test model has stuck because it's convenient for everyone, regardless of its clear weaknesses.
I'm one of those weird outliers I guess, but even when there were only 1-2 exams for the whole course I stayed diligent throughout. The night before a test I just did a bit of review, and got a good nights sleep. I never understood those who tried to cram because it's not something that would ever work for how I learn and understand.
Likely unpopular opinion here, but it seems a little unfair to fault YC founders for believing "that the way to win was to hack the test", when the YC application itself would seemingly select for founders that exhibit this behavior with questions like "When have you most successfully hacked a non-computer system to your advantage?"
I'd posit the YC application in and of itself shares some of the facets that the article is critiquing in a "test". The fact there are paid services popping up to review YC applications reminds me of SAT Prep services.
I'd posit the YC application in and of itself shares some of the facets that the article is critiquing in a "test". The fact there are paid services popping up to review YC applications reminds me of SAT Prep services.
> In theory you shouldn't have to prepare for a test in a class any more than you have to prepare for a blood test.
The key here is the “in theory” part. It is very hard to design exams that actually test your understanding of some topic as opposed to your memorization skills.
As somebody who recently got his bachelor’s CS degree (currently working on my master’s degree), and who also puts more emphasis on actually learning rather that getting good grades, I can tell you that understanding a subject might be enough to pass, but is seldom enough to get good grades.
The key here is the “in theory” part. It is very hard to design exams that actually test your understanding of some topic as opposed to your memorization skills.
As somebody who recently got his bachelor’s CS degree (currently working on my master’s degree), and who also puts more emphasis on actually learning rather that getting good grades, I can tell you that understanding a subject might be enough to pass, but is seldom enough to get good grades.
This is especially true in a domain that has no practical purpose. In computer science, the "exam" could be to go solve some hard problem, or write a program with a particular effect. In engineering, you can have students build a bridge and then stress test it. In athletics, the exam is competition. In business, it's making money. In dating, it's finding a mate (or whatever your personal goals are in that space). In farming, it's making food come out of the ground. For writers, it's a compelling or profitable story or book. For painters, it's a painting. Woodworkers can build a chair.
All those domains are testable. But what is the practical work product of a deep knowledge of medieval history? It's extremely difficult to test the past, so it's certainly not predictions about what already happened. Nobody needs knowledge of medieval history for any practical purpose in the modern day. There is no possible way to test for this knowledge in a practical scenario because there is no practical outlet for the knowledge. Literary criticism is the same, along with much that is called "liberal arts" today. So for subjects like this, exams and essays are the only possible work product, and it's true that at that point you're going to have to use contrived tests, since real tests don't exist.
All those domains are testable. But what is the practical work product of a deep knowledge of medieval history? It's extremely difficult to test the past, so it's certainly not predictions about what already happened. Nobody needs knowledge of medieval history for any practical purpose in the modern day. There is no possible way to test for this knowledge in a practical scenario because there is no practical outlet for the knowledge. Literary criticism is the same, along with much that is called "liberal arts" today. So for subjects like this, exams and essays are the only possible work product, and it's true that at that point you're going to have to use contrived tests, since real tests don't exist.
Athletics and engineering have exactly same problem as medieval history. For engineering, just because it is practical does not mean you cant memorize it. If there is defined set of problems, you can memorize solutions. The same programmer can be able to solve one difficult problem and fail with other difficult problem.
The bridge test can be passed by memorizing one bridge.
The history course from American university required students to write few paragraphs as answers to around 5 questions that covered topics from lectures. Which is pretty close to what actual historians do.
Practical usage of medieval history: know when people are bullshiting for ideological reasons when they blable about how people in history behaved, what values they had and origins of certain traditions. Pretty much all "in the past people would" is a fantasy.
The bridge test can be passed by memorizing one bridge.
The history course from American university required students to write few paragraphs as answers to around 5 questions that covered topics from lectures. Which is pretty close to what actual historians do.
Practical usage of medieval history: know when people are bullshiting for ideological reasons when they blable about how people in history behaved, what values they had and origins of certain traditions. Pretty much all "in the past people would" is a fantasy.
>But what is the practical work product of a deep knowledge of medieval history? It's extremely difficult to test the past, so it's certainly not predictions about what already happened.
This strikes me as a really sad way to see the world.
Would you then agree that say, archaeology or palaeontology are things that nobody needs knowledge of for any practical purpose in the modern day? If you disagree, why?
This strikes me as a really sad way to see the world.
Would you then agree that say, archaeology or palaeontology are things that nobody needs knowledge of for any practical purpose in the modern day? If you disagree, why?
I didn't say we shouldn't study those things. I just said that there is not a much of a practical work product, and especially not one that a student could produce.
But, I also think that probably more people are studying them than need to be. If that is a student's interest, then well enough. Otherwise, I'm not sure that, e.g., a middle school medieval history class is a good use of a student's time.
But, I also think that probably more people are studying them than need to be. If that is a student's interest, then well enough. Otherwise, I'm not sure that, e.g., a middle school medieval history class is a good use of a student's time.
There is a practical work product: research, formation of theories, etc. Edit: Possible new ways to understand the modern world. Just like archaeology or palaeontology.
In my opinion, the issue in liberal arts education is with grade inflation, along with the scalability of assessing learning discussed in the essay/these comments.
I suspect that really rigorous liberal arts education is much harder to do at scale than really rigorous science/engineering education because to be blunt, there is less room for argument in scientific concepts.
For example, properly grading student essays and research is more difficult at scale in the liberal arts and the level of the whole thing gets dumbed down as a result.
In my opinion, the issue in liberal arts education is with grade inflation, along with the scalability of assessing learning discussed in the essay/these comments.
I suspect that really rigorous liberal arts education is much harder to do at scale than really rigorous science/engineering education because to be blunt, there is less room for argument in scientific concepts.
For example, properly grading student essays and research is more difficult at scale in the liberal arts and the level of the whole thing gets dumbed down as a result.
Hackathons are a great example of this. Ostensibly they should just be about the project that is the coolest or most interesting. But participants started to realize that it wasn't really about the most fascinating project. It was about the project that got the usually non-technical, naive judges to go "ooooh". People started hacking hackathons by building projects that were totally infeasible and completely fake in terms of technical implementation, but had some sort of wow factor through buzzwords (AI/ML/CV/VR/etc) or "making the world a better place".
It's no coincidence that some of the largest hackathons are run and attended by students from the top universities pg mentions in the essay.
Which is why YC's hackathon was so satisfying. It was great seeing people pitch the normal hackathon-y ideas, then getting simple but direct questions such as "Why would someone use this app?" or "Why wouldn't I use <alternative> instead?".
It's no coincidence that some of the largest hackathons are run and attended by students from the top universities pg mentions in the essay.
Which is why YC's hackathon was so satisfying. It was great seeing people pitch the normal hackathon-y ideas, then getting simple but direct questions such as "Why would someone use this app?" or "Why wouldn't I use <alternative> instead?".
Interesting essay, however it is written from a perspective of having basic needs met and having the resources to spend time 'hacking' a system. When you are struggling to pay the bills for the basics, you don't focus on the hacking just on the surviving.
The question I have is how do you hack the system such that you help those who are struggling to survive?
The question I have is how do you hack the system such that you help those who are struggling to survive?
I love Paul Graham's articles, but isn't this Goodhart's Law but with more words?
(When a measure becomes a target, it ceases to be a good measure)
Searched the page for Goodhart and found this. There's added value here as this is a discussion of why Goodhart's Law is bad and how it's so prevalent in our education and business systems. I knew about the law but still got a lot out of this essay.
I did a Ctrl+f "goodhart", too. Glad to see that there is growing awareness of this principle... I think it underlies a lot of basic problems with society/tech/government in general. There is a related real-life parable I like about the ww2 plane designer who put more armor where returning bombers _weren't_ shot up, but I forget his name.
The problem is not that fundraising feels like a test, it's that fundraising is a test posed by authority figures, and by pg's own definition, is probably hackable. How hackable is proportional to the quality of the VC.
Luckily startups themselves are indeed not a test. So you can avoid any test by making enough revenue to not need external funding, but the second best strategy is take the VC's test without trying to hack it as your own test of the investor. Assuming you're a good investment, you'll beat those trying to hack the test unless the investor's test is susceptible to hacking. If you're not a good investment (which is by definition not obvious in the early days), maybe you should just try to learn to hack the test? Ironically, one of YC's biggest value adds is that it helps you hack those tests by lending you their name.
Luckily startups themselves are indeed not a test. So you can avoid any test by making enough revenue to not need external funding, but the second best strategy is take the VC's test without trying to hack it as your own test of the investor. Assuming you're a good investment, you'll beat those trying to hack the test unless the investor's test is susceptible to hacking. If you're not a good investment (which is by definition not obvious in the early days), maybe you should just try to learn to hack the test? Ironically, one of YC's biggest value adds is that it helps you hack those tests by lending you their name.
Paul, though he doesn't reference his previous essay [0] (which has a section on gaming the system) in the current one, did write then that fooling the investors and treating it like a test is only going to delay the inevitable.
[0] http://paulgraham.com/before.html
[0] http://paulgraham.com/before.html
I've found about how to hack the tests quite early on. I've had straight A's until I no longer cared about the grades. I've also realized that entering and pursuing a career at a tech corporations is a game as well.
What has been driving me to startups is the fact that creating a profitable business has no place for hacks.
Fundraising is a game that could be easily hacked though. I thought it was only possible to hack the fundraising game for a couple of rounds but Adam Neumann et al. changed my mind that hacks could carry a business even to post-IPO.
What has been driving me to startups is the fact that creating a profitable business has no place for hacks.
Fundraising is a game that could be easily hacked though. I thought it was only possible to hack the fundraising game for a couple of rounds but Adam Neumann et al. changed my mind that hacks could carry a business even to post-IPO.
> What has been driving me to startups is the fact that creating a profitable business has no place for hacks.
Not sure that's true. Marketing is a hack, for one. If you have the best product but nobody knows about it, it's no use.
More generally, there's a reason that the tech career ladder is hackable: performance reviews are both important and also too expensive to do right. If you want a real evaluation of what I've done over the past year, what you really need is someone following me 40 hours a week, noting how I contributed (positively or negatively) at small meetings, keeping track of whether my projects are late because I helped someone with something truly important or I spent my time on HN, etc. But you can't assign one reviewer per employee, so you make an approximate process where employees self-report what they did and managers report the fraction they've seen. You also can't get rid of the process, because a simple profit motive demands you incentivize employees for actually delivering business value. So you have a process that's vulnerable to exploits like flashy launches that will wither in a year.
If you as a business owner aren't evaluating your employees, you won't be profitable. If you're watching each moment of your employees, you're wasting time. If you spend your time developing a fairer review process, you're not working on your actual business. If you don't hire employees and just put yourself out there in the market, your customers certainly aren't evaluating you fairly. (And of course your employees and potential hires are evaluating you on partial data, and it's in your interest to hire and retain good employees.) So whatever you do, it's hackable, and if you don't play the game you'll forfeit it.
Not sure that's true. Marketing is a hack, for one. If you have the best product but nobody knows about it, it's no use.
More generally, there's a reason that the tech career ladder is hackable: performance reviews are both important and also too expensive to do right. If you want a real evaluation of what I've done over the past year, what you really need is someone following me 40 hours a week, noting how I contributed (positively or negatively) at small meetings, keeping track of whether my projects are late because I helped someone with something truly important or I spent my time on HN, etc. But you can't assign one reviewer per employee, so you make an approximate process where employees self-report what they did and managers report the fraction they've seen. You also can't get rid of the process, because a simple profit motive demands you incentivize employees for actually delivering business value. So you have a process that's vulnerable to exploits like flashy launches that will wither in a year.
If you as a business owner aren't evaluating your employees, you won't be profitable. If you're watching each moment of your employees, you're wasting time. If you spend your time developing a fairer review process, you're not working on your actual business. If you don't hire employees and just put yourself out there in the market, your customers certainly aren't evaluating you fairly. (And of course your employees and potential hires are evaluating you on partial data, and it's in your interest to hire and retain good employees.) So whatever you do, it's hackable, and if you don't play the game you'll forfeit it.
> What has been driving me to startups is the fact that creating a profitable business has no place for hacks.
I feel that this hack/not hack distinction comes from confusion of goals. If your end goal really is creating a profitable business then hack/not hack is not a useful distinction. Everything you do either gets you closer to your goal (then it is good, hack or not) or not (then it is bad, even if it "feels right"). But suppose your real goal is "doing most good for society" or whatever and you think that you can achieve this goal by building a profitable business. Then if you e.g. create some bullshit product and market the hell of it (thus achieving your proxy goal), it will feel like a hack that doesn't get you closer to your real goal.
TLDR: hack/not hack distinction arises when there are two goals: a proxy and the real one. A hack is something that gets you closer to your proxy goal but not the real one.
I feel that this hack/not hack distinction comes from confusion of goals. If your end goal really is creating a profitable business then hack/not hack is not a useful distinction. Everything you do either gets you closer to your goal (then it is good, hack or not) or not (then it is bad, even if it "feels right"). But suppose your real goal is "doing most good for society" or whatever and you think that you can achieve this goal by building a profitable business. Then if you e.g. create some bullshit product and market the hell of it (thus achieving your proxy goal), it will feel like a hack that doesn't get you closer to your real goal.
TLDR: hack/not hack distinction arises when there are two goals: a proxy and the real one. A hack is something that gets you closer to your proxy goal but not the real one.
> If getting into college were merely a matter of having the quality of one's mind measured by admissions officers the way scientists measure the mass of an object, we could tell teenage kids "learn a lot" and leave it at that.
It is equally naive to think some idea of "quality of mind" or what someone has already learned in their first ~17 years is what they should go by. This is just shifting from one bias to another.
Evaluating a human being, reducing them to a number or a boolean, is hard, and people tasked with doing this, no matter how confident or how much thought and intelligence they put into it, will get it wrong, often with bad consequences. So perhaps we should take such systems lightly, stay humble, and not get overconfident in our judgements.
It is equally naive to think some idea of "quality of mind" or what someone has already learned in their first ~17 years is what they should go by. This is just shifting from one bias to another.
Evaluating a human being, reducing them to a number or a boolean, is hard, and people tasked with doing this, no matter how confident or how much thought and intelligence they put into it, will get it wrong, often with bad consequences. So perhaps we should take such systems lightly, stay humble, and not get overconfident in our judgements.
And yet I’ve interviewed with 2 or 3 different y combinator companies that hire based on coding tests that test nothing other than that you’ve memorized “cracking the coding interview”.
This is one of the reasons I quit academia and am skeptical of much of the research coming out of universities. As a grad student, at some point, it became pretty clear that I would have to choose between doing good science or doing career advancing work (by ignoring biases in my research, p-hacking etc.). Universities have self selected for people who do bad but publishable science.
The problem might be a lot bigger.
Possibly most of one's life is spent (over?) fitting to some "test" in a system, or "expectation" someone else has, instead of doing what they truly want: trying to pass a test at school, trying to land that job (you think) you want, trying to be socially acceptable in the schoolyard, at a party, or in the office, trying to please a parent, a friend, a partner, trying to conform to some unconsciously shared vision of "success", trying to conform to your own vision of "success", etc.
The list is long.
Fairer, unhackable systems would definitely help in reducing the amount of such "tests" we artificially created as a society. We should strive for this.
However is it possible to solve human nature? To solve "affinity bias", "confirmation bias", "psychological needs", etc.? Is it possible to create a society in which everyone is truly free? Or at least as free as possible from all this? Is it possible to solve freedom & wisdom? :-)
Possibly most of one's life is spent (over?) fitting to some "test" in a system, or "expectation" someone else has, instead of doing what they truly want: trying to pass a test at school, trying to land that job (you think) you want, trying to be socially acceptable in the schoolyard, at a party, or in the office, trying to please a parent, a friend, a partner, trying to conform to some unconsciously shared vision of "success", trying to conform to your own vision of "success", etc.
The list is long.
Fairer, unhackable systems would definitely help in reducing the amount of such "tests" we artificially created as a society. We should strive for this.
However is it possible to solve human nature? To solve "affinity bias", "confirmation bias", "psychological needs", etc.? Is it possible to create a society in which everyone is truly free? Or at least as free as possible from all this? Is it possible to solve freedom & wisdom? :-)
Hacking tests is a lesson much older than modern schools. It's a fundamental part of being human, or indeed, an animal.
When a peacock is being judged for his feathers or song or mating dance, these are all being used as proxies for the actually important thing, the quality of his genes. And it's not just a male thing; women do all kinds of work to appeal to arbitrary male preferences.
On the other hand, survival of the fittest is an honest judge, so maybe these peacock tests work better than we think?
When a peacock is being judged for his feathers or song or mating dance, these are all being used as proxies for the actually important thing, the quality of his genes. And it's not just a male thing; women do all kinds of work to appeal to arbitrary male preferences.
On the other hand, survival of the fittest is an honest judge, so maybe these peacock tests work better than we think?
I feel like Paul Graham is unlearning his own experience. Maybe, he's at the top of the game and tries to figure out why it's messed up. Life is full of hacks. People hack stuff: school, test, career. The site we're on now called itself Hacker News. I suppose Paul Graham gave it the name.
When I was in school, I did all kinds of hacks. I didn't graduate high school in the US. I took the GED to go to college. I did challenge tests to skip classes and save money. I got to know people who collected old tests. They trade old tests with me for homework help. Everyone graduated, got jobs, and continued their life hacks.
The focus on learning and doing is important. As long as we're delivering, we can hack some stuff. The philosophy debate between Kant and Locke will never end. There's a balance for each person. We need to discover it ourselves.
When I was in school, I did all kinds of hacks. I didn't graduate high school in the US. I took the GED to go to college. I did challenge tests to skip classes and save money. I got to know people who collected old tests. They trade old tests with me for homework help. Everyone graduated, got jobs, and continued their life hacks.
The focus on learning and doing is important. As long as we're delivering, we can hack some stuff. The philosophy debate between Kant and Locke will never end. There's a balance for each person. We need to discover it ourselves.
It seems like you're responding to the idea that "hacking is bad" and that's not at all what the essay got across to me. In fact, it outright agrees that "the focus on learning and doing is important."
Instead of "hacking is bad", the essay was saying "incentivizing young people with a single easily-hacked methodology is bad".
Instead of "hacking is bad", the essay was saying "incentivizing young people with a single easily-hacked methodology is bad".
> If you merely read good books on medieval history, most of the stuff you learned wouldn't be on the test. It's not good books you want to read, but the lecture notes and assigned reading in this class.
It is the professor's job to decide which parts of medieval history are the most important to learn. Therefore, those are what I need to study.
Maybe not all professors do this effectively—but as long as we're talking about ideals, this is how it should work.
It is the professor's job to decide which parts of medieval history are the most important to learn. Therefore, those are what I need to study.
Maybe not all professors do this effectively—but as long as we're talking about ideals, this is how it should work.
Right, I had a similar reaction.
Autodidacts often end up reading a bunch of stuff from here and there with sub-par results. One of the main benefits of a (university) course is focus and guidance. That the prof or teacher selects specific topics they deem important for that stage of education. As a student it can be difficult to judge that yourself.
Autodidacts often end up reading a bunch of stuff from here and there with sub-par results. One of the main benefits of a (university) course is focus and guidance. That the prof or teacher selects specific topics they deem important for that stage of education. As a student it can be difficult to judge that yourself.
Yeah, this came across to me as overly simplistic not really thought out argument too. I dont think that college medieval history not being the "students can pick any part of it and anything you learn about medieval history is good to go" is some kind of massive issue with medieval history course.
It does not mean that you did not learned or that grade does not measure learning. It just means the topic is not whole medieval history (which would be ridiculously broad) but only selected parts of it.
It is kind of like complaining that algorithms 101 are allowing students to pick any algorithms to learn and require everyone to learn the same set of algorithms.
It does not mean that you did not learned or that grade does not measure learning. It just means the topic is not whole medieval history (which would be ridiculously broad) but only selected parts of it.
It is kind of like complaining that algorithms 101 are allowing students to pick any algorithms to learn and require everyone to learn the same set of algorithms.
> And even most of that you can ignore, because you only have to worry about the sort of thing that could turn up as a test question.
That part just did not describe my experience in college. Maybe I was bad at knowing what "could turn up as a test question."
My college also did not have any old tests floating around, at least as far as I was aware.
My college also did not have any old tests floating around, at least as far as I was aware.
In classes I have seen, things you could ignore were minority of it and usually side stuff like funny anecdotes, minor players that are there for you to get feel for the context, famous nun dick tree and such.
This stuff helps you learn important stuff and makes it all less dry, but it absolutely does not make sense to penalize students for not learning it actively.
This stuff helps you learn important stuff and makes it all less dry, but it absolutely does not make sense to penalize students for not learning it actively.
I think we should just drop book work questions altogether and allow students to take notes into the exam. Just make the tests about analysis and synthesis, which is the stuff you should be able to do if you truly understood something and didn't just memorize some facts.
I personally don’t think this is _the_ problem with education. Education has a lot of shortcomings but, as some siblings point out, having a standardized measure of learning is a _hard_ problem.
The thing that had always left me a bit uneasy after my rocky road through education was just how damned certain everyone is about everything. Our education system, at least up through undergraduate unless you dip your toes into undergraduate research (and even sometimes still then!), teaches you things we “know” about the world we live in. Humans have discovered an _incredible_ amount about the world around us, but we have a _tonne_ left to figure out. We don’t lean into the unknown with our education system until late stage. The masses miss out on the beauty of the unknown.
The thing that had always left me a bit uneasy after my rocky road through education was just how damned certain everyone is about everything. Our education system, at least up through undergraduate unless you dip your toes into undergraduate research (and even sometimes still then!), teaches you things we “know” about the world we live in. Humans have discovered an _incredible_ amount about the world around us, but we have a _tonne_ left to figure out. We don’t lean into the unknown with our education system until late stage. The masses miss out on the beauty of the unknown.
Good grades = Passing well paying jobs resume filter = startup initial capital for 90% of the population without access to the bank of mom and dad or a large inheritance.
PG starting to lose touch with regular joe.
PG starting to lose touch with regular joe.
Yeah, I had a similar take. A lot of what he writes seems to come back to keeping the YC pipeline full.
Half way through the essay I was thinking, surely this can't be the first time PG has realized the idea of "playing the game", which is what I've known it as all my life. But then at the end of the essay he uses that exact same phrase to hit home the point that "playing the game" is antiquated now if one desires to become rich.
While I'd agree there are more opportunities for individuals or small groups to get rich now than in 1960, for the vast majority of us, the best bet to become rich is to "play the game".
Half way through the essay I was thinking, surely this can't be the first time PG has realized the idea of "playing the game", which is what I've known it as all my life. But then at the end of the essay he uses that exact same phrase to hit home the point that "playing the game" is antiquated now if one desires to become rich.
While I'd agree there are more opportunities for individuals or small groups to get rich now than in 1960, for the vast majority of us, the best bet to become rich is to "play the game".
Had to chime in with a nod of agreement to both brador and 0x445442 .
I'm not being normally someone to comment on such things ( why bother with the effort?). I am making concrete efforts to get noticed by identifying groups/people who might align with my own thoughts/beliefs for my own vested interests.
I'm not being normally someone to comment on such things ( why bother with the effort?). I am making concrete efforts to get noticed by identifying groups/people who might align with my own thoughts/beliefs for my own vested interests.
Paul Graham is one of the most powerful people on the on the planet. Of course he sees "hacking bad tests" (aka politics) as something that will become obsolete. He has the privilege of ejecting from any political situation he finds unpleasant.
Meanwhile, the other 7B population with substantially less power, we have to deal with each other. Politics are, and always will be, a thing. As long as people exist.
Meanwhile, the other 7B population with substantially less power, we have to deal with each other. Politics are, and always will be, a thing. As long as people exist.
Love it. It has been a while since I read a P.G essay, although this was what first attracted me to HN. It also means that it is OK to work on your idea without entering YC. After all, being accepted into YC is just another test by authority at this point.
Artificial "hackable" tests are opportunities to see progress at scale. While I appreciate the recognition that bespoke, perhaps Montessori, education might better prepare people for the open ended ness of life, there aren't enough unicorn startup opportunities in the world to use that as a primary means of knowing how well you know things. Evaluation through essays is better than multiple choice, the Socratic method is better than essays, but we don't know how to scale those things yet. Perhaps some day we will have a Young Ladies Illustrated Primer, but until then we have hackable tests.
I think the elephant in the room is the question of purpose. What is the purpose guiding your actions? Getting good grades is a mean that /can/ be necessary to achieve the purpose you have chosen in life. But it’s not sufficient for most purposes. On the other hand, once you have a purpose, you truly make the most of education.
That is a great essay. Thank you for sharing.
I've worked in tech for quite a while now. I used to buy into the meritocracy cant but as the years wore on, I realized how much bullshit it was.
I think the biggest moment came when I started to think about how to explain to people in my new home, South Africa, that working in tech would free you economically. There's some truth to that. But that's not the whole truth.
The truth is that all those years ago when I got my first coding job in financial services, I ticked 3 important boxes for the interview: white, male, and university-educated.
I've worked in tech for quite a while now. I used to buy into the meritocracy cant but as the years wore on, I realized how much bullshit it was.
I think the biggest moment came when I started to think about how to explain to people in my new home, South Africa, that working in tech would free you economically. There's some truth to that. But that's not the whole truth.
The truth is that all those years ago when I got my first coding job in financial services, I ticked 3 important boxes for the interview: white, male, and university-educated.
As an engineer, this makes me wish there were a better path to FANG employment than hacking their whiteboarding interviews by leetcoding for weeks. They probably do this because it's more objective and simpler than something like the "long conversation with a professor" test that pg suggests.
This is a great essay. But, I find it odd that YC's famous "cohorts" are invented by PG. Cohorts work because startups in that batch compare against each other, which in itself is a grading system. Getting into YC, topping the batch and raising money are inevitable symptoms of "cohort" thinking. We could argue YC uses cohorts to make startups compete for meaningful metrics instead of vanity metrics. But still, YC is based on this innate human nature to compete and get social acceptance.
I think at various points in life, you worry about the whole system and its effects on people, not just maximizing your local niche. This seems good, both morally and from a systems perspective.
For me when younger this was about Ivan Illich (deschooling society), Feynman (the Brazilian light polarization story) etc. Now I'm older it's reading the anger in 'Excellent Sheep', watching the pain in low-trust workplaces, and telling my kids I value real learning over test-passing (they listen, but huge pressure to conform).
It seems true that a new field is more intellectually stimulating and liberating than a codified one (I've talked to older female analyst programmers from the 70s/80s who had a blast with computers and left when it got rigid and boring, I worked in Gurgaon early 00s when those interested in IT were self-educating geeks doing it for love, who later got moved aside + down when the money arrived). Maybe this is why IT people chase the new shiny, as a defensive strategy against codification, a red queen game to keep the game interesting ( though inefficient).
Anyway, glad Paul is publishing these viewpoints, even if the actions to be taken are not clear. Also, society is over-geeked currently. It's not obvious that hackers ( or intellectuals or logistics experts or technocrats) should have too much power instead of History students, or politicians or sports-stars or whoever. Life is both a game and not a game, short term gaming has long-term consequences. History and Biology and Literature teach us this clearly.
For me when younger this was about Ivan Illich (deschooling society), Feynman (the Brazilian light polarization story) etc. Now I'm older it's reading the anger in 'Excellent Sheep', watching the pain in low-trust workplaces, and telling my kids I value real learning over test-passing (they listen, but huge pressure to conform).
It seems true that a new field is more intellectually stimulating and liberating than a codified one (I've talked to older female analyst programmers from the 70s/80s who had a blast with computers and left when it got rigid and boring, I worked in Gurgaon early 00s when those interested in IT were self-educating geeks doing it for love, who later got moved aside + down when the money arrived). Maybe this is why IT people chase the new shiny, as a defensive strategy against codification, a red queen game to keep the game interesting ( though inefficient).
Anyway, glad Paul is publishing these viewpoints, even if the actions to be taken are not clear. Also, society is over-geeked currently. It's not obvious that hackers ( or intellectuals or logistics experts or technocrats) should have too much power instead of History students, or politicians or sports-stars or whoever. Life is both a game and not a game, short term gaming has long-term consequences. History and Biology and Literature teach us this clearly.
> No one was pulling all-nighters two weeks into the semester.
As a physics student, I can say we definitely pulled 2am-ers 2 weeks into the semester. Our Condensed Matter professor gave us some of the most random questions that no one could anticipate. To be honest, I was one of those "diligent" students who got good grades. I'll tell you right now his quizzes were not hackable. The following held true:
On the other hand, I've also had professors who give exams pulled from the internet and allow use of the internet while taking those exams. I hacked that immediately during the exam, then almost got in trouble for it.
> ...if the professor tells you that there were three underlying causes of the Schism of 13... you'd better know them.
A professor like my Condensed Matter Physics professor would go on and on about 3 fundamental causes and then quiz students about the 5 fundamental causes, giving a zero to anyone who couldn't think of all the base 3, then grading the remaining students who thought creatively of 2 additional causes against each other on a bell curve. :)
As a physics student, I can say we definitely pulled 2am-ers 2 weeks into the semester. Our Condensed Matter professor gave us some of the most random questions that no one could anticipate. To be honest, I was one of those "diligent" students who got good grades. I'll tell you right now his quizzes were not hackable. The following held true:
|quiz subject matter| >> |lecture matter|
On the other hand, I've also had professors who give exams pulled from the internet and allow use of the internet while taking those exams. I hacked that immediately during the exam, then almost got in trouble for it.
> ...if the professor tells you that there were three underlying causes of the Schism of 13... you'd better know them.
A professor like my Condensed Matter Physics professor would go on and on about 3 fundamental causes and then quiz students about the 5 fundamental causes, giving a zero to anyone who couldn't think of all the base 3, then grading the remaining students who thought creatively of 2 additional causes against each other on a bell curve. :)
> A professor like my Condensed Matter Physics professor would go on and on about 3 fundamental causes and then quiz students about the 5 fundamental causes, giving a zero to anyone who couldn't think of all the base 3, then grading the remaining students who thought creatively of 2 additional causes against each other on a bell curve. :)
To me, this sounds like a plausible argument against such tests.
The issue of creatively grading responses to an ill-defined question often pops up in discussion here about interview practices. In those discussions, typically someone will say “I don’t do a generic whiteboard interview. Instead I do [idiosyncratic thing x]. It really gives me amazing insight into the candidate.”
Then someone else says “Yeah right, a weird test with unclear metrics just gives you a big empty space to fill in with all your biases and pick someone who answers the way you would”.
Of course, both sides are exaggerated here. But it’s not clear to me that “creative” tests are necessarily any better.
To me, this sounds like a plausible argument against such tests.
The issue of creatively grading responses to an ill-defined question often pops up in discussion here about interview practices. In those discussions, typically someone will say “I don’t do a generic whiteboard interview. Instead I do [idiosyncratic thing x]. It really gives me amazing insight into the candidate.”
Then someone else says “Yeah right, a weird test with unclear metrics just gives you a big empty space to fill in with all your biases and pick someone who answers the way you would”.
Of course, both sides are exaggerated here. But it’s not clear to me that “creative” tests are necessarily any better.
Forget the grade. Impress your profs. Let them learn from you.
For example, you could work on analog computers (look it up). The early AI systems were all analog, using motors on "volume knobs" to adjust the weights.
Analog is MUCH faster than digital. You could probably train an analog neural net "at movie speeds" because it all happens in parallel. Analog weights are trivial (capacitors), backprop is just differentiation which analog does trivially. Function inversion just involves putting the function in the feedback loop of an op-amp. A board with 100x100 mini-caps would be dirt cheap and would have 10,000 "weights".
Besides being "up to speed on AI NNs" you would also be breaking new ground (well, really old ground but nobody remembers anything before last year).
Imagine talking the department into an "analog AI lab" in cooperation with the EE dept and you get to be "the person". Heck, now you've nearly "joined the department" rather than being a student.
Think of college as a big company that lets you do ANYTHING. Profs have to prove themselves to get tensure. Students have to "get good grades" and "learn to hack the FAANG interviews so they can be a 'good cultural fit (aka grade hackers)". But YOU are at college as a "free agent" and can use all the resources to excel at what you love. Imagine having 4 years at a company with the resources of a college that lets you do ANYTHING.
You will never have it this good again. Forget the grade.
For example, you could work on analog computers (look it up). The early AI systems were all analog, using motors on "volume knobs" to adjust the weights.
Analog is MUCH faster than digital. You could probably train an analog neural net "at movie speeds" because it all happens in parallel. Analog weights are trivial (capacitors), backprop is just differentiation which analog does trivially. Function inversion just involves putting the function in the feedback loop of an op-amp. A board with 100x100 mini-caps would be dirt cheap and would have 10,000 "weights".
Besides being "up to speed on AI NNs" you would also be breaking new ground (well, really old ground but nobody remembers anything before last year).
Imagine talking the department into an "analog AI lab" in cooperation with the EE dept and you get to be "the person". Heck, now you've nearly "joined the department" rather than being a student.
Think of college as a big company that lets you do ANYTHING. Profs have to prove themselves to get tensure. Students have to "get good grades" and "learn to hack the FAANG interviews so they can be a 'good cultural fit (aka grade hackers)". But YOU are at college as a "free agent" and can use all the resources to excel at what you love. Imagine having 4 years at a company with the resources of a college that lets you do ANYTHING.
You will never have it this good again. Forget the grade.
What if we moved to pass/fail grading?
Every assignment and test is just marked pass or fail. Realistically tempered by what percentage you're allowed to fail for the class, e.g. 40% for university calculus - no different than grading on a curve today.
Fail any N assignments/tests and fail the course. Maybe some are weighted to count extra, e.g. Final project, midterm, final exam.
This is really not much different than the current system, but if you stop caring about the difference between C and A, students can either slack off or focus more on learning (if the subject is interesting).
Nobody cares about your grades after your first job anyway. I had nearly a perfect GPA in college, and it was a terrible waste of time in hindsight. If I'd spent more time socializing maybe I would have made some lifelong contacts that would have benefited my career or become co-founders with me.
I've always thought to tell my hypothetical kids to not sweat the grades and remember that the benefit of university is really the friends (and sometimes life partner) who you meet there. It's total shit when it comes to learning stuff. A self motivated person can learn more in a year than a 4 year degree. Somewhat depends on the subject though - I wouldn't want a self taught doctor who didn't do the practical parts of the education.
Every assignment and test is just marked pass or fail. Realistically tempered by what percentage you're allowed to fail for the class, e.g. 40% for university calculus - no different than grading on a curve today.
Fail any N assignments/tests and fail the course. Maybe some are weighted to count extra, e.g. Final project, midterm, final exam.
This is really not much different than the current system, but if you stop caring about the difference between C and A, students can either slack off or focus more on learning (if the subject is interesting).
Nobody cares about your grades after your first job anyway. I had nearly a perfect GPA in college, and it was a terrible waste of time in hindsight. If I'd spent more time socializing maybe I would have made some lifelong contacts that would have benefited my career or become co-founders with me.
I've always thought to tell my hypothetical kids to not sweat the grades and remember that the benefit of university is really the friends (and sometimes life partner) who you meet there. It's total shit when it comes to learning stuff. A self motivated person can learn more in a year than a 4 year degree. Somewhat depends on the subject though - I wouldn't want a self taught doctor who didn't do the practical parts of the education.
So while I agree with what he is saying directly, to me, testing is a function of filtering at large scale. To a secondary degree, measuring as feedback for that system.
Testing and grades in general are a society level filtering mechanism and is a measuring proxy for how the work force filters for candidates at the entry level. As most know, once you've got about 10 years, most of that is largely ignored at the bottom of your resume.
Entrepreneurs, in the eyes of this system, are deviants of sorts. Hacks who couldn't/didn't want to make it in the system, but turns out that system is not the only way. A non-western example of this are a lot of the families that came up in money in China the past few decades. They didn't make it in the rigor of Chinese school system, but when they said fuck-it, I'll survive by starting something, they were wildly successful.
Controversially there's evidence pointing to education doesn't help people at scale[1]. So that whole structure, debatably, has diminishing returns.
So, I agree with Paul Graham 100%, schooling isn't the only way, and I mean that by - if my children decide they do not want to attend school anymore because they have a plan that, while unconventional, can do great things; Support all the way.
Testing and grades in general are a society level filtering mechanism and is a measuring proxy for how the work force filters for candidates at the entry level. As most know, once you've got about 10 years, most of that is largely ignored at the bottom of your resume.
Entrepreneurs, in the eyes of this system, are deviants of sorts. Hacks who couldn't/didn't want to make it in the system, but turns out that system is not the only way. A non-western example of this are a lot of the families that came up in money in China the past few decades. They didn't make it in the rigor of Chinese school system, but when they said fuck-it, I'll survive by starting something, they were wildly successful.
Controversially there's evidence pointing to education doesn't help people at scale[1]. So that whole structure, debatably, has diminishing returns.
So, I agree with Paul Graham 100%, schooling isn't the only way, and I mean that by - if my children decide they do not want to attend school anymore because they have a plan that, while unconventional, can do great things; Support all the way.
There is a parallel with one of pg's other essays, 'Why nerds are unpopular'[1]. He says (I paraphrase): 'nerds care more about being smart than being popular'. But that's another way of saying nerds think the tests you have to pass to be popular are (on balance) a waste of time, ie bad tests.
[1] http://paulgraham.com/nerds.html
[1] http://paulgraham.com/nerds.html
I can't find whose quote it is that "the most difficult thing is to teach people to think simply".
Unfortunately education cannot test that, because tests are by definition testing only a fragment of a system. Knowing and understanding one thing end-to-end is meaningful knowledge, and qualitatively better than knowing fragments of 100 things. Perhaps you can assess the former by grading a diploma thesis on a very specific subject. Tests may have an important role in motivating students to delve more deeply in subjects but they don't go beyond that.
There's similarly more value to knowing some technology "end-to-end" rather than specializing in only one level and never understanding what's below.
It's also interesting to read this:
> There are now ways to get rich by doing good work, and that's part of the reason people are so much more excited about getting rich than they used to be.
Which may reflect how things were 15 years ago. There were 2 threads in "Ask HN" last week which basically concluded that these days you're much better off (money wise) working for an uncreative role in FANG.
Unfortunately education cannot test that, because tests are by definition testing only a fragment of a system. Knowing and understanding one thing end-to-end is meaningful knowledge, and qualitatively better than knowing fragments of 100 things. Perhaps you can assess the former by grading a diploma thesis on a very specific subject. Tests may have an important role in motivating students to delve more deeply in subjects but they don't go beyond that.
There's similarly more value to knowing some technology "end-to-end" rather than specializing in only one level and never understanding what's below.
It's also interesting to read this:
> There are now ways to get rich by doing good work, and that's part of the reason people are so much more excited about getting rich than they used to be.
Which may reflect how things were 15 years ago. There were 2 threads in "Ask HN" last week which basically concluded that these days you're much better off (money wise) working for an uncreative role in FANG.
> Tests may have an important role in motivating students to delve more deeply in subjects but they don't go beyond that.
I feel like the university doesn't allow for time to stroll from the beaten path and explore topics that you're interested in. Most people I talked to have a very stressful studying experience. You know people care about your grade in the end, and you pay a lot of money for your degree. So it makes sense to also spent the majority of your time to learn the course material in and out.
I feel like the university doesn't allow for time to stroll from the beaten path and explore topics that you're interested in. Most people I talked to have a very stressful studying experience. You know people care about your grade in the end, and you pay a lot of money for your degree. So it makes sense to also spent the majority of your time to learn the course material in and out.
the gordian knot is more of a brutish non-solution to a hard problem. "thinking simply" is more about developing intuitive understanding to the point where you can also explain it simply
Interesting enough, I have the same philosophy - I never cared about grades, not even a degree really. I took what interested me and what I felt would help me. For me I always took 16-20 credit hours Per semester, including summers and averaged a GPA if 2.8-3.4 per semester or something. I also took enough classes in 5 years to get two bachelors.
Unfortunately, during job interviews many companies ask for GPA on graduation. I always had to explain “I learned a lot! I just pushed myself to the limit and when you do that you get B’s”
It’s always an interesting discussion and occasionally will lose my potential interviews. The truth is that school is really about certifying what you know, not learning. Personally, I think that should change, because building the skill of curiosity and The ability to research and understand a topic is far more valuable. But to learn to be curious you also have to be able to explore - can’t do that craming for test.
Unfortunately, during job interviews many companies ask for GPA on graduation. I always had to explain “I learned a lot! I just pushed myself to the limit and when you do that you get B’s”
It’s always an interesting discussion and occasionally will lose my potential interviews. The truth is that school is really about certifying what you know, not learning. Personally, I think that should change, because building the skill of curiosity and The ability to research and understand a topic is far more valuable. But to learn to be curious you also have to be able to explore - can’t do that craming for test.
Wonder what this guy thinks of Asian schools, where you literally have to dedicate yourself to studying 9 hours a day, 6 days a week, through middle and high school (where you have to dedicate 11+ hours regularly) for one test that literally determines your entire career path, unless you want to move to a different country that is.
"When a measure becomes a target, it ceases to be a good measure" -Charles Goodhart
I've read most of PG's letters but I think this is now my favorite because while I've noticed each of these issues separately I've never connected them together through this common flaw in our education system. I find it really profound.
I've read most of PG's letters but I think this is now my favorite because while I've noticed each of these issues separately I've never connected them together through this common flaw in our education system. I find it really profound.
I actually have found that, at least for math and science classes, focusing on learning was a less stressful way to do well on tests, even if cramming was potentially a workable way to get the grade too. The remaining problem is that often times the course load dictated that some classes had to be undertaken in hack mode.
I think the worst thing school teaches you -- especially STEM -- is that 99%+ questions have a certain right answer.
IRL most interesting and important questions are not anywhere close to that.
IRL most interesting and important questions are not anywhere close to that.
For anyone who enjoyed this, a lot of these ideas were really well discussed a few years back in Neil Postman's "Teaching as a Subversive Activity" (1969) and John Dewey's "Democracy & Education" (1916). As you can see these books are over 50 years old. These ideas aren't new, which goes to say how long we've known this and what little have we done.
The real question Graham poses here is: what is the point of schooling? At the moment, it teaches and conditions people to "hack tests" as Graham puts it. So people continue to do that throughout their lives.
It also conditions people to think there is a "correct" answer to problems. Which in the open-ended nature of the real questions we have to grapple with in life is a complete lie. A teacher will rarely say: "It depends."
What passes for "education" today is really just "training" and very far removed from real learning. People only learn about things that interest them. And you learn by asking questions, which most of the time will lead to further questions to which you will only ever find tentative, temporary answers. Not that different from how scientists treat their "solutions" to big questions as tentative.
What can we do to change that? Change schools. It's the one place where we have kids kept for 15 years by mandate! It's the biggest lever we have to change perceptions and attitudes. Remove tests and grading. Learning will free up when that happens. It's remarkable what happens when at the beginning of the year the teacher gives everyone an A. Let go of the idea that permeates American society that everything needs to be measured. Some things do, but things like intelligence can't and shouldn't.
I know the immediate response is that this is unpractical. And to that I ask why? Because we need a way to control? Control what? What district is "performing" better? When was a score on a math test a true measure of human potentiality?
We need to go back to core questions as to what is the point of schools and what is worth teaching and learning in today's world. The world has changed immensely, but our "curriculums" are the same. The only difference is we use tablets and Microsoft Word to hand in assignments.
I agree with Graham's conclusion. When we liberate ourselves from these useless idealogical metaphors we will free people and society up to stop hacking tests and start living.
The real question Graham poses here is: what is the point of schooling? At the moment, it teaches and conditions people to "hack tests" as Graham puts it. So people continue to do that throughout their lives.
It also conditions people to think there is a "correct" answer to problems. Which in the open-ended nature of the real questions we have to grapple with in life is a complete lie. A teacher will rarely say: "It depends."
What passes for "education" today is really just "training" and very far removed from real learning. People only learn about things that interest them. And you learn by asking questions, which most of the time will lead to further questions to which you will only ever find tentative, temporary answers. Not that different from how scientists treat their "solutions" to big questions as tentative.
What can we do to change that? Change schools. It's the one place where we have kids kept for 15 years by mandate! It's the biggest lever we have to change perceptions and attitudes. Remove tests and grading. Learning will free up when that happens. It's remarkable what happens when at the beginning of the year the teacher gives everyone an A. Let go of the idea that permeates American society that everything needs to be measured. Some things do, but things like intelligence can't and shouldn't.
I know the immediate response is that this is unpractical. And to that I ask why? Because we need a way to control? Control what? What district is "performing" better? When was a score on a math test a true measure of human potentiality?
We need to go back to core questions as to what is the point of schools and what is worth teaching and learning in today's world. The world has changed immensely, but our "curriculums" are the same. The only difference is we use tablets and Microsoft Word to hand in assignments.
I agree with Graham's conclusion. When we liberate ourselves from these useless idealogical metaphors we will free people and society up to stop hacking tests and start living.
Several schools of economic thought claims "Money" also operates as a test for "value for society". It's also as porous as a colander.
> I suspect many people implicitly assume that doing working in a field with bad tests is the price of making lots of money.
> Now you can make lots of money by making cool things.
An analogous set of incentive structures, which compels people in elite universities to play the tests applies to intra & inter-company interactions as well: companies who play for the tests will enjoy structural advantages over companies which do not; and this lever allows winner-takes-most mechanisms via buyouts, consolidations, private equity.
While founders, individually, may make out of this wealthy, companies, and society as a whole suffers, and playing for the tests, as a behavior, saturates.
> I suspect many people implicitly assume that doing working in a field with bad tests is the price of making lots of money.
> Now you can make lots of money by making cool things.
An analogous set of incentive structures, which compels people in elite universities to play the tests applies to intra & inter-company interactions as well: companies who play for the tests will enjoy structural advantages over companies which do not; and this lever allows winner-takes-most mechanisms via buyouts, consolidations, private equity.
While founders, individually, may make out of this wealthy, companies, and society as a whole suffers, and playing for the tests, as a behavior, saturates.
>For me, as for most students, the measurement of what I was learning completely dominated actual learning in college.
Well I never had this problem in school.
I remember a high school calculus exam where I got the answer to the derivatives of the fundamental functions wrong (simple memorization - boring!) but I got the delta epsilon part of the test perfect.
I got a C- on the exam, but the teacher wrote a comment that always stuck with me: "You understood the most complicated part , as usual!"
Same thing in University, I never did homework in groups. I wasn't looking for the answer to the questions on my problem sets, I wanted a unique answer, or a more elegant answer.
I would outdo the professors sometimes. One time I saved a ton of busywork on a problem by setting up a homomorphism with another vector space, proof done in 5 steps!
Well I never had this problem in school.
I remember a high school calculus exam where I got the answer to the derivatives of the fundamental functions wrong (simple memorization - boring!) but I got the delta epsilon part of the test perfect.
I got a C- on the exam, but the teacher wrote a comment that always stuck with me: "You understood the most complicated part , as usual!"
Same thing in University, I never did homework in groups. I wasn't looking for the answer to the questions on my problem sets, I wanted a unique answer, or a more elegant answer.
I would outdo the professors sometimes. One time I saved a ton of busywork on a problem by setting up a homomorphism with another vector space, proof done in 5 steps!
You need a metric to measure performance. It's hard to get such a metric right. You need this metric because the government institutionalized schooling and need to have aggregate data on performance.
However, for the student, it'd be better if they'd focus on the intersection of what they're interested in and what is societally relevant and not to be graded at all.
I know a friend of mine who went to a school for 1 year where he didn't get graded, all he learned was soft business skills. I can clearly see that he grew a lot from it. But I can also see that it is tough for anyone else to see that he learned anything at all. Partially this is because he isn't graded and the behaviors he picked up are now natural to him and it just looks like it was always there.
However, for the student, it'd be better if they'd focus on the intersection of what they're interested in and what is societally relevant and not to be graded at all.
I know a friend of mine who went to a school for 1 year where he didn't get graded, all he learned was soft business skills. I can clearly see that he grew a lot from it. But I can also see that it is tough for anyone else to see that he learned anything at all. Partially this is because he isn't graded and the behaviors he picked up are now natural to him and it just looks like it was always there.
The most interesting class I took in my entire studies (Soft Matter physics) was also the one with the most interesting exams.
Our year's 2-hour exam was the following. The professor gave us a small cup of water, a piece of paper, and a piece of toilet paper. There was only 2 questions - place one end of the paper in the water. Describe what happens. Explain - place one end of the toilet paper in the water. Describe what happens. Explain
The year before us was with a small piece of glass and a small piece of potatoe - breathe on the glass. Describe. Explain - rub the potatoe on the glass. Breathe on it. Describe. Explain
You REALLY needed to understand the class material to succeed in these seemingly simple tests
Our year's 2-hour exam was the following. The professor gave us a small cup of water, a piece of paper, and a piece of toilet paper. There was only 2 questions - place one end of the paper in the water. Describe what happens. Explain - place one end of the toilet paper in the water. Describe what happens. Explain
The year before us was with a small piece of glass and a small piece of potatoe - breathe on the glass. Describe. Explain - rub the potatoe on the glass. Breathe on it. Describe. Explain
You REALLY needed to understand the class material to succeed in these seemingly simple tests
As a new parent I think about this a lot in terms of how to educate my son.
I find it especially relevant as my first job was starting my own SAT tutoring company. Anecdotally I found it interesting that immigrant parents & most of the wealthiest US-born parents shared a recognition that this was a game. They were direct and explicit to their kids that this was a system that they must participate in, so they might as well win at it.
I think that is the key: make sure your kids can recognize when something is a bs system they need to hack (like selling enterprise software) and when something is actually about craft and the best will win (like building great products that end-users want to use).
I find it especially relevant as my first job was starting my own SAT tutoring company. Anecdotally I found it interesting that immigrant parents & most of the wealthiest US-born parents shared a recognition that this was a game. They were direct and explicit to their kids that this was a system that they must participate in, so they might as well win at it.
I think that is the key: make sure your kids can recognize when something is a bs system they need to hack (like selling enterprise software) and when something is actually about craft and the best will win (like building great products that end-users want to use).
It took me several years to learn how to get good grades. I was not that great a student in the beginning, and was motivated by things that were interesting to me. The two eventually converged, and I ended up with good grades by the end.
One test I had was a counterexample to the hackable tests that Paul talked about. And it was the only one of that kind that was of that nature. It was in the EE Static Fields course. The final exam was one question, and you could hear the groans as the test was passed out:
Derive Maxwell's Equations
This test was a great indicator of how well you got the basic facts of the course.
One test I had was a counterexample to the hackable tests that Paul talked about. And it was the only one of that kind that was of that nature. It was in the EE Static Fields course. The final exam was one question, and you could hear the groans as the test was passed out:
Derive Maxwell's Equations
This test was a great indicator of how well you got the basic facts of the course.
One of my favorite college courses was a computer science course that taught C++. It only had 4 graded components, each worth 25% of your final grade. There were two tests (programming tests with paper and pencil only) and two projects.
It wasn't hard, but it was the fifth most failed course in the university because the tests were extremely difficult to "hack" and the projects were big enough that a student could not rush them at the last minute. There was no homework or attendance to pad grades.
It was one of the few times in my college career that I felt like a professor actually cared whether or not students actually learned about the subject.
It wasn't hard, but it was the fifth most failed course in the university because the tests were extremely difficult to "hack" and the projects were big enough that a student could not rush them at the last minute. There was no homework or attendance to pad grades.
It was one of the few times in my college career that I felt like a professor actually cared whether or not students actually learned about the subject.
This is the difference between project-based learning and standardized testing.
When you learn through projects your ability to apply concepts is tested by the results you generate, connecting to our intrinsic motivations.
Standardized testing is, functionally a disease. Check out “DIY University” to see the effect it has on society long-term.
If you think projects can’t be the path to teaching highly abstract disciplines, such as physics, check out “Surely You’re Joking Mr. Feynman.”
“Most Likely to Succeed” is probably the best book written on this topic.
I spent 10 years as an edtech entrepreneur. It is hard to overstate the degree to which we have built a Testing system at the expense of an educational one.
When you learn through projects your ability to apply concepts is tested by the results you generate, connecting to our intrinsic motivations.
Standardized testing is, functionally a disease. Check out “DIY University” to see the effect it has on society long-term.
If you think projects can’t be the path to teaching highly abstract disciplines, such as physics, check out “Surely You’re Joking Mr. Feynman.”
“Most Likely to Succeed” is probably the best book written on this topic.
I spent 10 years as an edtech entrepreneur. It is hard to overstate the degree to which we have built a Testing system at the expense of an educational one.
The solution seems to be - avoid people with degrees from top universities? Or, in more extreme form, all college graduates.
Less extreme variant of it would be - OK, you went to college - but show me what you did on your own (personal projects, small attempts at startups etc.) during that time. That's what the game dev industry is doing. If the answer to this question is "nothing", then you're probably facing a person who's happy to take directions and land himself a cushy job in some large organization, where he will use his intelligence to navigate towards maximum pay and minimum workload.
Less extreme variant of it would be - OK, you went to college - but show me what you did on your own (personal projects, small attempts at startups etc.) during that time. That's what the game dev industry is doing. If the answer to this question is "nothing", then you're probably facing a person who's happy to take directions and land himself a cushy job in some large organization, where he will use his intelligence to navigate towards maximum pay and minimum workload.
> Why did founders tie themselves in knots doing the wrong things when the answer was right in front of them? Because that was what they'd been trained to do.
Maybe I'm overly cynical, but I see another possibility: They want to create the appearance of running a hot startup without actually having to do the work. Why make something people want when you can just dazzle some investors and get them to dump a million dollars in your lap, no strings attached?
I wonder if that also might explain why such a tiny fraction of startups generate essentially all the wealth -- most of them aren't really trying.
Maybe I'm overly cynical, but I see another possibility: They want to create the appearance of running a hot startup without actually having to do the work. Why make something people want when you can just dazzle some investors and get them to dump a million dollars in your lap, no strings attached?
I wonder if that also might explain why such a tiny fraction of startups generate essentially all the wealth -- most of them aren't really trying.
A corollary: software engineering interviews are tests. When you are young you can memorize more things. When you are older you've experienced many different ways of building software. But, interviews, most of them, generally just ask you to regurgitate concise facts, just like PG points out tests do. And, our brains are tuned to pick out mistakes in those facts and make decisions based on that. I'm not saying hiring for engineers is biased against older engineers, but it makes me think after reading this essay.
TL; DR;
Einstein said sth along the lines of "Don't let your education get in the way of your knowledge."
Finish uni with high grades then spend the rest of your life learning.
> Finish uni with high grades then spend the rest of your life learning.
I'd rather not waste 30% of my life on that nonsense. Better to use uni for what it is meant for: Learn whatever seems interesting and don't mind the grades.
I'd rather not waste 30% of my life on that nonsense. Better to use uni for what it is meant for: Learn whatever seems interesting and don't mind the grades.
I completely agree with the the theory, however what 90% or more of us do when they finish university is to go work for an established (i.e. large) company, where winning means hacking bad tests (promotion cycles).
So in some perverse sense school does prepare most of us well for working life, as most of the time doing well in BigCo isn't to actually perform well or to actually make a difference, it's to leave the impression of it at the right time with the people that matter for your next promotion.
So in some perverse sense school does prepare most of us well for working life, as most of the time doing well in BigCo isn't to actually perform well or to actually make a difference, it's to leave the impression of it at the right time with the people that matter for your next promotion.
This isn't fair to tests. They are actually far more fair and objective than BigCo promo processes. You can't fake doing a variety of hard math questions the way you can move up the ladder due things like nepotism.
>The real problem is that most tests don't come close to measuring what they're supposed to
This is what happens when the teacher's hidden goal is to make marking/grading easier.
This is what happens when the teacher's hidden goal is to make marking/grading easier.
Couple that with students' hidden goal being "not ruining my career prospects and/or angering my parents by getting bad grades", and you arrive at modern school.
>angering my parents by getting bad grades
At secondary school level, it's no longer an issue. Parents just change schools. A few months ago, an uber unserious secondary girl altered her end of term results. The father brought the slip to school to complain about the unprofessional look of the slip due to multiple alterations.
Found out what his daughter did and change schools.
At secondary school level, it's no longer an issue. Parents just change schools. A few months ago, an uber unserious secondary girl altered her end of term results. The father brought the slip to school to complain about the unprofessional look of the slip due to multiple alterations.
Found out what his daughter did and change schools.
Coming back to this discussion a few days later as I've just started an online course on Kubernetes from the Linux Foundation and I came across this section in the introduction materials that seemed very relevant here. Quoting from it:
Training/Certification Firewall
The Linux Foundation has two separate training divisions:
Course Delivery and Certification. These two divisions are
separated by a firewall.
The curriculum development and maintenance division of the
Linux Foundation Training department has no direct role in
developing, administering, or grading certification exams.
Enforcing this self-imposed firewall ensures that
independent organizations and companies can develop third
party training material, geared towards helping test
takers pass their certification exams.
Furthermore, it ensures that there are no secret "tips"
(or secrets in general) that one needs to be familiar with
in order to succeed.
It also permits the Linux Foundation to develop a very
robust set of courses that do far more than teach the
test, but rather equip attendees with a broad knowledge
of the many areas they may be required to master to have a
successful career in open source system administration.
Perhaps one of the most noticeably hackable tests I've participated in is democratic elections. I ran for office a few times and learned a lot about how the system actually works. I believe this is why politics has such a negative connotation. Ultimately, voters want politicians to do what's right and politicians learn they have to gain the genuine love of one group and appease a large enough cross-section of groups without offending an important one.
If there's one thing I learned in politics, the biggest hack is in having a great narrative. Ultimately, brands, companies, and politicians are characters in our consciousness that only achieve meaning through the stories they tell.
Essentially, the story must be believable and ring true to the audience being targeted according to their past experience. For example, evangelicals love a good redemption and hard work story. Each group has their own highest morals. I'd recommend "Don't Think of An Elephant" by George Lakoff, which explains how politicians hack the human mind by crafting an entire language that plays into their hopes and fears.
I'd love to have every test in life be genuine and unhackable. But when tests are clearly so hackable, it doesn't help to not call out how the hack works. Call it responsible cultural exploit disclosure.
If there's one thing I learned in politics, the biggest hack is in having a great narrative. Ultimately, brands, companies, and politicians are characters in our consciousness that only achieve meaning through the stories they tell.
Essentially, the story must be believable and ring true to the audience being targeted according to their past experience. For example, evangelicals love a good redemption and hard work story. Each group has their own highest morals. I'd recommend "Don't Think of An Elephant" by George Lakoff, which explains how politicians hack the human mind by crafting an entire language that plays into their hopes and fears.
I'd love to have every test in life be genuine and unhackable. But when tests are clearly so hackable, it doesn't help to not call out how the hack works. Call it responsible cultural exploit disclosure.
Ivan Illichlready said it 50 years ago. Read 'Deschooling Society', its all there. We are schooled to confuse process with essence, and the bad results are apparent.
> The real problem is grades
Bingo! The real problem is the using incorrect metrics. If you judge a fish by its ability to climb a tree, it will spend its entire life thinking it is stupid!
Bingo! The real problem is the using incorrect metrics. If you judge a fish by its ability to climb a tree, it will spend its entire life thinking it is stupid!
This article reminds me of my interview for workstations at HPi. I came away from it knowing that I had to work there. The interview was 5 hours: 4 one hour sessions and lunch with the team. I didn't write a single line of code the whole day. Every question came with the disclaimer that the answer was less important than the thought process.
Disclaimer: I work there now and my opinions don't necessarily represent my employer's etc.
Disclaimer: I work there now and my opinions don't necessarily represent my employer's etc.
As someone who's not entirely convinced VC funding start-ups isn't the best model for how society can allocate resources, I'm always impressed with how pg thinks, but I find he misses the boundaries of the world he lives in. He's right about artificial tests, but he fails when he says the current situation really allows "people doing good things" to get ahead. I mean, just look WeWork. It's hard to claim WeWork won at an artificial test, unless you count convincing others with your confidence as your test. I'm not sure that's what he thinks it is though.
You can claim non-artificial tests are "better," but humans (VC's) still hold the purse-strings so to speak. If you can convince them you're winning the non-artificial tests say of the market, then you can get money and do coke and party for a while, whilst sure your company will crash and burn but you escape unscathed with millions of sweet free money. My point is the problem is deeper than just "artificial" vs. "non-artificial", although it is an important aspect.
You can claim non-artificial tests are "better," but humans (VC's) still hold the purse-strings so to speak. If you can convince them you're winning the non-artificial tests say of the market, then you can get money and do coke and party for a while, whilst sure your company will crash and burn but you escape unscathed with millions of sweet free money. My point is the problem is deeper than just "artificial" vs. "non-artificial", although it is an important aspect.
It seems that hackability of tests, along with their purpose can be generalized. Tests, like big company politics, are signalling mechanisms. The totality of all tests, culminating in a degree from an institution of a certain prestige, is a signal of potential to contribute.
Companies and other big institutions need very strong assurance that supplicants will indeed "work out" by comporting themselves to the culture and making valuable contributions to their goals. And the reason such absurdly strong signals of assurance are needed is the ridiculous level of friction inherent in admission to and ejection from these institutions and their hierarchy of stations.
So, to get into Big Company X, you need a prestigious degree, and to advance once you're there, you must identify and play their internal signalling games. Such is the attachment to signalling, that you'd often do much better to lead the holiday decoration committee than to create actual value by solving some important problem.
The relative lack of friction in open source is why, given the ability and inclination, even a dog can contribute at a high level.
Companies and other big institutions need very strong assurance that supplicants will indeed "work out" by comporting themselves to the culture and making valuable contributions to their goals. And the reason such absurdly strong signals of assurance are needed is the ridiculous level of friction inherent in admission to and ejection from these institutions and their hierarchy of stations.
So, to get into Big Company X, you need a prestigious degree, and to advance once you're there, you must identify and play their internal signalling games. Such is the attachment to signalling, that you'd often do much better to lead the holiday decoration committee than to create actual value by solving some important problem.
The relative lack of friction in open source is why, given the ability and inclination, even a dog can contribute at a high level.
Love it. I wanna put this in the context of innovation in education:
IMHO, a radical innovation in education is one that can address at least two of the following flaws and failures in the current education system, at any growable scale:
–Duration and cost –Content –Delivery
If I were to satisfy a VC, I could also add a 4th bullet point called "Supply/Demand", but since I'm not writing for a VC, then I'll write what actually matters. In the case of education, that is, it's okay to ignore the existing supply and demand, and focus on the demand that must exist that doesn't yet.
Wrote a memo on these and elaborated more https://www.linkedin.com/pulse/q4-2019-what-i-see-current-st...
P.S: I think I got lucky that my mom was a teacher, so I grew up having a definition of right and wrong different from what was expected of me, and that stayed with me throughout my education. I didn't do what was asked, but what I believe was right for my growth.
IMHO, a radical innovation in education is one that can address at least two of the following flaws and failures in the current education system, at any growable scale:
–Duration and cost –Content –Delivery
If I were to satisfy a VC, I could also add a 4th bullet point called "Supply/Demand", but since I'm not writing for a VC, then I'll write what actually matters. In the case of education, that is, it's okay to ignore the existing supply and demand, and focus on the demand that must exist that doesn't yet.
Wrote a memo on these and elaborated more https://www.linkedin.com/pulse/q4-2019-what-i-see-current-st...
P.S: I think I got lucky that my mom was a teacher, so I grew up having a definition of right and wrong different from what was expected of me, and that stayed with me throughout my education. I didn't do what was asked, but what I believe was right for my growth.
Considering that "education" is being used for daycare and conformity first, it might not be "fixable" when these goals are still requirements...
In theory, tests are merely what their name implies:
tests of what you've learned in the class. In theory
you shouldn't have to prepare for a test in a class any
more than you have to prepare for a blood test. In
theory you learn from taking the class, from going to
the lectures and doing the reading and/or assignments,
and the test that comes afterward merely measures how
well you learned.
I was constantly amazed at how many of my peers, who were paying good money to learn stuff at school, were utterly shocked by that philosophy. I really wish more students understood it.
> I would explain that what makes a startup promising, not just in the eyes of investors but in fact, is growth.
Growth is like cancer. Eventually a company will get big enough to bully others for the sake of growth.
Last night I had dinner with a friend who is building a (yet another) language learning app. Simple fact about language learning, or learning in general: You must practice almost every single day. Even if it is only 5-10 minutes.
PG himself said, if you don't die (aka give up), you will eventually succeed: http://www.paulgraham.com/die.html
To me, the best, most sensical way for learning applications to measure success is to delete user accounts with > 3 days of inactivity. This way you can measure the absolute effectiveness of your program. Students who stick through with your application should 100% become fluent. Target the users who actually want to learn, and measure your success by how many users you made successful.
"NO no no", he says. You'll have no users if you do that.
Growth is like cancer. Eventually a company will get big enough to bully others for the sake of growth.
Last night I had dinner with a friend who is building a (yet another) language learning app. Simple fact about language learning, or learning in general: You must practice almost every single day. Even if it is only 5-10 minutes.
PG himself said, if you don't die (aka give up), you will eventually succeed: http://www.paulgraham.com/die.html
To me, the best, most sensical way for learning applications to measure success is to delete user accounts with > 3 days of inactivity. This way you can measure the absolute effectiveness of your program. Students who stick through with your application should 100% become fluent. Target the users who actually want to learn, and measure your success by how many users you made successful.
"NO no no", he says. You'll have no users if you do that.
There's a lot of evidence that taking breaks helps with language learning. And there are several other ways to learn languages. I'll sometimes stop using duo lingo but listen to Spanish language podcasts instead for a couple weeks
> No, I would explain, that is not how to get lots of users. The way you get lots of users is to make the product really great. Then people will not only use it but recommend it to their friends, so your growth will be exponential once you get it started.
> At this point I've told the founders something you'd think would be completely obvious: that they should make a good company by making a good product. And yet their reaction would be something like the reaction many physicists must have had when they first heard about the theory of relativity: a mixture of astonishment at its apparent genius, combined with a suspicion that anything so weird couldn't possibly be right.
The essay context is students and startup founders, but it turns out most multi-billion dollar enterprises have forgotten this as well.
This notion does not come embedded in the heads of most “senior management” from either world.
> At this point I've told the founders something you'd think would be completely obvious: that they should make a good company by making a good product. And yet their reaction would be something like the reaction many physicists must have had when they first heard about the theory of relativity: a mixture of astonishment at its apparent genius, combined with a suspicion that anything so weird couldn't possibly be right.
The essay context is students and startup founders, but it turns out most multi-billion dollar enterprises have forgotten this as well.
This notion does not come embedded in the heads of most “senior management” from either world.
The point is that big companies have other, easier roads to success.
[deleted]
[deleted]
> How does one get lots of users? They had all kinds of ideas about that. They needed to do a big launch that would get them "exposure." They needed influential people to talk about them. They even knew they needed to launch on a tuesday, because that's when one gets the most attention.
> No, I would explain, that is not how to get lots of users. The way you get lots of users is to make the product really great. Then people will not only use it but recommend it to their friends, so your growth will be exponential once you get it started.
Ironically enough, if there is one thing I've heard from HN, it is to not believe in the "if you build it, they will come" myth. That no matter how good a product you build as an engineer, it is all pointless if you aren't willing to hit the ground and start aggressively marketing and selling it. That a mediocre product with great marketing/sales, will win over a better product with poor marketing/sales.
I've heard this directly from YC partners themselves, when they keep telling early-stage founders to market themselves via "do things that don't scale". At Reddit, one of YC's early successes, the founders were literally spending their time spamming their own site with fake posts and comments using sock-puppets, in order to create the illusion of activity. As someone who loves building products, I hate the idea of spending time and energy on tactics like that. I'd love to spend all that time and energy on making my product better. But I've learnt grudgingly from YC that product development is pointless unless I'm using guerilla tactics to hawk my product in front of users.
Maybe I'm in the minority, but I've actually had the opposite experience as what PG describes in his essay. Back in school and University, I did study very hard to ace tests. But the way I studied was to genuinely learn and understand the material as well as possible. Not to "hack" it in some way. Whereas once I graduated, and especially once I tried out entrepreneurship, I realized that just building a great product was insufficient. I now had to "hustle" and use "street smarts" and "growth hacking" in order to get people to notice what I'm doing. The lesson I had to unlearn from school was that the quality of your work will speak for itself, and will win the day. I never needed marketing and sales in school, but it seems indispensable in the real world.
> No, I would explain, that is not how to get lots of users. The way you get lots of users is to make the product really great. Then people will not only use it but recommend it to their friends, so your growth will be exponential once you get it started.
Ironically enough, if there is one thing I've heard from HN, it is to not believe in the "if you build it, they will come" myth. That no matter how good a product you build as an engineer, it is all pointless if you aren't willing to hit the ground and start aggressively marketing and selling it. That a mediocre product with great marketing/sales, will win over a better product with poor marketing/sales.
I've heard this directly from YC partners themselves, when they keep telling early-stage founders to market themselves via "do things that don't scale". At Reddit, one of YC's early successes, the founders were literally spending their time spamming their own site with fake posts and comments using sock-puppets, in order to create the illusion of activity. As someone who loves building products, I hate the idea of spending time and energy on tactics like that. I'd love to spend all that time and energy on making my product better. But I've learnt grudgingly from YC that product development is pointless unless I'm using guerilla tactics to hawk my product in front of users.
Maybe I'm in the minority, but I've actually had the opposite experience as what PG describes in his essay. Back in school and University, I did study very hard to ace tests. But the way I studied was to genuinely learn and understand the material as well as possible. Not to "hack" it in some way. Whereas once I graduated, and especially once I tried out entrepreneurship, I realized that just building a great product was insufficient. I now had to "hustle" and use "street smarts" and "growth hacking" in order to get people to notice what I'm doing. The lesson I had to unlearn from school was that the quality of your work will speak for itself, and will win the day. I never needed marketing and sales in school, but it seems indispensable in the real world.
> That no matter how good a product you build as an engineer, it is all pointless if you aren't willing to hit the ground and start aggressively marketing and selling it.
You have pretty much summarized another one of his talks : http://paulgraham.com/ds.html
(Or even two : http://paulgraham.com/schlep.html )
You have pretty much summarized another one of his talks : http://paulgraham.com/ds.html
(Or even two : http://paulgraham.com/schlep.html )
In my opinion the most insidious feature of the education system is not tests. After all a proper hard STEM program is not that easily hackable - the easiest way to complete it may be to actually study!
What is worse (and although it is related to grades it is not the same), you only do what you are told to do. Your goals are set for you - read this chapter, solve this problem, answer this question. Thus your inherent creativity is slowly but surely gets sapped away until you become a "professional" - someone who solves problems of others without questioning them much.
Not sure what to do with this though - every way to overcome this problem seems inherently not scalable. Also not sure if this is really a bad thing - a society where everybody suddenly becomes more creative can suddenly collapse because of all the excess creativity!
What is worse (and although it is related to grades it is not the same), you only do what you are told to do. Your goals are set for you - read this chapter, solve this problem, answer this question. Thus your inherent creativity is slowly but surely gets sapped away until you become a "professional" - someone who solves problems of others without questioning them much.
Not sure what to do with this though - every way to overcome this problem seems inherently not scalable. Also not sure if this is really a bad thing - a society where everybody suddenly becomes more creative can suddenly collapse because of all the excess creativity!
> When I was a kid, you could either become an engineer and make cool things, or make lots of money by becoming an "executive." Now you can make lots of money by making cool things.
Software is outrageously accessible. The threshold cost for make-lots-of-money-by-making-cool-things has fallen to a used laptop and a wifi password. The prerequisite talent is mostly the ability to focus relentlessly while illegitimi non carborundum.
But since those are rare talents the laptop can only elevate rare people. It's a path to relative wealth for only a pareto distribution of people, like pretty much anything else. But it's a path to absolute wealth for most of the rest of us. Cool things that make money tend to make our lives better.
Make-lots-of-money-by-making-cool-things is a recipe for both more absolute and unequal wealth.
Software is outrageously accessible. The threshold cost for make-lots-of-money-by-making-cool-things has fallen to a used laptop and a wifi password. The prerequisite talent is mostly the ability to focus relentlessly while illegitimi non carborundum.
But since those are rare talents the laptop can only elevate rare people. It's a path to relative wealth for only a pareto distribution of people, like pretty much anything else. But it's a path to absolute wealth for most of the rest of us. Cool things that make money tend to make our lives better.
Make-lots-of-money-by-making-cool-things is a recipe for both more absolute and unequal wealth.
I'm too late to comment, but my 2 cents:
As long as there's something important at stake people will try to hack the test. There's no way around that.
The best that can be done is to try to keep the correlation between the "test" and what it's trying to measure add high as possible and to teach everyone how to hack it, that way everyone is on a level play field. If everyone hacks it then the difference in scores will be mostly about what is being measured and not about who hacked it and who didn't.
That's why big companies actually advice interview candidates to practice on leetcode and give them example interview questions.
But I agree with the main point that being good only at "hacking tests" in life instead of being good at your profession is very bad.
The best that can be done is to try to keep the correlation between the "test" and what it's trying to measure add high as possible and to teach everyone how to hack it, that way everyone is on a level play field. If everyone hacks it then the difference in scores will be mostly about what is being measured and not about who hacked it and who didn't.
That's why big companies actually advice interview candidates to practice on leetcode and give them example interview questions.
But I agree with the main point that being good only at "hacking tests" in life instead of being good at your profession is very bad.
I actually did what this article suggests in college. It didn't go well grade wise. However, I had money and time to study and learn about things I wanted to learn. I found that my path often was interwoven with the path of the class, but that the perspective was different and sometimes professors are only grading on perspective--specifically their own. The biggest dividing line is, I suspect, that college does not, at this point teach one how to think. How to think and good grades are not necessarily the same thing at all. I was busy learning how to think, and deciding what could be relegated to "It's in a book, I can find it if I need it" vs "I should memorize this because it will be useful in life for me later".
I’m returning to school for a BFA in art,
and I’m constantly surprised by exactly how un-hackable the assignments are, compared to my experience studying engineering. It’s finals season, and many people (myself included) take it as an excuse to push and make work that is harder and more complex than they could during the rest of the semester. Most classes are pass/fail, so the only reward is having better work (and a better portfolio). You could duct tape a banana to the wall, but that’s on you to live with.
Maybe there aren’t a lot of fine artists starting YC companies, but how about designers who went to art school? Do they treat these problems differently? Do they have less to unlearn, or different bad habits?
Maybe there aren’t a lot of fine artists starting YC companies, but how about designers who went to art school? Do they treat these problems differently? Do they have less to unlearn, or different bad habits?
The managerial class will keep on embarrassing themselves sweating bullets over facile things like tests, college admissions, and school districts until they stop feeling precarious and downwardly mobile.
There's no point looking at one of their idiosyncrasies in isolation.
But then Paul Graham plays himself because all the startups are looking to get or softbanked. Absorbed into the monopolies or put in permanent luxery lift support. What a joke. Playing the VCs is more important than playing the increasingly poorer consumer hoards of the decreasing yield suburban consumer farms. We're years deep into the startup fad, and blinded test takers appealing to their blinded test taker money holders is all that's left.
There's no point looking at one of their idiosyncrasies in isolation.
But then Paul Graham plays himself because all the startups are looking to get or softbanked. Absorbed into the monopolies or put in permanent luxery lift support. What a joke. Playing the VCs is more important than playing the increasingly poorer consumer hoards of the decreasing yield suburban consumer farms. We're years deep into the startup fad, and blinded test takers appealing to their blinded test taker money holders is all that's left.
I think "hackability" is a spectrum [0]. The idea is that VC funding is less hackable than college admissions, and that market growth/acceptance is very far on the un-hackable end, along with fairly-referee'd sporting events.
He doesn't admit any ways to actually hack market growth (as if there were none), which is weird. Seems like Use Dark Patterns, Pad Your Numbers, Fake It 'Til You Make It, Regulatory Capture, &c would all make the list.
I agree with the general idea, though: I think the least hackable tests are either Wimbledon/tennis or GSL/StarCraft tournaments; you simply cannot schmooze your way to even the lower brackets.
[0] like most things people put in 2 categories
He doesn't admit any ways to actually hack market growth (as if there were none), which is weird. Seems like Use Dark Patterns, Pad Your Numbers, Fake It 'Til You Make It, Regulatory Capture, &c would all make the list.
I agree with the general idea, though: I think the least hackable tests are either Wimbledon/tennis or GSL/StarCraft tournaments; you simply cannot schmooze your way to even the lower brackets.
[0] like most things people put in 2 categories
Nothing said here about how entrepreneurs think who have not been to university.
It would have been an interesting contrast.
I wonder is that because almost all people who do YC are university educated.
Perhaps Paul graham doesn’t have much experience of entrepreneurs who are not university people.
It would have been an interesting contrast.
I wonder is that because almost all people who do YC are university educated.
Perhaps Paul graham doesn’t have much experience of entrepreneurs who are not university people.
> At this point I've told the founders something you'd think would be completely obvious: that they should make a good company by making a good product.
Sure, ok. How do you make a good product? That definitely is not "completely obvious"
Sure, ok. How do you make a good product? That definitely is not "completely obvious"
At least one user is going to love it.
So i have been looking at architecture and security lately. Is it a thing that you automatically start noticing what is not right in a sight? For instances first thing i saw in that link was that the site runs on http not https.
I'm not sure if the correct takeaway here is "unlearn how to hack systems" but something more akin to "prioritize the right hacks you need to do".
Making a good product (with whatever shortcuts/hacks come with it) is necessary for doing well. Getting exposure and launching on a Tuesday are neither necessary nor sufficient, but they will give a marginal edge.
Both the former and the latter are 'hacks', the only difference is degree of relevance. pg throws the baby out with the bathwater a bit by saying don't do both types of hacks.
Making a good product (with whatever shortcuts/hacks come with it) is necessary for doing well. Getting exposure and launching on a Tuesday are neither necessary nor sufficient, but they will give a marginal edge.
Both the former and the latter are 'hacks', the only difference is degree of relevance. pg throws the baby out with the bathwater a bit by saying don't do both types of hacks.
This post reminded me of Eliezer Yudkowsky's Guessing the Teacher's Password https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-t..., which focuses more on the need to unlearn answering questions in the style that school expects you to answer them in, since that style has been optimized to make testing easy, rather than demonstrate in-depth understanding
A related observation: what is the point of a “B” grade? It means “this person didn’t learn everything.” If you’re going to say “oh ok they got a B so we need to teach them some more”, that’s really useful.
But if you just given someone a “B” and send them up a year at school/university, then presumably they find it harder and harder to do subsequent courses. And over time they learn they’re a B student. Which is demoralising, and possibly also insulting as it ascribes failure of the learning process to the student and not the teacher.
But if you just given someone a “B” and send them up a year at school/university, then presumably they find it harder and harder to do subsequent courses. And over time they learn they’re a B student. Which is demoralising, and possibly also insulting as it ascribes failure of the learning process to the student and not the teacher.
This is actually wrong and surprising to hear from PG. For the most part, life is hackable. Sure, being the “real deal,” whatever that is, is great. But most of “success” in life (again, whatever that is) comes from hacking. (And luck.) Look at WeWork, for instance, one of the greatest and most successful hacks of our time. YC itself became a hack as soon as it acquired useful signal. If you’re optimizing for success, you should teach students to hack more not less.
I went to a small liberal arts school, and tests were rare. Papers were common, presentations as well. Projects, and experiments in the science courses. But not tests.
Most of the time, our grades were based on the work we produced, and our participation in discussions. Even in Art History, when the job was to study and memorize slides and identify them, the test was not to just say what they were, but to write a paragraph about how they fit into the cultural context from which they came.
Most of the time, our grades were based on the work we produced, and our participation in discussions. Even in Art History, when the job was to study and memorize slides and identify them, the test was not to just say what they were, but to write a paragraph about how they fit into the cultural context from which they came.
Great and inspirational article that’s pretty practical.
That said, the website is a pain to use on mobile safari and difficult to properly capture when sharing it to Notion or Bear. Notion doesn’t capture it at all, and Bear can’t capture the hyphens or line/paragraph breaks. It’s probably because of the read more button. It also has some jaggy scrolling issue where if you switch between apps it scrolls up and sometimes collapses the article or jumps to the bottom of the page.
That said, the website is a pain to use on mobile safari and difficult to properly capture when sharing it to Notion or Bear. Notion doesn’t capture it at all, and Bear can’t capture the hyphens or line/paragraph breaks. It’s probably because of the read more button. It also has some jaggy scrolling issue where if you switch between apps it scrolls up and sometimes collapses the article or jumps to the bottom of the page.
Studying, getting hired, career promotions, raising funds... It’s all about beating the system. You’re evaluated based on certain criteria and you optimize for those.
The following paragraph is a good summary, I think:
> Anyone who cares about getting good grades has to play this game, or they'll be surpassed by those who do. And at elite universities, that means nearly everyone, since someone who didn't care about getting good grades probably wouldn't be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades.
It's a race to the bottom.
> Anyone who cares about getting good grades has to play this game, or they'll be surpassed by those who do. And at elite universities, that means nearly everyone, since someone who didn't care about getting good grades probably wouldn't be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades.
It's a race to the bottom.
I wonder why pg didn’t launch this article on a Tuesday.
School grades are more a measure of conformity and ability to adapt a wage-work environment. Performing well in both wage-work and school requires an individual to jump through arbitrary hoops (performance reviews, office politics requiring certain procedures, etc). Teachers are just as capricious as bosses and upper management with their directives.
Perhaps if we move away from wage work then school will change too.
Perhaps if we move away from wage work then school will change too.
I never cared about grades, but I was never very good or very bad. Just slightly above average
In academia, having good grades is sometimes helpful to climb the ladder. Not all people can study medicine. Not all people can get a master degree.
Often these gigs are given to the best students only, but sometimes you can circumvent the grading issue by waiting longer, going to the military, be friends with a prof. or working for free.
In academia, having good grades is sometimes helpful to climb the ladder. Not all people can study medicine. Not all people can get a master degree.
Often these gigs are given to the best students only, but sometimes you can circumvent the grading issue by waiting longer, going to the military, be friends with a prof. or working for free.
> Not all people can study medicine
I think you are falling for the fallacy that the essay is trying to expose?
Surely those that pass medicine are those that are good at medicine exams. Plenty of people would make great doctors that don’t due to the exam system, not due to a lack of capability.
I think you are falling for the fallacy that the essay is trying to expose?
Surely those that pass medicine are those that are good at medicine exams. Plenty of people would make great doctors that don’t due to the exam system, not due to a lack of capability.
I've got one kid in college, the other is finishing high school. Here's what I'm telling them, although not in these words: If you get something of real lasting value out of your education, then you're the one who is hacking the system.
Figuring out the tricks to pass tests is not hacking, it's going with the flow. Or at best, it's hacking in the "script kiddie" fashion.
Figuring out the tricks to pass tests is not hacking, it's going with the flow. Or at best, it's hacking in the "script kiddie" fashion.
It sounds a lot like raising venture funding. Make sure there's a slide that says "$10 billion" because that's the new billion. Talk about your "AI" which was formerly "blockchain" or "big data" and make sure you fit the mold of whatever the funding is looking for. Leave creativity at the door please.
As a teacher, the way to fix this is to give "aligned tests". That is, where the test tests for exactly what you want the student to learn. See https://en.m.wikipedia.org/wiki/Constructive_alignment.
I am surprised this book is not mentioned. Talks about how optimization doesn't lead to optimal goals in complex environments (e.g. life).
https://www.amazon.com/Why-Greatness-Cannot-Planned-Objectiv...
https://www.amazon.com/Why-Greatness-Cannot-Planned-Objectiv...
[deleted]
TLDR: The way to succeed in school is by gaming the system, but the way to succeed in business is by making a great product.
But PG is wrong. Startups can, and do, game the system. By showing unsustainable, artificial growth with no real value behind it, they can fool investors long enough to make a lucrative exit by IPO or acquisition.
But PG is wrong. Startups can, and do, game the system. By showing unsustainable, artificial growth with no real value behind it, they can fool investors long enough to make a lucrative exit by IPO or acquisition.
an IPO or acquisition without proving their market value should count as startup failure.
And Lesson to Learn is when real life problems come, people will ask can you solve the problem not what grade you got.
I wonder if there will be any changes made to the YC application process now that Paul is thinking about these things.
Good point, but there have been countless amazing products that failed to raise money and/or to attract enough users. Many have even succeeded just to be bought and terminated. Doing the actual job is not enough, in real the real world you are doomed to hack users, investors, laws and many other things anyway.
Did he ever pretend otherwise?
Getting tested accelerates learning. That’s why we learn more from actual work, every action is tested by reality. Or why we learn language by using it, every interaction is a test.
> I was genuinely interested in most of the classes I took, and I worked hard. And yet I worked by far the hardest when I was studying for a test.
> I was genuinely interested in most of the classes I took, and I worked hard. And yet I worked by far the hardest when I was studying for a test.
Tsunebaro Makiguchi, an early 20th century Japanese educator, was striving towards deconstruction of rote education but his work was disrupted by authorities who worried that his educational revolution would disrupt orderly Japanese society. Careful, PG, you might be rocking the boat too much with this. ;)
"how to raise money ... read as the test. It came at the end of YC. It had numbers attached to it, and higher numbers seemed to be better. It must be the test."
This is a great sharing of internal mental processes. The same process evolves around investment reports. "Oh look! I got a good grade!"
This is a great sharing of internal mental processes. The same process evolves around investment reports. "Oh look! I got a good grade!"
No, no, no, experienced students are saying to themselves.
Oh, no no no, it was too cold always
(Still the dead one lay moaning)
I was much too far out all my life
And not waving but drowning.
Paul should talk to Seth about education. He seems to have had the insights before him[1].
[1] https://seths.blog/2018/09/education-needs-to-be-inconvenien...
[1] https://seths.blog/2018/09/education-needs-to-be-inconvenien...
This makes me wonder if pg has ever read any of the writings of John Taylor Gatto?
Various quotes: https://www.goodreads.com/author/quotes/41319.John_Taylor_Ga...
Various quotes: https://www.goodreads.com/author/quotes/41319.John_Taylor_Ga...
I feel like he definitely has, he seems like he reads a lot and Gatto is not unpopular for people with thoughts on school
This post glosses over something important, which is that many college classes don't assign most of your grade based on tests. In Stanford's CS 229 Machine Learning class (which I am TAing right now), 20% of the grade is the midterm, 40% is HW, and 40% is a big project that students do throughout the quarter (and for which they get to decide the problem, approach, and most anything with just feedback and guidance from TAs). In my experience, "In practice, the phrase "studying for a test" was almost redundant, because that was when one really studied." is absolutely false, because studying also happened when doing assignments and projects. Of course one could cheat if they just cared about the grade, but I did not because I don't think most smart people really only care about grades, it feels bad to cheat.
So TLDR is, I dont think the situation is so bad, because grades are not only a function of tests but also assignments and projects. I think the deeper issue is that people learn to be told how to learn (which assignments or projects to do), and not taught how to learn by themselves. There are exceptions (at Stanford a ton of CS classes have projects where the students map out the problem and solution themselves), but I think this is pretty important too.
So TLDR is, I dont think the situation is so bad, because grades are not only a function of tests but also assignments and projects. I think the deeper issue is that people learn to be told how to learn (which assignments or projects to do), and not taught how to learn by themselves. There are exceptions (at Stanford a ton of CS classes have projects where the students map out the problem and solution themselves), but I think this is pretty important too.
There's a passage in Zen and the Art of Motorcycle Maintenance about exactly this.
---
Phaedrus' argument for the abolition of the degree and grading system produced a nonplussed or negative reaction in all but a few students at first, since it seemed, on first judgment, to destroy the whole University system. One student laid it wide open when she said with complete candor, "Of course you can't eliminate the degree and grading system. After all, that's what we're here for."
She spoke the complete truth. The idea that the majority of students attend a university for an education independent of the degree and grades is a little hypocrisy everyone is happier not to expose. Occasionally some students do arrive for an education but rote and the mechanical nature of the institution soon converts them to a less idealistic attitude.
The demonstrator was an argument that elimination of grades and degrees would destroy this hypocrisy. Rather than deal with generalities it dealt with the specific career of an imaginary student who more or less typified what was found in the classroom, a student completely conditioned to work for a grade rather than the knowledge the grade was supposed to represent.
Such a student, the demonstrator hypothesized, would go to his first class, get his assignment and probably do it out of habit. He might go to his second and third as well. But eventually the novelty of the course would wear off and, because his academic life was not his only life, the pressure of other obligations or desires would create circumstances in where he just would not be able to get an assignment in.
Since there was no degree or grading system he would incur no penalty for this. Subsequent lectures which presumed he'd completed the assignment might be a little more difficult to understand, however, and this difficulty, in turn, might weaken his interest to a point where the next assignment, which he would find quite hard, would also be dropped. Again no penalty.
In time his weaker and weaker understanding of what the lectures were about would make it more and more difficult for him to pay attention in class. Eventually he would see that he wasn't learning much; and facing the continual pressure of outside obligations, he would stop studying, feel guilty about this and stop attending class. Again, no penalty would be attached.
But what had happened? The student, with no hard feelings on anybody's part, would have flunked himself out. Good! This is what should have happened. He wasn't there for a real education in the first place and he had no real business there at all. A large amount of money and effort had been saved and there would be no stigma of failure and ruin to haunt him the rest of his life. No bridges had been burned.
The student's biggest problem was a slave mentality which had been built into him by years of carrot-and-whip grading, a mule mentality which said, "If you won't whip me, I won't work." He didn't get whipped. He didn't work. And the cart of civilization, which he supposedly was being trained to pull, was just going to have to creak along a little slower without him.
This is a tragedy, however, only if you presume that the cart of civilization, "the system," is pulled by mules. This is a common, vocational, "location" point of view, but it's not the Church attitude.
The Church attitude is that civilization, or "the system" or "society" or whatever you want to call it, is best served not by mules but by free men. The purpose of abolishing grades and degrees is not to punish mules or get rid of them but to provide an environment in which that mule can turn into a free man.
The hypothetical student, still a mule, would drift around for a while. He would get another kind of education quite as valuable as the one he'd abandoned, in what used to be called the "school of hard knocks." Instead of wasting money and time as a high-status mule, he would now have to get a job as a low-status mule, maybe as a mechanic. Actually his real status would go up. He would be making a contribution for a change. Maybe that's what he would do for the rest of his life. Maybe he'd found his level. But don't count on it.
In time - six months; five years, perhaps - a change could easily begin to take place. He would become less and less satisfied with a kind of dumb, day-to-day shop-work. His creative intelligence, stifled by too much theory and too many grades in college, would now become reawakened by the boredom of the shop. Thousands of hours of frustrating mechanical problems would have made him more interested in machine design. He would like to design machinery himself. He'd think he could do a better job. He would try modifying a few engines, meet with success, look for more success, but feel blocked because he didn't have the theoretical information. He would discover that when before he felt stupid because of his lack of interest in theoretical information, he'd now find a brand of theoretical information which he'd have a lot of respect for, namely, mechanical engineering.
So he would come back to our degreeless and gradeless school, but with a difference. He'd no longer be a grade-motivated person. He'd be a knowledge motivated person. He would need no external pushing to learn. His push would come from inside. He'd be a free man. He wouldn't need a lot of discipline to shape him up. In fact, if the instructors assigned him were slacking on the job he would be likely to shape them up by asking rude questions. He'd be there to learn something, would be paying to learn something and they'd better come up with it.
Motivation of this sort, once it catches hold, is a ferocious force, and in the gradeless, degreeless institution where our student would find himself, he wouldn't stop with rote engineering information. Physics and mathematics would come within his sphere of interest because he'd see he needed them. Metallurgy and electrical engineering would come up for attention. And, in the process of intellectual maturing that these abstract studies gave him, he would be likely to branch out into other theoretical areas that weren't directly related to machines but had become part of a larger goal. This larger goal wouldn't be the imitation of an education in Universities today, glossed over and concealed by grades and degrees that gave the appearance of something happening when, in fact, almost nothing is going on. It would be the real thing.
---
Phaedrus' argument for the abolition of the degree and grading system produced a nonplussed or negative reaction in all but a few students at first, since it seemed, on first judgment, to destroy the whole University system. One student laid it wide open when she said with complete candor, "Of course you can't eliminate the degree and grading system. After all, that's what we're here for."
She spoke the complete truth. The idea that the majority of students attend a university for an education independent of the degree and grades is a little hypocrisy everyone is happier not to expose. Occasionally some students do arrive for an education but rote and the mechanical nature of the institution soon converts them to a less idealistic attitude.
The demonstrator was an argument that elimination of grades and degrees would destroy this hypocrisy. Rather than deal with generalities it dealt with the specific career of an imaginary student who more or less typified what was found in the classroom, a student completely conditioned to work for a grade rather than the knowledge the grade was supposed to represent.
Such a student, the demonstrator hypothesized, would go to his first class, get his assignment and probably do it out of habit. He might go to his second and third as well. But eventually the novelty of the course would wear off and, because his academic life was not his only life, the pressure of other obligations or desires would create circumstances in where he just would not be able to get an assignment in.
Since there was no degree or grading system he would incur no penalty for this. Subsequent lectures which presumed he'd completed the assignment might be a little more difficult to understand, however, and this difficulty, in turn, might weaken his interest to a point where the next assignment, which he would find quite hard, would also be dropped. Again no penalty.
In time his weaker and weaker understanding of what the lectures were about would make it more and more difficult for him to pay attention in class. Eventually he would see that he wasn't learning much; and facing the continual pressure of outside obligations, he would stop studying, feel guilty about this and stop attending class. Again, no penalty would be attached.
But what had happened? The student, with no hard feelings on anybody's part, would have flunked himself out. Good! This is what should have happened. He wasn't there for a real education in the first place and he had no real business there at all. A large amount of money and effort had been saved and there would be no stigma of failure and ruin to haunt him the rest of his life. No bridges had been burned.
The student's biggest problem was a slave mentality which had been built into him by years of carrot-and-whip grading, a mule mentality which said, "If you won't whip me, I won't work." He didn't get whipped. He didn't work. And the cart of civilization, which he supposedly was being trained to pull, was just going to have to creak along a little slower without him.
This is a tragedy, however, only if you presume that the cart of civilization, "the system," is pulled by mules. This is a common, vocational, "location" point of view, but it's not the Church attitude.
The Church attitude is that civilization, or "the system" or "society" or whatever you want to call it, is best served not by mules but by free men. The purpose of abolishing grades and degrees is not to punish mules or get rid of them but to provide an environment in which that mule can turn into a free man.
The hypothetical student, still a mule, would drift around for a while. He would get another kind of education quite as valuable as the one he'd abandoned, in what used to be called the "school of hard knocks." Instead of wasting money and time as a high-status mule, he would now have to get a job as a low-status mule, maybe as a mechanic. Actually his real status would go up. He would be making a contribution for a change. Maybe that's what he would do for the rest of his life. Maybe he'd found his level. But don't count on it.
In time - six months; five years, perhaps - a change could easily begin to take place. He would become less and less satisfied with a kind of dumb, day-to-day shop-work. His creative intelligence, stifled by too much theory and too many grades in college, would now become reawakened by the boredom of the shop. Thousands of hours of frustrating mechanical problems would have made him more interested in machine design. He would like to design machinery himself. He'd think he could do a better job. He would try modifying a few engines, meet with success, look for more success, but feel blocked because he didn't have the theoretical information. He would discover that when before he felt stupid because of his lack of interest in theoretical information, he'd now find a brand of theoretical information which he'd have a lot of respect for, namely, mechanical engineering.
So he would come back to our degreeless and gradeless school, but with a difference. He'd no longer be a grade-motivated person. He'd be a knowledge motivated person. He would need no external pushing to learn. His push would come from inside. He'd be a free man. He wouldn't need a lot of discipline to shape him up. In fact, if the instructors assigned him were slacking on the job he would be likely to shape them up by asking rude questions. He'd be there to learn something, would be paying to learn something and they'd better come up with it.
Motivation of this sort, once it catches hold, is a ferocious force, and in the gradeless, degreeless institution where our student would find himself, he wouldn't stop with rote engineering information. Physics and mathematics would come within his sphere of interest because he'd see he needed them. Metallurgy and electrical engineering would come up for attention. And, in the process of intellectual maturing that these abstract studies gave him, he would be likely to branch out into other theoretical areas that weren't directly related to machines but had become part of a larger goal. This larger goal wouldn't be the imitation of an education in Universities today, glossed over and concealed by grades and degrees that gave the appearance of something happening when, in fact, almost nothing is going on. It would be the real thing.
Is it ironic that I'm going to quote this passage in my grad school admissions essay?
Is anyone else not seeing the footnotes inline? I'm only seeing the notes section at the end.
Funny to read and enjoy this article this morning...
Open up HN again this afternoon for a quick work break...
And find my eyes IMMEDIATELY glancing at my karma score in the top right before anything else.
Perhaps PG should practice what he preaches and rip that out of the UI?
I would encourage it... damnit... I just looked again.
Open up HN again this afternoon for a quick work break...
And find my eyes IMMEDIATELY glancing at my karma score in the top right before anything else.
Perhaps PG should practice what he preaches and rip that out of the UI?
I would encourage it... damnit... I just looked again.
Just inject:
td:nth-child(3) > span.pagetop { color: transparent; }
Hides karma, leaves user and logout intact (as they're links)
td:nth-child(3) > span.pagetop { color: transparent; }
Hides karma, leaves user and logout intact (as they're links)
[deleted]
How is the YC interview not hackable? It seems to me that it consists of ticking boxes mostly...
I actually had to drop out of Spanish courses in university as they were going to crush my GPA if I didn’t drop them. I was learning a ton and enjoying them, but for my targeted profession my resume would be tossed if my GPA didn’t meet an arbitrary cutoff.
>In the mid-twentieth century, when the economy was composed of oligopolies, the only way to the top was by playing their game. But it's not true now.
It is still true. A FAANG job is the easiest path for a engineer to millionaire status in about 5 - 10 years.
The interview process is eminently hackable.
Or perhaps you want to do a startup, get funding. You build a this for that .com. Do some social media, find a few users who love it. Read pg's advice, pitch deck advice, go to some demo days.
Hack the test. Get funding. The incubators, the vc fiends are themselves, oligopolic. A handful of firms with common expectations. Mmr. Hockey stick user growth. It's the team! Here's your A good chap.
Those are the tests here. But there are other ways to get an A+. You don't have to start a business the way software people start a business. You can do it like the cabinet maker, or the electrician, or the store owner.
You can buy one increasingly easier today because the baby boomers who started than decades ago want to retire. Imagine how you can take that business into the future with the software you can use to automate it.
Business brokers abound. Go talk to one.
Stop thinking go big or go home.
Go talk to a business broker. Go ask about owner absent business options. Residual income and you didn't even have to be there. Use automation to make the employees lives better and more prodictive. And your customers' lives.
Package that software you write into a SaaS and sell it to you competitors. Profit more.
You are your own case study.
It is still true. A FAANG job is the easiest path for a engineer to millionaire status in about 5 - 10 years.
The interview process is eminently hackable.
Or perhaps you want to do a startup, get funding. You build a this for that .com. Do some social media, find a few users who love it. Read pg's advice, pitch deck advice, go to some demo days.
Hack the test. Get funding. The incubators, the vc fiends are themselves, oligopolic. A handful of firms with common expectations. Mmr. Hockey stick user growth. It's the team! Here's your A good chap.
Those are the tests here. But there are other ways to get an A+. You don't have to start a business the way software people start a business. You can do it like the cabinet maker, or the electrician, or the store owner.
You can buy one increasingly easier today because the baby boomers who started than decades ago want to retire. Imagine how you can take that business into the future with the software you can use to automate it.
Business brokers abound. Go talk to one.
Stop thinking go big or go home.
Go talk to a business broker. Go ask about owner absent business options. Residual income and you didn't even have to be there. Use automation to make the employees lives better and more prodictive. And your customers' lives.
Package that software you write into a SaaS and sell it to you competitors. Profit more.
You are your own case study.
To use a fun analogy, the teachers write the unit tests and you pass them by writing a mock or fake implementation of knowledge.
edit: and now I got to a point in the article where that's pretty much already said. Well, too late to delete this comment now.
edit: and now I got to a point in the article where that's pretty much already said. Well, too late to delete this comment now.
I suspect that some of the things people do to get good grades, like proofread their essays and edit them down to reduce redundancy, are good things to teach in school, and are broadly applicable outside of the university.
Recently I have been charged with helping someone setup paperwork for their startup. I am not a lawyer so I simply have the paperwork, standard agreements, and general company structure that any lawyer would (I assume) give away for free. Avodocs has pretty much all that paperwork covered, carta has the captable, and pilot has the books under wrap. After reading this article I'm somewhat torn about mentioning things like c corps and filing in Delaware to my friend since in the end it really is only the product that matters. Corporate structure steeped in law/history is a hackable test which everyone must take...resulting in either getting bitten or by running to attorneys and tax advisors (because they were bitten in the past).
TLDR opinion. The hackable c corp structure Y pushes has payed off well for Ycombinator yet it is the very thing that can distract from building a good product.
Then again I'm sure someone might make the claim that c corps are also a way to easily share equity with your employees.
TLDR opinion. The hackable c corp structure Y pushes has payed off well for Ycombinator yet it is the very thing that can distract from building a good product.
Then again I'm sure someone might make the claim that c corps are also a way to easily share equity with your employees.
One solution might be to silo the people who evaluate the students from those that teach the materials. Treat the test more about evaluating the effectiveness of instructors and less about the students.
I always shot for a “B+” in college. My logic was that in 5 years I’d remember the content at a B+ level, and any incremental effort was better spent socializing or building student organizations.
I disagree with this essay, I have no knowledge in the field that would allow me to argue against this essay, I think this essay lacks brevity, and the moral of the story seems weak.
The point is summarized well by one of my favorite aphorisms (often attributed incorrectly to Mark Twain): "I have never let my schooling interfere with my education."
The free market itself is the ultimate hackable test, no? Our society has examples where superb work goes unrewarded financially.
Does pg argue that they simply should ignore this test grade?
Does pg argue that they simply should ignore this test grade?
He doesn't offer help there but...
Yes and the job interview.
Yes and the job interview.
Makes me hopeful, and why I am so excited about the Recurse Center. https://www.recurse.com
I think this is the difference between education in many countries In Asia and the US: they care only about the grades, not what you actually learn.
Yeah, but what you learned can be changed in the future (by learning more when you're in a more comfortable position), grades can't.
Is the latest essays the in-text links to notes are missing.
(My favorite from this one: "Learning is the naive algorithm for getting good grades.")
(My favorite from this one: "Learning is the naive algorithm for getting good grades.")
[deleted]
As someone who got very high grades in my CS degree I agree, transitioning to the real world with my first job was 12 months of pain.
Once in a while there will be a unicorn that will spread from mouth to mouth simply because its better then everything out there.
The hoop-jumping that most shapes society is the need to maximize profit, at the expensive of all other goals.
It gets worse:
In my view schools/universities are institutions for mass indoctrination. Noam Chomsky has said something along similar lines. I borrowed the words from him.
A bunch of my other observations:
https://realminority.wordpress.com/observations-of-the-world...
In my view schools/universities are institutions for mass indoctrination. Noam Chomsky has said something along similar lines. I borrowed the words from him.
A bunch of my other observations:
https://realminority.wordpress.com/observations-of-the-world...
Thus the famous quote - Don't let your schooling get in the way of your education.
I think one solution to the problem PG describes is self-directed study.
Unit tests are unit tests. Sometimes the unit is the worker.
I never let my schooling get in a way of my education :)
The way we filter candidates into any stream be it college or companies need to change. People are inherently pressurized by capitalistic mindsets and competition that makes them to not question things but just follow the herd mindset of setting up papers and interviews mechanically. We always are chasing the next than solving the now.
Some related thoughts I wrote a while ago on the state of Education https://medium.com/@thallukrish/why-the-why-matters-d7b8170b...
Listen to this essay here: https://listle.app.link/v0haf9qve2
"A"s are for cowards.
[deleted]
I think it is characterized by using the ideal solution
Is there any tl;dr ? Thank you in advance.
When I was teaching math in college, it was easy enough to give tests that (i) really tested knowledge of the material and (ii) were not hackable. Here's how I did that: Just announce at the beginning of the class that (i) the homework assignments were just to work enough of the exercises in the book to be good at working the exercises and (ii) the tests would be just some of the exercises in the book with just minor or no modifications. Worked fine.
In D. Knuth's The TeXBook is:
"The traditional way is to put off all creative aspects until the last part of graduate school. For seventeen or more years, a student is taught examsmanship, then suddenly after passing enough exams in graduate school he's told to do something original."
In particular, there can be some comeuppance to too much course hacking and not enough learning: Without enough actual learning, may have too much trouble doing "something original".
On the course hacking Paul describes:
(1) Dad had his Master's in education and to me emphasized just learning and never grades. So, I emphasized learning. He was a bit extreme in ignoring my grades: A few times my neglect of grades cost me, never much but a little.
And for Paul's remarks on studying for tests, that's what I did: I never studied for tests. Essentially as Paul explained, all along in the course I'd been trying to learn the material so no way would just cramming and losing sleep help me.
Broadly Dad was correct: Most of what I got out of education that actually helped me was just the learning. And one of the best things I did learn was how to learn, i.e., pick good subjects, topics, and sources and learn independently from the sources.
E.g., the college I went to as a freshman was selected because it was nearly for free and I could walk to it. But the college was so much like just a grade 13 that they didn't let me start with calculus. But I'd been a math major at a relatively good college prep high school, e.g., that the year before me had sent three students to Princeton and my year sent one to MIT. And in my year, of #1, #2, #3 on the Math SAT, the MIT guy was #3 and I was #2. Similarly on the CEEB Math knowledge test. Sooooo, I SHOULD have been in calculus, and the course they had me in was 90% lower than what I'd already learned in high school and for the other 10% I just taught that to myself in a few days. But I wanted to be a college math major and not fall behind so got a good calculus book and dug in and did okay. The next year I went to a college with a quite good math department, continued with their sophomore calculus from the same text Harvard was using, pretty good text, made As, and did fine. So, I never took freshman calculus; my ability to teach myself saved my tail feathers.
On Paul's remarks on grades, there was a cute story: When the SAT scores came back, one of the teachers, I'd had in the sixth grade, was appointed the counselor for giving the scores to the students; so, she knew me well and knew my reputation among the teachers. She read my Verbal SAT score and had a nice smile, said it was "Very good". It wasn't so good, but she was happy because the score was MUCH better than my reputation! Then she read my Math SAT score and became afraid, upset, confused, speechless. When she recovered she said "There must be some mistake." Yup, there had been, hers and the other teachers for 12 painful years.
E.g., there was one really good day for learning: In plane geometry, my favorite subject, the teacher was competent but really nasty. No way did I want her to have any credit for my learning; so no way would I ever admit to doing any homework. Each day she assigned three not very difficult exercises. I outlined solutions in the margin of the book for the most difficult exercises, maybe some of hers, and then turned to the more difficult exercises in the back and did them ALL, never once missing one. One of them took me from Friday afternoon to Sunday evening. In class on Monday, she worked at the board an easy exercise with the same figure I'd worked on all weekend. So, I woke up from having my head down resting as usual and for the first and last time spoke in class: "There's an exercise in the back with the same figure." She took the bait! About 15 minutes later, with frustration, she was exhorting the class "Think, class, think. Think about the given ....". Heck, I didn't know that I was the only one doing ALL the homework and that even the teacher had not done all the homework! I didn't want to be accused of ruining the class so blurted out "Why don't we ..." and in a few more words, with great anger, she interrupted me. NO WAY did she want my solution! With anger she said "You knew how to do it all along!". Of COURSE I did: No way would I ask HER for help in math! This story might make a good scene in a movie!
There was one more with her: On the state test, the last problem was how to inscribe a square in a semi-circle. After the test I kept thinking about a good solution so asked to see her after school. She didn't like that. I mentioned the problem, and she asked "Why are you working on that?". Well, it was curious, and I'd already solved everything else in sight! So, I went off on the side and constructed a square, circumscribed the semi-circle, constructed a fourth proportional, so had the crucial length in the original figure and did that construction. She said it was not legal to construct the other figure. In college, a student in a geometry for teachers class showed me, given triangle ABC, construct D on AB and E on BC so that the lengths of AD = DE = EC. So, off on the side I started with the angle BAC, picked any length for the AD for that figure, finished constructing the figure on the side similar to the given figure, found the crucial fourth proportional, and completed the construction in the original figure. She said, "How'd you know to do that? That is smilimitude, an advanced technique we are studying." I said "I rediscovered it in high school, and the teacher said can't do that.".
(2) I started my career in applied math, computing, and physics for mostly problems in US national security near DC. There no one cared what course grades I'd made but cared a LOT what I could DO that was important and based on what I'd learned or could learn quickly. And I was learning like drinking from a fire hose. Net, what I knew and learned got me a good career, at one time an annual salary six times what a new high end Camaro cost.
(3) I did well in freshman physics, led the class. One of the guys I'd beaten got some respect for me so contacted me when FedEx was struggling with scheduling the fleet. I'd been teaching computer science at Georgetown University (had never taken a course in computing but had been learning from that fire hose) so joined FedEx, typed software furiously, got a schedule for the fleet, thrilled the Founder Smith, pleased the BoD, enabled some crucial funding, and saved the company, all based on what I'd learned, in physics class and in computing. Later did a little calculus for some revenue projections, pleased two crucial BoD Members, and saved the company again.
(4) In grad school, what I'd learned before entering helped a LOT: On the five Ph.D. qualifying exams, I did the best in the class on four of the exams, and for three of them what I did was all from what I'd learned in class or taught myself before entering. Later my interest in learning and neglect of formalities torqued off my department Chair. So, in a course in optimization, I saw a question with no answer in the course. I found no answer in the library. I got some rough ideas for how to get an answer so signed up for a reading course to address the problem, not necessarily solve it. In two weeks I had a nice solution with a nice new general theorem and also a nice solution to a related problem stated but not solved in the famous paper of Arrow, Hurwicz, and Uzawa in mathematical economics. My work looked publishable (it was; I did publish it later in JOTA). Word of my solution spread in the department. Then I had a halo: No one cared about formalities and let me just LEARN and, then, do my Ph.D. research, with no faculty direction. Again, what I had LEARNED saved my tail feathers. And for Knuth's remark about something "original", the learning let me do that. I did find that the thinking techniques I'd used to do the more difficult exercises in Rudin, Royden, Neveu, Loeve, Halmos, Fleming, etc. were sufficient to do publishable original research.
So, net, just learn and do that by doing the more difficult exercises! E.g., back to high school a challenging test question got me to reinvent similitude!
In D. Knuth's The TeXBook is:
"The traditional way is to put off all creative aspects until the last part of graduate school. For seventeen or more years, a student is taught examsmanship, then suddenly after passing enough exams in graduate school he's told to do something original."
In particular, there can be some comeuppance to too much course hacking and not enough learning: Without enough actual learning, may have too much trouble doing "something original".
On the course hacking Paul describes:
(1) Dad had his Master's in education and to me emphasized just learning and never grades. So, I emphasized learning. He was a bit extreme in ignoring my grades: A few times my neglect of grades cost me, never much but a little.
And for Paul's remarks on studying for tests, that's what I did: I never studied for tests. Essentially as Paul explained, all along in the course I'd been trying to learn the material so no way would just cramming and losing sleep help me.
Broadly Dad was correct: Most of what I got out of education that actually helped me was just the learning. And one of the best things I did learn was how to learn, i.e., pick good subjects, topics, and sources and learn independently from the sources.
E.g., the college I went to as a freshman was selected because it was nearly for free and I could walk to it. But the college was so much like just a grade 13 that they didn't let me start with calculus. But I'd been a math major at a relatively good college prep high school, e.g., that the year before me had sent three students to Princeton and my year sent one to MIT. And in my year, of #1, #2, #3 on the Math SAT, the MIT guy was #3 and I was #2. Similarly on the CEEB Math knowledge test. Sooooo, I SHOULD have been in calculus, and the course they had me in was 90% lower than what I'd already learned in high school and for the other 10% I just taught that to myself in a few days. But I wanted to be a college math major and not fall behind so got a good calculus book and dug in and did okay. The next year I went to a college with a quite good math department, continued with their sophomore calculus from the same text Harvard was using, pretty good text, made As, and did fine. So, I never took freshman calculus; my ability to teach myself saved my tail feathers.
On Paul's remarks on grades, there was a cute story: When the SAT scores came back, one of the teachers, I'd had in the sixth grade, was appointed the counselor for giving the scores to the students; so, she knew me well and knew my reputation among the teachers. She read my Verbal SAT score and had a nice smile, said it was "Very good". It wasn't so good, but she was happy because the score was MUCH better than my reputation! Then she read my Math SAT score and became afraid, upset, confused, speechless. When she recovered she said "There must be some mistake." Yup, there had been, hers and the other teachers for 12 painful years.
E.g., there was one really good day for learning: In plane geometry, my favorite subject, the teacher was competent but really nasty. No way did I want her to have any credit for my learning; so no way would I ever admit to doing any homework. Each day she assigned three not very difficult exercises. I outlined solutions in the margin of the book for the most difficult exercises, maybe some of hers, and then turned to the more difficult exercises in the back and did them ALL, never once missing one. One of them took me from Friday afternoon to Sunday evening. In class on Monday, she worked at the board an easy exercise with the same figure I'd worked on all weekend. So, I woke up from having my head down resting as usual and for the first and last time spoke in class: "There's an exercise in the back with the same figure." She took the bait! About 15 minutes later, with frustration, she was exhorting the class "Think, class, think. Think about the given ....". Heck, I didn't know that I was the only one doing ALL the homework and that even the teacher had not done all the homework! I didn't want to be accused of ruining the class so blurted out "Why don't we ..." and in a few more words, with great anger, she interrupted me. NO WAY did she want my solution! With anger she said "You knew how to do it all along!". Of COURSE I did: No way would I ask HER for help in math! This story might make a good scene in a movie!
There was one more with her: On the state test, the last problem was how to inscribe a square in a semi-circle. After the test I kept thinking about a good solution so asked to see her after school. She didn't like that. I mentioned the problem, and she asked "Why are you working on that?". Well, it was curious, and I'd already solved everything else in sight! So, I went off on the side and constructed a square, circumscribed the semi-circle, constructed a fourth proportional, so had the crucial length in the original figure and did that construction. She said it was not legal to construct the other figure. In college, a student in a geometry for teachers class showed me, given triangle ABC, construct D on AB and E on BC so that the lengths of AD = DE = EC. So, off on the side I started with the angle BAC, picked any length for the AD for that figure, finished constructing the figure on the side similar to the given figure, found the crucial fourth proportional, and completed the construction in the original figure. She said, "How'd you know to do that? That is smilimitude, an advanced technique we are studying." I said "I rediscovered it in high school, and the teacher said can't do that.".
(2) I started my career in applied math, computing, and physics for mostly problems in US national security near DC. There no one cared what course grades I'd made but cared a LOT what I could DO that was important and based on what I'd learned or could learn quickly. And I was learning like drinking from a fire hose. Net, what I knew and learned got me a good career, at one time an annual salary six times what a new high end Camaro cost.
(3) I did well in freshman physics, led the class. One of the guys I'd beaten got some respect for me so contacted me when FedEx was struggling with scheduling the fleet. I'd been teaching computer science at Georgetown University (had never taken a course in computing but had been learning from that fire hose) so joined FedEx, typed software furiously, got a schedule for the fleet, thrilled the Founder Smith, pleased the BoD, enabled some crucial funding, and saved the company, all based on what I'd learned, in physics class and in computing. Later did a little calculus for some revenue projections, pleased two crucial BoD Members, and saved the company again.
(4) In grad school, what I'd learned before entering helped a LOT: On the five Ph.D. qualifying exams, I did the best in the class on four of the exams, and for three of them what I did was all from what I'd learned in class or taught myself before entering. Later my interest in learning and neglect of formalities torqued off my department Chair. So, in a course in optimization, I saw a question with no answer in the course. I found no answer in the library. I got some rough ideas for how to get an answer so signed up for a reading course to address the problem, not necessarily solve it. In two weeks I had a nice solution with a nice new general theorem and also a nice solution to a related problem stated but not solved in the famous paper of Arrow, Hurwicz, and Uzawa in mathematical economics. My work looked publishable (it was; I did publish it later in JOTA). Word of my solution spread in the department. Then I had a halo: No one cared about formalities and let me just LEARN and, then, do my Ph.D. research, with no faculty direction. Again, what I had LEARNED saved my tail feathers. And for Knuth's remark about something "original", the learning let me do that. I did find that the thinking techniques I'd used to do the more difficult exercises in Rudin, Royden, Neveu, Loeve, Halmos, Fleming, etc. were sufficient to do publishable original research.
So, net, just learn and do that by doing the more difficult exercises! E.g., back to high school a challenging test question got me to reinvent similitude!
What is funny about this is that he doesn't realise most medieval historians have actually recognised this and solved it...answer: no exams. Just get students to write essays. Most history classes do exams but, where it is taught well (i.e. not the US), they give you the questions that will be on the exams...these aren't exams like in maths. This tests for understanding, and actual ability to conduct research yourself (you still get bias from professors, I had a Chinese politics professor who had some fairly inflexible views about the Chinese economy...but it does average out).
I still have no idea why this isn't used more generally in life. Want to see if someone can do this job...hm, rather than actually seeing them do the job...have you ever thought about how many windows there are in New York? All you are testing is whether someone has either been asked that kind of question before or, more likely, whether they went online and looked at what you would ask them.
It is truly stunning how stupid people are about this. And, tbh, I think it largely reflects what most students about the world at university where certain professors seem to revel in the utter banality of pointless tests.
I still have no idea why this isn't used more generally in life. Want to see if someone can do this job...hm, rather than actually seeing them do the job...have you ever thought about how many windows there are in New York? All you are testing is whether someone has either been asked that kind of question before or, more likely, whether they went online and looked at what you would ask them.
It is truly stunning how stupid people are about this. And, tbh, I think it largely reflects what most students about the world at university where certain professors seem to revel in the utter banality of pointless tests.
Q:
>You're investing time in the same company you're asking them to invest money in. If it's not a good investment, why are you even doing it?
A: Because you and them don’t have the same circumstances, and thus don’t have the same utility function. You have different values.
The whole (long) thing is a mess of obvious misses. Too bad.
A: Because you and them don’t have the same circumstances, and thus don’t have the same utility function. You have different values.
The whole (long) thing is a mess of obvious misses. Too bad.
Fwiw I responded vi Twitter thread https://twitter.com/p10q/status/1203348614338113536
Are you in EECS? Most big data centers use FPGAs "on the wire" to do things like encryption on the fly or compression so that they get "wire speedups" without burning CPU cycles.
So FPGA solutions that "live on the wire" and can be "plugged into the LAN port" are a really useful area you can hack in your dorm room.
I worked in the security area. One of the key problems is "exfiltrations" where someone tries to copy valuable files off a server. I created a startup to attack this "on the wire" with hardware so it can't be hacked. (It failed because nobody knew what an FPGA was, nor how TCP packets worked. Sigh).
The idea is to create crypto-hashes of the valuable files. Hand-install the hashes into an FPGA using a read-only micro card. The FPGA sits on the wire, hashing files being sent. If the hash matches, scream.
THe same idea of "unhackable" malware virus scans can be done on an FPGA. Only someone with physical access to the FPGA card can modify it. It operates "at wire speeds".
The hard part is finding people who can spell FPGA and know what malware means. Be that person.
You could do this in your dorm room.
The side effects are that you learn a LOT very quickly, it is grounded in real code, and you could actually convince YC that your "homework" is worth a passing grade.
Besides, FPGAs are just plain fun. Make your own hardware Neural Nets. Hack your bike with a camera and an NN to warn of approaching cars.
So FPGA solutions that "live on the wire" and can be "plugged into the LAN port" are a really useful area you can hack in your dorm room.
I worked in the security area. One of the key problems is "exfiltrations" where someone tries to copy valuable files off a server. I created a startup to attack this "on the wire" with hardware so it can't be hacked. (It failed because nobody knew what an FPGA was, nor how TCP packets worked. Sigh).
The idea is to create crypto-hashes of the valuable files. Hand-install the hashes into an FPGA using a read-only micro card. The FPGA sits on the wire, hashing files being sent. If the hash matches, scream.
THe same idea of "unhackable" malware virus scans can be done on an FPGA. Only someone with physical access to the FPGA card can modify it. It operates "at wire speeds".
The hard part is finding people who can spell FPGA and know what malware means. Be that person.
You could do this in your dorm room.
The side effects are that you learn a LOT very quickly, it is grounded in real code, and you could actually convince YC that your "homework" is worth a passing grade.
Besides, FPGAs are just plain fun. Make your own hardware Neural Nets. Hack your bike with a camera and an NN to warn of approaching cars.
For example, it's long been known in the physics education research community that students come away from introductory courses with very little physical understanding, even if they can do the plug and chug problems on typical tests just fine. Students can all recite Newton's third law, but immediately afterward claim that when a truck hits a car, the truck exerts a bigger force. They know the law for the gravitational force, but can't explain what kept astronauts from falling off the moon, since "there's no gravity in space". Another common claim is that a table exerts no force on something sitting on it -- instead of "exerting a force" it's just "getting in the way".
For research purposes, we measure physical understanding using a battery of tests, such as the Force Concept Inventory, containing only simple conceptual questions with unambiguous answers. So then everybody asks: if ordinary tests are so hackable, why not just switch to these conceptual ones? But that wouldn't work. There are less than ~100 distinct FCI-style questions. If these conceptual tests were the norm, students would just memorize the answers and parrot them back, with a flimsy understanding that crumples the second any follow-up question is asked. It would be just the same problem as before, except they would be worse computationally, too. The FCI only works as long as it doesn't count for a grade.
The problem isn't tests, it's scale. If the people aren't motivated, any standardized measure will miss the mark -- even the entrepreneurship Paul Graham advocates for. God knows I've seen a lot of bullshit in that direction.