Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)(signalsandthreads.com)
signalsandthreads.com
Finding Signal in the Noise: Machine Learning and the Markets (Jane Street)
https://signalsandthreads.com/finding-signal-in-the-noise/
156 comments
Been following signals and threads for a while, and it’s amazing. Have to be in a learning mood, but it gets pretty technical. Podcasts are a tough medium to do that in but if you really zone in, signals and threads is fantastic
This one and Machine Learning Street Talk are my two favorites at the moment.
> In Young Cho thought she was going to be a doctor but fell into a trading internship at Jane Street
Genuinely perplexing how they always try to show each multi-million earning engineer as some normal person and not someone that went to Exeter and Harvard
Genuinely perplexing how they always try to show each multi-million earning engineer as some normal person and not someone that went to Exeter and Harvard
[deleted]
I don't really see a lot of elitism in these circles - there's a chunk of such engineers that did not in fact go to Exeter and Harvard - but what they have in common is they are all to a fault very bright technical people that can produce complicated things quite quickly, and can communicate effectively with people around them - to make sure that what they produce is in fact useful :)
ever meet any such engineers from the bottom 20% of the world population? just some food for thought :)
Personally know of 1 from Ghana and 1 from Nigeria - both engineers in HFT - both emigrated to the US post-college - so not that rare. Is it proportional to population? Of course not. Clearly there's going to be an english-speaking bias and an education availability bias and whatever bias du jour one wants to bake in. I don't think it is controversial to say that being born into poverty hurts - as naively as just accounting for malnutrition, for example, and going from there...
idk to me bottom 20% is a household thing more so than geographic if you live in Ghana or Nigeria but grew up driving a Maybach or BMW that would't apply if you grew up homeless in the US it would
perhaps ancedotal but the bottom 20% more or less stay the same across generations no matter how welcoming an organization is or no matter if an individual is bright or has a good idea
perhaps ancedotal but the bottom 20% more or less stay the same across generations no matter how welcoming an organization is or no matter if an individual is bright or has a good idea
You can levy plenty of criticism at finance. But they certainly take smart kids from wherever they can get them (for certain jobs; for others you need certain polish).
I genuinely don't think people that went to Exeter and Harvard (or MIT or Stanford or Penn or other JS feeder schools) see the rest of us as human unless there's another common factor like being employed by Jane Street and having those quirky hobbies the rest of them do - and then they rationalize it away by saying that the exceptions didn't apply to top schools, or were admitted but didn't attend for "financial reasons".
People like me are "NPCs" in their parlance.
People like me are "NPCs" in their parlance.
You're putting words in their mouth
And yet, am I wrong? This is basically how they think
Yes, you are.
Yeah I don’t think so. They think the rest of us are insects.
I interact with those people regularly without having the same background as them and this has basically never been the case. I think you've just been talking to horrible people.
Graduating from a SUNY school, Jane Street gave me an interview and a fair shake. I didn't get the sense of elitism there. There does happen to be a lot of really smart engineers, mathematicians and scientists going to Harvard, MIT, etc.
There are places that without that Ivy League or Target School degree you don't hear back, I don't think Jane Street is one of them.
There are places that without that Ivy League or Target School degree you don't hear back, I don't think Jane Street is one of them.
Might be different with a SUNY PhD or another trading firm on the resume.
I don't think they'd ever call someone like me back, not that I'd ever be able to pass a coding round with them.
I don't think they'd ever call someone like me back, not that I'd ever be able to pass a coding round with them.
There are some firms that are very elitist in terms of their resume review requirements. You should note that for a place like Jane Street, "Harvard" helps, but it won't get you an interview. Someone who has done well at USAMO or IMO or has meaningful open-source contributions and goes to SUNY or a community college will generally have a better chance than a run-of-the-mill Harvard student. The median Harvard Math/CS student isn't getting an interview at Jane Street.
JS absolutely does filter on school though.
Seconding another commenter. I went to a state-school (maybe in the top 5 US state schools) and got an offer without any elite background.
Obviously Michigan, UCB, UDub aren’t in the same category, especially with prior trading firm experience.
And it was none of the mentioned schools!
Would be surprised if it was not a union of the above schools and {GTech, UT, UCLA, UIUC}.
I went to an "elite" school, but I worked at one HFT where the local team of 5 people had only me with a background from a similar school, and another HFT where I was one of three on my immediate team of ~12 engineers. At one point, one of my colleagues at the second firm asked me why I was there as opposed to the alternatives of (1) a company with more of a focus on the elite and cultivating power or (2) being an artist supported by my well-off parents.
I will eternally find it sad how much talent is wasted on trading. So much money, so much intelligence, so much time and effort, all the provide almost no tangible value to society.
I take the silver lining. Smart people getting rich is better than dumb people getting rich. Hopefully some day they'll do something good with that money.
The problem is that smart people are the ones being employed, not the ones getting rich.
I am pretty sure anyone who sticks around Jane Street for a while is getting “rich” by any popularly accepted measure.
Sam Bankman-Fried and Caroline Ellison did for a while…
I'm pretty sure the parent is implying that there is quite a disproportionate distribution.
Sorry, implying what? That Jane Street quants aren’t getting rich enough?
Read the comment again
> The problem is that smart people are the ones being employed, not the ones getting rich.
I would prefer good people get rich and bad people don't.
I’m not sure that’s true either.
See for example “effective altruism”, which turns out to have been neither, but more of a self-deluded justification for insane greed, coupled with a god complex.
See for example “effective altruism”, which turns out to have been neither, but more of a self-deluded justification for insane greed, coupled with a god complex.
Ridiculous, uninformed comment. There is no question that Effective Altruism has done tremendous good overall by saving many, many lives. I say this as someone who disagrees with important aspects, such as valuing distant lives equally to local ones. They have also been more correct than anyone else about AI, pandemics, etc.
It’s also been used as a self-justification by the likes of Sam Bankman-Fried, who exemplifies the qualities I mentioned above.
It’s clearly a very flexible philosophy that “smart people” can use to justify pretty much whatever, and given the presence of such people as SBF and cronies, certainly doesn’t support the notion that it’s a great thing for “smart people” to get rich, or really that we should expect that to be any more than neutral.
“Smart People” can be scumbags, and some prominent EA folks were, all the while shouting about how good they were.
It’s clearly a very flexible philosophy that “smart people” can use to justify pretty much whatever, and given the presence of such people as SBF and cronies, certainly doesn’t support the notion that it’s a great thing for “smart people” to get rich, or really that we should expect that to be any more than neutral.
“Smart People” can be scumbags, and some prominent EA folks were, all the while shouting about how good they were.
I agree, how dull and uninspired. I'm very much with Paul Graham, who believes in the creation of wealth (as opposed to the extraction of money).
Can you codify the difference? I seem to be fundamentally misunderstanding the difference between wealth creation and wealth extraction if voluntary market activity constitutes extraction.
I'd definitionally describe all voluntary transactions free of coercion to imply the buyer values the utility of what they're buying (i.e. true wealth - piles of currency are not true wealth, they're what you exchange for true wealth) more than the currency they're trading for it, no?
Involuntary transactions featuring coercion on the other hand, like the government demanding you pay taxes under threat of imprisonment (enforced at gunpoint, if necessary) are clearly extractive, by my definition.
I'd definitionally describe all voluntary transactions free of coercion to imply the buyer values the utility of what they're buying (i.e. true wealth - piles of currency are not true wealth, they're what you exchange for true wealth) more than the currency they're trading for it, no?
Involuntary transactions featuring coercion on the other hand, like the government demanding you pay taxes under threat of imprisonment (enforced at gunpoint, if necessary) are clearly extractive, by my definition.
Money Extraction: fighting over a slice of pie.
Wealth Creation: creating new pie.
In practice, a lot of things that look extractive (e.g., designing better high frequency trading algorithms) potentially have some marginal utility (e.g., creating market liquidity), but the money high performing people make is likely larger than the utility they add to the system (because most of the money in having the best high frequency trading algorithm comes from beating other people's high frequency trading algorithms).
Wealth Creation: creating new pie.
In practice, a lot of things that look extractive (e.g., designing better high frequency trading algorithms) potentially have some marginal utility (e.g., creating market liquidity), but the money high performing people make is likely larger than the utility they add to the system (because most of the money in having the best high frequency trading algorithm comes from beating other people's high frequency trading algorithms).
While there's many definitions, I'd concentrate on zero-sum vs non-zero-sum games. Lots of games in trading are effectively zero-sum games - if I make 100USD, you lose 100USD (there's details about transaction costs going to exchanges that make this more nuanced, but the principle applies). A chunk of financial engineering games are not : for example risk pooling games. A big part of finance for example is liquidity provisioning games - which kindof boil down to risk pooling games in the limit. But unfortunately - a _very_ big chunk of the financial markets is zero-sum.
Even in a purely digital world - most of the economy is not zero sum - i.e., it _creates_ wealth. I pay you 100USD for 1 million LLM tokens - a purely digital transaction - the net result of this is the 1 million tokes that I can consume and use - net of the transaction.
Even in a purely digital world - most of the economy is not zero sum - i.e., it _creates_ wealth. I pay you 100USD for 1 million LLM tokens - a purely digital transaction - the net result of this is the 1 million tokes that I can consume and use - net of the transaction.
Financial markets are not zero sum. Wealth is created or destroyed in financial markets despite each trade having a buyer and a seller.
I mean - of course the entirety of financial markets are not zero sum - it would indeed be almost lunacy to claim that :) My point is that there's a big part of the financial markets that _are_ zero-sum - not that all financial markets are. One can argue about EMH and that the zero-sum games are in fact injecting information into the market by providing better price discovery, and that is indeed an argument - but one is left with the intellectually unsatisfying statement of "Well, anything the market does is information extraction, and the market is an information extraction machine, so prima facie, it works" - which is basically restating the EMH axiom.
For example, the added value provided by sub-millisecond arbitrage between NY and Chicago - while making the prices converge a fraction of a millisecond faster, makes the overall set of people playing that game in excess of 1B USD in aggregate. I'd argue that such a ratio of profit vs value-added gain is very bad, bordering on 0 - thus making that effectively a zero-sum game.
For example, the added value provided by sub-millisecond arbitrage between NY and Chicago - while making the prices converge a fraction of a millisecond faster, makes the overall set of people playing that game in excess of 1B USD in aggregate. I'd argue that such a ratio of profit vs value-added gain is very bad, bordering on 0 - thus making that effectively a zero-sum game.
Price discovery and liquidity. But yeah, I agree sub-millisecond arbitrage maybe less so.
Sorry but this is a shit definition.
You’ve defined scamming people, a totally voluntary error, as creative not extractive.
You’ve also defined building roads as extractive not creative.
I think you’re latching onto some very misguided principles.
You’ve defined scamming people, a totally voluntary error, as creative not extractive.
You’ve also defined building roads as extractive not creative.
I think you’re latching onto some very misguided principles.
Building toll roads is creation of value. People voluntarily pay to drive on them because they offer superior qualities to garbage public roads. If I could exclusively drive on toll roads, I would.
Public roads, OTOH, are extractive because someone is taking money from you to build them even if you don't consent to it and deliberately avoid them at every opportunity.
Public roads, OTOH, are extractive because someone is taking money from you to build them even if you don't consent to it and deliberately avoid them at every opportunity.
Did he really advocate for that? Then why did he support the GOP, a very-much pro-financialization party?
Are you proposing a universal hierarchy of value where the goals individuals set for themselves are inherently worth less than the goals you want to set for them, if the goal they want for themselves is prosperity?
Who elected you Supreme Leader?
Who elected you Supreme Leader?
Smart people who grew up poor making their own way, isn’t that the idea? Or do we just assume they’re all rich, privileged folks?
If it didn’t provide value, nobody would pay for it.
The alternative to highly technical, agile quantitative trading is fat middle men, wide spreads, and capital sitting in 8%* stupider places than it would otherwise. That’s a pretty big deal even if the observable effects are extremely diffuse.
* Made up number, but if we woke up on Monday with nothing but the tech we used to trade in 1984 it would probably hit much worse.
The alternative to highly technical, agile quantitative trading is fat middle men, wide spreads, and capital sitting in 8%* stupider places than it would otherwise. That’s a pretty big deal even if the observable effects are extremely diffuse.
* Made up number, but if we woke up on Monday with nothing but the tech we used to trade in 1984 it would probably hit much worse.
Lots of things provide no value that people pay for.
The inverse is true too: lots of things provide value that no one pays for.
I find it shocking that anyone that programs would think this way considering how widespread and common open source tooling is. I don't know how you can get through your day without using OSS or even free websites like stack overflow.
The inverse is true too: lots of things provide value that no one pays for.
I find it shocking that anyone that programs would think this way considering how widespread and common open source tooling is. I don't know how you can get through your day without using OSS or even free websites like stack overflow.
Of course the inverse is true, and you don’t have to stick to software. Raising children produces value despite no one exchanging currency.
But generally nobody pays quants to lose money. They expect value over the other opportunities they have to use that same money.
Is this really controversial?
But generally nobody pays quants to lose money. They expect value over the other opportunities they have to use that same money.
Is this really controversial?
>But generally nobody pays quants to lose money. They expect value over the other opportunities they have to use that same money. Is this really controversial?
As with your other comment, this has nothing to do with what the original parent comment was remarking on. They didn't claim, nor did anyone else, that proprietary trading firms weren't making money and that people weren't well compensated due to helping firms make money. Their remark was that the value these firms are providing to society is questionable and that the highly intelligent employees could likely be providing greater value to society elsewhere.
As with your other comment, this has nothing to do with what the original parent comment was remarking on. They didn't claim, nor did anyone else, that proprietary trading firms weren't making money and that people weren't well compensated due to helping firms make money. Their remark was that the value these firms are providing to society is questionable and that the highly intelligent employees could likely be providing greater value to society elsewhere.
Money and value are different things.
[deleted]
Your comment comes off as "people pay, therefore it has value." Which the inverse would need to be true in that case.
But you are conflating money and value. As you stated, raising children produces value, and as you imply, this is not generating revenue. Which you are also conflating exchanging money with generating value. Sure, liquidity can generate value but don't confuse these things.
People seem to be forgetting that money is a proxy and that a proxy is not the same as the thing you are proxying.
But you are conflating money and value. As you stated, raising children produces value, and as you imply, this is not generating revenue. Which you are also conflating exchanging money with generating value. Sure, liquidity can generate value but don't confuse these things.
People seem to be forgetting that money is a proxy and that a proxy is not the same as the thing you are proxying.
[deleted]
>The alternative to highly technical, agile quantitative trading is fat middle men, wide spreads
No, this would be the alternative to securities and derivatives markets being electronic.
>If it didn’t provide value, nobody would pay for it.
The parent was remarking on the value provided to society, not the value provided to the firms.
No, this would be the alternative to securities and derivatives markets being electronic.
>If it didn’t provide value, nobody would pay for it.
The parent was remarking on the value provided to society, not the value provided to the firms.
they make the mistake of paying for it unlike those who know what to do https://www.investopedia.com/articles/investing/030916/buffe...
My actual gripe is with where that value comes from, and what it's true cost to society is.
I'd argue that pulling these people out of moving society forward is just another form of externalized cost. The stanford guy who figured out how to halve the cost of solar in 2012 never got to realize it because $750k/yr to do stochastic modeling of cattle feed to (secretly) overcharge farmers for futures contracts was just too enticing.
I'd argue that pulling these people out of moving society forward is just another form of externalized cost. The stanford guy who figured out how to halve the cost of solar in 2012 never got to realize it because $750k/yr to do stochastic modeling of cattle feed to (secretly) overcharge farmers for futures contracts was just too enticing.
The way I see HFT, is that previously this money was going to brokers and banks.
Now, most of this money is going to HFT shops, but you could also make an argument that some of it stays with the investors, since spreads are much smaller now.
There is 0 cost to society, maybe even a small gain.
As for moving society forward, I don't know. If not in finance, most of these guys would work for big tech companies trying to actively make kids and adults addicted to screens.
Now, most of this money is going to HFT shops, but you could also make an argument that some of it stays with the investors, since spreads are much smaller now.
There is 0 cost to society, maybe even a small gain.
As for moving society forward, I don't know. If not in finance, most of these guys would work for big tech companies trying to actively make kids and adults addicted to screens.
One benefit I've seen is that commission free trades wouldn't be possible without payment for order flow. I remember the old days when it cost a lot to do a trade. Also the stock price tick increments used to be higher. That's been steadily eroded.
These are just first order effects. I'm talking about second and third order effects that I would argue make the value prop of tighter spreads enormously negative to society on the whole.
Liken it to the government deciding to spend 40% of the budget on roads. Suddenly the roads quickly get pot holes filled, the lines are seemingly always freshly painted, cracks are all filled, and even mildly uneven roads gets freshly paved.
The first order effects of this would all be positive. The transportation dept. could talk all day about benefits to everyone. Everyone would love the great roads.
But spending 40% of the budget on roads is insane, and the second and third order effects would be disastrous to that society.
Liken it to the government deciding to spend 40% of the budget on roads. Suddenly the roads quickly get pot holes filled, the lines are seemingly always freshly painted, cracks are all filled, and even mildly uneven roads gets freshly paved.
The first order effects of this would all be positive. The transportation dept. could talk all day about benefits to everyone. Everyone would love the great roads.
But spending 40% of the budget on roads is insane, and the second and third order effects would be disastrous to that society.
If we're talking second and third order effects, then we should bring up the capitalism vs socialism debate.
In a separate single example, it's fairly easy to point out what's the right thing to do - as you do with your Stanford guy example. However, history show us that it usually turns out to be disastrous in the long term. Maybe this Stanford guy will make several millions $, and then go back to desiging how to produce those half price solars at scale.
In a separate single example, it's fairly easy to point out what's the right thing to do - as you do with your Stanford guy example. However, history show us that it usually turns out to be disastrous in the long term. Maybe this Stanford guy will make several millions $, and then go back to desiging how to produce those half price solars at scale.
Tyler Cowen did an 'ask me anything' at Jane Street when I interned in 2016. One of the interns asked him exactly this: "What do you think of the fact that we all work here instead of, I don't know, curing cancer?"
He replied with, roughly, "Those of you who work here probably couldn't do anything else other than perhaps math research. Arguably, working here is the economically efficient use of your time."
I think about whenever I see a comment like this. Quant firms select for a very specific set of skills. In particular, I've found that many traders/software engineers in quant are very smart but not very self-directed. Places like Jane Street work well for people who can excel, but only when given a lot of structure and direction. I think this is not unrelated to why so many people 'accidentally' end up as traders after going to an Ivy League school!
He replied with, roughly, "Those of you who work here probably couldn't do anything else other than perhaps math research. Arguably, working here is the economically efficient use of your time."
I think about whenever I see a comment like this. Quant firms select for a very specific set of skills. In particular, I've found that many traders/software engineers in quant are very smart but not very self-directed. Places like Jane Street work well for people who can excel, but only when given a lot of structure and direction. I think this is not unrelated to why so many people 'accidentally' end up as traders after going to an Ivy League school!
General intelligence is overstated sometimes, but it is a thing. Someone who is smart enough to work for Jane Street probably could at least be an intelligence analyst or software developer at the NSA contributing to national security. (Jim Simmons literally was a code breaker during the Vietnam War)
I don't think there's a gene for playing esoteric minigames on the options market while you literally suck at everything else
I don't think there's a gene for playing esoteric minigames on the options market while you literally suck at everything else
Ya, I’m very surprised by the argument “you’re some of the best at numerical analysis and high frequency trading, and you’d be bad at anything else”, lol. That said, I think there are better reasons to work at such a place. Providing liquidity to the market is a good thing, and has real world value, it’s just hard for us to connect it to concrete outcomes. But as a simplified “toy” example, when they orchestrate a trade for/with a retirement fund, they help the fund improve its holdings at very low cost. That benefits everyone impacted by the retirement fund
I think the amount of money that is creamed off for this service is disproportionate, and the amount of benefit to wider society a case of diminishing returns.
Allocation of capital, market liquidity etc are useful, but the size of the financial sector and the rewards it gives out for this shuffling of money are insane.
Allocation of capital, market liquidity etc are useful, but the size of the financial sector and the rewards it gives out for this shuffling of money are insane.
I think the liquidity argument is overrated.
The market has two functions:
1. Incentivizing convergence of the price towards fundamental value, to support proper asset allocation decisions.
2. Supporting buying and selling (ie "liquidity") to shift consumption in time (and enable productive investments with the delayed consumption).
Suppose a retirement fund holds their investments over a 20 year period on average, growing at a modest 4%. A 1% wider spread would reduce the return from 119% to 118%. I'm not sure avoiding that is worth the financial sector constituting 30% of GDP.
What would happen if equity markets were only open a very short period a day? Say you have one auction a day, or maybe two, and no continuous trading?
Everyone whose advantage is speed would lose out (HFT, some prop traders). Their current gain would instead accrue to those on the other side of the trades.
The market has two functions:
1. Incentivizing convergence of the price towards fundamental value, to support proper asset allocation decisions.
2. Supporting buying and selling (ie "liquidity") to shift consumption in time (and enable productive investments with the delayed consumption).
Suppose a retirement fund holds their investments over a 20 year period on average, growing at a modest 4%. A 1% wider spread would reduce the return from 119% to 118%. I'm not sure avoiding that is worth the financial sector constituting 30% of GDP.
What would happen if equity markets were only open a very short period a day? Say you have one auction a day, or maybe two, and no continuous trading?
Everyone whose advantage is speed would lose out (HFT, some prop traders). Their current gain would instead accrue to those on the other side of the trades.
This comment is weird to me. Are you saying that the "financial sector" (incredibly vague term) makes 30% of US GDP? Not even close. A trivial Google search proves that it is way off base.
According to this source: https://www.statista.com/statistics/248004/percentage-added-...
Since the original article is about Jane Street's financial market making business, let's focus on investment banks. What percent of US GDP do you think that investment banks and trading hedge funds represent? It is tiny. I would be shocked if it is more than 5%.
Finally, professional financial market makers have an important role to play in reducing the size of bid-ask spread. I recently bought some 1Y US Treasury bills using Interactive Brokers. I was stunned by how tight are the spreads, and I am a "Retail Normie/Nobody". Absolutely, this was not available to people like me 30 years ago. Who do you think is providing this liquidity that keeps bid-ask spreads so tight?
According to this source: https://www.statista.com/statistics/248004/percentage-added-...
> finance, insurance, real estate, rental, and leasing ... is 20.7% of US GDP.
Since the original article is about Jane Street's financial market making business, let's focus on investment banks. What percent of US GDP do you think that investment banks and trading hedge funds represent? It is tiny. I would be shocked if it is more than 5%.
> What would happen if equity markets were only open a very short period a day? Say you have one auction a day, or maybe two, and no continuous trading?
Finally, professional financial market makers have an important role to play in reducing the size of bid-ask spread. I recently bought some 1Y US Treasury bills using Interactive Brokers. I was stunned by how tight are the spreads, and I am a "Retail Normie/Nobody". Absolutely, this was not available to people like me 30 years ago. Who do you think is providing this liquidity that keeps bid-ask spreads so tight?
The finance sector contributes only 7% of GDP, correct.
One of the sources I had in mind [0] cites 31%, but that was revenue as a proportion of GDP, which doesn't really make sense. However, the financial sector takes about 30% of US profits [1].
With respect to trading only once a day, I don't think your ad absurdum counterarguments hold water.
The HK stock exchange used to trade 4.5 hours a day, from 10 to 12, and then after a generous 2 hour lunch break from 2 to 4:30. How is trading 8 or 12 hours a day better for pension funds?
Price discovery with a limit order book that's cleared once a day might work just as fine as when it's spread out over hours, maybe even better.
Think about some major event affecting a company happening on the weekend. Then everyone can put in buy/sell orders with adjusted prices, and they'll be cleared during the morning auction. How is that worse than if the event happens intraday, and only the most switched on automatic HFT will pick off people still quoting at the old price, then the professional traders in hedge funds will pick off people?
As I said, the difference would be that the gains of the fast movers would instead be distributed among those on the "wrong side" of the news. Why should price discovery be worse?
Finally, what makes you think that liquidity and bid-ask spread concentrated in a minute a day would be worse than spread out over many hours a day?
[0] https://www.investopedia.com/ask/answers/030515/what-percent...
[1] https://conversableeconomist.com/2022/09/13/financial-servic...
With respect to trading only once a day, I don't think your ad absurdum counterarguments hold water.
The HK stock exchange used to trade 4.5 hours a day, from 10 to 12, and then after a generous 2 hour lunch break from 2 to 4:30. How is trading 8 or 12 hours a day better for pension funds?
Price discovery with a limit order book that's cleared once a day might work just as fine as when it's spread out over hours, maybe even better.
Think about some major event affecting a company happening on the weekend. Then everyone can put in buy/sell orders with adjusted prices, and they'll be cleared during the morning auction. How is that worse than if the event happens intraday, and only the most switched on automatic HFT will pick off people still quoting at the old price, then the professional traders in hedge funds will pick off people?
As I said, the difference would be that the gains of the fast movers would instead be distributed among those on the "wrong side" of the news. Why should price discovery be worse?
Finally, what makes you think that liquidity and bid-ask spread concentrated in a minute a day would be worse than spread out over many hours a day?
[0] https://www.investopedia.com/ask/answers/030515/what-percent...
[1] https://conversableeconomist.com/2022/09/13/financial-servic...
> What would happen if equity markets were only open a very short period a day?
I think about it like this:
How stable are prices of low liquidity instruments compared to the most liquid instruments?
What happens in the first 10-15 mins after the markets open? EXTREME volatility.
Longer trading sessions, higher volume and more liquidity lowers volatility on average.
A hypothetical X percent worse spread on my mortgage bonds, means I have to borrow X% more to buy the house. That’s meaningful money for most people.
Market makers will still earn the spread. More trading just means it gets lowered because of competition and volume.
I think about it like this:
How stable are prices of low liquidity instruments compared to the most liquid instruments?
What happens in the first 10-15 mins after the markets open? EXTREME volatility.
Longer trading sessions, higher volume and more liquidity lowers volatility on average.
A hypothetical X percent worse spread on my mortgage bonds, means I have to borrow X% more to buy the house. That’s meaningful money for most people.
Market makers will still earn the spread. More trading just means it gets lowered because of competition and volume.
I.e., they work to increase the return on capital. Since I believe one of the biggest problems in the world today is economic inequality stemming from an increasing gulf in the remuneration of labour and the remuneration of capital, I don't think this is a good thing. Let alone a good use of stupendous amounts of talent and money.
The only reason software engineers are able to be paid so much at all is because of “capitalism” and “free trade” I.e. vacuuming up money from the rest of the world, we sell them ads and “SaaS” and “intellectual property” while they sell us food and clothes. Would be a bad day for me personally and our profession in the US when that stops
My ethics is not defined by what is personally beneficial for me. I don't suddenly stop thinking something is wrong just because I benefit from it.
We’ve seen what happens when the price of eggs goes up a little bit, never before seen levels of prosperity seems to be the only way to convince Americans to treat each other kindly, pick your poison I guess
While I’m sympathetic to the “people working hard to build out ad markets instead of cancer research” argument, I only believe in a weak version of this.
Why? I have met many “smart with computers” people. Many of them have terrible people skills, don’t show up to things on time, are unable to keep their workspace clean, don’t know how to explain anything, and cry about how they can absolutely never ever be interrupted because their workspace is so hard. There are also people who are “good at it all”, of course, but I have the impression that the math/computer people tend to be fairly unwilling to deal with even mild inconveniences.
People who can barely deal with the tyranny of daily standups probably would struggle a lot in a world where you need to write grant proposals continuously to justify your existence.
I’m being glib for effect, but there’s so much involved in getting work done beyond “being smart”!
Besides… it’s not like the reason we don’t do more cancer research is because smart people didn’t go into that. “Cancer research” is limited by funding for positions into that domain!
So “this quant should have been a cancer researcher” is saying “this person who decided to become a quant will be a better cancer researcher than a cancer researcher who went into that domain directly”. I don’t know the prestige vectors there but it’s a stretch in my book!
Why? I have met many “smart with computers” people. Many of them have terrible people skills, don’t show up to things on time, are unable to keep their workspace clean, don’t know how to explain anything, and cry about how they can absolutely never ever be interrupted because their workspace is so hard. There are also people who are “good at it all”, of course, but I have the impression that the math/computer people tend to be fairly unwilling to deal with even mild inconveniences.
People who can barely deal with the tyranny of daily standups probably would struggle a lot in a world where you need to write grant proposals continuously to justify your existence.
I’m being glib for effect, but there’s so much involved in getting work done beyond “being smart”!
Besides… it’s not like the reason we don’t do more cancer research is because smart people didn’t go into that. “Cancer research” is limited by funding for positions into that domain!
So “this quant should have been a cancer researcher” is saying “this person who decided to become a quant will be a better cancer researcher than a cancer researcher who went into that domain directly”. I don’t know the prestige vectors there but it’s a stretch in my book!
> People who can barely deal with the tyranny of daily standups probably would struggle a lot in a world where you need to write grant proposals continuously to justify your existence.
I'm continuously writing grant proposals to justify my existence, and have been quite successful (lucky) in it. But I do bitch about the pointless grant game and about the pointless meetings.
Perhaps the problem is that to survive in academia you have to be able and willing to waste your time on all the bullshit that is not research. And it selects for people who are good and willing at the grantwriting and politics game, which is not the same as being good at research.
Maybe there's some point in bitching about the tyranny. Having tech people to do sales and marketing on the side like researchers have to do probably isn't an ideal division of labor.
I'm continuously writing grant proposals to justify my existence, and have been quite successful (lucky) in it. But I do bitch about the pointless grant game and about the pointless meetings.
Perhaps the problem is that to survive in academia you have to be able and willing to waste your time on all the bullshit that is not research. And it selects for people who are good and willing at the grantwriting and politics game, which is not the same as being good at research.
Maybe there's some point in bitching about the tyranny. Having tech people to do sales and marketing on the side like researchers have to do probably isn't an ideal division of labor.
I didn’t take that to be his point. I assume he says “economically efficient” because he means their strongest skillsets don’t have (m)any other uses and they wouldn’t be realizing their potential by leaving those skills unused.
He probably overstates that case, especially talking to early career interns that haven’t yet narrowed their specialization and could pivot to other highly quantitative roles that use other high level math.
He’s also probably flattering his audience, to whom “math research” is more likely to be status-bearing.
Doubt he’s saying they’d suck at anything else.
He probably overstates that case, especially talking to early career interns that haven’t yet narrowed their specialization and could pivot to other highly quantitative roles that use other high level math.
He’s also probably flattering his audience, to whom “math research” is more likely to be status-bearing.
Doubt he’s saying they’d suck at anything else.
I would rather people work in trading than for the NSA
[deleted]
Oof. Interesting take.
To build on / extend on this - quants / finance folks need to cultivate an image of only taking the very brightest, to justify the shit working conditions (even if pay is often decent) but honestly the brightest tend not to apply. Working in those environments is neither rewarding nor stimulating.
> He replied with, roughly, "Those of you who work here probably couldn't do anything else other than perhaps math research. Arguably, working here is the economically efficient use of your time."
Complete garbage. The same way that Jane Street hires smart people that don't know anything about trading and those people contribute, the same would be true if there was money in curing cancer.
Complete garbage. The same way that Jane Street hires smart people that don't know anything about trading and those people contribute, the same would be true if there was money in curing cancer.
Plus, this post is about someone who quit being a doctor to work in trading!
Tyler Cowen, master of the ‘cum hoc ergo propter hoc’ fallacy. He frequently mistakes the occurrence of phenomena for causative proof of that phenomena. He particularly exhibits this inclination when his confidence rises amid scant data (like the rest of us).
This error seems to be a particularly common (and often lauded!) trait among those who work in high-conjecture low-evidence fields (eg, economics). The prominent thinkers become skillful at deploying this fallacy: “see, it’s there, therefore <insert-personal-belief> is certainly the cause!”, using their credentials and esteem to mask the error. Listeners think, “well he’s a smart, respected guy,” and nod along despite the missing logical link.
I greatly appreciate Cowen’s podcast, and I definitely respect him as a thinker and inquisitor – so I don’t mean to discard his work or opinions (in fact, I appreciate his occasional brashness because it exposes the underlying thought/principle). However, many of his aggressive-yet-speculative statements (like the one you roughly quoted) are best received with an understanding of the error.
This error seems to be a particularly common (and often lauded!) trait among those who work in high-conjecture low-evidence fields (eg, economics). The prominent thinkers become skillful at deploying this fallacy: “see, it’s there, therefore <insert-personal-belief> is certainly the cause!”, using their credentials and esteem to mask the error. Listeners think, “well he’s a smart, respected guy,” and nod along despite the missing logical link.
I greatly appreciate Cowen’s podcast, and I definitely respect him as a thinker and inquisitor – so I don’t mean to discard his work or opinions (in fact, I appreciate his occasional brashness because it exposes the underlying thought/principle). However, many of his aggressive-yet-speculative statements (like the one you roughly quoted) are best received with an understanding of the error.
I plus-one the last paragraph, well explained.
At least it’s arguably better than working for giant ad engines.
This reminds me of Renaissance Technologies (Jim Simons, PhD in math from uc berkeley).
He started the company in 1982 after he left academia for finance and leveraged quant models for training, they mostly hire PhD mathematicians, physicists and scientists, working on algorithms.
They have a fund called Medallion that is closed off from outsiders (you have to be an employee I believe), and it averages 66% annual gross returns (39% net after fees). Generated hundreds of billions in profits.
He started the company in 1982 after he left academia for finance and leveraged quant models for training, they mostly hire PhD mathematicians, physicists and scientists, working on algorithms.
They have a fund called Medallion that is closed off from outsiders (you have to be an employee I believe), and it averages 66% annual gross returns (39% net after fees). Generated hundreds of billions in profits.
Well. Read about him all the way to the end, then.
Theres large portions of the tech industry that are pretty useless to society too while paying incredibly high salaries
random examples include:
- facebook/google ads teams
- various SaaSLOP companies
Lots of smart engineers that work on making buttons pretty and A/B testing crap rather than pushing the boundaries of science.
Lots of smart engineers that work on making buttons pretty and A/B testing crap rather than pushing the boundaries of science.
This is also bad. Huge amounts of our human capital are invested in fundamentally anti social enterprises. This sort of market failure seems to be endemic to our system as it stands today, and the only answers I've seen are "well, but that other bunch are even less useful!"
Eh, hard to say that when the main alternative (math, physics or CS research) requires jumping through a lot of selective hoops, barely has positions available, and absolutely no trouble filling all the positions they do have.
Why would it be wasted? I like apples and I like being able to buy them any day of year, even out of season. The only reason I can do so is because there is an entire industry of people who manage the inventory, distribution, try to predict supply and demand, and take on a risk doing that. The principal reason why I can buy and sell equities at very small spreads any day of the year is similar: traders are competing to take the other side of the bet.
I also encourage everyone to read CFTC response to the Vatican's (!) Congregation for the Doctrine of the Faith https://www.cftc.gov/PressRoom/SpeechesTestimony/giancarlore... It talks about about the social utility of derivatives and refutes the narrative that they are basically tools of "speculation." Traders take on risks others can't bear, creating massive economic value that ripples through the entire system.
I also encourage everyone to read CFTC response to the Vatican's (!) Congregation for the Doctrine of the Faith https://www.cftc.gov/PressRoom/SpeechesTestimony/giancarlore... It talks about about the social utility of derivatives and refutes the narrative that they are basically tools of "speculation." Traders take on risks others can't bear, creating massive economic value that ripples through the entire system.
Everyone understands that there's a pricing utility to markets and market speculation. But consider that, like in all things, the free market has its 'biases': wouldn't you expect that something like trading, which directly produces profit with very little overhead and maximal ability to hedge against rush, would draw an inordinate amount of capital -- in the same way that things like infrastructure, sans government intervention, naturally repel it?
We have far, far too much of our economy sunk into a sector that fundamentally produces nothing of value, one that only shuffles value around and chooses to whom it will be allocated (coincidentally, often the people doing the allocating!).
Compare our economy and its woes to China. Do they spend nearly as much of their human capital figuring out how to get an extra bp on some statistical arb? Thethe origins of DeepSeek are illustrative in this regard.
We have far, far too much of our economy sunk into a sector that fundamentally produces nothing of value, one that only shuffles value around and chooses to whom it will be allocated (coincidentally, often the people doing the allocating!).
Compare our economy and its woes to China. Do they spend nearly as much of their human capital figuring out how to get an extra bp on some statistical arb? Thethe origins of DeepSeek are illustrative in this regard.
> I like apples and I like being able to buy them any day of year, even out of season. The only reason I can do so is because there is an entire industry of people who manage the inventory, distribution, try to predict supply and demand, and take on a risk doing that.
This has been the case for well over 200 years and is not done by the same kind of people that modern finance/fintech employs.
Imagine the world 150 years ago and imagine were you'd allocate the smart people that's the highest paid these days. They'd be doing science and developing technology instead of doing finance. Why is finance that much more important now? (is it just because it makes rich people richer?)
This has been the case for well over 200 years and is not done by the same kind of people that modern finance/fintech employs.
Imagine the world 150 years ago and imagine were you'd allocate the smart people that's the highest paid these days. They'd be doing science and developing technology instead of doing finance. Why is finance that much more important now? (is it just because it makes rich people richer?)
I do think finance is important in making markets more efficient, predictable, and liquid. In turn, this improves the amount of capital available and also improves the efficiency and availability of international trade.
It's also a sophisticated "export" product itself.
However, I think there's a point where, absent sufficient regulation, it overshoots and most of the effort goes to squeezing nickels out of everyone else in underhanded ways. I feel like we could get 90-95% of the benefits with a finance industry that's half the current size, and also avoid a whole lot of indirect costs that come with an oversized finance industry.
It's also a sophisticated "export" product itself.
However, I think there's a point where, absent sufficient regulation, it overshoots and most of the effort goes to squeezing nickels out of everyone else in underhanded ways. I feel like we could get 90-95% of the benefits with a finance industry that's half the current size, and also avoid a whole lot of indirect costs that come with an oversized finance industry.
> I like apples and I like being able to buy them any day of year, even out of season. The only reason I can do so is because there is an entire industry of people who manage the inventory, distribution, try to predict supply and demand, and take on a risk doing that.
In practice, if the availability of the apples you're eating is dependent on finance people, they're commodity grade, not specialty varieties.
I don't eat apples, but I'd rather have a good heirloom tomato from the farmer's market than the commodity grade stuff that's at the grocery store.
In practice, if the availability of the apples you're eating is dependent on finance people, they're commodity grade, not specialty varieties.
I don't eat apples, but I'd rather have a good heirloom tomato from the farmer's market than the commodity grade stuff that's at the grocery store.
How do you get to the farmers market?
Walk?
If not then how was your vehicle produced and powered and how were its components got to you?
Ok, but what do high frequency trading bots possibly do to help me eat apples in winter (a deeply unecological mode of consumption, btw)?
I believe that they provide liquidity. That’s a fancy way of saying that they ensure that any time someone wants to sell he can find a buyer, and every time someone wants to buy he can find a seller. That in turn means that folks are more confident of their ability to enter and exit a position, which means that they are more likely to enter and exit positions, which makes the market more efficient at finding prices.
And of course prices are hugely valuable: the market is a conversation about the relative value of everything, and the language it speaks is prices. So by enabling liquidity, high-frequency traders are enabling efficient allocation of resources towards those things which mankind most dearly needs and desires.
And of course prices are hugely valuable: the market is a conversation about the relative value of everything, and the language it speaks is prices. So by enabling liquidity, high-frequency traders are enabling efficient allocation of resources towards those things which mankind most dearly needs and desires.
There is an extreme case of diminishing returns at play here, and unfortunately the amount of money that flows through these markets is so incredible that it becomes worth it (more than worth it) to commission the top minds of society to push the limit as close to 1 as absolutely possible.
Derivatives have great value to industry. Derivatives that require the fuel of 15 math Phds to lock in fractions of a percentage pricing inefficiencies so their firm can pocket the difference do not have much value at all.
It's like employing Harvard Med surgeons to remove gold dust from gold market sidewalks, and calling it "gold market efficiency".
Derivatives have great value to industry. Derivatives that require the fuel of 15 math Phds to lock in fractions of a percentage pricing inefficiencies so their firm can pocket the difference do not have much value at all.
It's like employing Harvard Med surgeons to remove gold dust from gold market sidewalks, and calling it "gold market efficiency".
And yet if there was so much gold dust lying on the sidewalk that you could pay 15 Harvard med surgeons enough to pick it up and still have some left over, would you just not?
That is the issue.
The amount of gold recovered by them as a function of the entire global gold market is a minuscule rounding error. A loss so small that when distributed across every market participant (as it would be if left alone on the ground), would amount to no practical discernible difference in anyone's life.
But having 15 less top notch surgeons not doing surgery? There stands to be many practical discernible differences in many people's lives.
Keep in mind, the surgeons are not the only ones out there. There are large armies of grunts combing those streets 24/7 picking up all the easy pieces. The surgeons are there to get the dust that everyone else misses. It's an enormous waste of talent.
The amount of gold recovered by them as a function of the entire global gold market is a minuscule rounding error. A loss so small that when distributed across every market participant (as it would be if left alone on the ground), would amount to no practical discernible difference in anyone's life.
But having 15 less top notch surgeons not doing surgery? There stands to be many practical discernible differences in many people's lives.
Keep in mind, the surgeons are not the only ones out there. There are large armies of grunts combing those streets 24/7 picking up all the easy pieces. The surgeons are there to get the dust that everyone else misses. It's an enormous waste of talent.
Gold is not analogous to trading.
If you subscribe to the theory that markets allocating capital based on supply and demand is beneficial for society (even if detrimental sometimes in the short term), then traders provide the utility of contributing to the proper allocation of resources in society (which is constantly in flux).
The fact that smart people opt to go into trading (or selling advertising) rather than research is a consequence of government underpaying scientists (or the volatility is too high, or the path to quality of life at work is too low).
Either way, if the situation is that society needs more scientists or doctors or whatever, then the government should be paying more to incentivize those choices.
If you subscribe to the theory that markets allocating capital based on supply and demand is beneficial for society (even if detrimental sometimes in the short term), then traders provide the utility of contributing to the proper allocation of resources in society (which is constantly in flux).
The fact that smart people opt to go into trading (or selling advertising) rather than research is a consequence of government underpaying scientists (or the volatility is too high, or the path to quality of life at work is too low).
Either way, if the situation is that society needs more scientists or doctors or whatever, then the government should be paying more to incentivize those choices.
Markets influenced by traders can lead to misallocation of resources, as traders often prioritise short-term profits through speculation rather than investing in productive, long-term projects beneficial to society. Traders frequently increase market volatility, contributing little to meaningful innovation or economic growth.
Additionally, even if governments improved pay for essential roles like scientists or doctors, the outsized financial rewards from trading would still attract talent away from these critical areas. Therefore, depending on markets and government incentives alone ignores the negative impact traders’ profit-driven strategies can have on society’s overall well-being.
Additionally, even if governments improved pay for essential roles like scientists or doctors, the outsized financial rewards from trading would still attract talent away from these critical areas. Therefore, depending on markets and government incentives alone ignores the negative impact traders’ profit-driven strategies can have on society’s overall well-being.
I think I’d rather have the surgeons saving peoples lives tbh.
Then make it so one does not have to sacrifice their 20s and harm their health to become one.
Expand the number of medical schools, the number of residency positions, etc. Reduce tuition or pay graduates and residents more. Reduce unnecessary learning requirements so that one can expect to have a life and become a surgeon. Reform tort law.
None of these are under the purview of trading firms or the people that work there.
Expand the number of medical schools, the number of residency positions, etc. Reduce tuition or pay graduates and residents more. Reduce unnecessary learning requirements so that one can expect to have a life and become a surgeon. Reform tort law.
None of these are under the purview of trading firms or the people that work there.
> Derivatives that require the fuel of 15 math Phds to lock in fractions of a percentage pricing inefficiencies so their firm can pocket the difference do not have much value at all.
They have at least as much value as the total compensation of 15 math PhDs, otherwise that work would not be done.
They have at least as much value as the total compensation of 15 math PhDs, otherwise that work would not be done.
It is bizarre that people conflate money with value.
A scammer who makes $1m dollars a year does not generate any value, in fact he removes value from society.
A scammer who makes $1m dollars a year does not generate any value, in fact he removes value from society.
It's remarkable how on every single discussion on this topic the Circular Argument is eventually brought up lol
That isn't really what the math people are doing. Pricing is easy.
Amazing link. Including the car mechanic joke.
Sure, some derivatives have some utility.
But, ok, Spread Networks spent $300m around 2010 to build a somewhat straighter glass fibre data connection between NY and Chicago. Other companies spent more to build a micro wave connection, since microwaves move faster through air than light through fibre.
Is that money really well spent? And I'm afraid a lot of what happens in finance is similarly private gain, without much social welfare increase.
But, ok, Spread Networks spent $300m around 2010 to build a somewhat straighter glass fibre data connection between NY and Chicago. Other companies spent more to build a micro wave connection, since microwaves move faster through air than light through fibre.
Is that money really well spent? And I'm afraid a lot of what happens in finance is similarly private gain, without much social welfare increase.
That money is very well spent considering that the microwave network technologies developed for trading have huge applications for military and emergency response use. Similarly, millimeter wave technology built for trading firms in New Jersey could rightly be considered as one of the precursors to Starlink.
It's fun to see how every time a finance topic pops up, discussion steers towards lamenting on how "talent is wasted" by doing this and not that.
See, if you're a fishmonger and someone comes to you and say "why don't you trade flowers instead", you ignore them because they are not your customer. They won't trade fish with you, and in fact they wouldn't trade flowers either. They're just useless relative to your trade and only yapping so just turn your back on them and leave them be. Tell them to fuck off and find a flower monger if they really want it or mong those those flowers themselves because you are sticking to monging fish.
See, if you're a fishmonger and someone comes to you and say "why don't you trade flowers instead", you ignore them because they are not your customer. They won't trade fish with you, and in fact they wouldn't trade flowers either. They're just useless relative to your trade and only yapping so just turn your back on them and leave them be. Tell them to fuck off and find a flower monger if they really want it or mong those those flowers themselves because you are sticking to monging fish.
At the end of the day, it’s fine to ignore the critics or people who don’t understand your work, but dismissing the conversation entirely without considering the underlying point might close off an opportunity for meaningful reflection. Just my two cents.
Reflect on what? Some utopian cvasi-religious bullshit which has no real applicability except virtue-signalling? Why don't we all do nice things instead of being caught in the capitalistic rat race to make a living? (Not that other systems fared better). Why do we have to work shit, meaningless jobs instead of doing the grand things of life, engineering, arts, etc?
Well first of all because all those niches are already full beyond capacity. All the engineering, arts, cancer research, you name it, that the society can pay for is stuffed to bursting point and in no lack of pipelines of fresh wannabe recruits. Also cancer research is a deeply unprofitable enterprise overall, much in contrast to a domain like finance. One can labor a lifetime and get nothig, at least in finance they'll shovel some money from one pocket to the other and pocket the commission themselves.
Please leave people alone and stop suggesting alternate careers, that you yourselves did not choose. That's the uttermost hypocrisy. I don't wanna see anymore programmers with fat paychecks jumping on whatever latest fad like flies on a fresh laid turd doing AI and crypto and advertising and just bullshit pretend work at FAANG shedding crocodile tears on how the finance guys should be poor and suffering. Quit your high paying jobs yourselfes and go starve while doing biology PhDs then work as baristas while competing with the other 10,000 highly skilled Phds hoping to catch one of the 10 currently open positions in the world that pay a miserable salary with no stability in the future that does cancer research.
Well first of all because all those niches are already full beyond capacity. All the engineering, arts, cancer research, you name it, that the society can pay for is stuffed to bursting point and in no lack of pipelines of fresh wannabe recruits. Also cancer research is a deeply unprofitable enterprise overall, much in contrast to a domain like finance. One can labor a lifetime and get nothig, at least in finance they'll shovel some money from one pocket to the other and pocket the commission themselves.
Please leave people alone and stop suggesting alternate careers, that you yourselves did not choose. That's the uttermost hypocrisy. I don't wanna see anymore programmers with fat paychecks jumping on whatever latest fad like flies on a fresh laid turd doing AI and crypto and advertising and just bullshit pretend work at FAANG shedding crocodile tears on how the finance guys should be poor and suffering. Quit your high paying jobs yourselfes and go starve while doing biology PhDs then work as baristas while competing with the other 10,000 highly skilled Phds hoping to catch one of the 10 currently open positions in the world that pay a miserable salary with no stability in the future that does cancer research.
Nobody is saying finance people should be poor and starve. The same people critiquing the finance industry in this thread have been happy to critique ad tech as well, and I agree that they're both largely parasitic 'faux frais'. But regardless of which industries you choose to put in that category, the point isn't to make individuals choose to work in one or another -- that's not how serious change happens. It's to build support and hopefully eventually consensus on the idea that our economic system is broken and in serious need of change.
Your perspective is totally valid - people should do what works best for them. But dismissing the conversation out of hand could limit opportunities for a deeper understanding of what kind of work aligns with both personal fulfilment and societal good.
Yes this. There's no shortage of people working in quant finance who used to work in research who quit because they literally could not afford to start a family.
People imagine it's like prestigious professors at top research institutions leaving their posts because of the allure of money, when it's more like the 100x people just like them who were denied tenure and realized they chased a career in science like an inner city youth chases getting into the NBA: most people fail. Then what?
I even know people who worked in quant finance, struck it rich, quit to finish becoming doctors, and then came screaming back because of how horrible medicine was.
People imagine it's like prestigious professors at top research institutions leaving their posts because of the allure of money, when it's more like the 100x people just like them who were denied tenure and realized they chased a career in science like an inner city youth chases getting into the NBA: most people fail. Then what?
I even know people who worked in quant finance, struck it rich, quit to finish becoming doctors, and then came screaming back because of how horrible medicine was.
Thank you for the link, it was very insightful.
While it’s true that trading and financial markets provide value, the conversation around “wasted talent” isn’t necessarily about denying the value of these industries. It’s more about the broader context - are all forms of financial activity contributing to the greater good, or are some activities just enriching a small group of people without providing societal benefits?
Would you rather people spend their time on centering divs? Or come up more ways to make people click on ads. I see this type of view often on HN from big tech employees. Get off your high horses people.
Are those the only other options to being a quant? That's a false dichotomy if I ever saw one.
You haven’t contested the substance of the argument at all. It’s as though you agree.
There is nothing to contest here. This is HN forum and there is wide held belief that capital markets tech work is somehow below what the folks are doing in SV.
Funny enough I don't work in tech, but rather manufacturing.
From a technical challenge standpoint, trading is extremely difficult. But it provides high upside which only starting a business can exceed. However, starting a business has all the other uncertainty (regulatory, accounting, insurance, licensing, taxes, customers, branding, sales, product-market-fit), making it far more psychologically challenging overall. Startup founders need to learn and juggle a lot of things instead of hyper-fixating on markets. Basically, if you know you are sufficiently intelligent and talented as an engineer, your expected return is likely much higher in trading (especially via a big quant firm), with much lower risk.
If we want more talented traders to become founders building economic value (which we should all agree on), we need to make it less onerous to start a business.
If we want more talented traders to become founders building economic value (which we should all agree on), we need to make it less onerous to start a business.
I'd argue it provides negative value even. The goal of these people is to extract money from the markets, without actually producing anything. In biology, we'd call that parasitism.
It's a zero-sum game. People don't understand this: https://community.intercoin.app/t/intercoin-compared-to-the-...
I guess advertising is too, in the limit. So a lot of the business model on the web is also not much value to society.
I guess advertising is too, in the limit. So a lot of the business model on the web is also not much value to society.
[deleted]
Society has the problem, not the individuals.
Please share what industry you work in? And if it's social media/ad tech or military, get off your high horse you smug hypocrite.
This sentiment is also covered in 'Zero to One' by Peter Thiel.
[deleted]
I recommend you read the life of Jim Simons (through The Man who Solved the Market) as well as a recent HN thread on DeepSeek showing how their trading arm finances the whole company.
There is much more nuance in this world than your post implies.
There is much more nuance in this world than your post implies.
Ten years ago, all the $1mn+ earning engineers were in trading. Now they are in LLM/AI. Glad to see this happen, Jane Street has increased their advertising here because of this.
I love this podcast but I can’t listen to it while running. It’s too interesting so I risk falling off the treadmill :D
I read this and think: "I would love to spend a day with Jane Street and teach them how to use notebooks." So much effort is wasted because of knowledge gaps or systems that don't encourage best practices.
I just taught a course to a client this week helping them with this (and other best practices for Python).
I just taught a course to a client this week helping them with this (and other best practices for Python).
I'm intrigued - I didn't listen to the post but wonder what kind of best practices you're talking about.
Do you have this written down anywhere or on video anywhere? I'd love to learn more about what you mean.
Do you have this written down anywhere or on video anywhere? I'd love to learn more about what you mean.
I’m intrigued as well.. My experience is notebooks struggle as a format for production code. We encourage people who work heavily in notebooks to use them for exploratory work, but choose other tools when it comes time to ship.
When you are exploring something, experimenting, showing.. it’s great; train-of-thought structure, APIs like Pandas optimised for writing and terseness etc.
But when you have a piece of code that will lose a million dollars a minute if someone ships a bug, and which will be maintained by many engineers over many years, then you really want a format that’s optimised for long-term maintenance, incremental change, testability, and APIs optimised for readers.
When you are exploring something, experimenting, showing.. it’s great; train-of-thought structure, APIs like Pandas optimised for writing and terseness etc.
But when you have a piece of code that will lose a million dollars a minute if someone ships a bug, and which will be maintained by many engineers over many years, then you really want a format that’s optimised for long-term maintenance, incremental change, testability, and APIs optimised for readers.
I write production code, I also work lots in Jupyter notebooks.
Personally, I think the fact that notebooks are usually easier/funner for me to work with is a big problem. I'm by no means a Clojure expert, but I did do a semi-large project in Clojure a few years ago, and some of the ideas of true REPL-driven development that exist there are things I wish that Python supported.
It's hard to explain without actually learning it for real (and most Python devs mistakenly think Python has REPL-driven development; I sure did before learning Clojure!). But once you get used to being able to interact with your actual source code, and at any point just being able to write new code and immediately print out its value, then with one shortcut make it part of the regular codebase... that just blurs the distinction that exists between Jupyter Notebooks and production code in a way that makes everything much better.
Personally, I think the fact that notebooks are usually easier/funner for me to work with is a big problem. I'm by no means a Clojure expert, but I did do a semi-large project in Clojure a few years ago, and some of the ideas of true REPL-driven development that exist there are things I wish that Python supported.
It's hard to explain without actually learning it for real (and most Python devs mistakenly think Python has REPL-driven development; I sure did before learning Clojure!). But once you get used to being able to interact with your actual source code, and at any point just being able to write new code and immediately print out its value, then with one shortcut make it part of the regular codebase... that just blurs the distinction that exists between Jupyter Notebooks and production code in a way that makes everything much better.
I'd love to hear more about this REPL-driven development. I've heard people bring it up from time to time, but it's clearly very different from the typical "stateless horizontal micro-service" that has become common practice.
What tools are used for "write new code and immediately print out its value, then with one shortcut make it part of the regular codebase" and how does that square with working on a team and getting code reviewed?
What tools are used for "write new code and immediately print out its value, then with one shortcut make it part of the regular codebase" and how does that square with working on a team and getting code reviewed?
My book, Effective Pandas 2, has many of them. There's a few conference talks of mine floating around on YouTube that also mention some.
Writing clean data code is one aspect. Filling in knowledge gaps is another. Covertly teaching software engineering best practices to folks who "aren't programmers" yet sit down and write code in Jupyter all day is another.
I should probably write a blog post or record a short video. (I just taught a week long course on this for a client last week.)
Writing clean data code is one aspect. Filling in knowledge gaps is another. Covertly teaching software engineering best practices to folks who "aren't programmers" yet sit down and write code in Jupyter all day is another.
I should probably write a blog post or record a short video. (I just taught a week long course on this for a client last week.)
I've been back and forth whether notebooks are really "best practice".
Currently leaning towards no, as they are hard to test, compose, and version control (it is possible, but you need bespoke notebook-specific tools, and most just don't).
Currently leaning towards no, as they are hard to test, compose, and version control (it is possible, but you need bespoke notebook-specific tools, and most just don't).
Why are you trying to test and compose notebooks? Notebooks don't really fit into that lifecycle of work, they're implicitly script-like code.
Any time a notebook needed to be tested, refactoring it into a module and testing that was the better choice.
Notebooks in my opinion are by far the best tool for interactive and exploratory data work. I've been using Jupyter/IPython for 10 years and it takes very little discipline to keep them clean and clear. I've never bothered trying to deploy them in any meaningful way.
The quirks with version control are annoying, that's one reason I've switchex to marimo in the last few months.
Any time a notebook needed to be tested, refactoring it into a module and testing that was the better choice.
Notebooks in my opinion are by far the best tool for interactive and exploratory data work. I've been using Jupyter/IPython for 10 years and it takes very little discipline to keep them clean and clear. I've never bothered trying to deploy them in any meaningful way.
The quirks with version control are annoying, that's one reason I've switchex to marimo in the last few months.
Very interesting podcast, I find that the guest was very candid so it was great to hear what it is really like working at Jane Street.
The reproducible-Python notebook problem/notebook for researchers mentioned in the podcast inspired me to create a new project, branch-pad https://github.com/alexyorke/branch-pad which is an interactive Python notebook environment that allows you to create and explore multiple branches of code execution.
The reproducible-Python notebook problem/notebook for researchers mentioned in the podcast inspired me to create a new project, branch-pad https://github.com/alexyorke/branch-pad which is an interactive Python notebook environment that allows you to create and explore multiple branches of code execution.
Keynesian for machines
Notebooks: just use Marimo. They've fixed python notebooks.