HFTs certainly have an edge on micro/milli/seconds timescales, where price moves are more predictable. But that expertise doesn't necessarily generalize to mid/low-frequency trading where human traders and the real world come into play.
Eh, don't bother reading any financial literature that other commentators suggest you for this competition. With anonymized input data and even the nature of the problem (is it even short or long term prediction? the two are very different beasts), you're not going to use any financial domain knowledge here. Besides, the good stuff is rarely if ever openly published. Just keep in mind two useful unchanging truths about finance here: it's time series data and with very low signal-to-noise ratio, so overfitting is a major concern.
Start by reading/running notebooks ("kernels") and winners solutions from this and previous competitions with tabular/time series data, building models and then grid searching, stacking, bagging, boosting 'em up. Here's a bit aged but still useful guide how pros do it: https://mlwave.com/kaggle-ensembling-guide/
It's privately owned - the firm trades the executives' (partners) capital. Wlb depends on the role. As a quant you won't have a life in any firm, as an SWE I heard it's decent for the pay.
How about being able to go buy stocks/ETFs paying nothing in commissions and razor thin spreads today? A few decades ago you'd pay O($10-100) per trade and then some in bid/ask
This is all largely thanks to HFT. Robinhood is only viable because big HFT firms are willing to pay dearly for the privilege to serve retail order flow
On a frozen feature set and evaluation dataset, sure. 3000+ competitors vs a few employees at JS who prepped this data. In light of very noisy data, the chances are not really on JS side here. But would the best model still be as useful on novel out of sample data is very questionable
Market prediction is low signal-to-noise problem, what's much more likely the contest will be an exercise in overfitting and the final shake up will be yuuuge.
Jane Street is a high frequency firm. In this domain execution & infrastructure matters as much if not more than your algo secret sauce.
What good is kagglers' favorites giant boosted bagged lightgbm/xgboost/neural ensembles which will take seconds to predict, when my FPGA or ASIC running in the same rack as exchange's matching engine can make a million trades in the meantime with a much simpler strategy involving nothing beyond freshman math (take a cue in XTX Markets' firm name, for example)
And in the longer timeframes, the edge is more often than not in the data than training algo.