100%. It really depends on the application domain and system complexity.
I see a lot of people speaking about a 10x productivity improvement and so on. When I work on hobby projects, I do see that. Just last weekend, I set up a hobby project that I've been thinking about for a while. I'm pretty sure it would have taken me at least a week to implement manually, but instead, it took me three hours.
But some of the systems I have to work on during the day are so big and complicated that you can spend multiple days on a small feature or even just tracking down a bug, even with the support of Opus.
Expecting even a 2x productivity improvement on some those systems is wildly unrealistic. I'm seeing a lot of people get stressed out because the productivity gains from simple application building trickle into the expectations for these complex systems.
That said, if things keep improving at this rate it might just be a matter of time.
I set up mine with WSL and used a Linux terminal over xserver. Once you have a decent unix shell (and a good terminal) it's fine. VSCode etc work fine. This was 2020ish, things might've gotten easier now.
But I do prefer a Linux or Mac for development, just because it's so much less hassle to set up.
Windows developers who don't set up a good terminal environment... I honestly don't know how they manage.
I think there are no safe harbor investments at this time. Even gold is unpredictable.
Personally I went 80% world excl US and 20% equal weight S&P500 to hedge against what I think is an AI bubble. But if the market decides to adjust Nvidia's valuation 20% downward next week, I expect there to be ripple effects throughout the economy.
(Like the .com bubble, I think the tech is genuinely transformative and here to stay, but the valuations are just ridiculous.)
The notation is supposed to mean: you have a matrix Q, and also a shared K=V matrix.
I agree with GP that it's super confusing to us the minus sign as a delimiter between formulas. The tuple notation suggested elsewhere would be way clearer.
I'm afraid you're mistaken. He hired a cartographer to iterate over the design, but from the images, he likely used that feedback to create a map style.
Where I live, your employer basically has to give you notice (weeks to months, depending where you live). It's common for that notice period to turn into "garden leave" though, i.e. get paid but don't show up.
Mass layoffs, or RIFs, operate under slightly different rules, but I still saw a stark difference between US and EU employees when I went through one at a different corp.
US accounts were deactivated same day. EU employees were given until end of week to look over the proposed terms etc.
Note that any supervised fine-tuning following the Pretraining stage is just swapping the dataset and maybe tweaking some of the optimiser settings. Presumably they're talking about this kind of pre-RL fine-tuning instead of post-RL fine-tuning, and not about swapping out the Pretraining stage entirely.