If you're just building a non-security-sensitive frontend to validate market traction, it makes total sense to go full AI. The commenter working at a 10-person company getting flamed for not using AI to iterate aggresively against competitors has a super valid point.
Validation methods will evolve to accommodate human laziness. Insisting on doing it the hard way is no different than the old-timers who used to claim engineers 'weren't skilled' if they didn't know how to use punch cards.
Do you also inspect and study what assembly code your program was compiled to?
// Obviously LLMs are non-determenistic etc and it depends on your domain, but your VP's point 100% makes sense if you folks are trying to cook up another demo-CRUD apps to convince investors for another funding round
It surprises me how JetBrains managed to lose such a great market opportunity.
I don't think they ever going to be able to re-claim large chunk of developers who are now fine with thin VSCode-like + Terminal for non-JVM languages.
Perfect example of how large corp with research capacity failed to navigate their product changes.
Solid q. I think the part of it is that it’s really easy to attract some “mass” (capital) of users, as there are definitely quite a few of idle Macs in the world.
Non-VC play (not required until you can raise on your own terms!) and clear differentiation.
If you want to go full-business-evaluation, I would be more worried about someone else implementing same thing with more commission (imo 95% and first to market is good enough).
That solution actually makes great sense. So Apple won in some strange way again?
Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns.
I think batch-evals for non-sensitive data has great PMF here.
You absolutely could. It'd be cool (not easy from security/compliance perspective) to be able to deeply "scan" your prod-deployed app.
There are a quite a few startups created by connecting relevant eBPF/OTel traces e.g. in response to uncaught exceptions (traditional RAG-based bug-fix generation).
I think that works well for smaller orgs, but in larger organizations (especially where department headcount growth is not expected) it might be more complicated and more meta/political. I wish that were not the case, but in reality, trying to "do the job" of your manager can backfire.
I extracted text quests from Space Rangers 2 after Claude failed a simple riddle I gave it when playing. Ran frontier LLMs through the 'easiest' quest, got 1 success in 60 attempts. Humans don't really have any problems solving it.
I agree. I never understood LeCun's statement that we need to pivot toward the visual aspects of things because the bitrate of text is low while visual input through the eye is high.
Text and languages contain structured information and encode a lot of real-world complexity (or it's "modelling" that).
Not saying we won't pivot to visual data or world simulations, but he was clearly not the type of person to compete with other LLM research labs, nor did he propose any alternative that could be used to create something interesting for end-users.
Ehh, pretty sad there's almost no information on FACEIT anti-cheat. One of the most impactful out there. Wonder if it's just the invasiveness that separates it.
Valve can't replicate even part of it, while CS2 game modes are flooded with cheaters. Most people who chase competitiveness (which CS used to be all about – now it's also skins) just install FACEIT directly and ignore 90% of built-in game content.
Maybe Valve just doesn't want to make the game more difficult to install and sacrifice several % of their user base.
"You can actually read up on how these things work."
While you can definitely read about how some parts of a very complex neural network function, it's very challenging to understand the underlying patterns.
That's why even the people who invented components of these networks still invest in areas like mechanistic interpretability, trying to develop a model of how these systems actually operate. See https://www.transformer-circuits.pub/2022/mech-interp-essay (Chris Olah)