I've used RLAIF to build out heuristic based non-LLM models for various decision systems and achieved like, 95% F1 on certain projects. We're in a place where models can be used to fine tune a lot of stuff via loops.
LLM architectures need to fundamentally change or inference needs to be used in constrained trusted environments. Nothing surprising here. Filtering and sanitizing, relying on tags around input strings that can be intercepted and replayed is like, childs play security theatre. As long as prompts accept abitrary user input nothing is changing here. Non-deterministic security is never going to be acceptable.
I'm quite certain that Google's AI services are likely the most used in the world right now by virtue of having the widest distribution. It's in the search box. It's on your Android phone. Just because they aren't the preferred coding or research agent does not mean they are losing - that's a pretty small slice.
Would love to see this benchmark tested on more perceivably LLM friendly frameworks/ORM (e.g. is NestJS or Drizzle / Kysely more performant than their choice of Sequelize) and more frontier model vs just GPT 5.2.
Anyone read whether these tests include any validation loops? What happens if the models get back test failures, for instance? Understanding how many turns to hit full passing behavior suite would also be interesting. Great methodology in the study though.
This is fairly standard practice for device fingerprinting. LI is probably using this to protect its platform from scraping etc, and extension lists have sufficient enough entropy to help identify users and form a useful component of a fingerprint.
The worst has been the post-covid assignment of seating and QR code driven ordering in bars. So few opportunities to mingle. I miss standing in bars, talking to bartenders, chatting with random patrons. This has recovered much better in large cities but I find that restaurants and bars in US suburban environments are deeply impersonal now. It’s no wonder singles are stuck meeting partners on apps with so little unstructured social opportunities left. Not to mention no one is going to bars anymore anyway.
I'm still stuck on superpowers. Can't seem to get better plans out of native claude planning - superpowers ensures I have a reviewed design that actually matches my mental model. Typical claude planning doesn't confirm assumptions sufficiently for my weak brain dumps/poorly spec'd tickets.
I think that, for possibly a very long time, AI will just increase the quality bar and scale of expectations when we produce things. We might take the same amount of time (or longer) to produce something, but with significantly better outcomes. Ultimately human preferences and tastes prevail and the world is full of problems that are not simple I/O, that are not repeatable, and that require human taste to improve. The people who will immediately survive economically are the ones who leverage AI to produce stuff that wasn't possible before.
I've tried all the Q&A skills, confidence meters and little hacks to get agents to clarify and propose better solutions. Clarification and planning has gotten a lot better using some skills (e.g. obra/superpowers), but counterproposals and negative feedback are rarely up to snuff with something a staff level colleague would come up with - this seems to be amplified when you already have an extensive PRD or plan together. If a plan is already fleshed out but is inefficient or contains some anti-patterns, I've had better results just throwing these out, taking what I've learned and summarizing tradeoffs in a brand new chat.
Once you have a comprehensive plan together, or a fairly full context window, agents have a lot of issues zooming out. This is particularly painful in some coding agents since they're loading your existing code into context and get weighted down heavily by what already exists (which makes them good at other tasks) vs. what may be significantly simpler and better for net-new stuff or areas of your codebase that are more nascent.
Yes, we're in for more headless interfaces and there are existing products that will struggle to serve these new interaction models due to organizational constraints. But I don't think it's as simple as asking "are they a system of record" as we think about the companies that will adapt and thrive and the new ones that will come. Enterprises are investing AI spend into improving core processes and responding to competitive pressure, not saving money and introducing risk into areas they have historically delegated to vendors. AI is going to give us more software, and increase spending as firms seek efficiency in new areas, and they're going to continue to knock on doors of vendors to do it as they always have. Not to mention the demand for auditable, repeatable workflows is still there and always going to be there and dedicated systems are needed to solve this in each problem domain.
Yeah, I guess this take is tempting for a technologist, but Gen Z is buying iPods and walking around in wired headphones because it's cool and nostalgic, not because of usability. Cycles of nostalgia are well understood to be getting smaller. The creative industry is creating new things less frequently and referring back sooner (the old 20 year cycle of fashion repeating itself is contracting). There is an element of disenchantment, of wanting to disconnect from the present, but that has always sort of been there as people reached for vintage cameras, record players, and old clothes in the niche cultural movements that have preceded the current Gen Z 2000's obsession that's happening.