> stylelint beeps you can't just pass hex colors directly, that color is not in our design system, you need to write a design doc for custom color tokens and get approval from the frontend platform team, open a PR in their repo, make sure you have storybook tests covering all the color <cross product> button variants, ask in their slack channel for approve, ask their manager, wait. someone from their team leaves a comment: "we should make this an approved custom colors enum not a string, so if you want to add custom colors you also have to update this enum", fix the PR, staff engineer from sister team drive by request for changes: "we are currently implementing custom themes and changing colors will be done through the ColorSwatch service", ask for timelines, "maybe next week behind a feature flag", give up, close the PR, open a new PR with "stylelint-disable", force-merge it.
If you play with these models long enough, you realize there is more to them than just "model X is smarter than model Y" or "model Y is cheaper than model Z". They are different tools and the prompting technique is different. It is very much like playing an instrument.
With Claude, you sometimes want to under-specify or phrase things more indirectly to give a color to the implementation or elicit something creative. Also (you might raise an eyebrow at this) being nice to Claude will be rewarded and being mean to Claude will be punished. Claude tends to mirror your tone more aggressively and you don't want to get into negative loops with it.
With GPT, you have to be precise and reduce ambiguity. GPT will often try to resolve ambiguity in a min-max style "I'm going to do X, but make sure it is not quite Y". It will tend to be more paranoid and overengineer to catch all edge cases if you don't tell it precisely what the scope is.
With Qwen, you have to give it a shape and let it fill it in. Qwen likes XML, JSON and lists. Qwen likes to be shown a bunch of examples of previous work.
So this is the other side of banning American models for non-Americans? And how exactly do they plan on enforcing all of this? Great Firewall of America?
This is a complete joke. The malicious clowns behind this should be removed from power and prevented from ever holding any position of power in any form of governance system.
Technology is deflationary. I am holding in my hand a device that would have been a supercomputer 30 years ago. It costed me a couple of hundreds of dollars.
These models and the hardware they are running on will get even more efficient. We are nowhere near the physical limits of what we can achieve.
This mammalian hardware we are running on is merely a bootloader, larva state, don’t take it too seriously. It was good enough to run us and I am grateful I woke up running on it.
This looks really great, more thoughtful than any benchmark that I've seen until now!
I'm curious if you're only interested in scoring frontier models or you would accept submission from custom harnesses? I am working on multi-model harnesses and would love to test them against your benchmark. Do you plan on releasing the tasks publicly?
> Eventually you have to come back to the real world and learn to accept the harsh truths of the reality you find yourself inhabiting.
I don't know why I feel so much contempt for people like you. I kinda get it, you want to feel intellectually superior. You're the reasonable one, the adult in the room, etc. but seriously bro, this reality is so fucking harsh. What keeps you going? Why are you still here and going through with it?