Back when GPT-2 was released, I tried figuring out how to fine tune it. I found a google notebooks template, scraped a bunch of data from r/ChangeMyMind and asked it to change my mind on different topics.
I was dumbfounded that it actually tried doing that. Obviously GPT-2 wasn’t great at it, but the writing was on the wall quite literally.
Unfortunately, I was too broke to invest in stocks, but I did pivot my career quite a bit.
For everyone in the EU: Copying and pasting sensitive data (like customer data) into AI tools is a violation of the GDPR, and potentially the AI act, which will be enforced soon.
I keep wondering when this discussion comes up… If I take an apple and paint it like an orange, it’s clearly not an orange. But how much would I have to change the apple for people to accept that it’s an orange?
This discussion keeps coming up in all aspects of society, like (artificial) diamonds and other, more polarizing topics.
It’s weird and it’s a weird discussion to have, since everyone seems to choose their own thresholds arbitrarily.
For YouTube, this already exists and I‘m using it. The extension is caller DeArrow and aims to reduce sensationalism via crowdsourcing, though I wouldn’t be surprised if top contributors are bots using LLMs.
AI in it‘s current phase, definitely. However, we‘ve been seeing the transformer architecture plateauing in the last couple of years. There are still improvements, but open source models are catching up.
I feel like at this point it’s an inevitability that given enough time, capable models will be cheap enough for everyone.
Having a tool that instantly searches through the first 50 pages of google and comes up with a reasonable solution is just speeding up what I would have done manually anyways.
Would I have learned more about (and around) the system I‘m building? Absolutely. I just prefer making my system work over anything else, so I don’t mind losing that.
I think this is actually the correct way to move forward.
We should be able to verify facts about people on the internet without compromising personal data. Giving platforms the ability to select specific demographics will, in my view, make the web a better place. It doesn’t just let us age restrict certain platforms, but can also make them more authentic. I think it’s really important to be able to know some things to be true about users, simply to avoid foreign election interference via trolling, preventing scams and so much more.
With this, enforcement would also be increasingly easy: Platforms just have to prove that they’re using this method, e.g. via audit.
The fact that a language model can „reason“ (in the LLM-slang meaning of the term) about 3D space is an interesting property.
If you give a text description of a scene and ask a robot to perform a peg in hole task, modern models are able to solve them fairly easily based on movement primitives. I implemented this on a UR robot arm back in 2023
The next logical step is, instead of having the model output text (code representing movement primitives), outputting tokens in action space. This is what models like pi0 are doing.
> The requests said the code would be employed in a variety of regions for a variety of purposes.
This is irrelevant if the only changing variable is the country. From a ML-perspective adding any unrelated country name shouldn’t matter at all.
Of course there is a chance they observed an inherent artifact, but that should be easily verified if you try this same exact experiment on other models.
I was dumbfounded that it actually tried doing that. Obviously GPT-2 wasn’t great at it, but the writing was on the wall quite literally.
Unfortunately, I was too broke to invest in stocks, but I did pivot my career quite a bit.