There is still a difference though. If some company decides to raise prices on OR, you can just switch to any other provider of the same model since there is no moat.
As always: it depends on your needs. Here's a very basic heuristics rundown:
- More RAM: bigger models, more intelligence.
- More FLOPs: higher pre-fill (reading large files and long prompts before answering, the so-called "time to first token").
- More RAM bandwidth: higher token generation (speed of output).
So basically Macs (high RAM, okay bandwidth, lowish FLOPs) can run pretty intelligent models at an okay output speed but will take a long time to reply if you give them a lot of context (like code bases). Consumer GPUs have great speed and pre-fill time, but low RAM, so you need multiple if you want to run large intelligent models. Big boy GPUs like the RTX 6000 have everything (which is why they are so expensive).
There are some more nuances like the difference of Metal vs. CUDA, caching, parallelization etc., but the things above should hold true generally.
For the same reason that a human who is fluent in five languages can probably express themselves better in either one compared to human that only speaks one, while also having a more nuanced understanding of general grammar.
From what I know, learning on a more diverse set makes a model better overall.
I'm trying to get back to verifying some of my old fun ideas. I want to finally build my 3D QR cube (https://deriese.net/qrcubes.html?s=hn) by sending a design to a laser shop, and I also want to find someone with a few termocouples to verify my results to the coffee cup cooling problem (https://deriese.net/coffee.html?s=hn). If anyone wants to help, feel free to send me a message.
Ever since I read about this, I have been thinking about the next logical step: train a NN to route the internal loops dynamically after each layer. Instead of just choosing a given set of layers that are repeated, let the new classifier decide whether it wants to loop, where it wants to loop, whether to loop multiple times, to loop a big part, or to just jump to the final layers straight away. Each token could loop more or less based on its relevance.
It has some similarities of a MoE architecture, but instead of choosing experts, it chooses layer routes. Training this NN classifier together with the LLM could condense the required amount of layers for a given intelligence down drastically if it works. If anyone wants to work on this, feel free to send me a message.
So they use a valid and valuable company to hide a giant dumpster fire company. To add to that, their best argument is "AI in space", which has some real "solar roadways" energy to it. I honestly don't know how any SpaceX shareholder could approve this.
> If you post for ad revenue, I truly feel sorry for you.
I think this is a bit dismissive towards people who create content because they enjoy doing it but also could not do it to this level without the revenue, like many indie Youtubers.
There is this popular coffee cup cooling problem: assume you want to keep your morning cup of coffee hot for as long as possible. When do you add your milk? Immediately or later?
I am overengineering a simulation-based solution to this because I think there are scenarios based on cup shapes and environmental temperatures that allow either answer to be true. This will end up as a blog post I guess.
Haha cheers. Despite this being a fun project, I'd still like to be the first to do it, so forgive me for not telling yet. I will try another round of outreach tomorrow and post it on HN as soon as I get permission.
What a coincidence to see this on the HN front page. I want to use these catalogs for a project of mine, but I first wanted to speak to one of the people of the IKEA museum or IKEA itself to inquire about permissions (outside of the ones on the website). I have been trying to get a hold of them for weeks now, but with no luck so far. If anyone here knows someone at those places, please let me know.
Small things like this or the fact that AI studio still has issues with simple scrolling confuse me. How does such a brilliant tool still lack such basic things?