IIRC training starts with the initial state and the end state, and the end state is always oriented the same way. It would be interesting to see what would happen if the end state was rotated randomly though I suspect it wouldn't work so well.
I have yes.. You need to erase a larger amount of the center, but it almost always results in a collapse wheras erasing around the center typically regrows.
There's hidden state in the model which presumably it uses to communicate position, ie there's the 3 colors but then a bunch of other channels that the model can use how it wants.
With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.
Iirc, the model is still pre-trained on modern text before being fine tuned on 1930's material, so it's possible it still has some knowledge of words that didn't exist back then. Edit: looks like they make some attempt to filter out documents from the pre-training but it's still possible some sneak in.
AI (as it currently exists) will never be "creative" in the sense that it can only imitate or interpolate between past works. At some point we'll recognize that AI generated works are boring and predictable. AI will probably never invent a new musical genre or artform since it can only reproduce or recombine works from the past. I wonder what happens when the internet is so full of AI generated slop that that the only things worth "training" on were made in the time before AI generation became a thing. Will AI generations be full of dated references to a time gone by?
People may say it increases creativity but I see it more as lowering the bar to produce things. The same could be said for a lot of inventions like photography made producing images easier and I'm sure a lot of portrait painters lost their jobs to photographers. I think the danger is that we may see a very rapid erosion of jobs in the creative space that won't be easy to transition into new fields which I feel will have a detrimental effect on our society.
I'm struggling to understand why an LLM even needs to be involved in this at all. Can't you write a script that takes the last 10 slack messages and checks the github status for any URLs and adds an emoji? It could be a script or slack bot and it would work far more reliably and cost nothing in LLM calls. IMO it seems far more efficient to have an LLM write a repeatable workflow once than calling an LLM every time.
> Valve actually tried with the first Half-Life game in a decade, and even that didn't work.
Half Life Alyx is still considered to be one of the best VR games ever made and one that is still consistently recommended to new users even years after release. IMO people buy hardware because of the exclusive content. If a standard game console came out and it only had one AAA game on it, I probably wouldn't bother buying it. But if there were 3-4 games that looked really interesting it starts to look more worth the investment. Playing VR games takes a lot of committment (time / physical space / $$$) so the payoff has to be worth it or you'll lose people. With the huge amount of money spent on R&D for new hardware I think it's a valid argument to say that maybe funding content would have been a better investment in terms of ensuring platform growth.
Also, side note but not every game requires free motion. Plenty of hits had no movement or teleport etc. A lot of these were completely new (sub-)genres that didn't exist or hit the same as they would in a traditional pancake game. Plus lots of kids seem unaffected by free movement (maybe as high as 50% of users by my rough estimate).
Another way to look at it is parallel processing vs sequential processing.. our brains can make a judgement call about a thousand subtle variables and data points that we can't exactly put our fingers on unless we really dig into it, which we usually label as 'feelings', using the parallel part of our brain. The sequential (logical) part can only consider a limited number of variables at a time. I don't think either mode of thinking is inherently worse (we need both), but in our society the feelings part has traditionally been discounted as being 'illogical' by academics.. I think AI has shown us that parallel processing is actually incredibly important to thinking.
But back to the original post, I think 'having good taste' and knowing when something feels like the right solution is one of those hard to define qualities that can make the difference between average and great products (and has far reaching effects in any business).
IMO this is the big problem with the internet as it exists today - there is no incentive for producing accurate, unbiased information and non-sensationalist opinions. My greatest hope for the future is that somehow we can incentivize people to produce "good" information for AI based assistants and move away from the rage/shock based advertising model that most of the internet currently uses. Personally I would rather pay a few cents for a query that produces valuable results and doesn't put me in a bad mood than pay with my time and attention like we do today. AI systems will absolutely need to be able to identify the training sources with every result (even if it is coming from several sources) and those sources should be compensated. IMO that's the only fair model for both image and text generation that is based on authors and artists work.