I don't think it rigs the models - I think that video is comprised of models generated by Stable Zero123 that were then rigged/animated/postprocessed in Blender.
When I had it block the Twitter mobile app, it also blocked twitter.com - I assume the same thing will happen if you block Facebook. The founders have mentioned they're adding support for blocking specific websites soon.
The nag/friction that happens every time I try to open the app when I'm under buddget on this has cut down my doomscrolling so much. Really a fan of this approach - Twitter usage is down probably 95%. General stoke on this is really high!
Just adding my two cents here - I bought a quest pro hoping to use it as a monitor replacement while traveling for a couple of months. I found it pretty much unworkable after trying to make it work across multiple different virtual desktop apps.
You can get something up and running - but it feels akin to staring at a 1080p screen with a weird delay. I felt way more productive just looking at my tiny laptop screen.
Your mileage may vary, obviously. I really liked the idea of working in a distraction free virtual environment, but I feel like the tech isn’t there yet unless you’re an enthusiast whose willing to put up with all of the strangeness.
From my understanding the model in the linked video only stylizes existing meshes from text.
There’s plenty of papers that have tried text -> 3D model generation using photogrammetric-esque methods similar to what the parent comment suggested - the two minute papers video on one example is here.
Outside of cherry-picked examples this style of model tends to suffer from what people are calling the “Janus Problem” - the easiest way for it to satisfy the loss is to simply make the object look like the input prompt from as many angles as possible. So if you enter “a rubber duck”, it tends to generate yellow blobs with multiple head-like appendages sprouting off from it.
Google’s paper that tried this approach using Imagen as the text->image generator had great results, but they might be cherry-picked. Someone replicated it with stable diffusion as the text->image backend - still major Janus problems.
It's very possible they are doing something with them, just not the sort of thing OpenAI is doing. Google could have a model just as capable as Whisper running in their Google Cloud or Assistant transcription service - but they might have upgraded from their previous model without any announcement or blog post.
Are you saying they should have more bombast in their product announcements?
Google has models that are at parity with or outperform OpenAIs models at most tasks, they just don’t bring them to market quickly or build their brand around them as strongly as OpenAI does.
The common refrain in 2015 was that we were < 5 years away from having self-driving cars operating at scale without the need for a safety driver in most major cities in the US.
At this point we have a public robotaxi service in one suburban area and two robotaxi services operating under restricted domains in SF that are open to the public behind a waitlist. The cars have problems with rain, fog, or snow, and expanding to a new city still takes a ton of time. Having joined the industry after the hype of 5 years ago, the reality now is nowhere near the vision companies we’re selling back then.
My point wasn’t that these systems will never work in some capacity - my point is that they’re taking a lot longer to roll out than people predicted, and that the internals of these systems aren’t end-to-end ML models - it’s ML-based perception feeding in to a lot of traditional robotics code, with ML models handling certain prediction tasks in certain spots.
I’m a little skeptical - at least of the view that we’re a few years away from all animation jobs being dead and ML models producing entire movies or TV episodes e2e from text prompts.
What’s likelier is that ML-based tooling becomes a key part of the animation workflow. Used to generate assets, animations, backdrops, characters, etc, which are then combined by an animator/editor. The examples he cites in the article all fit this mold - the anime character generator he cites in the article uses separate models to generate the character then rig it from facial data.
After working in the self-driving car industry, I’m really skeptical of any claim that rapid advances in one modality or task mean we’re “just 3 years away” from all related tasks being done via ML models. Alexnet et al completely revolutionized perception in self driving cars - between 2014-2017 it was really common to hear predictions that we’d have end to end models driving our cars perfectly in “less than 5 years”. That reality never arrived because ML just wasn’t capable of handling more complex tasks the way it could with object detection. Lots of articles similar to this one talking about what we were going to do with all of the out of work truckers and Uber drivers.
And yeah obviously generative art is a different beast than autonomous driving. And I’ve seen examples from WIP text2video models. But I just want to caution that tons of progress in image and text doesn’t necessarily mean we’re just a year or two away from all related tasks being conquered.
This is just an off the shelf img2depth model run on top of stable diffusion - I don’t think there’s a novel model or research behind this. People have been doing the same thing in colab for a while.
I think this is just created by running a depth prediction model on the output of stable diffusion and then inserting the relevant mesh into a 3D scene. The output of stable diffusion isn’t seamless by default, so those jumps will happen.
Haven’t read the paper yet, but based on the videos it likely generates the rotation of each bone in the skeleton, which can then be used to animate a humanoid skeleton. So the videos you see are those poses being applied to a model.
This is par for the course - there have been other instances where an 'anonymous' paper mentioned training on a cluster of TPUs that weren't publicly available yet - dead giveaway it was Google.
This is crazy good - most prior text-to-3d models produced weird amorphous blobs that would kind of look like the prompt from some angles, but had no actual spatial consistency.
Blown away by how quickly this stuff is advancing, even as someone who's relatively cynical about AI art.