This is one of the most overlooked problems in generative AI. It seems so trivial, but in fact, it is quite difficult. The difficulty arises because of the non-linearity that is expected in any natural motion.
In fact, the author has highlighted all the possible difficulties of this problem in a much better manner.
I started with some simple implementation by trying to move segments around the image using some segmentation mask + ROI. That strategy didn't work out, probably because of some mathematical bug or data insufficiency data. I suspect the later.
The whole idea was to draw a segmentation mask on the target image, then draw lines that represent motion and give options to insert keyframes for the lines.
Imagine you are drawing a curve from A to A. You divide the curve into A, A_1, A_2... B.
Now, given the input of segmentation mask, motion curve, and whole image, we train some model to only move the ROI according to the motion curve and keyframe.
The problem with this approach is in sampling the keyframe and matching consistencies --making sure RoI represents same object-- across subsequent keyframes.
If we are able to solve some form of consistency, this method might be able to give enough constraints to generate viable results.
I am a recent graduate specialized in computer graphics (character animation) and computer vision (inverse rendering). I am currently in Germany. I would be happy to move as long as I get a proper visa sponsorship.
I am comfortable with anycode base of any kind. My personal website highlights the kind of projects I did in the past and my university.
Regarding LLMs, I am one of the co-authors on indirect prompt injection paper [1]. I also happen to have intuition of the maths behind diffusion. Meaning, I am also open to any GenAI related roles.
This is what a truly revolutionary idea looks like. There are so many details in the paper. Also, we know that transformers can scale. Pretty sure this idea will be used by a lot of companies to train the general 3D asset creation pipeline. This is just too great.
"We first learn a vocabulary of latent quantized embeddings, using graph convolutions, which inform these embeddings of the local mesh geometry and topology. These embeddings are sequenced and decoded into triangles by a decoder, ensuring that they can effectively reconstruct the mesh."
This idea is simply beautiful and so obvious in hindsight.
"To define the tokens to generate, we consider a practical approach to represent a mesh M for autoregressive generation: a sequence of triangles."
This field moves so fast. Blink an eye and there is another new paper. This is really cool and the learning speed of us humans is insane! Really excited on using it for downstream tasks! I wonder how easy it is to integrate animatediff with this model?
Also, can someone benchmark it on m3 devices? It would be cool to see if it is worth getting on to run these diffusion inferences and development. If m3 pro can allow finetuning it would be amazing to use it on downstream tasks!
Been using a splitkb for a year now. The only regret, I didn't buy it sooner.
Seriously nothing compares to the typing experience of split keyboard. Open shoulders are great, relaxed arms and wrists is a blessing, then there are layers. Those are sooooo awesome when used in correct way!
It has really been amazing experience. I can't imagine going to normal keyboards.
I switched to Colemak-DH on a split keyboard. My hands thank me too much. They are always rested!
And the shoulders, the best part of split keyboard is the open shoulders! It feels good to work on the split keyboard. One of the best investment!
I am glad you created a layout for yourself!
Considering speed, I am all recovered and even surpassed by QWERTY speed. I am average (~60wpm) when it comes to typing speed so it wasn't hard to catch up to it.
My switching experience was also similar to yours. Get to ~35 wpm and start using. It took me 14 hrs to reach there (1hr deliberate practice everyday for 2 weeks.)
Always wanted to create some fast paced game! Finally got to start it!
I will definitely evolve it over time.
All done in my spare time with help of agentic coding!