> i would fire any friend who attempted to use an AI as a proxy for talking with me.
Mitra does not replace calls that you already make to your friends where you want to talk to them. In fact, I encourage you to talk to your friends more directly.
Mitra is useful for 3 main reasons when it comes to friends:
1.) When you or the other person are not comfortable saying certain things or having certain, tough conversations.
2.) When you or the other person may not have the time to have a full fledged conversation about something.
3.) When you want to call your friends for pure entertainment purposes by having Mitra say funny things.
So if a friend used this on you it does not necessarily mean that they view you in a negative light.
Mitra, just like Snapchat or Twitter DMs or Instagram DMs, is simply another way you can communicate with your friends that is more engaging and fun than the others.
- "The Declaratory Ruling limits the use of AI-generated voices in robocalls, but it does not impose a total ban, as some headlines have suggested. Instead, it clarifies that callers who choose to employ AI-generated voices must comply with existing TCPA regulations."
- "The TCPA does not ban robocalls to residential phone lines (i.e., home landlines), even without consent from the called party as long as (1) the calls are not made for a commercial purpose..."
Virtual avatar generation models can act as world navigators in complex environments. This is a potential new direction for general purpose robotics as well.
We introduce a novel video model simulating human movement in rock climbing environments using a virtual avatar. Our diffusion transformer predicts the sample instead of noise in each diffusion step and ingests entire videos to output complete motion sequences. By leveraging a large proprietary dataset, NAV-22M, and substantial computational resources, we showcase a proof of concept for a system to train general-purpose virtual avatars for complex tasks in robotics, sports, and healthcare.
Of the full distribution of possible video qualities one can take on a modern phone camera, the vast majority of video qualities will be fine for the AI to understand fine details. Obviously, if you somehow or for some reason, take a video with really bad quality, it will not give you what you want.
Same explanation goes for the walls. If you take a video of just a really dark wall with really bad holds, it is probably won't give you what you want either.
This is an end-end system that just takes in video frames. Camera parameters are one of the things that is predicted. It gives promising results for a wide variety of environments (cliffs, diff types of bouldering walls, diff outdoor walls, etc.), though not always accurate. Path planning is also part of the end-end system. Will share more details in the paper.
Thanks! Still unsure if we will commercially pursue this specific idea or not. This was moreso meant to be a public progress marker from our broader work pursuing virtual avatar simulations. If enough interest is there, we will think about it. But, for now, this (will be) a paper and some open sourced code.
Ha! That was unintended.
"You can't connect the dots looking forward; you can only connect them looking backwards." - someone somewhere (likely Steve Jobs)
Yeah it performs best with indoor bouldering videos with spread out holds and great lighting. I will post examples of it climbing natural walls in the paper. Will send soon.
I will be open sourcing the codebase and model. Due to compute and training dataset size constraints, the model is not exactly "production ready". I have (and am) cherry picking rare examples of it where it is decent. You can play around with it when I open source it but truly good results require multiple millions of dollars (but it really is a linear scaling of compute and dataset).
Will attach more example outputs and make a detailed document about how the model was built and the research behind it. If this is interesting to you, feel free to sign up to the waitlist on www.climbing.ai (and make sure to sign up for our Discord!)
I originally planned on open sourcing the model, data, weights, and code. Only a few people (me and a couple friends) have access to the hosted model on the web app.
If enough people are on the waitlist, I will consider releasing access. This is very expensive to run so was only considering open sourcing it.
Note: the model works well sometimes, but most of the times it does not. This is an early research preview. Please tamper any expectations. A really good general model requires millions of videos and much more training time so it is really prohibitively expensive. As mentioned before, if someone has the compute, all they have to do is scale the existing dataset and training pipeline (which I will publish open-source in the coming weeks).
- it creates a 3D model of you based on the first second of your body in the video. How the video is recorded does affect how the 3D avatar looks.
- don't have that functionality but its really easy to implement.
- ideally, it works for anything. I will attach some outputs of it climbing outdoor routes. The model doesn't know what a wall means. It just has seen enough data of people climbing that it can somehow correlate certain features in videos to certain human movements.
Sources:
1.) https://www.bluedothq.com/blog/call-recording-laws#:~:text=S....
2.) https://www.avoma.com/blog/call-recording-laws
Here, the audio cue can be labeled as the AI voice itself as there is no regulation surrounding what exactly the audio cue should be.
So the recording is the default but I encourage you to always inform people and ask if that is okay before you ask Mitra to call them.
Mitra never makes calls to someone unless you specifically ask it to.