Aaah interesting, does stuff like this generalise to furniture moving around and different lighting conditions and stuff? Also sounds like if the route gets blocked it just wont move
Wow okay there is a lot here, just so that I understand this correctly:
1. Make a replica of my home/ room in a game engine or a simulator
2. Generate trajectories with RL where the reward is hand specified by me
3. Automate trajectory rewards using some proximity flags
Some stupid questions:
1. How do I build a replica of my home? Is there an SFM algorithm I could use to do this just from camera images?
2. Would this still work even if things/ furniture move around the house?
3. This data collection strategy will have a distribution shift compared to real data so it might struggle with different lighting conditions and stuff?
Apart from just detecting obstacles, we wanted to build a robot which is intelligent enough to take in semantic cues like this is a doorway so I can go through it, or this is a kitchen I can clean it this way and so on
Damn thats very cool! Thanks for sharing! I guess we would only need to detect dust somehow which believe it or not is really hard, the camera isnt great quality but I guess this could work for slightly larger debris?
Yeah agreed 100%, might also need to factor in the cost of the charging dock but the overall thesis is still sound.
Do you know any cheap wifi MCU with a little ML accelerator that we can buy off the shelf? The only one we could think of was the Jetson Orin Nano and thats not cheap
Thanks! Our main goal was to build a vacuum which understands semantics inside the house so that it can "clean the kitchen" or "clean the bedroom" so we wanted to do machine learning and since we were doing machine learning we were like why not try to do something E2E instead of first doing SLAM, optical flow etc..