I love this.
For years I had been thinking about creating a map where you can spot first and last sun in an area. Now that I’m actually tackling bigger and complexer projects I forgot all about that idea.
Great job!
There’s an app on sidequest (for the oculus quest) that can track your piano and finger moves. You can input a midi file and then it sees you playing it and shows your virtual fingers. It might be possible to do hand tracking like that with an iphone.
Have you tried concerting sound from a phone microphone to midi?
I would like to use this idea on a project of mine.