Long distance sound localization with the Raspberry Pi(medium.com)
medium.com
Long distance sound localization with the Raspberry Pi
https://medium.com/@kim_94237/tdoa-sound-localization-with-the-raspberry-pi-3e777469c4fa
42 comments
Fascinating. This reminds me of the WWII novel All The Light We Cannot See where the young German soldier is tasked with locating illegal radio transmissions by placing receivers at different places and using trigonometry calculations to estimate the distance and direction of the source. Of course, the culprits would invariably be killed when found, leaving the boy with a deep sense of guilt.
That was indeed a great movie.
I did similar long baseline acoustic localization for my dissertation a few years ago. We had about 15 microphones deployed in a wetland for bioacoustic monitoring.
https://resenv.media.mit.edu/pubs/theses/Resynthesizing_Volu...
Rather than manually finding the timestamps in the individual audio files you can do pretty well with using cross-correlation to find the relative delays between the mic signals. Any particular delay between a pair of microphones corresponds to a hyperbola through the mic array. If you visualize the hyperbola from all the pairs of microphones, the correct location pops up as the intersection of them. The cross-correlation peak can get pretty smeared out though, because each microphone is hearing a different version of the source signal.
https://resenv.media.mit.edu/pubs/theses/Resynthesizing_Volu...
Rather than manually finding the timestamps in the individual audio files you can do pretty well with using cross-correlation to find the relative delays between the mic signals. Any particular delay between a pair of microphones corresponds to a hyperbola through the mic array. If you visualize the hyperbola from all the pairs of microphones, the correct location pops up as the intersection of them. The cross-correlation peak can get pretty smeared out though, because each microphone is hearing a different version of the source signal.
Fun! 1-2m accuracy is surprisingly good (at least for me who has zero experience with such things).
You could probably use something like this in a war zone to locate from where somebody is trying to shoot you. A quick google later yields e.g. https://www.rheinmetall.com/en/products/c4i/reconnaissance-a... or https://transvaro.com/wp-content/uploads/akustik-eng.pdf (which lists accuracy of... 1-2m).
You could probably use something like this in a war zone to locate from where somebody is trying to shoot you. A quick google later yields e.g. https://www.rheinmetall.com/en/products/c4i/reconnaissance-a... or https://transvaro.com/wp-content/uploads/akustik-eng.pdf (which lists accuracy of... 1-2m).
Interesting. That looks like a system that uses phase difference.
Last weekend I localized an explosion to within 20m when two of the microphones were 3 km apart. I went to the location and they were firing fireworks as part of some religious ceremony.
Last weekend I localized an explosion to within 20m when two of the microphones were 3 km apart. I went to the location and they were firing fireworks as part of some religious ceremony.
Feed the RPI data to https://www.civtak.org/ and send some to Ukraine...
Saving lives was never so cheap.
The landing page doesn’t describe what TAK is and doesn’t even link to a page which does, even in a hidden menu.
To everyone: if you make something and put it on the web, please spend 10 minutes to write a single paragraph of introduction for people who just learned that your thing exists.
To everyone: if you make something and put it on the web, please spend 10 minutes to write a single paragraph of introduction for people who just learned that your thing exists.
The "about" box doesn't show on mobile. On desktop it says this:
The Android Team Awareness Kit (ATAK), for civilian use, or Android Tactical Assault Kit (also ATAK) for military use - is a suite of software that provides geospatial information and allows user collaboration over geography.
ATAK was originally developed by the Air Force Research Laboratory (AFRL) and is now maintained by the TAK Product Center (TPC).
There's more information on the About page (accessible via the navigation bar on desktop, or the menu on mobile): https://www.civtak.org/atak-about/
The Android Team Awareness Kit (ATAK), for civilian use, or Android Tactical Assault Kit (also ATAK) for military use - is a suite of software that provides geospatial information and allows user collaboration over geography.
ATAK was originally developed by the Air Force Research Laboratory (AFRL) and is now maintained by the TAK Product Center (TPC).
There's more information on the About page (accessible via the navigation bar on desktop, or the menu on mobile): https://www.civtak.org/atak-about/
Sad but true. The world is full of so many places like this :-(
That’s a nice read, thank you for that. If he sets up three or more recorders then next time he can find out within minutes of the first bang to within meters.
But his article presents a fun challenge. He doesn’t say where his receiver locations are but he does give a triangle dimensions. I wonder if it’s possible to map this onto the street names and then work out which farm it was. It seems likely that there is only one way to map that triangle so each vertex is on the street it is reported to be on.
Then you could invent a time, any time and add the number of seconds that he mentions to get the two other times. Now you have the times and the co-ordinates, within the constraints from the time inaccuracies you could run the docker version of my localization program (https://github.com/hcfman/sbts-aru) on this input and it would output the map link to the offending farm. With a largish degree of error.
Note. From the diagram, the sound source doesn’t fall within the triangle of receivers. This can still work, but the convergence drops off after a time so you can only practically localize this if the distance is not too far outside. But if you make battery powered receivers, you choose a better spot and move one if the recorders so it does fall within the triangle.
Also, just one second of error is a lot!
But his article presents a fun challenge. He doesn’t say where his receiver locations are but he does give a triangle dimensions. I wonder if it’s possible to map this onto the street names and then work out which farm it was. It seems likely that there is only one way to map that triangle so each vertex is on the street it is reported to be on.
Then you could invent a time, any time and add the number of seconds that he mentions to get the two other times. Now you have the times and the co-ordinates, within the constraints from the time inaccuracies you could run the docker version of my localization program (https://github.com/hcfman/sbts-aru) on this input and it would output the map link to the offending farm. With a largish degree of error.
Note. From the diagram, the sound source doesn’t fall within the triangle of receivers. This can still work, but the convergence drops off after a time so you can only practically localize this if the distance is not too far outside. But if you make battery powered receivers, you choose a better spot and move one if the recorders so it does fall within the triangle.
Also, just one second of error is a lot!
Note also, although my project details the construction of a Raspberry Pi recorder for high accuracy localization. If you are localizing over long distances and you are happy a second or two inaccuracy from the arrival times, then indeed you can run the docker version of the program listed on GitHub and perform localizations based on mobile phone determinations of time of arrival. You just have to match the correct input format as detailed in the article. Specifically like the following:
$ docker run -i localize_event 20
Enter GPS coordinates and timestamps. Press enter twice to finish.
51.014131667,5.813736667 2023-09-17_15-49-48.522301
51.015373333,5.811656667 2023-09-17_15-49-48.810330
51.016368332,5.814084879 2023-09-17_15-49-48.710824
51.015238333,5.815941667 2023-09-17_15-49-48.543376
Location: 51.014903961182846,5.814442712946816
Web links:
OpenStreetMap:
https://www.openstreetmap.org/?mlat=51.014903961182846&mlon=...
Google Maps:
https://www.google.com/maps?q=51.014903961182846,5.814442712...
Note also, you can simulate this easily by choosing locations on a map and calculating the expect time of arrivals. Then you can introduce one more seconds of error and see what difference it makes.
It’s lovely to work things out for yourself but he could have saved himself some weeks of sleep by googling sound localization if that was his primary interest :-)
It is weird that you can leg off air cannons all night for three weeks and not have the authorities on your head though.
$ docker run -i localize_event 20
Enter GPS coordinates and timestamps. Press enter twice to finish.
51.014131667,5.813736667 2023-09-17_15-49-48.522301
51.015373333,5.811656667 2023-09-17_15-49-48.810330
51.016368332,5.814084879 2023-09-17_15-49-48.710824
51.015238333,5.815941667 2023-09-17_15-49-48.543376
Location: 51.014903961182846,5.814442712946816
Web links:
OpenStreetMap:
https://www.openstreetmap.org/?mlat=51.014903961182846&mlon=...
Google Maps:
https://www.google.com/maps?q=51.014903961182846,5.814442712...
Note also, you can simulate this easily by choosing locations on a map and calculating the expect time of arrivals. Then you can introduce one more seconds of error and see what difference it makes.
It’s lovely to work things out for yourself but he could have saved himself some weeks of sleep by googling sound localization if that was his primary interest :-)
It is weird that you can leg off air cannons all night for three weeks and not have the authorities on your head though.
> GPS coordinates
*Coordinates, or lat/long
Other navigation systems are available, including other GNSS!
> 51.014903961182846,5.814442712946816
I'm not sure localisation to the level of the atomic lattice is entirely relevant in this use case.
*Coordinates, or lat/long
Other navigation systems are available, including other GNSS!
> 51.014903961182846,5.814442712946816
I'm not sure localisation to the level of the atomic lattice is entirely relevant in this use case.
Actually, chatgpt says that for 1mm resolution you need 9 decimal places anyway :-) Note a rtk gnss can resolve your location to 1cm. I don't know how much jitter there is in the USB buffering, but for this purpose, lets assume 0. Then if I want to do experiments with short distance localization then it would be good to be one decimal less than what I want to use in practice, so there is the 1 mm. So I could format this to a max of 9 decimal places. I see the above is still a little more. Something for the next time I make a push. Thanks for your concern :-)
I guess I could ask chatgpt how many decimals gets you to cm accuracy and fix the decimal places to that. Haven’t found it a priority yet. But if it makes details guys happy. You are not the first to say this.
Seems to have some similarity to "Finding the Air Cannon": https://news.ycombinator.com/item?id=39188040
In reference to that "Finding the Air Cannon" article. The author indicated the following facts:
* His listening posts were part of a triangle with sides of length 3.8km, 3.0km and 6.3km
* The vertices were on the roads in Corvallis, Bellfountain Rd, Brooklane Dr and Rivergreen Ave
* The relative times of arrival are 0, 4, 6
* The temperature was 35F or 1 degree celsius
The only way I can make a triangle that fits on those three roads yields a set of the following likely co-ordinates from where his listening posts are:
44.52421,-123.33492 44.53796,-123.29172 44.52975,-123.25571
Adding the relative times to the co-ordinates in a form whereby my localization program can work on it and putting it through my localization program I get the following result:
$ localize_event.sh 1
44.52421,-123.33492 2024-05-27_12-00-00.0
44.53796,-123.29172 2024-05-27_12-00-04.0
44.52975,-123.25571 2024-05-27_12-00-06.0
Enter GPS coordinates and timestamps. Press enter twice to finish. Location: 44.488866130770134,-123.31219694386117
Web links:
OpenStreetMap:
https://www.openstreetmap.org/?mlat=44.488866130770134&mlon=...
Google Maps:
https://www.google.com/maps?q=44.488866130770134,-123.312196...
Which results in a location south and west of the airport as described in the article :)
Knowing how localization works where the sound source is outside of the polygons it could be a bit further South as well.
I'd be very curious to know how close I have been in working this out.
* His listening posts were part of a triangle with sides of length 3.8km, 3.0km and 6.3km
* The vertices were on the roads in Corvallis, Bellfountain Rd, Brooklane Dr and Rivergreen Ave
* The relative times of arrival are 0, 4, 6
* The temperature was 35F or 1 degree celsius
The only way I can make a triangle that fits on those three roads yields a set of the following likely co-ordinates from where his listening posts are:
44.52421,-123.33492 44.53796,-123.29172 44.52975,-123.25571
Adding the relative times to the co-ordinates in a form whereby my localization program can work on it and putting it through my localization program I get the following result:
$ localize_event.sh 1
44.52421,-123.33492 2024-05-27_12-00-00.0
44.53796,-123.29172 2024-05-27_12-00-04.0
44.52975,-123.25571 2024-05-27_12-00-06.0
Enter GPS coordinates and timestamps. Press enter twice to finish. Location: 44.488866130770134,-123.31219694386117
Web links:
OpenStreetMap:
https://www.openstreetmap.org/?mlat=44.488866130770134&mlon=...
Google Maps:
https://www.google.com/maps?q=44.488866130770134,-123.312196...
Which results in a location south and west of the airport as described in the article :)
Knowing how localization works where the sound source is outside of the polygons it could be a bit further South as well.
I'd be very curious to know how close I have been in working this out.
Sorry. My reply to this are in the comments above. Thanks for posting, this is a nice read.
And also points out that the longer the distances involved the less critical time error is making mobile phone time of arrival viable.
And also points out that the longer the distances involved the less critical time error is making mobile phone time of arrival viable.
I did this for my senior design project a few years ago.
We had 4 raspberry pis with mic arrays. They were time synced over a wifi mesh network (indoor so no GPS). Each had a detector running, and would send detection segments to a base station to run generalized cross correlation.
It worked pretty well, but we were doing it in room scale 20' environments. Our biggest issue was that we could not disambiguate echos, and had to hack in a hold off period.
We did not end up having time but I wish we could have gotten beamforming working on each pis microphone array, then we could have combined angle of arrival with tdoa, and potentially better handled the echo.
We had 4 raspberry pis with mic arrays. They were time synced over a wifi mesh network (indoor so no GPS). Each had a detector running, and would send detection segments to a base station to run generalized cross correlation.
It worked pretty well, but we were doing it in room scale 20' environments. Our biggest issue was that we could not disambiguate echos, and had to hack in a hold off period.
We did not end up having time but I wish we could have gotten beamforming working on each pis microphone array, then we could have combined angle of arrival with tdoa, and potentially better handled the echo.
Indoor and thus smaller distances means time accuracy becomes much more important. In my software chain there is a very small bit of uncortekated latency due to unknown buffering size of the USB driver adding to time inaccuracy potentially.
Nice! I want to try and get something like that going as well.
I've had an unrealized pet project on my personal to-do list for quite some time in a similar area - to estimate the speed of neighborhood traffic based on the doppler effect of their traffic noise while passing my house. I can't use radar/IR devices since there's I'm not directly adjacent to the road and the intervening land doesn't belong to me, and the substantial hedges make any line-of-sight sensing impractical.
Guessing the problem with the air horn is in part due to how directional it is?
I understand wanting to avoid firing a gun, an explosive.
But my mind recalls putting dry ice in a 2-liter soda bottle containing already hot water. Lid goes on quickly and tightly....
Legal? Not sure. Definitely gives you shotgun-level decibels though. Not at all directional.
I understand wanting to avoid firing a gun, an explosive.
But my mind recalls putting dry ice in a 2-liter soda bottle containing already hot water. Lid goes on quickly and tightly....
Legal? Not sure. Definitely gives you shotgun-level decibels though. Not at all directional.
What's wrong with a good ol' fashioned cherry bomb?
I tried this too years ago, that was interesting!
How do you time-synchronize the 4 RPi? When you power them off, you lose the sync. So it is difficult to do it correctly.
Also it may be possible that the different RPi have different clock drift over time? i.e. 1.0000 hour on one of them = 1 hour + 20 milliseconds on another one
How do you time-synchronize the 4 RPi? When you power them off, you lose the sync. So it is difficult to do it correctly.
Also it may be possible that the different RPi have different clock drift over time? i.e. 1.0000 hour on one of them = 1 hour + 20 milliseconds on another one
My project, sbts-aru,
https://github.com/hcfman/sbts-aru
uses a GPS to synchronise the time with and then even when running completely disconnected from any network the clocks will be accurate to real time with less than 1 microsecond of error. Typically the system time hovers around 100 ns or less from the real time. And I’ve tested this by triggering interrupts on gpio’s on two devices with the same switch and printing the time.
Syncing to sub microsecond accuracy is usually achieved in under a minute.
https://github.com/hcfman/sbts-aru
uses a GPS to synchronise the time with and then even when running completely disconnected from any network the clocks will be accurate to real time with less than 1 microsecond of error. Typically the system time hovers around 100 ns or less from the real time. And I’ve tested this by triggering interrupts on gpio’s on two devices with the same switch and printing the time.
Syncing to sub microsecond accuracy is usually achieved in under a minute.
[deleted]
Oh. I’d like to point out that the localization algorithm itself that I use comes from the lovely opensoundscape project from Tessa Rhinehart and friends from the University of Pittsburg. A big thanks to them for their lovely project.
http://opensoundscape.org
http://opensoundscape.org
That's useful. It wold probably be more useful as a phone app, so that a few people with phones could locate a sound. This is especially useful for low-pitched noise, where human ears are not far enough apart to get any phase difference.
Over really long distances the time inaccuracy makes less difference so indeed a phone can do interesting things.
I’m a person that likes extremes so the extremely accurate clock time I could get with a GPS synched Oi really appeals to me. Although it’s not really necessary I want to get an rtk GNSS next year to experiment with. Together with ultrasonic microphones I’m pretty sure it could do a great job localizing bats.
I’m a person that likes extremes so the extremely accurate clock time I could get with a GPS synched Oi really appeals to me. Although it’s not really necessary I want to get an rtk GNSS next year to experiment with. Together with ultrasonic microphones I’m pretty sure it could do a great job localizing bats.
Some ham radio friends pointed out that the easy way to synchronize is to relay the sound over a cheap analog radio.
If I remember correctly, when you have 2 RPi located at F and F', the solutions M are on an hyperbola : MF = MF' + delta_distance where delta_distance is simply computed with delta_t in the two signals.
I just use the opensoundscape library that can use as many sound sources as you like in the equation.
What about having a better estimation of the speed of sound according to temperature and humidity? Do you think it can improve the accuracy ?
Temperature is already a parameter. Apparently humidity doesn’t have quite such an effect.
In the example further down the page where I attempt to work out where the air cannon was fired from, the “1” is for 1 degree Celsius.
In the example further down the page where I attempt to work out where the air cannon was fired from, the “1” is for 1 degree Celsius.
[deleted]
So this is how city wide gunshot detection works.. nice!!