How to Do Real Time Object Detection on Drones(blog.nanonets.com)
blog.nanonets.com
How to Do Real Time Object Detection on Drones
https://blog.nanonets.com/real-time-object-detection-for-drones/
34 comments
Is there a title you would recommend I change it to, to make it more clear?
Real Time Object Detection on Drone Video Stream
Changed to "How To Do Real Time Object Detection On Drone Video Streams" in the blog. Can't change it on HN.
yes, that would be a more accurate title.
Wow. Combine this with the ability to drop some of the latest nano-sensors (atoms strung together to build dust-sized sensors and batteries and power at scale), and we should be able to monitor absolutely everything everywhere. Let's see anyone even try to object to my rules or procure a weapon!
And given some on-board weapons, we would never have to worry about civil unrest again. Only people grateful for the food and job they are allowed to be in our society. Utopia!
And given some on-board weapons, we would never have to worry about civil unrest again. Only people grateful for the food and job they are allowed to be in our society. Utopia!
Hey MrTonyD. I understand where your coming from. But honestly most people are using this for monitoring things like Solar Panels, Wind Turbines, detecting seals and crop prediction. We can't have a sustainable future without a lot of these technologies in place.
https://nanonets.com/drone/wind-turbines/
https://nanonets.com/drone/solar-panels/
All new technology will have some deviant use case associated with it. The internet is one of the worst enablers of child pornography, human trafficking and everything disgusting with the world. However it just means you try and control the use of technology not throw the baby out with the bathwater.
All new technology will have some deviant use case associated with it. The internet is one of the worst enablers of child pornography, human trafficking and everything disgusting with the world. However it just means you try and control the use of technology not throw the baby out with the bathwater.
Well, I'm 60 now and I've worked in Silicon Valley for over 40 years. So I guess I hear about all the other uses of such technology. I've worked with NSA, CIA, DIA, all the branches of military, and a bunch of people who don't officially exist. So, while there are definitely things I can't say, I will say that the "dust sensors" have been in use for many years, and I'm not describing anything theoretical. We need to think about these things in order to address them. Simply saying that there also exist good uses isn't enough.
Given your expertise over the many years how would you recommend we control the use of such technology?
Maybe my question is based on the assumption that you agree: stopping development of the technology in the first place is not the solution. Hopefully this assumption is correct.
Maybe my question is based on the assumption that you agree: stopping development of the technology in the first place is not the solution. Hopefully this assumption is correct.
There is no way it can be stopped. Most of the development is secret and much further along than people believe. I think we need to regain community control over those who possess this technology - it will be deployed whether we like it or not. And, even more frightening, whoever deploys it first wins, and it will be so effective that it can no longer be defeated. This issue is more serious than global warming, jobs, healthcare, or World War 3 - it is about having any freedoms beyond those provided by whoever deploys these monitoring devices.
Infantry weapons powered by the kind of computer vision and facial recognition AI easily demonstrable today would be incredibly frightening, no matter how they were used. Of course there's no way of stopping this in an adversarial world.
Surely it would only be a matter of choosing to increase the efforts for a year or so in order to take these weapons from proof-of-concept to highly effective. E.g. autonomous flying gun platforms that can be deployed en masse and let loose with small or no human oversight (e.g. "hold the button to kill the person the machine has identified as a valid target", if even that level of control), or the kind of flying, target-seeking grenades envisioned in the linked video. With any level of autonomy, loitering and communication with command. It's only ethics and public perception that stops this today. In total war, a 20% mis-classification rate doesn't matter. Bombing doctrine in WW2 was much, much worse than this in its effect on civilians. Hell, many military leaders emphasized killing civilians as a desirable objective. Even present-day attacks consider "collateral damage" acceptable, even if politicians sometimes pretend otherwise.
Defending against weapons like that would be an arms race in itself, and completely beyond the reach of weaker actors. At least on a tactical level. Maybe there's strategies that could be employed as a low-cost defense, such as staying out of sensor range or confusing the sensors with adversarial patterns, but it would crush the resistance of anyone that tried a conventional defense.
Does the post-WW2 doctrine of emphasizing trade and the un-profitability of war, or ensuring that it is more economical for everyone to support a stable, free democratic economic system, still have any hope of preventing such weapons from being used?
Surely it would only be a matter of choosing to increase the efforts for a year or so in order to take these weapons from proof-of-concept to highly effective. E.g. autonomous flying gun platforms that can be deployed en masse and let loose with small or no human oversight (e.g. "hold the button to kill the person the machine has identified as a valid target", if even that level of control), or the kind of flying, target-seeking grenades envisioned in the linked video. With any level of autonomy, loitering and communication with command. It's only ethics and public perception that stops this today. In total war, a 20% mis-classification rate doesn't matter. Bombing doctrine in WW2 was much, much worse than this in its effect on civilians. Hell, many military leaders emphasized killing civilians as a desirable objective. Even present-day attacks consider "collateral damage" acceptable, even if politicians sometimes pretend otherwise.
Defending against weapons like that would be an arms race in itself, and completely beyond the reach of weaker actors. At least on a tactical level. Maybe there's strategies that could be employed as a low-cost defense, such as staying out of sensor range or confusing the sensors with adversarial patterns, but it would crush the resistance of anyone that tried a conventional defense.
Does the post-WW2 doctrine of emphasizing trade and the un-profitability of war, or ensuring that it is more economical for everyone to support a stable, free democratic economic system, still have any hope of preventing such weapons from being used?
I agree with you. But I think there is a bigger "weapon". When dust size monitoring is pervasive, how can anyone acquire weapons or coordinate any resistance? Dust is everywhere. And teams of nano-weapons can kill people (this has been reported as research for well over a decade, so anyone could reasonably assume that unreported research has gotten a lot further.) You don't need armies when nobody can effectively fight you or even acquire or build any weapon of any sort. So I don't worry as much about WW3 - since WW3 can be prevented with the pervasive monitoring technology on the near horizon. (There may be a WW3, but we have bigger problems coming up.)
When the map starts to reach the same size as the territory, don't you then need a map for the map? Pervasive monitoring sounds like one hell of a dataset to search through. Sure, you can use it to follow things you have already identified, but just because you have recorded something, doesn't mean that you know that you have recorded it.
Some people wonder why Cloudera (Hadoop and now Spark) had such high stock prices. But the real investors and the real clients were not public. Now, we all know that these systems are limited both in data volume and ability to run all the Machine Learning algorithms - but this problem is looking more and more solvable - and there are many who are trying outside of the public eye.
I think self-censorship itself is a problem. Just knowing that there are sensors around, many may remain silent.
Catharsis. I had been suspicious of similar things since early 2000s, thinking that the technology must be there. Surprised how only-ok sensors were while at research institutions (I avoided career paths even adjacent to military per principle).
Perhaps this is naive: One matter I never hear of, and is underestimated is automation of the legal system, and its equalizing effect. I have high hopes for it, although to be fair, our society is as unequal as we are. But those who cannot win the battle will have everything they need. And those who never have enough will be put in their places, and ideally technologists and scientists can take over from politicians, if we aren't automated ourselves. Then I wish to submit a request for democratization of computing power to the legal skynet, which grants it, and that in theory gives me my freedom as much as a gun claims it does. I should be able to own dust sensors to monitor intruding dust sensors. High hopes.
Some people make fun of elon musk for mars, but he has a point.
But thanks for turning my fanthom chills into real ones. The downvotes are from those who believe otherwise, which is not bad at all to see.
I wanted to mention something about certain areas of research on similar grounds, but I am not topping dust sensors by a long shot. Although my suspicions are probably correct then.
Again, I am probably on the naive-spectrum, but deep down, I feel if I don't ever go after money, I'll have less of a say in how the world plays out, and I am more fair than most I know, so I have a responsibility (I currently do research for research's sake, attic-style, and unfortunately I am only moderately bright, but bright enough to see through the facade where I am living that many many things only exist and function so the rich may remain rich, despite new technologies).
Perhaps this is naive: One matter I never hear of, and is underestimated is automation of the legal system, and its equalizing effect. I have high hopes for it, although to be fair, our society is as unequal as we are. But those who cannot win the battle will have everything they need. And those who never have enough will be put in their places, and ideally technologists and scientists can take over from politicians, if we aren't automated ourselves. Then I wish to submit a request for democratization of computing power to the legal skynet, which grants it, and that in theory gives me my freedom as much as a gun claims it does. I should be able to own dust sensors to monitor intruding dust sensors. High hopes.
Some people make fun of elon musk for mars, but he has a point.
But thanks for turning my fanthom chills into real ones. The downvotes are from those who believe otherwise, which is not bad at all to see.
I wanted to mention something about certain areas of research on similar grounds, but I am not topping dust sensors by a long shot. Although my suspicions are probably correct then.
Again, I am probably on the naive-spectrum, but deep down, I feel if I don't ever go after money, I'll have less of a say in how the world plays out, and I am more fair than most I know, so I have a responsibility (I currently do research for research's sake, attic-style, and unfortunately I am only moderately bright, but bright enough to see through the facade where I am living that many many things only exist and function so the rich may remain rich, despite new technologies).
Yeah, they worked amazingly well at stopping some terrorists.
Let's see them stop me from procuring a weapon.
FFS, even my ISP is more of a problem than dust sensors.
Let's see them stop me from procuring a weapon.
FFS, even my ISP is more of a problem than dust sensors.
In case it wasn't clear, we don't have pervasive monitoring deployed yet (not for lack of steps in that direction), so catching terrorists is still hard. But I guess you've already determined the limitations of future systems, so I don't even need to say that.
Flip that around, and anyone with $1k, some time on their hands and an internet connection can build a flying killbot with facial recognition. Which is equally bad.
[deleted]
I tried this on my i7 macbook pro, but my CPU usage went to 700% and basically i never got any moving image. Is this normal? What kind of computer you need to at least run this kind of real time?
Using the MobileSDK (https://developer.dji.com/mobile-sdk/) you can build this straight on a mobile app. With the OnboardSDK (https://developer.dji.com/onboard-sdk/) you can run it on the drone directly.
Has anyone attempted to do anything like this for SAR (Search and Resuce)? I”m not sure where one would gather a good dataset for training a model.
Here are a few more who have tried it or use deep learning for SAR:
https://www.planckaero.com/maritimedrone/
https://flytbase.com/flytstories/search-and-rescue-drone/
https://news.developer.nvidia.com/autonomous-search-and-resc...
Data for training such models for practical use are difficult to get. Often, one has to train a model on a different but related dataset that contains your object of interest and ensure that your model generalizes well. These models are highly sensitive to the data that they are trained on, so it is best if you can gather data which is closer to the actual environment (in which you want to use the model).
Data for training such models for practical use are difficult to get. Often, one has to train a model on a different but related dataset that contains your object of interest and ensure that your model generalizes well. These models are highly sensitive to the data that they are trained on, so it is best if you can gather data which is closer to the actual environment (in which you want to use the model).
Probably, I know of at least one project https://liu.se/en/news-item/har-testas-autonom-sjoraddning-i...
We at Nanonets have been building a lot of Machine Learning models for drone based image analysis. A frequent request from our customers is how do they use our models to do real time analysis. So we decided to create a how to on building real time object detection using a DJI drone.
Also useful if you're just looking to get the video stream to your laptop/computer.
Also useful if you're just looking to get the video stream to your laptop/computer.
Awesome, was looking for something like this. Will definitely try it out. What rate (fps) were you able to process the stream at?
If you really want this at low latency, you can also take the hdmi output of the dji controller and plug it into a capture card.
Depending on size of model between 30 and 5 fps
Very interesting! I really liked the result. Would you mind if I linked to this tutorial in WeeklyRobotics[1]?
[1] https://weeklyrobotics.com/
[1] https://weeklyrobotics.com/
sure
Skynet ist thanking you ;)
Does this work on parrot drones also?
Apps like `nanostream` allow you to forward the drone's live stream to any RTMP server, so this should work for Parrot drones too
There's some mention at the end of the article on how to attach a PC to the drone, but basically it would be the same thing: just run the processing on a PC.
Just by the title, I was expecting a clever way to run a pre-trained model directly on a drone.