Omniviolence Is Coming and the World Isn’t Ready(nautil.us)
nautil.us
Omniviolence Is Coming and the World Isn’t Ready
http://nautil.us/blog/omniviolence-is-coming-and-the-world-isnt-ready
128 comments
If we don't police everyone every second of the day, how will we stop them from joining fascist organizations?
Also you don't really need a full-ass panopticon to notice three million shaped charges being assembled together with three million drones.
I don't like this "panopticon" but I don't think suggesting it as a solution makes the author an "aspiring" fascist.
If your best idea is to refashion the world as a giant prison camp, that's pretty fascist whether or not one likes to think of oneself that way; if Bostrom thinks it's a necessity but doesn't believe it to be fascist because he personally does not wish to wield power over people then that's just idiotically fascist. (There's a viable side argument here about the difference between totalitarianism and fascism as one kind of totalitarianism, so if you prefer just use the latter term).
Now that I recall, Bostrom is the guy who came up with the 'simulation hypothesis' that was so in vogue a decade or so ago, and basically argued against the reality of the world on the theory that the possibility of sufficiently advanced simulations made the probability and ubiquity of such inevitable to the point that we were statistically more likely to be living in one. This struck me as a warmed-over version of the literary conceit that we're all just characters in a particularly elaborate-seeming novel or stage play. So now he's gone from reanimating a literary corpse to reanimating the economic corpse of Jeremy Bentham.
To be frank, I think we should stop listening to this guy. He's terribly clever but also deeply neurotic, like a pessimistic Ray Kurzweil (whose prediction record is rather better than he gets credit for, but similarly dwells a little too much in his own imagination). Utilitarianism aspires to be a benign philosophy but its proponents tend to overestimate their capacity for foresight and moral reasoning and end up creating deeply inhumane systems which are arguably worse than the problems they set out to solve.
Now that I recall, Bostrom is the guy who came up with the 'simulation hypothesis' that was so in vogue a decade or so ago, and basically argued against the reality of the world on the theory that the possibility of sufficiently advanced simulations made the probability and ubiquity of such inevitable to the point that we were statistically more likely to be living in one. This struck me as a warmed-over version of the literary conceit that we're all just characters in a particularly elaborate-seeming novel or stage play. So now he's gone from reanimating a literary corpse to reanimating the economic corpse of Jeremy Bentham.
To be frank, I think we should stop listening to this guy. He's terribly clever but also deeply neurotic, like a pessimistic Ray Kurzweil (whose prediction record is rather better than he gets credit for, but similarly dwells a little too much in his own imagination). Utilitarianism aspires to be a benign philosophy but its proponents tend to overestimate their capacity for foresight and moral reasoning and end up creating deeply inhumane systems which are arguably worse than the problems they set out to solve.
>This struck me as a warmed-over version of the literary conceit that we're all just characters in a particularly elaborate-seeming novel or stage play.
He also came up with the notion of apocalyptic "superintelligences"[0], which is a warmed-over version of 20th-century critiques of the corporation, starting with one written by an economist and a lawyer[1]. People like Cory Doctorow[2], Ted Chiang[3] and Charlie Stross[4] have pointed out the similarities.
>He's terribly clever but also deeply neurotic, like a pessimistic Ray Kurzweil (whose prediction record is rather better than he gets credit for, but similarly dwells a little too much in his own imagination).
He's a transhumanist[5] who co-founded the WTA (now Humanity+)[6] in 1998. I think he's probably realised along the way that doom and gloom sells: he founded FHI in 2005[7] and inspired Musk to fund CSER in 2012[8] with his book[0]. It's pretty much his day job now to be deeply pessimistic.
And it looks like doom-and-gloom think tanks are catching on: France is apparently hiring science fiction writers to dream up future threats[9]. Nice job if you can get it, I guess.
[0] https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
[1] https://en.wikipedia.org/wiki/The_Modern_Corporation_and_Pri...
[2] https://boingboing.net/2015/07/03/why-were-still-talking-abo...
[3] https://boingboing.net/2017/12/18/skynet-llc.html
[4] https://boingboing.net/2017/12/29/llcs-are-slow-ais.html
[5] https://en.wikipedia.org/wiki/Transhumanism#Growth_of_transh...
[6] https://en.wikipedia.org/wiki/Humanity%2B
[7] https://en.wikipedia.org/wiki/Future_of_Humanity_Institute
[8] https://en.wikipedia.org/wiki/Centre_for_the_Study_of_Existe...
[9] https://www.theverge.com/2019/7/24/20708432/france-military-...
He also came up with the notion of apocalyptic "superintelligences"[0], which is a warmed-over version of 20th-century critiques of the corporation, starting with one written by an economist and a lawyer[1]. People like Cory Doctorow[2], Ted Chiang[3] and Charlie Stross[4] have pointed out the similarities.
>He's terribly clever but also deeply neurotic, like a pessimistic Ray Kurzweil (whose prediction record is rather better than he gets credit for, but similarly dwells a little too much in his own imagination).
He's a transhumanist[5] who co-founded the WTA (now Humanity+)[6] in 1998. I think he's probably realised along the way that doom and gloom sells: he founded FHI in 2005[7] and inspired Musk to fund CSER in 2012[8] with his book[0]. It's pretty much his day job now to be deeply pessimistic.
And it looks like doom-and-gloom think tanks are catching on: France is apparently hiring science fiction writers to dream up future threats[9]. Nice job if you can get it, I guess.
[0] https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
[1] https://en.wikipedia.org/wiki/The_Modern_Corporation_and_Pri...
[2] https://boingboing.net/2015/07/03/why-were-still-talking-abo...
[3] https://boingboing.net/2017/12/18/skynet-llc.html
[4] https://boingboing.net/2017/12/29/llcs-are-slow-ais.html
[5] https://en.wikipedia.org/wiki/Transhumanism#Growth_of_transh...
[6] https://en.wikipedia.org/wiki/Humanity%2B
[7] https://en.wikipedia.org/wiki/Future_of_Humanity_Institute
[8] https://en.wikipedia.org/wiki/Centre_for_the_Study_of_Existe...
[9] https://www.theverge.com/2019/7/24/20708432/france-military-...
Damn, I do that for free and a large mug of coffee every morning (crossposted to 'if you're so smart how come you're not rich' thread)
Sorry. Still, it never ceases to amaze me what some people end up doing for work. "You mean they pay people for this?!" is usually my first reaction.
On the contrary, that's exactly what it makes him.
Not all totalitarianism is fascism.
What exactly do you think fascist means
Fascism (/ˈfæʃɪzəm/) is a form of far-right, authoritarian ultranationalism characterized by dictatorial power, forcible suppression of opposition, and strong regimentation of society and of the economy which came to prominence in early 20th-century Europe.
- from Wikipedia.
Pretty sure his proposal nails the dictatorial power, forcible suppression of opposition, and strong regimentation of society parts. After all, that's the whole point of all-encompassing surveillance.
- from Wikipedia.
Pretty sure his proposal nails the dictatorial power, forcible suppression of opposition, and strong regimentation of society parts. After all, that's the whole point of all-encompassing surveillance.
Parse the definition a bit more here. The things you describe are "characteristics of fascism", but the part that actually _is_ fascism is "far-right, authoritarian ultranationalism". Authoritarianism is one part of fascism, but not the defining characteristic.
Well, that's not why Google are doing it. Perhaps I'm mistaken about their motivations?!?
Motivations? Every fascist is convinced of the purity and necessity of their motivations. And eventually maintaining power becomes the overriding motivation...for the children, for the common good, for the poor, for the planet, and all that.
(Google are doing it for the money.)
Literally nothing in this article seems like something that will actually come to pass.
It's like a bad Black Mirror fanfic.
It's like a bad Black Mirror fanfic.
The scale may be exaggerated and the particulars may be wrong, but there definitely will be new problems that arise from cheap and effective remote/autonomous weapons being available. In addition to cheap IED drones which are already becoming available, what happens when you can pop a vision system + grasper arm securely around a rifle and effectively toss it up on a ridge somewhere? It's probably been technically possible for a while, but soon it'll be (relatively) cheap, and probably hard to monitor.
Agreed and the vaguely proposed solution for their hypothesized problem sounds far worse than the problem.
Came to say exactly this...
Not quite sure how the author went from "Technology is, in other words, enabling criminals to target anyone anywhere and, due to democratization, increasingly at scale" to "One strategy could be a superintelligent machine—essentially, an extremely powerful algorithm—that’s specifically designed to govern fairly".
Not quite sure how the author went from "Technology is, in other words, enabling criminals to target anyone anywhere and, due to democratization, increasingly at scale" to "One strategy could be a superintelligent machine—essentially, an extremely powerful algorithm—that’s specifically designed to govern fairly".
> Of course, this is a fantastical proposal. Even the real-world use of AI in the justice system is fraught with problems. But at this point, do we have a better idea for preventing the collapse of the state system under the weight of widespread technological empowerment?
I'm not sure what motivated this sort of soundbite-style quoting without digging in further, but I hope if you first read the comments without the article you realize this is supposed to pose a question, not offer a solution.
I'm not sure what motivated this sort of soundbite-style quoting without digging in further, but I hope if you first read the comments without the article you realize this is supposed to pose a question, not offer a solution.
I'm not judging the solution, just the logic that skipped from "Omniviolence" to the absurd proposal for an AI-based government authority. The two are quite unrelated in my mind.
For example, the article made no serious mention of counter-measures such as string (which can down a drone), detection (RF directional antennae), or even just "plant more trees" (as trees make drone navigation more complicated - I know from personal experience).
Just broadcasting random RF/VHF/UHF noise (there are only so many channels available) at drones can wreak havoc.
Quite seriously, do a web search for "can you spot the source of a drone controller", and you will find a wealth of information from RF source detection, to the GPS coordinates of the controller (so the drone can "return home") - there a TON of alternatives that any "scholar" should research before even hinting at a doom-and-gloom alternative such as AI-based government.
For example, the article made no serious mention of counter-measures such as string (which can down a drone), detection (RF directional antennae), or even just "plant more trees" (as trees make drone navigation more complicated - I know from personal experience).
Just broadcasting random RF/VHF/UHF noise (there are only so many channels available) at drones can wreak havoc.
Quite seriously, do a web search for "can you spot the source of a drone controller", and you will find a wealth of information from RF source detection, to the GPS coordinates of the controller (so the drone can "return home") - there a TON of alternatives that any "scholar" should research before even hinting at a doom-and-gloom alternative such as AI-based government.
And, on that note, sounds like it would do effectively nothing to even address the problem in the first place.
While basically all the omniviolence scenarios would induce the state to trust individuals less, which is kind of the thrust of the article, climate change can still induce more frequent heatwaves and make things difficult for more people. Whether it will reach the threshold of societal collapse is a more difficult question.
Really?
I'm just waiting for some dumbass to take a cheap chinese drone, attach a blasting cap with a shape charge laced with sodium cyanide. Anybody nicked with fragments would die.
And it'd cost what, $80-$100 ?
It's the endgame of asymmetric warfare.
I'm just waiting for some dumbass to take a cheap chinese drone, attach a blasting cap with a shape charge laced with sodium cyanide. Anybody nicked with fragments would die.
And it'd cost what, $80-$100 ?
It's the endgame of asymmetric warfare.
Drone + cluster bomblet, cheep.
That sort of thing scares me more than weapons of mass destruction. Because they are usable by non-nation state actors and rouge nation state actors.
It's not that I'm worried anyone's going to try to take me out (like why?). It's I'm worried what sort of society we end up with when people like Larry Ellison are terrified of the above.
That sort of thing scares me more than weapons of mass destruction. Because they are usable by non-nation state actors and rouge nation state actors.
It's not that I'm worried anyone's going to try to take me out (like why?). It's I'm worried what sort of society we end up with when people like Larry Ellison are terrified of the above.
It's I'm worried what sort of society we end up with when people like Larry Ellison are terrified of the above.
I would like to subscribe to your newsletter. Do you by chance have a manifesto I could peruse?
I would like to subscribe to your newsletter. Do you by chance have a manifesto I could peruse?
Larry Ellison in the grand scheme of things is a turd who while he has probably pissed enough people off hasn't pissed off anyone enough to try to flush him with extreme prejudice.
I'd be more worried about people of actual importance like police chiefs, prosecutors, senators, presidents.
I'd be more worried about people of actual importance like police chiefs, prosecutors, senators, presidents.
Not signalling anyone out, pick a name any name tho. Consider though where I live no one gives a flying fuck what I say. Because nothing I say is going to upend anyone with any real power.
There are other places where I'd stick out like a nail.
There are other places where I'd stick out like a nail.
I'm actually quite a bit more worried about some dumbass tossing things off the overpass at my car. Rocks are free.
I thought drones in warzones have already been used to target people and explode at head level.
Maybe not crates of miniaturized ones with facial recognition, but precursors.
Maybe not crates of miniaturized ones with facial recognition, but precursors.
The scale is so different that it’s not really the same thing. It’s like comparing a mortar to an orbital bombardment with rail-guns. Yes both involve sending a projectile at a target at high speed but they aren’t remotely similar.
Consumer grade drones are already being used in combat in the middle east: https://www.militarytimes.com/news/your-military/2017/01/11/...
(All the better sources for these stories are paywalled, so have fun reading them)
(All the better sources for these stories are paywalled, so have fun reading them)
[deleted]
I would like to see a mini quad copter take the recoil of a 1 g "shaped charge" (aka bullet).
EDIT: ok, i found a clip of (a fairly big) drone firing a gun and it looks... not stable:
https://youtu.be/FI--wFfipvA
still, pretty worrisome.
EDIT: ok, i found a clip of (a fairly big) drone firing a gun and it looks... not stable:
https://youtu.be/FI--wFfipvA
still, pretty worrisome.
Shaped charges are not bullets. They're bits of specially-molded explosives, such that the blast energy of the explosive forms the charge into a jet of molten metal. They don't rely on any kinetic energy at all to penetrate, and can be delivered by sticking a small candle-sized (the little ones, like you might see at a fancy dinner) device onto the targeted surface.
https://en.wikipedia.org/wiki/Shaped_charge
https://en.wikipedia.org/wiki/Shaped_charge
What you said directly contradicts the wiki article you linked...
"Contrary to a widespread misconception (possibly resulting from the acronym HEAT) the shaped charge does not depend in any way on heating or melting for its effectiveness; that is, the jet from a shaped charge does not melt its way through armor, as its effect is purely kinetic in nature"
"Contrary to a widespread misconception (possibly resulting from the acronym HEAT) the shaped charge does not depend in any way on heating or melting for its effectiveness; that is, the jet from a shaped charge does not melt its way through armor, as its effect is purely kinetic in nature"
No, it doesn't contradict it. What I mean by "doesn't require any kinetic energy to penetrate" is that the explosive doesn't need to be moving at all to penetrate. For example, you can stick a shape charge directly underneath a parked tank, detonate it, and it will blow up the tank. (This is exactly what a landmine or IED does.) You could also just drop it by parachute from a drone (this is a cluster bomb).
What the Wikipedia article is saying is that the explosive forms its own projectile out of the molten metal casing, which is expelled at high velocity from the explosive and then penetrates through kinetic energy. It's a matter of where the energy for penetration comes from: what both Wikipedia and I are saying is that it comes from the explosive charge itself, not the velocity of the projectile, and hence there's no need for recoil when fired from a drone.
What the Wikipedia article is saying is that the explosive forms its own projectile out of the molten metal casing, which is expelled at high velocity from the explosive and then penetrates through kinetic energy. It's a matter of where the energy for penetration comes from: what both Wikipedia and I are saying is that it comes from the explosive charge itself, not the velocity of the projectile, and hence there's no need for recoil when fired from a drone.
Yes, now that you've completely changed what you said, from...
They don't rely on any kinetic energy at all to penetrate
to...
"which is expelled at high velocity from the explosive and then penetrates through kinetic energy"
I agee. But these projectiles only work under 2 conditions. (1) They have something very solid behind them, like the ground, or a metal gun, or (2) they are hurling toward the object at great speed. And if the drones are simply going to be kamikaze diving from 1000 feet above, almost anything you attach to it will cause significant damage. However that'd be a pretty low-precision maneuver, and sort of defeats the purpose of all those weeks spent training to perform image recognition.
They don't rely on any kinetic energy at all to penetrate
to...
"which is expelled at high velocity from the explosive and then penetrates through kinetic energy"
I agee. But these projectiles only work under 2 conditions. (1) They have something very solid behind them, like the ground, or a metal gun, or (2) they are hurling toward the object at great speed. And if the drones are simply going to be kamikaze diving from 1000 feet above, almost anything you attach to it will cause significant damage. However that'd be a pretty low-precision maneuver, and sort of defeats the purpose of all those weeks spent training to perform image recognition.
Whether the impactor mass is a solid shell, or the molten ejecta of a shaped charge, Sir Isaac is going to prosecute you under his Third Law.
Reactionless motion is not possible. The drone will recoil.
That said, it only need fire once given the mission in question.
Reactionless motion is not possible. The drone will recoil.
That said, it only need fire once given the mission in question.
It seems like we are all nearly on the same page. n.b. when the author mentioned millions of tiny drones, I immediately pictured my own...
https://i.imgur.com/LU4jzeY.jpg
It probably weighs about 5 grams, mostly due to the battery, which lasts all of 3 minutes (i.e. I wasn't thinking of millions of drones the size of that in the video I linked above)
https://i.imgur.com/LU4jzeY.jpg
It probably weighs about 5 grams, mostly due to the battery, which lasts all of 3 minutes (i.e. I wasn't thinking of millions of drones the size of that in the video I linked above)
If you actually watch the original video the drones aren't intended to survive.
OK, drones are cheap. How about that projectile firing apparatus?
shaped charges are made of a metal cylinder with explosives inside and a concave copper cone on one end. insurgents can and do manufacture them. the explosives and detonation rigging are the only part that has a meaningful cost, and if you're talking about one gram of explosive that isn't much.
I don't think you technically need to absorb the recoil.
Check out the EFP:
https://en.wikipedia.org/wiki/Explosively_formed_penetrator
EFPs were what turned out Humvees to swiss-cheese in Iraq before we switched over the MRAPs.
Check out the EFP:
https://en.wikipedia.org/wiki/Explosively_formed_penetrator
EFPs were what turned out Humvees to swiss-cheese in Iraq before we switched over the MRAPs.
Ah this is what I wanted to know. If you ignite a shaped projectile with nothing to push on but air, will it go
"An EFP 20 cm (8 inches) in diameter threw a 3 kg (7 lbs) copper slug at Mach 6, or 2,000 meters per second. (A .50-caliber bullet, among the most devastating projectiles on the battlefield, weighs less than two ounces and has a muzzle velocity of 900 meters per second.)
— Rick Atkinson, The Washington Post"
"An EFP 20 cm (8 inches) in diameter threw a 3 kg (7 lbs) copper slug at Mach 6, or 2,000 meters per second. (A .50-caliber bullet, among the most devastating projectiles on the battlefield, weighs less than two ounces and has a muzzle velocity of 900 meters per second.)
— Rick Atkinson, The Washington Post"
They're disposable. They don't need to take anything.
And shaped charges are entirely besides the point. They could instead carry a small auto-injector (like an EpiPen) containing a few mg of ricin [1]. It'd be like getting stung by an insect. Within days the symptoms would start to appear.
If this happened to even a few hundred people, let alone thousands, then hospitals would be completely overwhelmed and people would be dying everywhere. It would cause a mass panic.
Injection with ricin was how Georgi Markov was assassinated [2]. It was shocking to say the least.
[1] https://en.wikipedia.org/wiki/Ricin
[2] https://en.wikipedia.org/wiki/Bulgarian_umbrella
If this happened to even a few hundred people, let alone thousands, then hospitals would be completely overwhelmed and people would be dying everywhere. It would cause a mass panic.
Injection with ricin was how Georgi Markov was assassinated [2]. It was shocking to say the least.
[1] https://en.wikipedia.org/wiki/Ricin
[2] https://en.wikipedia.org/wiki/Bulgarian_umbrella
Have you ever had to use or administer an auto injector like an EpiPen? It's not trivial. It requires deliberate force against a skin surface. A lightweight drone simply couldn't do it. Not could any drone, on its own. It would take a dedicated system to find and latch on to the skin while the injection loaded. At that point, it would be far easier to just have drone release sarin has in a crowd. And that sort of deployment is a threat society has existed with for decades and hasn't collapsed under it's weight.
A few hundred is within the scope of what people are doing with conventional explosives. The article is talking about the ability to target millions of people.
I just don't believe we will see a day when tiny drones with epipens pose a greater threat than guns.
I just don't believe we will see a day when tiny drones with epipens pose a greater threat than guns.
What sets it apart from conventional explosives is that it's decentralized. A single attacker can distribute these things all over a city. This makes it much more difficult to mitigate the threat, let alone apprehend the attacker.
A single attacker can't distribute them in this way because it takes many more for that scale of resources. Absent a self-funded evil genius comic book villain, you're looking at terrorist organizations and nation states, not lone individuals.
Get a...
"very, very small quadcopter, one inch in diameter that can carry a one-or two-gram shaped charge,”
...and fire a bullet from it, and let me know what goes farther, the bullet or the quad.
"very, very small quadcopter, one inch in diameter that can carry a one-or two-gram shaped charge,”
...and fire a bullet from it, and let me know what goes farther, the bullet or the quad.
> In this scenario, the K/K ratio could be perhaps 3/1,000,000, assuming a 10-percent accuracy and only a single one-gram shaped charge per drone.
So you "only" need 10 million drones and 10 million grams of plastic explosive? An attack of that magnitude probably requires the resources of a nation-state, so I doubt it will happen. Targeted assassinations seem more plausible.
So you "only" need 10 million drones and 10 million grams of plastic explosive? An attack of that magnitude probably requires the resources of a nation-state, so I doubt it will happen. Targeted assassinations seem more plausible.
I mean if you barred the doors to a theater and tossed a few gallons of bleach and vinegar in you could also just gas hundreds of people, but it doesn't happen.
We worry a lot about technology because the attacks sound novel, but not about the conventional ones which still (mostly) don't happen because the reality is that the actual logistical challenges of implementing them without security services noticing still hasn't fundamentally changed all that much.
We worry a lot about technology because the attacks sound novel, but not about the conventional ones which still (mostly) don't happen because the reality is that the actual logistical challenges of implementing them without security services noticing still hasn't fundamentally changed all that much.
I think the real limiter is people willing and stable enough to murder indiscriminately at scale in novel ways. Mowing down crowds of people with a truck is bizarrely recent, and the peak of mass murder is still one guy sniping at a concert from a nearby hotel. Everyone was relatively surprised that passenger planes could be used as destructive weapons.
The really effective means of mass murder I think would be subtle, though, and mass murderers don't seem to do subtle. E.g. leaving some radioactive material in an innocuous public place, worsening visibility at crash prone intersections, subtle and slow poisons mixed into food and drink…
The really effective means of mass murder I think would be subtle, though, and mass murderers don't seem to do subtle. E.g. leaving some radioactive material in an innocuous public place, worsening visibility at crash prone intersections, subtle and slow poisons mixed into food and drink…
Or maybe something like developing, manufacturing, and prompting a new pain pill, that gets millions of people addicted to opioids, thus ultimately resulting of millions of overdose death over a decade.
Agree with all that, but when it comes to synbio, the article is 100% right to raise alarms.
But even then, historical analogues haven't been exploited that often. We've had probably thousands of people with access to weaponizable Smallpox etc in the US and former USSR for many decades now and no one has made any known attempt to use it, sell it, ...
Now turn that thousands to millions and you’ll see where the problem lies.
>if you barred the doors to a theater and tossed a few gallons of bleach and vinegar in you could also just gas hundreds of people, but it doesn't happen.
Well during the 2012 Aurora movie theater shooting the guy used homemade tear gas before killing 12 and shooting another 60. But you're right he didn't lock the doors. The 2016 Pulse shooter did lock the doors and killed 50 and shot an additional 50. So it's not as far fetched as you make it sound.
Well during the 2012 Aurora movie theater shooting the guy used homemade tear gas before killing 12 and shooting another 60. But you're right he didn't lock the doors. The 2016 Pulse shooter did lock the doors and killed 50 and shot an additional 50. So it's not as far fetched as you make it sound.
Cyberlockers are an example are they not? Criminals are killing people in hospitals by locking up hospital computers.
Fortunately hospitals do have "outdated" manual technology to keep running even when worst happens. Reliability is a key consideration.
The records are also still kept on paper just in case.
Triage is a possibility when truly overwhelmed.
The main resource is a thinking and resourceful doctor and nurse. They can improvise (in ways governed by law) if there is a real need, considering patient welfare first.
The main resource is a thinking and resourceful doctor and nurse. They can improvise (in ways governed by law) if there is a real need, considering patient welfare first.
was going to make a similar comment. In their hypothetical scenario they conveniently skip passed the need to obtain/manufacture 10 metric tones of high explosive.
I'd say it's not access to drones/AI that's the most frightening in this scenario, it's the access to massive amounts of high explosives. If 3 crazies can get their hands on that quantity of explosives, they don't need drones/AI. A few selectively placed trucks/cars loaded with explosives are a long standing tool of mass-murderers.
I amused myself with the math that results if our 3 crazies have to hand load 1g of shaped charge on each drone (assuming that you can't exactly subcontract out this "please make our killer drones for us and don't ask any questions). It came to a mere 4.75 years of 16 hour days for the 3 people, assuming it takes 30 seconds per drone. No time off for holidays.
It's pretty amazing that someone can spout off about this so confidently, with zero ability to do simple common sense back of the envelope calculations.
It's pretty amazing that someone can spout off about this so confidently, with zero ability to do simple common sense back of the envelope calculations.
lol. And no weekends.
Not to mention that the fucking things will fall from the sky after ten minutes, because of our battery tech...
Yeah, the 3/1000000 case is kind of a pointless extrapolation. 3/1000 is way more threatening if those 1000 are local political/business leaders - you could cripple an entire city (or with careful timing, an entire state).
[deleted]
And if nation-states want to kill 10 million people, that's what nuclear weapons programs are for.
A more reasonable solution to a bunch of drones terrorizing humanity (at least more reasonable than "hey let's make those drones' lives easier by setting up a giant artificially-intelligent surveillance monstrosity! What could possibly go wrong?") would be to stock up on shotguns, birdshot, butterfly nets, and crowbars.
Hope you've got a bunch of nets and a bunch of ammo cause drones are pretty cheap and there's nothing stopping someone from sending 5 or 10 your way.
As cheap as drones may be, a single drone costs more than a box of cartridges: https://www.cabelas.com/product/shooting/ammunition/shotgun-...
When it comes to defending one's citizens, the attitude of "that'll probably never happen, it'll be fine" doesn't exactly sound like a solid foundation. And I doubt any government around the world, worth its salt, has that attitude.
True, it all sounds like scaremongering, but the question is: what's stopping people from actually doing this?
Crime and violence already exist. And if we look at history, it is clear that it has changed with the times. What logical sense is there in thinking they will not follow the changes in technology today?
The events described are but a possibility. Yet, I do feel we should not dismiss it, simply because we don't like it.
True, it all sounds like scaremongering, but the question is: what's stopping people from actually doing this?
Crime and violence already exist. And if we look at history, it is clear that it has changed with the times. What logical sense is there in thinking they will not follow the changes in technology today?
The events described are but a possibility. Yet, I do feel we should not dismiss it, simply because we don't like it.
The article drums up a hypothetical scenario ("a semi-trailer full of autonomous drones carrying small shaped charges could kill a million New Yorkers, just for instance!") and then suggests we could counteract the Omniviolence that The World Is Not Ready for by submitting to a Panopticon society or making an AI our Philosopher King.
This is not serious minded analysis, but it's not fun either. It's just bad clickbait.
No system can stop every dedicated attacker to begin with. Technology gives people leverage in all their activities, including doing violence to one another. Writing an article in Nautilus that amounts to idle musing about a comic book villain's murderbot scheme and then trying to burnish it as Very Serious at the end with name dropping and the author's academic credentials is so lame.
This is not serious minded analysis, but it's not fun either. It's just bad clickbait.
No system can stop every dedicated attacker to begin with. Technology gives people leverage in all their activities, including doing violence to one another. Writing an article in Nautilus that amounts to idle musing about a comic book villain's murderbot scheme and then trying to burnish it as Very Serious at the end with name dropping and the author's academic credentials is so lame.
Abstract of a different work by the same author:
>the only plausible escape from this conundrum, at least from our present vantage point, is the creation of a “supersingleton” run by a friendly superintelligence, founded upon a “post-singularity social contract.”
I didnt read the paper, but that abstract sounds firmly rooted in sci-fi with a toe in nonfiction, instead of the other way around as im sure the author believes.
K/K has been very low for at least 50 years and probably longer. The idea that a terrorist would smuggle millions of drones to cause... Car crashes... Is the stuff of a high school writing class. i am 100% positive there are cheaper and more effective tools available to cause more serious damage that currently actually exist (and have for decades), and we dont have the society the author claims.
>the only plausible escape from this conundrum, at least from our present vantage point, is the creation of a “supersingleton” run by a friendly superintelligence, founded upon a “post-singularity social contract.”
I didnt read the paper, but that abstract sounds firmly rooted in sci-fi with a toe in nonfiction, instead of the other way around as im sure the author believes.
K/K has been very low for at least 50 years and probably longer. The idea that a terrorist would smuggle millions of drones to cause... Car crashes... Is the stuff of a high school writing class. i am 100% positive there are cheaper and more effective tools available to cause more serious damage that currently actually exist (and have for decades), and we dont have the society the author claims.
The organizing of any neural network training program to create AI that can identify targets would also be a very laborious process.
Is this really just another advertisement for Bostrom's ridiculous surveillance state saviour idea?
Every one of these "individual actor catastrophes" has just assloads of barriers to it. Take the "swarm of exploding drones" or "slaughterbots" if you're a douchebag. We do not have the technology for a swarm of a dozen one-inch drones to avoid crashing into each other, let alone carry out the intensely complicated task of tracking down human beings. Also, drones that size have a flight time of about five minutes. Also you would need to manufacture and attach millions of shaped charges, which, unless you're a state actor, is likely to kill the person doing it considering the poor life expectancy of people trying to make only handfuls of IEDs.
Let's take something that's entirely possible using current technology. A small group of motivated individuals could purchase thousands of toasters and turn them into bombs and mail them all to politicians or schools or _____ and set them to all explode at the same time. But that doesn't happen. A group could use agar and antibiotics to breed up a bunch of highly-resistant MRSA and use syringes to infect half the grocery section at dozens of stores on the same day and in doing so kill tens of thousands of people. But that won't happen either.
Just because you can imagine something and not think of a reason it's impossible doesn't mean it's a realistic threat.
Let's take something that's entirely possible using current technology. A small group of motivated individuals could purchase thousands of toasters and turn them into bombs and mail them all to politicians or schools or _____ and set them to all explode at the same time. But that doesn't happen. A group could use agar and antibiotics to breed up a bunch of highly-resistant MRSA and use syringes to infect half the grocery section at dozens of stores on the same day and in doing so kill tens of thousands of people. But that won't happen either.
Just because you can imagine something and not think of a reason it's impossible doesn't mean it's a realistic threat.
The described scenario involves billions of dollars which in most cases represents the output of thousands or 10s of thousands. The attack described is somewhat novel but the fact that the work of thousands can oppose the work of millions isn't.
This makes the nation state more relevant not less. The state can best fund countermeasures and preemptive investigations. They can work with nations who will work to police themselves so they don't have to deal with the backlash involved in locals killing a million people far from home and if needed punish nation states who fail in such policing.
This makes the nation state more relevant not less. The state can best fund countermeasures and preemptive investigations. They can work with nations who will work to police themselves so they don't have to deal with the backlash involved in locals killing a million people far from home and if needed punish nation states who fail in such policing.
> One strategy could be a superintelligent machine—essentially, an extremely powerful algorithm—that’s specifically designed to govern fairly.
I volunteer to write that fair governing powerful algorithm. You can trust me to make it fair and not make it favor my biases and interests.
I volunteer to write that fair governing powerful algorithm. You can trust me to make it fair and not make it favor my biases and interests.
I think we need to use machine learning, feeding it a corpus of all fair government decisions across a selection of successful governments over the last 1000 years.
I would be fine with being able to select which "fair decisions" to train them on.
The silver lining of deepfakes is that now everyone has plausible deniability for any 'leaked nudes': anyone with a gaming computer and a lot of free time can put anyone else's head onto a pornographic actor's body.
Within a decade, 'nudes being leaked' will no longer be controversial or damaging because they'll cost a couple hundred dollars or less to manufacture and won't have credibility.
I'd argue that one of the best things a 'chaotic good' actor could do today to fight revenge porn is to manufacture a huge quantity of deepfake porn of random targets and release it, so speeding along this process.
Within a decade, 'nudes being leaked' will no longer be controversial or damaging because they'll cost a couple hundred dollars or less to manufacture and won't have credibility.
I'd argue that one of the best things a 'chaotic good' actor could do today to fight revenge porn is to manufacture a huge quantity of deepfake porn of random targets and release it, so speeding along this process.
We already have photoshop and this isn't the case.
But I am. I was born ready, and I have been waiting. Preparing. Training.
"Will emerging technologies make the state system obsolete? It’s hard to see why not."
Yeah, it's hard to see why not if your view of technology is something like that of medieval demonology, which appears to be the thesis of this article.
Meanwhile, in reality land, you can make explosives and poison gas out of stuff in your kitchen, automobiles put a ridiculous amount of kinetic energy in the hands of individuals, this has been true for 100 years, and somehow "the state system" has done just fine; strengthened even in that period of time.
Also Bostrom is a nincompoop and no article that mentions him deserves to be taken seriously.
Yeah, it's hard to see why not if your view of technology is something like that of medieval demonology, which appears to be the thesis of this article.
Meanwhile, in reality land, you can make explosives and poison gas out of stuff in your kitchen, automobiles put a ridiculous amount of kinetic energy in the hands of individuals, this has been true for 100 years, and somehow "the state system" has done just fine; strengthened even in that period of time.
Also Bostrom is a nincompoop and no article that mentions him deserves to be taken seriously.
And I think that the author's view is totally and completely backwards. If anything, it will solidify the importance of the state as a law enforcement entity. If someone from Brazil attacks an Italian citizen in his home, is Japan going to help the Italian or bring justice to the criminal in Brazil?
>automobiles put a ridiculous amount of kinetic energy in the hands of individuals, this has been true for 100 years, and somehow "the state system" has done just fine; strengthened even in that period of time.
Yep.
States figured out how to turn automobiles into tanks, essentially strengthening their power. Yet the article implicitly posits that somehow the same won't happen with microprocessors and nanomachines? The entire article seems a bit fanciful. States aren't going anywhere.
Yep.
States figured out how to turn automobiles into tanks, essentially strengthening their power. Yet the article implicitly posits that somehow the same won't happen with microprocessors and nanomachines? The entire article seems a bit fanciful. States aren't going anywhere.
Yeah, this is a common fallacy, I'm sure there would be a name for it. Basically "my strategy will work, assuming my opponent doesn't act at all", and ignores the fact the opponent is a capable, well resourced entity.
Many bitcoin proponents have the same fallacy - i.e. "this is better than banks, therefore banks are obsolete", ignoring the fact that banks will work with new technology as it suits them.
Many bitcoin proponents have the same fallacy - i.e. "this is better than banks, therefore banks are obsolete", ignoring the fact that banks will work with new technology as it suits them.
>States figured out how to turn automobiles into tanks
So did this disgruntled citizen:
https://youtu.be/PZbG9i1oGPA
So did this disgruntled citizen:
https://youtu.be/PZbG9i1oGPA
And which tank is superior?
The "killdoser", designed and built by the non-state actor inside the US nation state?
Or the M1 Abrams, designed and built by the US nation state itself?
Even if you build yourself a "killdozer", you are powerless against the US state. Because the tanks that the US nation state built substantially increased the strength and power of that nation state years ago.
The "killdoser", designed and built by the non-state actor inside the US nation state?
Or the M1 Abrams, designed and built by the US nation state itself?
Even if you build yourself a "killdozer", you are powerless against the US state. Because the tanks that the US nation state built substantially increased the strength and power of that nation state years ago.
>Also Bostrom is a nincompoop and no article that mentions him deserves to be taken seriously.
Agreed, and as I've pointed out elsewhere[0] (see also the parent comment[1]), his ideas are also not that new. I think it's quite sad that people somehow think that, because it shows how history -- even the development of ideas in the 20th century -- can be so easily forgotten.
[0] https://news.ycombinator.com/item?id=21319970
[1] https://news.ycombinator.com/item?id=21319584
Agreed, and as I've pointed out elsewhere[0] (see also the parent comment[1]), his ideas are also not that new. I think it's quite sad that people somehow think that, because it shows how history -- even the development of ideas in the 20th century -- can be so easily forgotten.
[0] https://news.ycombinator.com/item?id=21319970
[1] https://news.ycombinator.com/item?id=21319584
"Will emerging technologies make the state system obsolete? It’s hard to see why not."
Honest question - could the U.S. Government be considered an Abstract State Machine?
Honest question - could the U.S. Government be considered an Abstract State Machine?
If you squint, I could buy that as a way of describing the execution of policy (a state machine could, for example, oversee wealth redistribution payments) but not as a description of policy making entities.
It’s far easier to build a zip gun than a killer drone, yet we don’t see too many of the former in countries with strong gun regulation. Why should we believe that hunter-killer drones carrying explosives will be any more of a large-scale problem?
I’m way more worried about home gamers buying gene editing kits and tinkering with pathogens.
Can you tag a gene with your twitter handle?
Can you tag a gene with your twitter handle?
It is at least somewhat novel to see a technological alarmist jump entirely over the "encryption enables child porn" argument all the way to "robot drones are going to murder your family in broad daylight."
Go big or go home, I guess.
Go big or go home, I guess.
Amusing to picture our "3" terrorists carefully assembling 10m drones, each with a 1g shaped charge on it - or does the author think this is something that you can just order from some factory in China, along with the 10 tons of plastic explosive.
It seems to be that anyone sophisticated enough to put 10 tons of plastic explosive in any form wherever they feel like is already quite capable of causing massive casualties.
Somehow, to make this even remotely plausible, we need to intersect a lot of unlikely combinations (a bunch of really bad guys who are hugely resourced, very technical and intelligent, completely uninfiltrated, fighting for some goal that isn't going to be radically harmed by the optics of killing thousands of soft targets).
It seems to be that anyone sophisticated enough to put 10 tons of plastic explosive in any form wherever they feel like is already quite capable of causing massive casualties.
Somehow, to make this even remotely plausible, we need to intersect a lot of unlikely combinations (a bunch of really bad guys who are hugely resourced, very technical and intelligent, completely uninfiltrated, fighting for some goal that isn't going to be radically harmed by the optics of killing thousands of soft targets).
The more I think about this, the funnier it gets. How long would it take to unpack a drone, put a power source in it, and install a 1g shaped charge on a drone and pack it away? 30 seconds? It seems like something you wouldn't exactly want to rush.
So with 3 terrorists, assuming they worked 16 hour days, they'd have their devices ready in 4.75 years, assuming they didn't take any holidays.
So with 3 terrorists, assuming they worked 16 hour days, they'd have their devices ready in 4.75 years, assuming they didn't take any holidays.
Sounds like a James Bond plot to me.
[deleted]
[deleted]
[deleted]
The premise of this article: Because a super-villain like attack “could” occur, we need to solve a technology threat, with more technology... that is more likely to be hacked and misused (eventually). Don’t agree.
If the “state” can’t defend me from fanastical threats, it serves no purpose and we should abolish it? Nope, don’t agree again. I accept that my state can’t guarantee ultimate protections from a Vin Diesel-like movie threat situation. However, my state provides other benefits like: funding for education, business regulations and tax credits, lending regulation, infrastructure build/repair, funding construction projects (my industry), wildlife protection, park systems, etc.
Joke of an article.
If the “state” can’t defend me from fanastical threats, it serves no purpose and we should abolish it? Nope, don’t agree again. I accept that my state can’t guarantee ultimate protections from a Vin Diesel-like movie threat situation. However, my state provides other benefits like: funding for education, business regulations and tax credits, lending regulation, infrastructure build/repair, funding construction projects (my industry), wildlife protection, park systems, etc.
Joke of an article.
This is the basic feeling I left with as well. In terms of the hypothetical threat presented here (millions of autonomous kill drones), there are actually solid defensive measures that can be taken, even if they may not be absolute (aka stopping evil superman). Some are direct - I.e. microwave/laser/kinetic countermeasures which a nation state could quite easily deploy, whereas others (the more important ones IMO) are indirect - I.e. Monitoring supply chains and persons of interest., following up on anonymous tips (aka clerk at hardware store concerned with customer purchase, or the guy complaining about the secret drone factory noise next door).
The value of the nation state, even confined to this hypothetical scenario, is still there for me. There is far more to this than the one perspective of "techno-horror vs hapless government" presented in the article.
The value of the nation state, even confined to this hypothetical scenario, is still there for me. There is far more to this than the one perspective of "techno-horror vs hapless government" presented in the article.
[deleted]
“A very, very small quadcopter, one inch in diameter can carry a one-or two-gram shaped charge,” he says. “You can order them from a drone manufacturer in China. You can program the code...
Or you could start a meme about sneaking up on people and punching them in the head to try to knock them out, styling it as some sort of game.
Guess which one I'm more worried about.
Or you could start a meme about sneaking up on people and punching them in the head to try to knock them out, styling it as some sort of game.
Guess which one I'm more worried about.
Wow.
The one you're worried about is even more hare-brained than the article's worry.
At this point people really are out here creating reasons to be fearful. Don't walk the street, you'll be punched. Don't go to walmart/target, you'll be gunned down by the terrorists. Take off your shoes before you get on this plane, so we can search for the bomb you put in them. And of course this article now adds, don't go outside, the nano swarm will annihilate you.
You guys should really go for a walk and smell some flowers or something.
The one you're worried about is even more hare-brained than the article's worry.
At this point people really are out here creating reasons to be fearful. Don't walk the street, you'll be punched. Don't go to walmart/target, you'll be gunned down by the terrorists. Take off your shoes before you get on this plane, so we can search for the bomb you put in them. And of course this article now adds, don't go outside, the nano swarm will annihilate you.
You guys should really go for a walk and smell some flowers or something.
This is a huge step backward from gray goo alarm-ism. Disappointing.
Nanotechnology at least had the possibility of being the singularity if only you would learn to relax and enjoy it. Bostrom seems bent on realizing the Inferno and adding a 10th circle of people saying 'it's your fault from not stopping me.'
As the cost of offensive technology decreases, so does the cost of defensive technology. The 10 million drones could be countered by a larger number or AI-powered defensive drones.
The real threat of technology comes from the power it gives to governments. Bad actors can do more damage through the political process than any other.
The real threat of technology comes from the power it gives to governments. Bad actors can do more damage through the political process than any other.
> Nick Bostrom argues that the only way to defend against a global catastrophe is to employ a universal and invasive surveillance system, what he calls a “High-tech Panopticon.”
I suspected as much since the simulation argument but looks like Bostrom has really lost his marbles.
I suspected as much since the simulation argument but looks like Bostrom has really lost his marbles.
This is a lot of words to communicate a fairly shallow fantasy.
The solution to potential threats is to surrender ourselves to actual totalitarianism. What a novel idea.
This is why I wish HN had downvotes. This article/author deserves no attention/spread. It is a waste of our time.
WTF am I reading?
Despite the article's weaknesses, I think its spot on to assert that political nation-states may not survive technology. I think we are living in nationalism's dying breath.
Aspiring fascists are always trying to save us from ourselves.