ShipMap(shipmap.org)
shipmap.org
ShipMap
https://www.shipmap.org/
79 comments
I see a lot of these points going over land, even where there aren't any big rivers. Maybe this is a plot of the containers rather than the ships?
What you’re witnessing is a self reported dataset from an international community (read as “unconcerned with The Protestant Work Ethic”) of ship captains and owners.
There are so many edge cases and oddities that the more you look the more you’ll find.
The moral of the story here is that you cannot accept AIS data, or any single data source, to be God’s Truth; you must always skeptical of the data before your eyes, lest you fall victim to “The Analyst’s Fallacy” (believing that the data presented is all the data there is) and create false stories in your mind.
There are so many edge cases and oddities that the more you look the more you’ll find.
The moral of the story here is that you cannot accept AIS data, or any single data source, to be God’s Truth; you must always skeptical of the data before your eyes, lest you fall victim to “The Analyst’s Fallacy” (believing that the data presented is all the data there is) and create false stories in your mind.
I would wager that the data is imperfect and the points are tweened to discrete samples rather than having a high detail point-to-point path. You'll probably see traveling over land more like cutting a corner. Just a hunch.
Canals. Figure they have probably fixed any wobbly data.
Some of the (likely) fixes look pretty weird, moving at a ridiculous speed, through land and sea in a straight line, only to reach a certain point and continue as if nothing had happened.
e.g. 8 June 2012 05:00 from western Sahara to English Channel, through Sahara, right the middle of Spain, parts of France.
Another on 9 May 2012 from Southeast Asia, straight through the south of India and right through the middle of Africa to somewhere on the Guinean Gulf in about half a day.
e.g. 8 June 2012 05:00 from western Sahara to English Channel, through Sahara, right the middle of Spain, parts of France.
Another on 9 May 2012 from Southeast Asia, straight through the south of India and right through the middle of Africa to somewhere on the Guinean Gulf in about half a day.
That sounds to me like those ships are missing transponder data for some reason, and the visualisation is extrapolating between to distant data points.
Not sure why this would be the case. Transponder turned off? Data missing for other reasons?
Not sure why this would be the case. Transponder turned off? Data missing for other reasons?
Quite possible.
I've never seen a perfectly clean dataset of any significant size. Big datasets have all sorts of weirdness unless there was an extraordinary effort to keep them pristine.
I've never seen a perfectly clean dataset of any significant size. Big datasets have all sorts of weirdness unless there was an extraordinary effort to keep them pristine.
[deleted]
Does anyone know where they get the data for these shipping map sites?
Modern vessels are required to have a AIS transponder. This is a radio that sends the ships speed, position, angle and optionally their destination around by marine (VHF) radio. These maps are probably made by having thousands of receivers recording the AIS data.
There are satellite networks that can pick up AIS broadcasts from almost anywhere on the planet. So you can track a ship in the middle of the ocean with a few minutes delay. Pretty amazing to imagine how many signals these satellites have to process at once and how weak they must be after making it all the way into space.
This site [1] allows you to get their real time global feed if you push your local feed in exchange. AFAIR equipment costs a few hundred bucks tops. I think other sites like [2] offer to provide a base station. Not sure what data and licensing you get in exchange though.
[1] https://www.aishub.net/
[2] https://www.marinetraffic.com/
[1] https://www.aishub.net/
[2] https://www.marinetraffic.com/
In this case, the underlying data is from exactEarth. It’s not free data. I believe exactEarth obtains the data by reading the pings from the ship’s transponders.
https://www.exactearth.com/product-exactearth-shipview
https://www.exactearth.com/product-exactearth-shipview
It’s free in the sense that there are projects where as long as you are operating an antenna feeding data, you get access to a feed of all data for free with the only restriction being you can’t compete by just restreaming the complete feed.
Oh that's interesting, thanks. So I guess it's similar to ADS-B exchange[1] and the like but for ships.
Have you operated an antenna, by chance? I've considered building an ADS-B receiver to satisfy my curiosity about the planes I see out my window all day.
[1] https://www.adsbexchange.com/
Have you operated an antenna, by chance? I've considered building an ADS-B receiver to satisfy my curiosity about the planes I see out my window all day.
[1] https://www.adsbexchange.com/
If you want to try the ADS-B receiver, you can do it for < $40. It's out of stock currently, but last year I bought the first one on this page https://www.rtl-sdr.com/buy-rtl-sdr-dvb-t-dongles/
I got busy with other things so haven't done much with it, but there are some Python libraries that made ADS-B easy.
I got busy with other things so haven't done much with it, but there are some Python libraries that made ADS-B easy.
There are pirates (thieves, but I like Pirate better?) whom were renting luxury boats in the Mediterranean, and stealing them.
They would fly to Europe, and dress like rich fancy boys.
They would then rent the ships for vacation.
They would take the transponder off the luxury vessel, and put it on a toy boat. The owner would check on his vessel, and it all looked fine.
Meanwhile, the thieves would have the boat in Russia in a few days.
(I believe they got 100 boats. This article reminded me of the caper.)
They would fly to Europe, and dress like rich fancy boys.
They would then rent the ships for vacation.
They would take the transponder off the luxury vessel, and put it on a toy boat. The owner would check on his vessel, and it all looked fine.
Meanwhile, the thieves would have the boat in Russia in a few days.
(I believe they got 100 boats. This article reminded me of the caper.)
Not going to lie but this sounds far more entertaining than sitting at my dining room all day making computer programs.
I lol'd. Sometimes when I read these kind of stories, including phishing and hacking ones, there's this tiny inner evil in me that wants to do bad things, but then I remember consequences and that being good usually pays off.
The Imp of the Perverse, of course.
Some very sketchy back of a napkin in my head math says that my fake email job[1] amounts to pulling off multiple boat heists every year without the intervention of the authorities (whether legitimate or—more problematic—illegitimate). I'm getting away with a lot here.
[1] https://twitter.com/Eve6/status/1341031553078448131?s=20
[1] https://twitter.com/Eve6/status/1341031553078448131?s=20
In a broader sense, they can be considered as hackers.
Making business harder for tourists and renting places is neither cool nor "hacking".
What? Don't some computer hackers also make business harder for regular users? Maybe you have a very different definition of hacker.
It just means maybe there is a market for boat rental pen-testing and security consultancies. There are probably lots of similar vulns in the rental business's process, which can be demonstrated by white hats and corrected. In this case, it was the business incorrectly trusting the thieves' story and foolishly delivering a product they had no way to repossess.
How do they sail into Russia? They would need their own AIS transponder and a valid ship registration, the Greek/Turkish maritime zone and the Black Sea has a tone of coast guard and military activity and you need to Lee-register/declare traversal.
Going into Russia through the northern sea doesn’t seem to be viable either.
Going into Russia through the northern sea doesn’t seem to be viable either.
In a past life I used to take (smaller) boats from Miami into the Bahamas and back. Everyone always warned me that the US coast guard would stop any boat entering US waters but I couldn’t tell you a single time it happened.
Grant it I wasn’t traveling on multi million dollar yachts, but sometimes what you expect to happen and what actually happens are two different things when it comes to enforcing laws.
Grant it I wasn’t traveling on multi million dollar yachts, but sometimes what you expect to happen and what actually happens are two different things when it comes to enforcing laws.
The Dardanelles are the narrowest straights you can’t just pass them, you need to book passage and you cant just sail into the Sea of Marmara with your AIS turned off, in fact you can’t enter Turkish waters with it turned off and you will get boarded.
The US has a much larger coast line and even then you usually have to communicate with someone if you don’t want to get a visit from the coast guard.
Also I haven’t seen a single luxury yacht you can rent without a crew, this isn’t any different than chartering a private jet you get a pilot, co-pilot and a flight attendant as part of the package. A yacht isn’t a small boat you need a proper captain, an engineer and a service crew.
A boat that can sail form anywhere in the med to Russia needs to be quite big and have quite a bit of fuel and supplies.
This doesn’t seem to be an operation that can be executed so simply you need to make sure there is enough fuel/supplies which may raise flags on shore when you book it, you need to find a way to deal with the crew, you need to be able to replace the AIS and have a valid ship registration to pass through the Dardanelles into the Black Sea.
While I’m sure an organized crime cartel might be able to pull this off but the resources and planning this job would require would be quite insane.
The US has a much larger coast line and even then you usually have to communicate with someone if you don’t want to get a visit from the coast guard.
Also I haven’t seen a single luxury yacht you can rent without a crew, this isn’t any different than chartering a private jet you get a pilot, co-pilot and a flight attendant as part of the package. A yacht isn’t a small boat you need a proper captain, an engineer and a service crew.
A boat that can sail form anywhere in the med to Russia needs to be quite big and have quite a bit of fuel and supplies.
This doesn’t seem to be an operation that can be executed so simply you need to make sure there is enough fuel/supplies which may raise flags on shore when you book it, you need to find a way to deal with the crew, you need to be able to replace the AIS and have a valid ship registration to pass through the Dardanelles into the Black Sea.
While I’m sure an organized crime cartel might be able to pull this off but the resources and planning this job would require would be quite insane.
Or they could just turn off AIS broadcasts?
Try doing that in the Aegean not to mention the Sea of Marmara / Dardanelles and lets see how that works out.
They might be able to sail into North Africa and then arrange for fake papers and ownership transfer but I would really like to know how they would sail into Russia without raising any alarms.
They might be able to sail into North Africa and then arrange for fake papers and ownership transfer but I would really like to know how they would sail into Russia without raising any alarms.
http://turkishnewstuhaf.blogspot.com/2019/07/slick-russian-p...
Be curious no more
Be curious no more
I would assume that all bets are off once they reach international waters?
The Russian coast of the Black Sea can only be accessed by passing through Turkish waters. And the other Russian coasts are relatively far away, challenging, and dangerous to sail to in a pleasure yacht.
To get through to Russia they need to pass territorial waters, you ain’t getting into the Black Sea or through the Aegean really with your AIS turned off.
Now it’s even harder due to the migrant crisis the waters are very heavily patrolled and anything bigger than a dingy will light up on radar.
Now it’s even harder due to the migrant crisis the waters are very heavily patrolled and anything bigger than a dingy will light up on radar.
Hmm, this seems almost like the perfect crime - easy to pull off, and difficult to get caught.
I wonder what protections can be made against this? Tamper-proofing the transponders might go some way, but a bullet-proof solution likely isn't possible.
I wonder what protections can be made against this? Tamper-proofing the transponders might go some way, but a bullet-proof solution likely isn't possible.
This is in large part due to the attacker having physical access of the asset. This complicates the security. Something like, how do you ensure the user is not running a modded app to send requests to the backend?
Deposits, ID checks, regular checks, mandatory security guards?
Deposits: would have to be a helluva deposit vs what you'd get for selling the boat
ID checks: presumably they must already do that - I mean, they check your ID for hiring a car/van. I wonder if the criminals used stolen ID to get around this?
Regular checks: on what?
Security guards: you mean when you hire a boat, a security guard comes along for the ride? I can't see many genuine clients being happy about that
ID checks: presumably they must already do that - I mean, they check your ID for hiring a car/van. I wonder if the criminals used stolen ID to get around this?
Regular checks: on what?
Security guards: you mean when you hire a boat, a security guard comes along for the ride? I can't see many genuine clients being happy about that
what did they do with the crews? most very high end yachts come with a dedicated crew
[deleted]
It would be interesting to see this with more recent data.
Ships taking the Northern Arctic route etc.
Ships taking the Northern Arctic route etc.
If you click to about the middle of May and wait a few seconds, you get a ship flying across the middle of Africa. I wonder what happened there :)
Saw that too, and across Europe in December.
Found the answer in the sites FAQ:
> Why do ships sometimes appear to move across land?
> In some cases this is because there are ships navigating via canals or rivers that aren’t visible on the map. Generally, though, this effect is an artefact of animating a ship between two recorded positions with missing data between, especially when the positions are separated by a narrow strip of land. We may develop the map to remove this effect in the future.
Found the answer in the sites FAQ:
> Why do ships sometimes appear to move across land?
> In some cases this is because there are ships navigating via canals or rivers that aren’t visible on the map. Generally, though, this effect is an artefact of animating a ship between two recorded positions with missing data between, especially when the positions are separated by a narrow strip of land. We may develop the map to remove this effect in the future.
I’d love it if I could fast forward to the Suez congestion
Or back to the 1967 - 74 Suez blockage.
[deleted]
Also happened to know about this site just today, thanks to the video by Johnny Harris (who covered a lot of geopolitical stuff recently). Awesome stuff.
cf. https://www.youtube.com/watch?v=9PbgYReEUO8
cf. https://www.youtube.com/watch?v=9PbgYReEUO8
Yeah that's where I found out about it too, was pretty surprised it was never posted on HN over the years.
It has been posted here before. I posted it when we first made it: https://news.ycombinator.com/item?id=11533539
Is there any economy/strategic game about delivering goods with cargo ships, containers, etc.? I would love to play it.
Theres a game called TransOcean: The Shipping Company on Steam: https://store.steampowered.com/app/289930/TransOcean_The_Shi...
Theres a sequel to it, although I've never played it. First one wasn't too bad, the economics wasn't too complicated, but worthy of playing.
Sequel: https://store.steampowered.com/app/350110/TransOcean_2_Rival...
Theres a sequel to it, although I've never played it. First one wasn't too bad, the economics wasn't too complicated, but worthy of playing.
Sequel: https://store.steampowered.com/app/350110/TransOcean_2_Rival...
https://store.steampowered.com/app/671440/Rise_of_Industry/
I found this to be a bit difficult though. I don't think there are necessarily boats of this, but it does go into basic business logistics
I found this to be a bit difficult though. I don't think there are necessarily boats of this, but it does go into basic business logistics
Offworld Trading Company is close to what you're looking for i think. But it's not ship-centric.
Good game overall, but terrible end-game. You're playing along, building and making money and no obvious mistakes, and then all of a sudden you get the big pink screen that says "Sorry, someone else made more money than you and decided to buy you. Game over. Better luck next time!"
there is a board game about exactly that:
https://www.boardgamegeek.com/boardgame/26990/container
https://www.boardgamegeek.com/boardgame/26990/container
[deleted]
The South China Sea looks like a traffic jam.
There's some interesting anomoly at the north east top bit of South America (both Colombia and Venezuela) where some really fast boats are regularly skipping across land.
Same thing seems to be happening to Great Lakes freighters crossing UP of Michigan over land.
Guess: Interpolation between actual GPS samples.
Great live / interactive data visualization.
Great voiceover narration.
Weird, weird choice of pondering, melancholy rainy-day ballerina-dance music. Is it a British thing?
Great voiceover narration.
Weird, weird choice of pondering, melancholy rainy-day ballerina-dance music. Is it a British thing?
I think the point is that we're supposed to feel sad about this economic activity because it's emitting tons of CO2.
I think so too. still seemed like only one of the points it was making there's a lot of other interesting information in there. Kind of weird that music but hey I guess everyone has got their editorial stance :)
Ha ha! I think this says more about Duncan’s music taste than anything else. I don’t think he was trying to make any sort of point: he just chose music he liked.
It’s lovely to see this here! I made it a few years ago; it was the last project we made under the Kiln banner before we devoted all our efforts to building a software platform (Flourish) instead.
If anyone has any questions, AMA!
If anyone has any questions, AMA!
Where does the data come from?
We were commissioned to make the map by the UCL Energy Institute, who had a research license from exactEarth for this historical AIS data.
If you press the little info button at the top right, you get some basic information, including this:
Where did you get the data and who paid?
Our data sources for shipping positions are exactEarth for AIS data (location/speed) and Clarksons Research UK World Fleet Register (static vessel information). We are very grateful to our funders, the European Climate Foundation.
If you press the little info button at the top right, you get some basic information, including this:
Where did you get the data and who paid?
Our data sources for shipping positions are exactEarth for AIS data (location/speed) and Clarksons Research UK World Fleet Register (static vessel information). We are very grateful to our funders, the European Climate Foundation.
If you go to the 12 of June 2012 you can see a ship making its way across the Sahara. I assume this is a data funny.
further discussion from 2016
https://news.ycombinator.com/item?id=11533539
https://news.ycombinator.com/item?id=11533539