Figuring out the best place to live in Helsinki(wanhala.net)
wanhala.net
Figuring out the best place to live in Helsinki
http://www.wanhala.net/post/156484186523/figuring-out-the-best-place-to-live-in-helsinki
49 comments
For those interested, there is a website that does this for a number of cities, mostly clustered in the US & Europe: http://www.mapnificent.net/
New url (moved off Azure as the quota was reached)
https://andersforsgren.github.io/commutemap/
https://andersforsgren.github.io/commutemap/
That's cool. Could I bother you for more information on how to do this? I'd like to try that for where I live.
Thanks
Thanks
I'll dig through the source tomorrow. Check back.
OK: I found the old source and I'll try to outline it (trust me it's a bit one-off since it's not usued in the production so it's easier to explain in english).
The most important bit is the public transit API: it allows returning travel times between addresses or geographic coordinates. For stockholm there is the trafiklab api which is excellent, and free. https://www.trafiklab.se/api/
A naive implementation would just generate the map by taking each map pixel, figuring out the lat/lon on the map, and calling the transit API to get the color of the pixel. That however will be painfully slow (a year?) since the API will throttle/limit the number of calls.
So a better approach is to sample some subset of points and interpolate. Most of the larger city area is not populated so a lot of time would be wasted trying to query transport times from places in water or forrests. The best solution I could come up with was to use a list of addresses, because those say where people live. So I needed a list of addresses either with lat/lon coordinates, or a service that could give me lat/lon from the addresses (Such as openstreetmap). I found a real estate api at Booli that provided a large number of addresses including lat/lon. Perfect. I just made a simple script to dump a large list of many thousand addresses to a little db I had.
So I loop all the addresses in my address list, using a timer to throttle the transit API calls to the allowed rate. The results (coordinate, transit time) I insert into a QuadTree for perf - a list would work just as well but finding the nearest point later would be slow as hell.
After that is done, I generate map tiles. For each pixel of each tile I get the lat/lon, and then do a lookup in the QuadTree for the closest known point within some max_walking_distance (I chose 2km) for which I know public transit (Taking the shortest travel time if there are several). This can now run completely offline so will complete generating maps for all the zoom levels in not many minutes (I chose zoom levels 5..14 which covers the use case nicely).
The app itself is then just a static html with some google maps api calls to serve the overlay map from the static tile images.
Note the demo moved: https://andersforsgren.github.io/commutemap/
The most important bit is the public transit API: it allows returning travel times between addresses or geographic coordinates. For stockholm there is the trafiklab api which is excellent, and free. https://www.trafiklab.se/api/
A naive implementation would just generate the map by taking each map pixel, figuring out the lat/lon on the map, and calling the transit API to get the color of the pixel. That however will be painfully slow (a year?) since the API will throttle/limit the number of calls.
So a better approach is to sample some subset of points and interpolate. Most of the larger city area is not populated so a lot of time would be wasted trying to query transport times from places in water or forrests. The best solution I could come up with was to use a list of addresses, because those say where people live. So I needed a list of addresses either with lat/lon coordinates, or a service that could give me lat/lon from the addresses (Such as openstreetmap). I found a real estate api at Booli that provided a large number of addresses including lat/lon. Perfect. I just made a simple script to dump a large list of many thousand addresses to a little db I had.
So I loop all the addresses in my address list, using a timer to throttle the transit API calls to the allowed rate. The results (coordinate, transit time) I insert into a QuadTree for perf - a list would work just as well but finding the nearest point later would be slow as hell.
After that is done, I generate map tiles. For each pixel of each tile I get the lat/lon, and then do a lookup in the QuadTree for the closest known point within some max_walking_distance (I chose 2km) for which I know public transit (Taking the shortest travel time if there are several). This can now run completely offline so will complete generating maps for all the zoom levels in not many minutes (I chose zoom levels 5..14 which covers the use case nicely).
The app itself is then just a static html with some google maps api calls to serve the overlay map from the static tile images.
Note the demo moved: https://andersforsgren.github.io/commutemap/
I had actually almost exactly the same problem that is why I created: http://crib.ninja
It allows to save apartments across different websites and automatically extracts the information (like rent, size, bedrooms, bathrooms, location, ...). The apartments can then also be displayed on a map with a kind of travel-time heatmap from https://www.route360.net.
Additionally is it possible to sort or filter by any of the properties and invite others to collaborate in real-time.
Disclaimer: Like written above I created it so I am obviously the founder
Disclaimer: Like written above I created it so I am obviously the founder
The heat map integration is nice, I wonder why real-estate listing websites don't provide something like this already.
Maybe there is a business opportunity for you there (to integrate this with some of the websites themselves that you support).
Agree that there is an opportunity but less for me than for Route360. The integration is quite simple and I am sure their sales people already talk with some of them. So there is not really a need for me anywhere there. Only if pages want to include data from other websites. So on crib.ninja are you for example able to also add your favorite restaurants from Yelp, the sights of your city from Wikipedia, ... and display all of them at once.
Would be nice to put that on the front page: Currently we concentrate on the US and Germany.
Just because that is what it currently concentrates on does not mean that it should not work perfectly fine for other countries ;-)
If somebody adds a page that does not work they can simply click on the (!) and inform us, we add then support asap.
If somebody adds a page that does not work they can simply click on the (!) and inform us, we add then support asap.
In this type of the task only payoff makes sense.
With 30B+ paths and additional f(x) of path and sum of paths, real life optimization is not really possible - I'll bet in the middle of the night that some simple heuristics (e.g. proximity to one of the major transport hubs) will perform much much better. And city prices are likely already reflecting this, contrary to what author claims.
That's considering commute time is somehow directly related to price, which isn't true.
Your commute depends on your job, where the price depends on safety, size, shops in the area, noise pollution, air pollution, parc proximity etc...
Connection is one factor but you can't just say "it's expensive, therefore faster to go to work"
Your commute depends on your job, where the price depends on safety, size, shops in the area, noise pollution, air pollution, parc proximity etc...
Connection is one factor but you can't just say "it's expensive, therefore faster to go to work"
Nice.
Nitpick: Rainbow (jet) color maps can be confusing. Better to use a perceptually uniform one such as viridis, see e.g. https://bids.github.io/colormap/
Nitpick: Rainbow (jet) color maps can be confusing. Better to use a perceptually uniform one such as viridis, see e.g. https://bids.github.io/colormap/
And if you don't like the green, you can use different flavors of cubehelix family color spaces[0].
[0] https://jiffyclub.github.io/palettable/cubehelix/
[1] http://www.ifweassume.com/2013/05/cubehelix-or-how-i-learned...
[0] https://jiffyclub.github.io/palettable/cubehelix/
[1] http://www.ifweassume.com/2013/05/cubehelix-or-how-i-learned...
Nice work, is the source available? I'd love to do something similar for Sydney, Australia.
Unfortunately not. I created it with various programs I've written for my data visualisation videos (https://www.youtube.com/user/laurivanhala )
Not at the moment, https://github.com/lvanhala?tab=repositories doesn't list it.
This reminds me of the "Space Syntax" school of urban planning and architecture (which basically adopts a computational/graph-theoretic/topological approach by computing the simplicity of paths between various points and all other points).
https://en.m.wikipedia.org/wiki/Space_syntax
https://en.m.wikipedia.org/wiki/Space_syntax
Genius. My only comment is in how you measure the value of an address. It seems very likely that your algorithm would converge to areas that are only centrally located.
Perhaps it could be improved if the algorithm took an input of common routes and times, then tried to find an optimal location for these routes. This way the algorithm could be scaled as needed and provide a more realistic scenario. Is this something you considered?
Perhaps it could be improved if the algorithm took an input of common routes and times, then tried to find an optimal location for these routes. This way the algorithm could be scaled as needed and provide a more realistic scenario. Is this something you considered?
Isn't his algorithm pretty bluntly designed to do this?
So I did my own analysis: I calculated the travel time from every address to every other address in Helsinki around 7:30-8:00am (about 30 billion searches total!). Then I calculated the (weighted) average travel time to anywhere in the city, using amount of jobs in the target area as weight.
That would seemingly bias towards centrally-located addresses (travel time & number of jobs), and his heat maps seem to show this. I believe you could pretty easily duplicate what he's doing with a few dozen randomly sampled routes. Or is there more to it?
So I did my own analysis: I calculated the travel time from every address to every other address in Helsinki around 7:30-8:00am (about 30 billion searches total!). Then I calculated the (weighted) average travel time to anywhere in the city, using amount of jobs in the target area as weight.
That would seemingly bias towards centrally-located addresses (travel time & number of jobs), and his heat maps seem to show this. I believe you could pretty easily duplicate what he's doing with a few dozen randomly sampled routes. Or is there more to it?
My thought was to make the algorithm a bit less blunt by sampling routes that are relevant to your lifestyle.
Just might spice it up if you partition the bucket of all routes into spaces defined by your lifestyle too. It would be interesting to see what the best address in Helsinki is for hipsters. Or families. Or athletes. Or business people...
This is so cool because anyone who has moved has faced this problem.
Just might spice it up if you partition the bucket of all routes into spaces defined by your lifestyle too. It would be interesting to see what the best address in Helsinki is for hipsters. Or families. Or athletes. Or business people...
This is so cool because anyone who has moved has faced this problem.
[deleted]
"My only comment is in how you measure the value of an address."
This. Having lived in the center (reddish area) and later also in southern Helsinki (bluish green), I strongly prefer the latter due to it's proximity to the sea and as it's way quieter area without busses, trams or general traffic. And judging by housing prices so do many others who live in the city.
Don't get me wrong. I found the map highly interesting but maybe in determining best places to live it's a bit of a stretch. There might more value here for businesses that aim to be easily reachable.
Don't get me wrong. I found the map highly interesting but maybe in determining best places to live it's a bit of a stretch. There might more value here for businesses that aim to be easily reachable.
From what I read there I think he assumes he is always visiting his homebase in between. But when he travels in one day from client A directly to client B (and even C) without visiting his homebase the stuff get's much more complex and results may vary. If you want to know more do research about the "traveling salesman problem". TL;DR: It's not so easy.
If you want to know more do research about the "traveling salesman problem". TL;DR: It's not so easy.
It's not easy to optimize the travelling salesman problem, but if you're happy to brute force it using 30B searches it's incredibly straightforward.
It's not easy to optimize the travelling salesman problem, but if you're happy to brute force it using 30B searches it's incredibly straightforward.
TSP is actually pretty easy as NP-hard problems go, because it's important in practice (chip layout!) and hence gets considerable attention. Instances with millions of cities can be solved nearly to optimality.
Could be that he only works by the day, so going from client A to client B will be rare.
Disregarding changes in travel time due to the different time of say, we know that A -> Origin + Origin -> B is an upper bound on A -> B so the solution is still good even if it's not quite optimal.
Disregarding changes in travel time due to the different time of say, we know that A -> Origin + Origin -> B is an upper bound on A -> B so the solution is still good even if it's not quite optimal.
Yes you're right. It all depends on amount of clients he visits in one day, if they are flexible in time or not and how to get from client A to client B (or C).
[deleted]
For a small number of clients tsp isn't so bad. The best known exact algorithm is n^2*2^n. And absolutely trivial for the approximate solution algorithms.
usually development contract jobs are for weeks/months so i think usually you would go only to one client per day.
>However, I have never seen any data on how well the public transit works in the different parts of the city.
Have you seen http://mak.hsl.fi/? It's kind of that, though you do have to specify the starting/end point.
Have you seen http://mak.hsl.fi/? It's kind of that, though you do have to specify the starting/end point.
Did you do it for /each address/ or each building? It seems like comparing each building (or even block?) to the rest would be faster in that case.
How long did it all take?
How long did it all take?
I did it for each street address number. Definitely an overkill but it only took a night or two (didn't check) so I didn't bother to optimize. :)
Does this data weigh the importance of certain locations? Because to me it seems that it doesn't which would seriously harm the informativeness of the set.
As a consultant you surely don't travel to every part of the city as frequently, and maybe the expensive neighbourhoods are rightfully expensive because that is the place where those who hire consultants live.
Nonetheless a really cool visualisation/idea!
As a consultant you surely don't travel to every part of the city as frequently, and maybe the expensive neighbourhoods are rightfully expensive because that is the place where those who hire consultants live.
Nonetheless a really cool visualisation/idea!
30B searches looks like an overkill... should find a way of creating a graph and use https://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorit... or something on that.
the end result is nice though.
the end result is nice though.
Just in case anyone missed it http://www.telegraph.co.uk/content/dam/Travel/2016/December/...
Really interesting. In the end, how did the results compare with the locations of established client firms, e.g. kpmg, accenture, pwc, e and y, deloitte, bcg, mckinsey, bain?
The lake by the "My Summer Car". :p
if your objective is to optimize the best place to buy a house, shouldn't you account about how the travel times of these 30B routes change over time?
It looks like a palm touch.
In this case I simply took a large number of addresses, ran them through the API for public transit for a random fixed workday (A tuesday morning 8 am commute) to a fixed central address. Then I plotted the times in a heatmap in the format Google maps uses and made a map overlay with the heat map.
It makes a nice spiral galaxy like map where the commuter train stations make little islands of short commute time far away from the city.
http://commutemap.azurewebsites.net/
(Static pic if you don't want to zoom in it and eat my free azure bandwidth... http://prnt.sc/e57bsc). Thinking about it now the dang thing ended up being completely static so it should be possible to just host for free somewhere (github?) rather than cloud. Uses my msdn "testing" sub now...
Outcome: bought place in the orange/red area and started working from home instead :)
Limitations: 1) target address is fixed, new target (e.g. new job location) needs a whole new map. But central stockholm is pretty small and walkable so this works pretty well if your office is central. 2) Interpolation between known points/addresses uses no kind of path finding. It assumes you can walk e.g. 200m in a straight line from an address to the closest bus stop. That might not be what you want to do if it e.g. means crossing water...