How Hyper built a 1M-accurate indoor GPS(andrewhart.me)
andrewhart.me
How Hyper built a 1M-accurate indoor GPS
https://andrewhart.me/hyper/
74 comments
Having used this and other localization techniques for large indoor robot fleets over the past decade, the hard part seems to be reliability. It was relatively easy to get a sensor fusion solution to work but much harder to get it to work consistently.
Drift is one issue but the other gremlins also cause problems with repeatability: initial localization errors (wrong aisle), electromagnetic environment change (wifi strength is not static), physical environment change (if using optical or lidar).
I think this page is a nice intro to the wifi localization space and the website has an exciting tech startup vibe. But I think if you want to really capture the undivided attention of my peers: find a clear, compelling way to demonstrate that it’s robust and repeatable. For example, walk the same real-world space twice a day for a month, stitch together the tracks in a time lapse, and promise you haven’t cherry picked the results. Especially important is demonstrating this in areas with long, repetitive aisles (grocery stores, hardware stores, warehouses, etc.) That would go far! Good luck on being acquired!
Drift is one issue but the other gremlins also cause problems with repeatability: initial localization errors (wrong aisle), electromagnetic environment change (wifi strength is not static), physical environment change (if using optical or lidar).
I think this page is a nice intro to the wifi localization space and the website has an exciting tech startup vibe. But I think if you want to really capture the undivided attention of my peers: find a clear, compelling way to demonstrate that it’s robust and repeatable. For example, walk the same real-world space twice a day for a month, stitch together the tracks in a time lapse, and promise you haven’t cherry picked the results. Especially important is demonstrating this in areas with long, repetitive aisles (grocery stores, hardware stores, warehouses, etc.) That would go far! Good luck on being acquired!
Whoa, 1m is really impressive! I've played around with stuff like https://github.com/dmsl/anyplace for https://www.emfcamp.org/ back in the day, but that was around 5m at best. Bet you a robotics startup will scoop it up!
Thanks. Very interested in robotics so that'd be an ideal home, given the SLAM + sensors tech stack.
How fast do the WiFi signal strength maps get out of date?
Just someone changing the angle of an antenna or shifting a pile of stock near the router has to have a pretty big impact on signal strength.
And obviously a WiFi system upgrade where all the Mac addresses change must be a fairly big change and effectively gives a full service outage for all the users till you remap.
Just someone changing the angle of an antenna or shifting a pile of stock near the router has to have a pretty big impact on signal strength.
And obviously a WiFi system upgrade where all the Mac addresses change must be a fairly big change and effectively gives a full service outage for all the users till you remap.
We've had the solution live in stores for years without any issue. Our algorithms can clean up and filter out unreliable data, based on all of the other data we have, from other APs, from previous data etc..
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
If they re-fit the WiFi, which they might spend a month doing once every 5-10 years, it would need a re-survey, which could take 1-2 hours. But thankfully no new infrastructure.
How does your system handle the massive variance in sensor quality (accelerometer, gyro, WiFi radio) between a high-end iPhone and a budget Android device? Does the 1m accuracy hold up across the board, or does it degrade gracefully? Getting this right seems critical for scaling to a 'billion people'
All of the videos shown on the speed-run of our technology is on 4 year-old Android devices, such as Samsung S21, Pixel 6 etc.. We always test and gather statistics on older devices to fairly represent what's available, rather than latest-and-greatest.
So, devices that were top of the line a few years ago. But what about budget devices? in the 100-200 range new?
I remember my old xiaomi started literally running in circles as the phone heated up 10-15 minutes after starting navigation: If i stood still the position would move in a circle of a few meters of radius.
Incidentally, the devices you metion are what i also use to develop, because those line of products actually behave as they should, per documentation. But most bugs and crashes always come from budget and no name devices because both the hardware and firmware is crap
Incidentally, the devices you metion are what i also use to develop, because those line of products actually behave as they should, per documentation. But most bugs and crashes always come from budget and no name devices because both the hardware and firmware is crap
Peripheral chips aren't differentiated by user values like the end products that use them. You don't get more preciser sensors in high end phones. Everyone gets the same thing. You pay more only for more materials.
Sensors that are actually a lot better than standard offerings would also be subject to and/ofs of ITAR or EAR or MTCR or local equivalents thereof, so progress in IMU appears to have been stagnating a bit due to that issue. Sony Semiconductor Solutions had a Arduino IDE compatible clustered IMU board that they say you can see rotation of Earth in data, they ended up selling it with scary warnings and without any of the cool stuffs.
Sensors that are actually a lot better than standard offerings would also be subject to and/ofs of ITAR or EAR or MTCR or local equivalents thereof, so progress in IMU appears to have been stagnating a bit due to that issue. Sony Semiconductor Solutions had a Arduino IDE compatible clustered IMU board that they say you can see rotation of Earth in data, they ended up selling it with scary warnings and without any of the cool stuffs.
There's a fair bit of quality difference between different chips and better chips have gotten cheaper. More importantly, default filtering quality has improved with more powerful uCs on the IMU package, which is what most cheap phone vendors are probably using.
The ITAR stuff is way more fun though. It's great to read between the lines for the intended customer in the datasheet.
The ITAR stuff is way more fun though. It's great to read between the lines for the intended customer in the datasheet.
This is the correct answer. They're all the same. The notion that Apple has some kind of edge here is farcical.
There's something called a Kalman filter:
https://en.wikipedia.org/wiki/Kalman_filter
It can combine several inaccurate sources and output a result that is more accurate than any one of them.
I was at an Amazon Fresh grocery store, and saw squares in the ceiling that look like QR codes. I guess that's how they are mapping the store.
https://en.wikipedia.org/wiki/Kalman_filter
It can combine several inaccurate sources and output a result that is more accurate than any one of them.
I was at an Amazon Fresh grocery store, and saw squares in the ceiling that look like QR codes. I guess that's how they are mapping the store.
Awesome breakdown of the challenges of indoor navigation! One thing I was curious about... given that many modern phones now have UWB radios built in, was UWB ever considered as part of the solution stack? From what I understand UWB can get to sub meter accuracy, and I know it's used in several sports applications where precise tracking is really critical.
Is the constraint more about infrastructure (installing anchors, device compatibility, power) or something else that made you lean towards WiFi + SLAM fusion instead?
Is the constraint more about infrastructure (installing anchors, device compatibility, power) or something else that made you lean towards WiFi + SLAM fusion instead?
Great question. Partly on device compatibility, but mostly that we can achieve an amazing, reliable experience right now without those anchors. If we weren't getting the accuracy we needed from our current methods, we would have to look at alternatives like UWB and consider the trade-offs.
This is a very cool project / startup! I'm curious, how do you get the ground truth data? Is that just you marking down where you are as you walk through the store?
Thank you! I tried to keep it interesting but not get lost in rabbit holes.
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
We have around 300 hours of ground-truth data now, in 1-second intervals, which we use for algorithm training and refinement. The same as performing a survey, our team marks their location on the map, then walks to the next location, and through post-processing we can correct any errors and interpolate the locations in between. You can see this process in the diagram with the large black dots, where the user marked their location.
Huge respect to anybody who ships a product, even software. I have yet to use any indoor navigation or even have that offered, but it would be nice. Question is, is SLAM the correct term? I thought mapping meant like the device doesn't have any prior knowledge of the environment and is, well, mapping it. This more feels like sensor fusion with a kalman filter.
The motion tracking system used by ARKit/ARCore is called SLAM, combination of sensors + camera data if available.
Sure, but my point was that just because something uses the same inputs and gets a similar output doesn't mean the processing is the same. This is just localization. Which is still impressive. But I'm not understanding there being any mapping going on.
Love reading about these types of R&D efforts, especially when they are successful. I started in robotics in 01 and was present at some of the first commercial vSLAM efforts and materially contributed to a hardware solution utilizing IR beacons on the ceiling. We also looked at RF radio mapping at the time, but the computational power wasn't there. Great to see how far it has come and LOVE that it doesn't require infrastructure!
I am curious how deeply you have had to study the impact of how busy the store is with your signal error? Considering that humans are bags of water which is quite detrimental to RF signals, my guess is that your error increases along with the density of people.
I am curious how deeply you have had to study the impact of how busy the store is with your signal error? Considering that humans are bags of water which is quite detrimental to RF signals, my guess is that your error increases along with the density of people.
This is very, very impressive. I have three questions. How do you know where you are within the store? Do you manually upload the floorplan? How does it know where all the items in the store are?
Very, very impressive, if (intentionally?) a little vague on how you got to 1m accuracy given you WiFi only gives you 3m.
I guess you must use the constraints on the directions a person can walk imposed by the shelves and other structures to give you orientation of the accelerometers. Which in turn means the person doing the ground truth mapping must walk down every aisle, and into every gap. That's not so difficult if your staff are doing it, but I bet you have trouble training the store staff gathering that data to do it well.
Best of British.
I guess you must use the constraints on the directions a person can walk imposed by the shelves and other structures to give you orientation of the accelerometers. Which in turn means the person doing the ground truth mapping must walk down every aisle, and into every gap. That's not so difficult if your staff are doing it, but I bet you have trouble training the store staff gathering that data to do it well.
Best of British.
Thanks for the feedback. I wanted to keep it balanced to be accessible but also insightful.
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
To answer your point: we have the digital map, can use that to understand obstacles etc in the space. In some of those larger stores you see in the visual, we typically survey the entire store within 2-3 hours, it's low-effort work, not a blocker.
The few examples they show do look pretty good for a wifi-based method, although who knows how cherry-picked they are. I wonder how much the "SLAM" part is contributing and how sensitive that is to the sensor quality on the phone. I would've assumed that they'd be using vision, which seems to be the method of choice for other companies like niantic. The ground-truth data part for vision would certainly be more onerous, though.
He explains it fairly well if you understand how you'd go from wifi accuracy to SLAM. THE WIFI was providing 3m accuracy and the SLAM down to 1M. how much it provides is those two numbers. I'm sure the algorithms are complex but he points out that SLAM is corrected by the actual maps made by the self service app. So it's fairly easy to understand: the map provides a probability space, the wifi puts you within 3m and the SLAM is use to fill in the blanks with help from the probability space.
Really interesting. Feels like it would be a natural addition to a company like Brain (https://www.braincorp.com/) who is already using robots to regularly perform those kinds of survey missions and have an overlapping customer base.
Congrats on the launch, that looks dope! I'm curious, will this be able to run on an embedded robotics hardware?
Yes - one of the limitations with mobile is engaging the camera/quality of the SLAM. With a robot, they're already using SLAM with strong tracking, and controlling the hardware stack means no device limitations (uncalibrated sensors, limited WiFi pings etc.)
In my view, you need to 'streetview' the world's shopping malls and make this a free app, the Hyper Browser. Add product search & reviews and drive traffic away from non-customer stores.
Do you think the basic physics and sensor tolerances would let you go to 10^-2 meters if the environment (e.g. wifi station placements, location of RF-interfering elements) was designed by you?
If you want high performance indoor positioning , look into 5G carrier phase positioning which claims cm-level accuracy. Fuse that with IMU and optical sensing and you certainly should get decent results
I've personally used one of these before in a performance where you needed individual based tracking, the chips were active UWB radios and sensors were placed and calibrated around the stage, I believe it was < 10cm accuracy and quite an interesting sensation to walk with lights following you perfectly and quickly.
https://kinexon.com/products/kinexon-rtls was what was used.
https://kinexon.com/products/kinexon-rtls was what was used.
So, a "Grocery Positioning System"?
I wonder how much better this does than CCC's attempts at indoor navigation for their conferences. Those are usually pretty fun to read about.
I wonder how much better this does than CCC's attempts at indoor navigation for their conferences. Those are usually pretty fun to read about.
FYI only one of your videos is displaying in HDR on your homepage, making the others seem dim in comparison.
It’s the Built for simple campus navigation video.
I’d recommend converting it to SDR.
It’s the Built for simple campus navigation video.
I’d recommend converting it to SDR.
This looks really cool. Can you also handle multi floor plans? I think that indoor multi floor plans is the most difficult challenge in this area…
Why do you need a human to do the initial mapping ? why can't you use a Roomba or a smaller hardware for this task
Many times these stores have floor cleaning machines - either robotic or driven by a human. An employee could zip-tie their sensor to it, let it do its cleaning trip around the store, and return to collect the data later.
This would allow an employee to do several stores in a town in a single day. And potentially less chance of a workers-compensation claim being filed if they fall down while walking around looking at their device.
This would allow an employee to do several stores in a town in a single day. And potentially less chance of a workers-compensation claim being filed if they fall down while walking around looking at their device.
It's been said in one of the comments that the initial mapping by a human takes like 2-3 hours. Knowing the speed of a Roomba, I guess that it would take much more time to do it. And 'humans' are COTS devices available on any department store (sorry ;P).
IIUC, the black dots have to be mapped by hand. I.e. the human has to drop pins on the map. A Roomba doesn't know where it is a-priori
If indoor is so inaccurate, how does Apple find AirTags then?
Apple use UWB[0] for nearby location sensing.
[0] https://en.wikipedia.org/wiki/Apple_silicon#Apple_U1
[0] https://en.wikipedia.org/wiki/Apple_silicon#Apple_U1
My understanding is they use bluetooth for coarse positioning (e.g. you're within 30 feet) then they use a special chip (ultra-wide band) for the precision location (within a foot or two).
Of course, there's a good bit of magic within all of that to make it work seamlessly.
Of course, there's a good bit of magic within all of that to make it work seamlessly.
Sounds to me like it is not a difficult problem except for the fact that they want to make it work with existing smartphones (both iOS and Android).
when I read "1m-accurate indoor GPS", my first gut reaction was along the lines of "so what? Nowadays outdoor differential/rover GPS/GNSS has <2cm precision with RTK+dual band receivers, even for civil use… and indoor allows for yet way better parameter control than outdoors, so it should be possible to achieve at least that precision" (besides, it would allow for robots to get a sufficiently precise location for indoor navigation and to shelve wares)
The thing is: that would require additional hardware though, whereas the relevant limiting criterion of the project presented in the article is obviously that it has to run with just the hardware provided by common mobile phones. Which is what makes the project interesting despite the comparatively lower precision.
The thing is: that would require additional hardware though, whereas the relevant limiting criterion of the project presented in the article is obviously that it has to run with just the hardware provided by common mobile phones. Which is what makes the project interesting despite the comparatively lower precision.
Honestly, I've wanted a system for a while where shops can provide a map, and I can search for an item and it will show me where in the shop that item is. I don't think I've ever been in a shop big enough where I'd need satnav to tell me how to get there though, is that an American thing?
Tesco in the UK literally has this but only for staff. If you work for Tesco you can access any shop, view a map, view where stock is on a shelf, check stock numbers and expected delivery and all that stuff.
Things that would absolutely be an amazing QOL improvement for any shopper. But they won't let you have it because they WANT you to bumble your way around a shop. They don't want you to know where things are. That's why they move shelves around seemingly at random.
Things that would absolutely be an amazing QOL improvement for any shopper. But they won't let you have it because they WANT you to bumble your way around a shop. They don't want you to know where things are. That's why they move shelves around seemingly at random.
I was going to shout out Tesco specifically, for a similar feature they have on their website/app in Ireland/UK.
If you find a product's page, there's an "other items on this shelf" section which shows items that are located next to it. It's probably not intended to help you locate things, but it's incredibly helpful for it.
Adds way more things for you to notice and tell you're in the right location. Same haystack, more needles.
If you find a product's page, there's an "other items on this shelf" section which shows items that are located next to it. It's probably not intended to help you locate things, but it's incredibly helpful for it.
Adds way more things for you to notice and tell you're in the right location. Same haystack, more needles.
I actually find this feature of supermarkets quite useful. Online shopping is far less discoverable - the end result is I forget things from my online shop quite often.
I find exceptionally annoying when I don't know if I need to look for my preferred moisturizer in "skincare" or "premium skincare" or some other section I've not seen yet (yes this is a common issue for me). If I could just load up their (horribly slow and memory heavy) website, go to the store locator and actually see where it is in the shop instead of "yeah we have it in stock, somewhere", that would be very useful
For me the end result is I buy random crap from the shop that I don't need. Sometimes it's good because it's new and I wanna try it, but sometimes it's just me being a pig.
I don't forget anything because I have a list.
I don't forget anything because I have a list.
Home Depot, Lowe's, Menards, and Wal-Mart are examples off the top of my head that have this product locator functionality on their websites.
I used this at Menards two days ago. The product location told me where the item was. Turned out that aisle g94 wasn’t aisle 94 but a kiosk at the end of aisle 27 on the other side of the store (these numbers are made up I don’t recall the specifics). I still had to ask a human where it was. So yeah not there yet and this type of service could really help.
How could one calculate the GPS of the floors?
Peoples at x, y (lat, lng) ain't the same when placed them in multiple floors?
Peoples at x, y (lat, lng) ain't the same when placed them in multiple floors?
does this require a camera to work at that 1m accuracy?
> They wanted to bring indoor maps and navigation to their retail stores... It turns out that this doesn’t just apply to retail. Every office, university campus, events venue, hotel, airport, warehouse, factory — basically everywhere indoors have some need to navigate people around, provide relevant information, and improve efficiency.
You'd think they would add this information to openstreetmap then or at least put a map on their website (and put it in the public domain so others like OSM can add it to their maps). Or put it in the store so people can take a picture. I go into target and there are posters saying to install an app for maps. Put the map on the poster!
> and they could pop up relevant promotions along the way
Oh, right, they don't want to provide information. They want to track people and spam them.
You'd think they would add this information to openstreetmap then or at least put a map on their website (and put it in the public domain so others like OSM can add it to their maps). Or put it in the store so people can take a picture. I go into target and there are posters saying to install an app for maps. Put the map on the poster!
> and they could pop up relevant promotions along the way
Oh, right, they don't want to provide information. They want to track people and spam them.
I was with them until they said that it's going to give me ads while I'm walking around
You might think shoppers finding their product in a store quickly would delight customers and pay for itself quickly.
But it seems instead of stores simply depending on the sale, they also now demand impulse purchases, which mean they want you wandering the store looking in multiple places for your quarry: the casino model. So if they delight a customer with direct route to the sale, they need to make up that windfall elsewhere?S
So they fall back on surveil, profile, and market plus selling your profile to others? Is this is why we can't have nice things?
But it seems instead of stores simply depending on the sale, they also now demand impulse purchases, which mean they want you wandering the store looking in multiple places for your quarry: the casino model. So if they delight a customer with direct route to the sale, they need to make up that windfall elsewhere?S
So they fall back on surveil, profile, and market plus selling your profile to others? Is this is why we can't have nice things?
Major grocery shops routinely swap their profitable items with the popular items. They do this to stop the customer going into auto-pilot and instead forcing them into actually looking for what they want. So no, shoppers finding their product does not pay for itself.
Little bit of a tangent, but as a customer I am not "delighted" when I find a product quickly and easily; I am merely not frustrated. Finding what I'm looking for is the baseline experience, having to search for what I'm looking for makes me annoyed and less likely to buy anything other than what I need so I can just get out of there.
In my experience, any product or service advertising itself as "delighting" customers actually means that they're overall making the baseline experience worse, and their product/service is just reducing the frustration they're introducing.
In my experience, any product or service advertising itself as "delighting" customers actually means that they're overall making the baseline experience worse, and their product/service is just reducing the frustration they're introducing.
It depends on the segment. All of the hardware stores near me have websites that list the exact aisle and bay product is in. They've seemed to figure out it's a competitive disadvantage to make their customers wander.
It's not entirely clear to me if grocers and other retail will end up taking the same route. Grocery service is increasingly move to hands-off (pickup or delivery) and other segments seem to be moving heavily on-line (including gig-delivery). It seems like they'll continue to punish foot traffic while encouraging customers to do online or hands-off buying.
It's not entirely clear to me if grocers and other retail will end up taking the same route. Grocery service is increasingly move to hands-off (pickup or delivery) and other segments seem to be moving heavily on-line (including gig-delivery). It seems like they'll continue to punish foot traffic while encouraging customers to do online or hands-off buying.
Finding a product quickly is actually the opposite of what a store owner would want, because it means you are spending less time on looking at the other products.
Yes I hate that attitude too.
Yes I hate that attitude too.
This is exactly why we can't have nice things. There has been a lot of study in how to manipulate shoppers to get them to buy more things in your shops, from playing in-store music that is slower so they walk slower, to putting expensive stuff at eye-level, to putting common things at the back of the shop so you have to walk past everything else to get to them
The dream of augmented reality was brilliant until the obvious consequences were recognized
If you had read the article you would see:
Yes, they mention promotions, but again the promotions would be opt-in – if I use their app to find a product I'm looking for, they might suggest other products along the way that I might also find useful, or they might take me in a route that passes right by them. This is no different to the way retailers stock up their shelves already, placing products next to others you might want, and moving necessity items around when they want to direct you to another part of the store.
I don't know, I think it's a bit harsh to criticise this when the technology has so many applications outside of retail. I would love this in a museum or library, and even in retail I absolutely hate those interactive map displays that modern shopping malls have, where only one person can use them at a time and you have to navigate through 200 store names for the one you actually want to visit
1/ they need to collect ground truth data for their algorithm to work, it doesn't magically work everywhere.
2/ the ground truth data was collected mostly by their clients, it is not their data to give away for free
Yes, they mention promotions, but again the promotions would be opt-in – if I use their app to find a product I'm looking for, they might suggest other products along the way that I might also find useful, or they might take me in a route that passes right by them. This is no different to the way retailers stock up their shelves already, placing products next to others you might want, and moving necessity items around when they want to direct you to another part of the store.
I don't know, I think it's a bit harsh to criticise this when the technology has so many applications outside of retail. I would love this in a museum or library, and even in retail I absolutely hate those interactive map displays that modern shopping malls have, where only one person can use them at a time and you have to navigate through 200 store names for the one you actually want to visit
The criticism was directed at retailers. If they want to provide indoor maps... why not just do that? For my Target example, there's even a convenient place to put them in store: the posters that say to download an app to see a map. There's also a standard place where they can add their indoor maps for free without needing anyone's permission (openstreetmap). Or put them online with a public domain disclaimer and someone else will eventually probably do it.
Edit: In Target's case, they do apparently also put it on their website if you go hunting for it, but the ubiquitous pushing of apps is still annoying vs just putting it right there in the store as well, and perhaps offering a QR code + text link to the online version. They're clearly using it as bait to install their tracking/ads trojan. Also their online map for my store is east-west inverted for some reason (the east end of the building is on the left, the north on top), which would be immediately obvious if they mapped it to their building in OSM.
Edit: In Target's case, they do apparently also put it on their website if you go hunting for it, but the ubiquitous pushing of apps is still annoying vs just putting it right there in the store as well, and perhaps offering a QR code + text link to the online version. They're clearly using it as bait to install their tracking/ads trojan. Also their online map for my store is east-west inverted for some reason (the east end of the building is on the left, the north on top), which would be immediately obvious if they mapped it to their building in OSM.
> If you had read the article you would see:
(1) It's clear from the use of quotes that the person you're replying to did read the article.
(2) from the official HN Guidelines[0]: Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".
[0]: https://news.ycombinator.com/newsguidelines.html
(1) It's clear from the use of quotes that the person you're replying to did read the article.
(2) from the official HN Guidelines[0]: Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".
[0]: https://news.ycombinator.com/newsguidelines.html
I go on the home depot website, pick my store, and find the product.
It tells me how many are in stock, aisle number, and bay number. No need for an app or walking advertisements.
It tells me how many are in stock, aisle number, and bay number. No need for an app or walking advertisements.
So they're not using GPS at all. Got it.
"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
I think an acquisition would be unfortunate. This could become really huge / useful to the world without being locked up as a private company's IP. Personally I would license it rather than sell it, as well as offering offline apps and limited SaaS. You don't need an established enterprise to sell it commercially; you just need a sexy product and some industry vets with contacts. If you do end up going that way, and need someone in IT Ops / DevOps / SysEng stuff to work on the "going enterprise with a billion users", give me a shout.