Learning Car Speed Using Inertial Sensors(arxiv.org)
arxiv.org
Learning Car Speed Using Inertial Sensors
https://arxiv.org/abs/2205.07883
44 comments
Any location estimator worth its salt is already doing this through the use of a multi-rate EKF. On top of that, it takes into account the vehicle dynamics (i.e. cars can't move perfectly horizontally) to improve estimates.
The "novelty" here is the ML approach although I am not sure if that is particularly novel as well.
The "novelty" here is the ML approach although I am not sure if that is particularly novel as well.
Exactly. I find it exceedingly odd that well-understood techniques like double integration and Kalman Filtering are being replaced by ML black boxes in the hope that ... what? They're going to rediscover what we already know? But in a format where we have no hope of figuring out what they discovered, so we can never bound their domains of acceptable performance?
Sometimes there are other regularities in the data that an ML algorithm can exploit; for example, if it can detect feet strike events in the IMU stream, it can compute an estimate of how fast someone is walking that is independent of integrating accelerations.
The benefit of ML over long-standing algos like kalman filtering for integrated navigation systems is they can can give you the occasional surprise when an unexplored edge case happens. People like surprises.
They can also give you the occasional surprise by diverging from the real trajectory, ex. trains doing barrel rolls
I've used ML to supplement straightforward heuristics before.
For instance, I built a fuzzy matcher for business addresses which was based on hand-rolled heuristics but used logistic regression to train a probability estimator that the match was correct and then hand-evaluated a few thousand matches so that the matcher knew the quality of results it was returning.
For instance, I built a fuzzy matcher for business addresses which was based on hand-rolled heuristics but used logistic regression to train a probability estimator that the match was correct and then hand-evaluated a few thousand matches so that the matcher knew the quality of results it was returning.
I am surprised that for dead-reckoning of an automobile position, one would not just use a something akin to a "mouse".
Why not compute optical flow from a camera on the underside of the car? Surely that would be more sensitive than an IMU?
Why not compute optical flow from a camera on the underside of the car? Surely that would be more sensitive than an IMU?
Factory systems almost always use the vehicle sensors in addition to GNSS, some early systems only had this ability since GPS wasn’t used.
Most cars have speed sensors, compass and steering angle sensors, but having that work with a smartphone is another problem.
Also there were these things: https://www.thedrive.com/news/34489/car-navigation-systems-b...
Most cars have speed sensors, compass and steering angle sensors, but having that work with a smartphone is another problem.
Also there were these things: https://www.thedrive.com/news/34489/car-navigation-systems-b...
I think that's a cool idea. But I think the answer is as simple as it would require mounting a camera on the underside of the car. If you are going that route why not just point a camera at the speedometer right? Or integrate into the can-bus of the car and read it right from the onboard computer. I think the presented idea was probably something that was constrained by what you could do on a phone without interfacing externally.
Cameras get dirty and the underside of a car doesn't tend to stay pristine.
In any event, GPS + measuring how fast the axial spins would be able to get some pretty precises measurements. Two things most vehicles already have. The only issue is calibration at the start of movement.
In any event, GPS + measuring how fast the axial spins would be able to get some pretty precises measurements. Two things most vehicles already have. The only issue is calibration at the start of movement.
Do you mean axle? I had the same thought, but I believe most every car has speed sensors on all 4 wheels for anti-lock breaks and I would imagine a way of telling the steering angle, so there are plenty of ways to get data; I'm sure more than I can think of since I'm just an occasional backyard mechanic, not an engineer.
Do you mean anti-lock brakes?
But yeah, most cars have sensors for all those things.
But yeah, most cars have sensors for all those things.
Yes, I mean brakes.
[deleted]
For optic flow, you also would have to know the height above the road. That varies with vehicle load, tire pressure, speed bumps, etc.
It also would have to work on all kinds of road surfaces, day and night, in rain and snow, with different amount of sunlight, and with varying locations of the sun in the sky.
⇒ I don’t think this is as simple as you make it look.
It also would have to work on all kinds of road surfaces, day and night, in rain and snow, with different amount of sunlight, and with varying locations of the sun in the sky.
⇒ I don’t think this is as simple as you make it look.
Fuse it with inertial or depth data, and illuminate the surfacE
Dead reckoning [1] is not new, particularly also in car navigation. Particularly the inertial zeroing trick is common in pedastrian dead reckoning. Neural networks seem to be en vogue to improve this as well [2]
[1] https://en.m.wikipedia.org/wiki/Dead_reckoning [2] https://www.sbir.gov/node/1696357
[1] https://en.m.wikipedia.org/wiki/Dead_reckoning [2] https://www.sbir.gov/node/1696357
>> Lots of noisy estimates, combined, can make for a pretty accurate estimate.
Or the people building the car could spring for the 100$ gyro rather than the 0.50$ phone-type gyro. With a reasonably-accurate gyroscope you can safely abandon most all the fuzzy data.
Or the people building the car could spring for the 100$ gyro rather than the 0.50$ phone-type gyro. With a reasonably-accurate gyroscope you can safely abandon most all the fuzzy data.
200x more expensive and a single point of failure? I can't see why they would pick that.
You'd need a better accelerometer too. For best results, you'd need three of each.
While I applaud the use of novel techniques to approach problems that have a traditional solution, I am reluctant to believe that they can outperform the traditional approaches for a certain type of problem (e.g. state estimation for systems where the dynamics are well understood and can be derived from first principles, like cars).
The traditional solutions still involve careful running. People are mentioning Kalman filters. How are the covariances of the noise processes chosen? The best choice of parameters is chosen by experimentation and measurement, not first principles.
Additionally, "first principles" can mean modeling the contact patch been the tire and road, and estimating the friction, based on weather and road conditions. Or it can mean a mixture of two models, one with and without slippage.
How do you detect which model is "active"? That's more parameters that require tuning.
Ideally, models are both data-driven and human-interpretable.
Additionally, "first principles" can mean modeling the contact patch been the tire and road, and estimating the friction, based on weather and road conditions. Or it can mean a mixture of two models, one with and without slippage.
How do you detect which model is "active"? That's more parameters that require tuning.
Ideally, models are both data-driven and human-interpretable.
I'm pretty sure that deriving position from acceleration data from an inertial sensor is not that cleanly done from first principles: it tends to drift a bunch.
There's no reason the trained model can't emulate the dynamics accurately and do just as well as the traditional approach, but then also learn from additional clues.
There's no reason the trained model can't emulate the dynamics accurately and do just as well as the traditional approach, but then also learn from additional clues.
You can compensate for drift with GPS updates, just like they're doing with the ML net. I don't think "add moar ML!" adds value to every domain.
At University I was involved in a project trying to estimate states, solely based on inertial sensors.
It is true, that estimates of states will drift off the ground truth very fast, but I don't think that ML approaches are the best solution to these kind of problem. In our tests we tried to estimate the position of a bicycle solely with an iPhone taped to the handlebar. My first idea was to update the state based on the fact, that the handlebar would always point in the riding direction. And the second idea was to update the altitude of the estimation based on a heightmap of the environment.
Of course, without these two approaches the estimation would drift off rapidly. But the first approach was enough to confine our estimation to the general area of riding (at least for the time of measuring). With this in place the most dirft would come from noisy gyrometer measurements and the angle of corners was not always right.
The position estimation did not work as nice as I hoped. But based on the fact, that the estimation did not drift away at breakneck speeds, I concluded that at least you could use the algorithm for a speedometer. If you would add a "zero velocity update" based on unchanged GPS positions it would work even better.
It was incredibly easy to eliminate a major part of noise in this problem. It is such a trivial idea, that I was struck when I first had a look at the great results. And I am sure that a machine learning approach would just learn the transformation between the IMU and the car and then ignore measurements that are implying that the car would drive in impossible directions.
It is true, that estimates of states will drift off the ground truth very fast, but I don't think that ML approaches are the best solution to these kind of problem. In our tests we tried to estimate the position of a bicycle solely with an iPhone taped to the handlebar. My first idea was to update the state based on the fact, that the handlebar would always point in the riding direction. And the second idea was to update the altitude of the estimation based on a heightmap of the environment.
Of course, without these two approaches the estimation would drift off rapidly. But the first approach was enough to confine our estimation to the general area of riding (at least for the time of measuring). With this in place the most dirft would come from noisy gyrometer measurements and the angle of corners was not always right.
The position estimation did not work as nice as I hoped. But based on the fact, that the estimation did not drift away at breakneck speeds, I concluded that at least you could use the algorithm for a speedometer. If you would add a "zero velocity update" based on unchanged GPS positions it would work even better.
It was incredibly easy to eliminate a major part of noise in this problem. It is such a trivial idea, that I was struck when I first had a look at the great results. And I am sure that a machine learning approach would just learn the transformation between the IMU and the car and then ignore measurements that are implying that the car would drive in impossible directions.
The lack of imagination in some of the negative responses in this thread as to how or why someone would want to do this is pretty ridiculous.
I'm at a startup where we do API as a service for insurance companies, and are specialty is acting as a platform for new insurance companies that are seeking to cater to drivers who might not have a credit history (new immigrants) or maybe even a bad one. Instead of each tiny company having to build apps that can allow them to gage driving behavior with GPS (not reliable, because shitty drivers who want to scam the company will turn off GPS every time they are behind the wheel), they could theoretically have an app installed on a customer's phone (required to have the insurance) that uses the phone's inertial sensors to estimate many factors, including speed in this case. It also allows these apps to not even have to collect location data at all, and that's a huge plus for privacy.
Maybe this isn't viable. Who knows? But the arrogance in this thread of assuming that there's no use for this is pretty absurd, and all in all negative.
I'm at a startup where we do API as a service for insurance companies, and are specialty is acting as a platform for new insurance companies that are seeking to cater to drivers who might not have a credit history (new immigrants) or maybe even a bad one. Instead of each tiny company having to build apps that can allow them to gage driving behavior with GPS (not reliable, because shitty drivers who want to scam the company will turn off GPS every time they are behind the wheel), they could theoretically have an app installed on a customer's phone (required to have the insurance) that uses the phone's inertial sensors to estimate many factors, including speed in this case. It also allows these apps to not even have to collect location data at all, and that's a huge plus for privacy.
Maybe this isn't viable. Who knows? But the arrogance in this thread of assuming that there's no use for this is pretty absurd, and all in all negative.
According to the abstract, this technology achieves a pretty good result over a 4-minute drive without satellite updates. It's not going to get much better than that, because double integration with phone-grade sensors is insanely error-prone, and that fact won't change no matter how much ML you throw at the problem. How many 4-minute drives do your customers take?
This is completely useless for your stated use case.
This is completely useless for your stated use case.
This is cool! I wouldn't be too surprised if Apple is doing something similar today.
Inertial navigation systems [0] use integration to calculation position, so errors add up quickly. The first in car navigation system [1] used inertial navigation. I think it used places where it knew the vehicle would (should?) stop as waypoints where the velocity could be set to zero. For example, if we think we're near a stop sign, and our calculated velocity is zero-ish, make it zero.
It's pretty amazing how accurate smartphones can be when you're looking at your position while moving in an area that doesn't have GPS, probably using similar techniques. The mobile device knows (is guessing) you're in a vehicle (using measured speed, Bluetooth connection to a vehicle, the fact that you're on a road, etc.) and can snap your location to a road on a map, as well as project your position.
It makes sense that inertial error could be similar for similar movement patterns, say driving over expansion joints in a concrete road. A ML model seems like a good way to try to compensate for that.
The reason I wonder if Apple is doing this already is they have all the pieces...they even know which vehicle I'm in, because I pair with the Bluetooth system in both vehicles. So they could actually build a model per vehicle, which I imagine would be more accurate.
They do have a patent pending for applying machine learning to the location domain, looks like this one is for GNSS though: https://patents.google.com/patent/US20200049837A1/en?oq=US20...
[0] - https://en.wikipedia.org/wiki/Inertial_navigation_system
[1] - https://en.wikipedia.org/wiki/Etak
Inertial navigation systems [0] use integration to calculation position, so errors add up quickly. The first in car navigation system [1] used inertial navigation. I think it used places where it knew the vehicle would (should?) stop as waypoints where the velocity could be set to zero. For example, if we think we're near a stop sign, and our calculated velocity is zero-ish, make it zero.
It's pretty amazing how accurate smartphones can be when you're looking at your position while moving in an area that doesn't have GPS, probably using similar techniques. The mobile device knows (is guessing) you're in a vehicle (using measured speed, Bluetooth connection to a vehicle, the fact that you're on a road, etc.) and can snap your location to a road on a map, as well as project your position.
It makes sense that inertial error could be similar for similar movement patterns, say driving over expansion joints in a concrete road. A ML model seems like a good way to try to compensate for that.
The reason I wonder if Apple is doing this already is they have all the pieces...they even know which vehicle I'm in, because I pair with the Bluetooth system in both vehicles. So they could actually build a model per vehicle, which I imagine would be more accurate.
They do have a patent pending for applying machine learning to the location domain, looks like this one is for GNSS though: https://patents.google.com/patent/US20200049837A1/en?oq=US20...
[0] - https://en.wikipedia.org/wiki/Inertial_navigation_system
[1] - https://en.wikipedia.org/wiki/Etak
Inertial navigation systems [0] use integration to calculation position, so errors add up quickly. The first in car navigation system [1] used inertial navigation. I think it used places where it knew the vehicle would (should?) stop as waypoints where the velocity could be set to zero. For example, if we think we're near a stop sign, and our calculated velocity is zero-ish, make it zero.
Not quite. Etak had wheel sensors. The sensor package was two wheel sensors, a vertical reference using fluid in a cup sensed electrostatically, a 2-axis magnetometer, and a 2-axis rate gyro which used a spinning disc that deformed slightly when turning. I still have one, somewhere.
All those sensors were very noisy and inaccurate. What made Etak work was map matching of turns. It tried to match its estimate of the recent path to the map data. It could get lost, but usually could find itself again after a few turns. The main problem was in grid cities. If estimated position became off by one block, turns would not help to disambiguate the problem, and so it would remain lost.
Not quite. Etak had wheel sensors. The sensor package was two wheel sensors, a vertical reference using fluid in a cup sensed electrostatically, a 2-axis magnetometer, and a 2-axis rate gyro which used a spinning disc that deformed slightly when turning. I still have one, somewhere.
All those sensors were very noisy and inaccurate. What made Etak work was map matching of turns. It tried to match its estimate of the recent path to the map data. It could get lost, but usually could find itself again after a few turns. The main problem was in grid cities. If estimated position became off by one block, turns would not help to disambiguate the problem, and so it would remain lost.
Did you have one? Or work on it? Really cool either way. Pretty amazing how far sensor technology has come. I remember going to an Analog Devices presentation for their MEMS accelerometers around 1997. They gave out a dev board I think I might still have somewhere.
Actually, the first automated in-car navigation systems used mechanical couplings to a wheel to determine progress along a set route: https://arstechnica.com/cars/2020/01/turn-by-turntables-how-...
Is this any better than a Kalman filter or just a lot more computationally intensive?
Kalman filter solves the problem of getting accuracy from noisy or infrequent speed data. However with only acceleration data you still can have integration errors build up over time. IN other words, Kalman filter can't solve that the driving steadily at 50 and 55mph are effectively indistinguishable from acceleration data.
OTOH I'm skeptical that this method really solves it either
OTOH I'm skeptical that this method really solves it either
I’d expect you could get some signal of speed from the periodic double-thumps of expansion joints, speed bumps, potholes, and the like. Combine that with acceleration integration from stops to find an estimate of the wheelbase.
It might not be practical or the easiest way, but I suspect that you could tell the difference between steady highway driving at 50 vs 55 from acceleration data only (due to inherent road imperfections).
It might not be practical or the easiest way, but I suspect that you could tell the difference between steady highway driving at 50 vs 55 from acceleration data only (due to inherent road imperfections).
That's really clever! That's also the kind of thing an AI could pick up on, so maybe this is a stronger technique than I initially thought.
Although autocorrelation is probably a better way to get that data than just throwing it into a AI model.
Although autocorrelation is probably a better way to get that data than just throwing it into a AI model.
I wonder if that generalizes to other kinds of vehicles or even other instances of the same kind of vehicle.
This.
Basically, fuck (MEMS) accelerometers.
I struggled with this in my early university days.
"Wait, we get acceleration? Why not just cumsum twice!" except get garbage.
So this is a fuck you to accelerometers for wasting my time
Basically, fuck (MEMS) accelerometers.
I struggled with this in my early university days.
"Wait, we get acceleration? Why not just cumsum twice!" except get garbage.
So this is a fuck you to accelerometers for wasting my time
Interesting, as they mention, it looks like the main advantage to an approach like this is that it can make use of "special events" (e.g. stop/starts, bumps, turning) which can null out the drift.
Combining this with other covariates/sensors could also provide more correction and better detection of "special events".
I wonder if something like this applied on top of traditional methods that incorporate the dynamics would make a better approach since you get the advantage of using the known dynamics + learnable "special event" corrections.
Combining this with other covariates/sensors could also provide more correction and better detection of "special events".
I wonder if something like this applied on top of traditional methods that incorporate the dynamics would make a better approach since you get the advantage of using the known dynamics + learnable "special event" corrections.
Assume there's good data in the time between bumps from the front and bumps at the back, if wheelbase is known or you just train for it.
Seems like an awful lot of work when there's a perfectly good speedometer already available.
The speedometer may be driven off the transmission, wheel speed sensors, or something else. It doesn't take into account slip.
Well, yes, a speedometer is perfectly fine if all you care about is "speed".
The point of the technique, however, is dead-reckoning of position which means integrating speed, which is prone to accumulate errors over time if you just look at the wheels.
The title of the paper, I think, could be better.
The point of the technique, however, is dead-reckoning of position which means integrating speed, which is prone to accumulate errors over time if you just look at the wheels.
The title of the paper, I think, could be better.
probably would get better performance with a microphone.
For example, GPS has a common problem in areas with tall buildings where satellite signals bounce off a large flat sides of buildings, putting you a block away from where you actually are. Uber has done some fascinating work in this area by actually knowing the location and shape of all tall buildings in a bunch of cities- very clever.
But what if instead, the location estimator took in both the GPS signal and the inertial sensor. Sure, GPS says you just changed blocks, but the inertial sensor says you haven't accelerated from your previous location and heading.
Lots of noisy estimates, combined, can make for a pretty accurate estimate.