PIDs: Creating Stable Control in Games(azeemba.com)
azeemba.com
PIDs: Creating Stable Control in Games
https://azeemba.com/posts/pids-creating-stable-control-in-games.html
40 comments
For every PID write up that reviews the theory we need an equally length follow up on the practical aspects of PID tuning and problem modelling.
Yup, and a good implementation that already handles Integral anti-windup, and damps Derivative bump when changing setpoint, variable cycle rates, etc.
Here’s a decent implementation in Python:
https://github.com/pjkundert/ownercredit/blob/master/pid.py
Here’s a decent implementation in Python:
https://github.com/pjkundert/ownercredit/blob/master/pid.py
There is a cool sandbox game on steam called “Plasma” that offers a graphical programming environment which includes a basic PID controller. I was really impressed with how easily I was able to make a cube float almost perfectly in 3D space within that game by only using 3 of these controllers (mainly the P and D portions) and some thrusters by monitoring the pitch and roll angles, and how high above the ground the cube was. I was even more impressed when adding some weight to one side of the cube and seeing the system compensate for it automatically.
The process of tuning the gains on the different inputs of the controllers, then seeing how the system responded made me think about how this is a similar yet much simpler process to training ML models.
I’ve always been a bit skeptical of how easily one would be able to make changes to something as complex as a large ML model, but looking at it like tuning a very complex PID controller kind of made it seem less like black magic to me for some reason
The process of tuning the gains on the different inputs of the controllers, then seeing how the system responded made me think about how this is a similar yet much simpler process to training ML models.
I’ve always been a bit skeptical of how easily one would be able to make changes to something as complex as a large ML model, but looking at it like tuning a very complex PID controller kind of made it seem less like black magic to me for some reason
The site explains the three parts of PID's very well.
On one hand I like PID controllers. It's a reusable concept which you can apply to a loot of stuff. On the other hand, in my experience it can be very tricky and cumbersome to tweak them.
In the given example, a PID makes perfect sense. There's a single, static target location you need to approach. Adjust velocity with a PID and you're done.
But what if the target location is not static? Imagine that the character needs to move to your mouse pointer? Do PID's still hold up? How do you handle the I and D term then?
It gets even trickier if you move to 3D. You maybe say, hmm, let's use a rigidbody this time instead of a kinematic body for the player character and use PID's to adjust forces to steer velocity and rotation.
But I never found a good way to do that, because in those other cases the error depends on the goal you need to reach, and if the goal constantly changes, I and D are kinda useless...
On one hand I like PID controllers. It's a reusable concept which you can apply to a loot of stuff. On the other hand, in my experience it can be very tricky and cumbersome to tweak them.
In the given example, a PID makes perfect sense. There's a single, static target location you need to approach. Adjust velocity with a PID and you're done.
But what if the target location is not static? Imagine that the character needs to move to your mouse pointer? Do PID's still hold up? How do you handle the I and D term then?
It gets even trickier if you move to 3D. You maybe say, hmm, let's use a rigidbody this time instead of a kinematic body for the player character and use PID's to adjust forces to steer velocity and rotation.
But I never found a good way to do that, because in those other cases the error depends on the goal you need to reach, and if the goal constantly changes, I and D are kinda useless...
Your example is really a classical real world PID where a motion system tracks a motion profile. The short answer is yes, PIDs work just as well when the target changes. The stability of a PID depends on the open loop response which command or error are not part of.
The theory isn't that simple (you can take a course in Control Theory which would cover that) but I think many/most people that use/tune PIDs in the real world just use rules of thumb and standard approaches to tune the different components of the loop. Somewhat more advanced, but still non-theoretical tools, are actually measuring the open loop response which lets you pretty much see the margins you have. Optimizing PIDs is really trying to get the loop to respond quickly without becoming unstable.
Another interesting thing that's used in the real world is feed-forward. That is if you have some estimate of how to perform the function of the PID directly you can just feed that to the output and add the PID in parallel. The motion control analogy of this is called acceleration feed forward. If you know the target acceleration you can basically multiply that by some constant and feed it directly to the output (which is generally current to the motor which is more or less linear to acceleration given a constant load). This dramatically improves the performance of your control system because the PID now just corrects the much smaller residual error.
The theory isn't that simple (you can take a course in Control Theory which would cover that) but I think many/most people that use/tune PIDs in the real world just use rules of thumb and standard approaches to tune the different components of the loop. Somewhat more advanced, but still non-theoretical tools, are actually measuring the open loop response which lets you pretty much see the margins you have. Optimizing PIDs is really trying to get the loop to respond quickly without becoming unstable.
Another interesting thing that's used in the real world is feed-forward. That is if you have some estimate of how to perform the function of the PID directly you can just feed that to the output and add the PID in parallel. The motion control analogy of this is called acceleration feed forward. If you know the target acceleration you can basically multiply that by some constant and feed it directly to the output (which is generally current to the motor which is more or less linear to acceleration given a constant load). This dramatically improves the performance of your control system because the PID now just corrects the much smaller residual error.
A tuned PID loop should follow a moving target with no issues and no advanced control theory. Here is a similar example from a Lego robot with a pixi camera [1]. The pixi camera recognizes the ball location in 2D, sends to arduino. There are two independent loops, one keeping the ball in the center of the horizontal visual field, by rotating the camera left-right, and the other is keeping the ball in the center vertically, by tilting the camera via two levers.
One thing you might notice is that the ball sometimes goes too fast for the robot to follow - this is due to the limitations of the power of the motors, and the rate of sampling in the loop. Games usually don't have the limitation of the power, but I suppose the rate of sampling and updating the loop could be limited by the game loop. In that case, it might be useful to update the PID multiple times per loop.
Other than that, there are some tricks to make the loops more stable. Limiting the error can be useful. Limiting the action of the actuator. Smoothing the action of the actuator is very useful. I tend to think of PD loops as damped springs. The P term is like the spring constant, and the D term is like friction or viscosity. The first one determines the speed of reaching the goal, and the second one reduces the oscillations when reaching the goal. I rarely use the I term.
[1]https://www.youtube.com/watch?v=7_IZb0RJN_U
Other than that, there are some tricks to make the loops more stable. Limiting the error can be useful. Limiting the action of the actuator. Smoothing the action of the actuator is very useful. I tend to think of PD loops as damped springs. The P term is like the spring constant, and the D term is like friction or viscosity. The first one determines the speed of reaching the goal, and the second one reduces the oscillations when reaching the goal. I rarely use the I term.
[1]https://www.youtube.com/watch?v=7_IZb0RJN_U
> the error depends on the goal you need to reach, and if the goal constantly changes, I and D are kinda useless...
Why do you think that? the error is the difference between actual and desired at every instant. If you move the desired then you enlarge the error term and PID reacts to close that gap, it might never reach if you are changing the goal too fast but that's something related to the concept of "bandwidth". OTherwise you are quite right and all is OK.
If you have multiple axes then the typical solution is to have n PID loops for n axes. This assumes that the dynamics of the system is decoupled. In english, if I do something in one axis it will not affect the other axes. Then you can tune the PID loops independently. Otherwise, dry math.
Why do you think that? the error is the difference between actual and desired at every instant. If you move the desired then you enlarge the error term and PID reacts to close that gap, it might never reach if you are changing the goal too fast but that's something related to the concept of "bandwidth". OTherwise you are quite right and all is OK.
If you have multiple axes then the typical solution is to have n PID loops for n axes. This assumes that the dynamics of the system is decoupled. In english, if I do something in one axis it will not affect the other axes. Then you can tune the PID loops independently. Otherwise, dry math.
Lol. There is an entire field dedicated to this question. It's called Automatic Control. Yes, you can do all of the above. You need to model it with a multivariable system. Google control systems and systems theory.
Usually in a physical control system (that is driven by a PID controller) your limiting factors are the execution element (how fast can you change the input) and your sensors (the resolution of the transducers/sensors dictate the resolution of what you can achieve). A lot of times the PID controller will have a way that it can autotune itself as it's extremely rare to be able to measure the properties of the physical system precisely and for those properties to not vary with temperature and with wear and tear.
Usually in a physical control system (that is driven by a PID controller) your limiting factors are the execution element (how fast can you change the input) and your sensors (the resolution of the transducers/sensors dictate the resolution of what you can achieve). A lot of times the PID controller will have a way that it can autotune itself as it's extremely rare to be able to measure the properties of the physical system precisely and for those properties to not vary with temperature and with wear and tear.
Well, you kind of prove my point, because you made me jump from a 5 minute lecture about PIDs to a study about control systems.
Yes, probably there are all kinds of systems to control and autotune a PID. But that's the problem. In gamedev, PID's get constantly recommended for so many things, but when you play around with them you soon realize, that they are quite hard to get right. So then there's two ways: you dive into control systems or you just lerp the value and call it a day.
Yes, probably there are all kinds of systems to control and autotune a PID. But that's the problem. In gamedev, PID's get constantly recommended for so many things, but when you play around with them you soon realize, that they are quite hard to get right. So then there's two ways: you dive into control systems or you just lerp the value and call it a day.
yup. i get it. it's a steep learning curve. and it's hard to grok if you don't understand what the theory behind them is.
My bigger point is that the articles like the one linked here just barely scratch the surface and you cannot really jump into it and get an intuitive understanding by just looking at it. that being said, if you want to learn more you can and they can be applied to a large range of problems.
My bigger point is that the articles like the one linked here just barely scratch the surface and you cannot really jump into it and get an intuitive understanding by just looking at it. that being said, if you want to learn more you can and they can be applied to a large range of problems.
Control theorist here. The articles are pretty good. There is nothing wrong with them.
What you are insinuating is not always correct that if you model stuff you'll get a better result. Sometimes PID is more then enough and in fact better since your model can get arbitrarily complex and you don't have any tools to deal with that behavior. It's a bit snobbish that you need to have math and all the shenanigans. The rest of the manufacturing world disagrees with your premise that is very common in academic circles.
What you are insinuating is not always correct that if you model stuff you'll get a better result. Sometimes PID is more then enough and in fact better since your model can get arbitrarily complex and you don't have any tools to deal with that behavior. It's a bit snobbish that you need to have math and all the shenanigans. The rest of the manufacturing world disagrees with your premise that is very common in academic circles.
Maybe you need a controller for how you interact over here?
Nobody was insinuating anything. I was pointing out that going past an intuitive understanding may be a harder leap than expected.
Nobody was insinuating anything. I was pointing out that going past an intuitive understanding may be a harder leap than expected.
I've done character controllers and AI with rigid bodies and I just tune damping values and naive functions until it's good enough and I achieve great results. Ie: players doing backflips with torque, AI steering and slowing to a stop within a small radius of a destination.
It's just a game!
It's just a game!
Simple example: You want to control torque with a PID to reach a target angular velocity. What if the player suddenly negates the target angular velocity (by going from left arrow key press to right arrow key press for example).
How do you handle the I and D term then?
What if the player can also use a gamepad and go from left to right gradually?
The fundamental problem is that the error accumulates while you try to reach the target. But then the target suddenly changes. Do you just drop the previous error? Do you create an upper bound on the error? Or something else? This is the question no PID post ever answers.
I don't say, that this is impossible, but just that PID's in this context are harder to use than they first appear.
How do you handle the I and D term then?
What if the player can also use a gamepad and go from left to right gradually?
The fundamental problem is that the error accumulates while you try to reach the target. But then the target suddenly changes. Do you just drop the previous error? Do you create an upper bound on the error? Or something else? This is the question no PID post ever answers.
I don't say, that this is impossible, but just that PID's in this context are harder to use than they first appear.
Yes drop the “previous error” by setting the integral to zero when the setpoint changes such that the sign of the error changes.
If the derivative is on the error when the setpoint is changed all of the sudden the error is increasing instead of decreasing so it works as desired.
There are many books on control systems that cover different forms of the PID algorithm and what applications they suit.
I prefer the velocity form or double derivative form for the simplicity of managing the integrator and applying real world limits on maximum and minimum outputs and rate of change of output. I basically never need such a fast response that it is worth bothering with D.
A mathematics professor solved the differential equations governing my processes so now I can calculate the exact correct gains for critical damping and simulate the response for different limitations such as input sensor resolution and measurement delays. It is wonderful.
If the derivative is on the error when the setpoint is changed all of the sudden the error is increasing instead of decreasing so it works as desired.
There are many books on control systems that cover different forms of the PID algorithm and what applications they suit.
I prefer the velocity form or double derivative form for the simplicity of managing the integrator and applying real world limits on maximum and minimum outputs and rate of change of output. I basically never need such a fast response that it is worth bothering with D.
A mathematics professor solved the differential equations governing my processes so now I can calculate the exact correct gains for critical damping and simulate the response for different limitations such as input sensor resolution and measurement delays. It is wonderful.
> How do you handle the I and D term then?
You recalculate the D term every iteration, so it doesn’t matter?
The ugly truth about the I term is that in practice people very often leave it at zero. We sometimes call these controllers PD controllers.
Especialy in a game where you don’t have sticky actuators and miscalibrated sensors this is usually more than enough for hapiness.
You recalculate the D term every iteration, so it doesn’t matter?
The ugly truth about the I term is that in practice people very often leave it at zero. We sometimes call these controllers PD controllers.
Especialy in a game where you don’t have sticky actuators and miscalibrated sensors this is usually more than enough for hapiness.
Seems like you misread my post. I don't use PIDs. I just tune random functions until the forces are good enough.
PIDs sound cool though. I should try em.
PIDs sound cool though. I should try em.
If you're learning about PID controllers, do yourself a favor and watch Brian Douglas's video series on the topic:
https://youtu.be/wkfEZmsQqiA?si=50WWz4kuber56JIU
Brian has great videos on other control theory topics as well.
https://youtu.be/wkfEZmsQqiA?si=50WWz4kuber56JIU
Brian has great videos on other control theory topics as well.
These videos were fantastic, thanks for linking them. The perfect balance of a little theory with practice and examples for someone who has an immediate need to control something but has never taken a control theory class.
PID get real fun when you model physical systems, like say filling a tank while trying to maintain liquid depth and temperature or maintaining a constant reaction rate. You get to start taking laplace transforms. I had a ChemE professor who’d tell stories from the 30’s when chemical engineering was forming as a discipline. Common practice back in the day (before the theory) was to manually adjust parameters until things went unstable, then back off 5%. Turns out the optimum point is the inflection between stable and unstable behavior.
My whole BSc degree was in systems engineering and automatic control. You make me nostalgic about all those classes that at the time I hated. What a weird feeling.
The most bang-for-the-buck feature of PIDs I've seen yet is to smooth out game camera movement so that the camera feels less 'mechanical' (works both for the 3rd-person camera behind a character, or the overhead camera in a strategy game which needs to jump between positions).
Just implement the camera positioning and lookat-pointing in the dumbest way possible (including non-continuous, sudden jumps), plug a PID into the position and look-at point, tweak the parameters a bit and voilà, pure magic :)
Just implement the camera positioning and lookat-pointing in the dumbest way possible (including non-continuous, sudden jumps), plug a PID into the position and look-at point, tweak the parameters a bit and voilà, pure magic :)
Feels like PIDs are the wrong tool for this problem. Stuff to do with matching animations up to physics is usually better solved with inverse kinematics.
My understanding is PIDs are vastly preferred in situations where very often real world measurements have a degree of error in them. Like having the wheels of your car slip extra distance when you put your foot on the paddle for a specific amount of time. It's a feedback system of sorts.
If there's no error in measurements, sure, a mix of IK and blending seems to be a lot easier to implement.
If there's no error in measurements, sure, a mix of IK and blending seems to be a lot easier to implement.
What are you imagining in this situation? How would one use IK with a running animation?
You could remove the root motion of the animation and control the position of runner in code. This would allow controlling the position/speed easily but to look natural you still have to tweak the animation speed dynamically. Just changing the foot position via IK would not be enough to make the animation look natural. So feels like you are still left with the original problem of tweaking the animation speed.
You could remove the root motion of the animation and control the position of runner in code. This would allow controlling the position/speed easily but to look natural you still have to tweak the animation speed dynamically. Just changing the foot position via IK would not be enough to make the animation look natural. So feels like you are still left with the original problem of tweaking the animation speed.
Not the person you asked, but here's a video you might like: https://www.youtube.com/watch?v=LNidsMesxSE&t=268s
In that example, they do the character's physics first (just a ball), then match the animation to it. The animation speed is controlled by a "surveyor wheel" approach — go to to 336s in the video for that part.
So, the animation speed is derived from the physics, rather than the physics speed being derived from the animation.
I wouldn't call that "IK" exactly, which would be more for things like matching the foot to individual stair steps as the character walks up them, or such. But I suppose in a broader sense it is, since you're deciding on a physical goal a-priori, then forcing the animation to match that goal.
In that example, they do the character's physics first (just a ball), then match the animation to it. The animation speed is controlled by a "surveyor wheel" approach — go to to 336s in the video for that part.
So, the animation speed is derived from the physics, rather than the physics speed being derived from the animation.
I wouldn't call that "IK" exactly, which would be more for things like matching the foot to individual stair steps as the character walks up them, or such. But I suppose in a broader sense it is, since you're deciding on a physical goal a-priori, then forcing the animation to match that goal.
Thanks for the link! The ideas are presented so well. Really a cool approach too
Using only PD (no I) when tracking also works well. Might be worth adding a short section about that to compare against the full PID.
Interestingly, there was another PID post earlier today and someone specifically commented on using a PID for camera tracking: https://news.ycombinator.com/item?id=39011630#39016836
I am surprised to hear about a PD controller though. In my testing, a PI controller seemed to behave much better than a PD controller. In my research, it seemed a common strategy to drop the D-component completely but I did not see the suggestion to drop the I-component.
I am surprised to hear about a PD controller though. In my testing, a PI controller seemed to behave much better than a PD controller. In my research, it seemed a common strategy to drop the D-component completely but I did not see the suggestion to drop the I-component.
In terms of the theoretical guarantees, a PD controller is (nearly) always stable and pretty easy to critically damp. However, if the set point is moving with a nonzero velocity or there is some load, then the PD controller will not converge. A PI and a PID controller can converge even if the target is moving.
Practically speaking, I would recommend encoding as much model information as possible before pulling out the I term: for instance, if you know the velocity of your tracked object and assume an intertidal model, you can just include some of that data in your feed forward rather than the controller, which simplifies things greatly.
Practically speaking, I would recommend encoding as much model information as possible before pulling out the I term: for instance, if you know the velocity of your tracked object and assume an intertidal model, you can just include some of that data in your feed forward rather than the controller, which simplifies things greatly.
Do you have some resources on that?
You can use your intuition in the sense that if what you measure just has a constant error without an integrator in the loop that error will never be closed. Think about a motion control system with a proportional gain where you are pushing the actuator with your hand away from the current position. The proportional gain will apply a force against your push but generally that will leave some residual error in steady state. This is where the integrator comes in. And I agree with the parent that I've mostly seen the D term dropped from controllers, definitely with motion control, though sometimes there are other weird components or multiple loops which may end up having a similar effect.
One thing that I think is often confused the matter is that P, I, and D all relate to what you measure and what you're controlling. E.g. you might measure velocity or position in a motion control system, and generally your output is going to be current. In a temperature control system your measurement would be temperature and your output is likely going to be current (which influences acceleration, not position or velocity). Ofcourse velocity being the derivative of position means that the meaning of "P" is different. I haven't done PID in a long while but I've always used to ground myself in motion control systems to get a sense what I want to put the loop over.
EDIT: E.g. IIRC it's common in motion control to have a PI loop over the position error and an additional proportional controller over the velocity (and then filters, feed-forward and a bunch of other components ;) ).
One thing that I think is often confused the matter is that P, I, and D all relate to what you measure and what you're controlling. E.g. you might measure velocity or position in a motion control system, and generally your output is going to be current. In a temperature control system your measurement would be temperature and your output is likely going to be current (which influences acceleration, not position or velocity). Ofcourse velocity being the derivative of position means that the meaning of "P" is different. I haven't done PID in a long while but I've always used to ground myself in motion control systems to get a sense what I want to put the loop over.
EDIT: E.g. IIRC it's common in motion control to have a PI loop over the position error and an additional proportional controller over the velocity (and then filters, feed-forward and a bunch of other components ;) ).
> Practically speaking, I would recommend encoding as much model information as possible before pulling out the I term
I was referring to that part. Ie: if I know more abouty system, how do I include that in my control loop.
E.g. if I want to control the temperature of a room, I might have an idea how much energy is lost due to the gradient between inside and outside temperatures.
I would assume that a PID controller is still suitable for that, but I have never seen novice level resources on that.
I was referring to that part. Ie: if I know more abouty system, how do I include that in my control loop.
E.g. if I want to control the temperature of a room, I might have an idea how much energy is lost due to the gradient between inside and outside temperatures.
I would assume that a PID controller is still suitable for that, but I have never seen novice level resources on that.
I think the parent is saying something along the lines of "if you can look at your problem and determine there's no need for an integrator, or you can take the need out using feed forward, then you can remove the integrator" (or set your integral gain to zero). Keep in mind though the integral gain isn't simply acting to take out constant error, it changes the transfer function of the loop.
If you have a mathematical model for the thing you're controlling you can derive the PID parameters using that model. That's very rarely the case though and even with a model reality almost always differs.
The PID controller doesn't "care" about your knowledge. What it does "care" about is how the system responds to the control signal, specifically with linear systems the gain and phase for the different frequencies. If you're tuning a PID using some heuristics then the process of tuning includes the response of the system you're controlling. So at least intuitively I don't see an easy way to take some partial knowledge of the system and incorporate it into the tuning process. Maybe someone with experience can say something like "using less proportional gain and more integral gain results in better <something> for temperature control systems where there's a lot of energy loss due to gradient" but otherwise there's no easy way to tell IMO. If the integrator doesn't help you're probably going to discover that in the process of tuning anyways.
I think this knowledge is more useful in the context of a feed forward, i.e. if you have a good model of how your system behaves and you incorporate it into the feed forward part of your control loop then the residual error is going to be much smaller and you'll get better performance.
If you have a mathematical model for the thing you're controlling you can derive the PID parameters using that model. That's very rarely the case though and even with a model reality almost always differs.
The PID controller doesn't "care" about your knowledge. What it does "care" about is how the system responds to the control signal, specifically with linear systems the gain and phase for the different frequencies. If you're tuning a PID using some heuristics then the process of tuning includes the response of the system you're controlling. So at least intuitively I don't see an easy way to take some partial knowledge of the system and incorporate it into the tuning process. Maybe someone with experience can say something like "using less proportional gain and more integral gain results in better <something> for temperature control systems where there's a lot of energy loss due to gradient" but otherwise there's no easy way to tell IMO. If the integrator doesn't help you're probably going to discover that in the process of tuning anyways.
I think this knowledge is more useful in the context of a feed forward, i.e. if you have a good model of how your system behaves and you incorporate it into the feed forward part of your control loop then the residual error is going to be much smaller and you'll get better performance.
I used a PD controller for a specific application where we were trying to maintain a specific temperature in the future, and cared more about the trajectory of the temperature over time to reach the goal. We didn’t much care for the integration of that temperature since that wasn’t important in this particular case. An integration factor would actually wind up the rate of temperature change too much!
Integral windup is the term for this issue, there's a few ways to deal with the windup if you need to keep the integral component (e.g. for tracking a moving set point, or to overcome a stable error).
https://en.wikipedia.org/wiki/Integral_windup
YMMV I guess :)
[deleted]