How can I get my engineers to accept being on call?(workplace.stackexchange.com)
workplace.stackexchange.com
How can I get my engineers to accept being on call?
https://workplace.stackexchange.com/questions/193511/how-can-i-get-my-engineers-to-accept-being-on-call
103 comments
I see two solutions to the conundrum. At one company where I previously worked, they required that employees receive 2x the time off, if you had to fix or work on anything during off-hours. Another company I’ve been at is pretty strict about not shipping things until they’re ready (but, that company doesn’t hire juniors in engineering). Pick your poison.
The fact is that no engineer wants to get paged. But we have to be responsible for our work and our mistakes. That said, I think it’s well understood, the dynamic of pushing rushed work. To which end, you have to construct a system where e.g. product is incentivized to align with engineering on some minimum level of reliability.
The fact is that no engineer wants to get paged. But we have to be responsible for our work and our mistakes. That said, I think it’s well understood, the dynamic of pushing rushed work. To which end, you have to construct a system where e.g. product is incentivized to align with engineering on some minimum level of reliability.
Sure, and "constructing that system" should mean paying your employees for non-work-hours work. And even if I never get paged, if I need to be within 15 minutes of a computer with an internet connection at all times, that is going to drastically limit what I can do with my free time. I should be compensated for that.
One company I worked at didn't have a formal policy of giving people time off if they got paged, but most/all managers would allow for it. That's nice, I guess, but to me still doesn't cover it. Even if the pager never goes off, the employee has made their off-hours worse (that is, not really off-hours anymore) in order to give the company value. Not compensating them for that is unethical.
Just to specifically address one bit:
> But we have to be responsible for our work and our mistakes.
Absolutely agree. But if you want me to be responsible for my work outside of regular work hours, I expect to be compensated for my time. Also consider that I'm being asked to be responsible for my teammates' work as well; while I do have some influence over the quality of their work, it's far from sufficient to guarantee their work is equivalent in quality to mine (and vice versa).
One company I worked at didn't have a formal policy of giving people time off if they got paged, but most/all managers would allow for it. That's nice, I guess, but to me still doesn't cover it. Even if the pager never goes off, the employee has made their off-hours worse (that is, not really off-hours anymore) in order to give the company value. Not compensating them for that is unethical.
Just to specifically address one bit:
> But we have to be responsible for our work and our mistakes.
Absolutely agree. But if you want me to be responsible for my work outside of regular work hours, I expect to be compensated for my time. Also consider that I'm being asked to be responsible for my teammates' work as well; while I do have some influence over the quality of their work, it's far from sufficient to guarantee their work is equivalent in quality to mine (and vice versa).
Agreed! I don't work at a job that requires on-call hours, but I do have to sit in front of a computer 40 hours a week. My job is fine, but I can enthusiastically say that when I am off work, I want to be far away from my computer. If I were looking at a job offer that required on-call time, it would have come with well defined time periods and extra pay.
This is an excellent point. I do recall that did on occasion alter my plans, inappropriately bring my laptop to functions, or limit my alcohol consumption for the evening just-in-case. Fortunately we were given generous on-call compensation even when not paged.
AKA "How can I squeeze employees even harder rather than paying someone to do the work?"
I much prefer the incentives around time off in lieu for working out of hours, rather than elevated pay.
We are paid a base rate just for being on call, it's less than our normal rate and the same for anyone in the company who goes on-call. It's payment for making ourselves available, not for doing any work. If I'm up in the night, I'll want a lie in the next morning.
We are paid a base rate just for being on call, it's less than our normal rate and the same for anyone in the company who goes on-call. It's payment for making ourselves available, not for doing any work. If I'm up in the night, I'll want a lie in the next morning.
Glad you get paid a base rate!
A company I worked at (informally, as there was no specific policy) gave people the day off if they got paged overnight and had to do work. That's certainly something. The problem with it being informal is that many employees didn't realize it'd be ok for them to take off the next day, so they didn't. And managers often wouldn't pro-actively offer it.
But the question to me is: if I have to wake up at 2am and work for 4 hours, is that "equivalent" to having the next day off? Maybe? But also: maybe not.
A company I worked at (informally, as there was no specific policy) gave people the day off if they got paged overnight and had to do work. That's certainly something. The problem with it being informal is that many employees didn't realize it'd be ok for them to take off the next day, so they didn't. And managers often wouldn't pro-actively offer it.
But the question to me is: if I have to wake up at 2am and work for 4 hours, is that "equivalent" to having the next day off? Maybe? But also: maybe not.
I find the 'in lieu' method breaks down when one day is not equivalent to another. A Sunday with family that I lose isn't something a Monday on my own can replace.
Then again the employer could just let the employee decide which one they want.
Then again the employer could just let the employee decide which one they want.
That might be a problem in theory, but it's not been a problem in practice for me. I've been on an on-call rota with my current employer for seven years or so, and it's never happened that I've lost even a significant fraction of a weekend. Stuff tends to break while folk are working and making changes, and I've lost evenings to serious outages. Issues out of hours tend to be solved by a roll-back, or a mitigation (like draining a region) that lets us leave the fix until regular working hours.
Probably the most severely interrupted non-working day was when I needed to take an hour between the main couse and pudding one Christmas day, when I was second line on-call and the person who was formally on-call (who doesn't celebrate) needed a second pair of eyes. While the most severely-interrupted working day was probably the one that included this outage: https://medium.com/@SkyscannerEng/how-a-couple-of-characters.... I'm quite proud of the response to that, especially that we quite carefully and deliberately stopped after one region was back up, running in a degraded state overnight in order to make sure we didn't burn out our teams by trying to do too much out of hours.
Probably the most severely interrupted non-working day was when I needed to take an hour between the main couse and pudding one Christmas day, when I was second line on-call and the person who was formally on-call (who doesn't celebrate) needed a second pair of eyes. While the most severely-interrupted working day was probably the one that included this outage: https://medium.com/@SkyscannerEng/how-a-couple-of-characters.... I'm quite proud of the response to that, especially that we quite carefully and deliberately stopped after one region was back up, running in a degraded state overnight in order to make sure we didn't burn out our teams by trying to do too much out of hours.
I agree if on call is being triggered regularly.
I strongly disagree if events that would trigger on call happen rarely (e.g. once per year).
Many companies have stable infrastructure but just need the extremely occasional coverage for things that rarely, if ever, happen.
I’ve seen people overreact when on call has been introduced, probably from negative experiences at prior jobs. As long as the on call events happen once in a blue moon, I don’t see why anyone would have a major problem with it.
I strongly disagree if events that would trigger on call happen rarely (e.g. once per year).
Many companies have stable infrastructure but just need the extremely occasional coverage for things that rarely, if ever, happen.
I’ve seen people overreact when on call has been introduced, probably from negative experiences at prior jobs. As long as the on call events happen once in a blue moon, I don’t see why anyone would have a major problem with it.
That doesn't matter at all to the base rate. Bottom line is that you don't know when that once-in-a-blue-moon page is going to happen, so you still have to curtail your leisure activities in order to be ready. Want to go see a movie? Want to go for a long drive? Want to go on a hike? Want to take your kids (who are young enough to need constant supervision) out somewhere? Want to go anywhere where you don't have cell service? Nope, can't do it. If the company wants me to change how I would otherwise spend my leisure time, they need to pay me for it.
So sure, a company fitting your description would only rarely pay out the elevated rate for someone actively handling an incident, but since an employee still has to do the exact same things when not handling an incident (regardless of how often incidents arise), the base rate needs to be there.
So sure, a company fitting your description would only rarely pay out the elevated rate for someone actively handling an incident, but since an employee still has to do the exact same things when not handling an incident (regardless of how often incidents arise), the base rate needs to be there.
Well, consider that "once in a blue moon" might actually be worse given that a) it is not predictable and b) likely to be less compensated.
Like, it doesn't matter if something almost never happens because I'd be "on call" regardless- that ---is--- what it means to be on call.
I did that for many years and it didn't matter so much when my kids were little and I didn't want to, like, go to a rave off the grid or perform music over a weekend or go climbing in BFE Utah.
But I quit a job that I'd had for almost 10 years because of that fact of the work. After a while it's a really bullshit thing to do to folks.
It doesn't matter if I never get called- I still have to be available and that really is something people need to be compensated for.
Like, it doesn't matter if something almost never happens because I'd be "on call" regardless- that ---is--- what it means to be on call.
I did that for many years and it didn't matter so much when my kids were little and I didn't want to, like, go to a rave off the grid or perform music over a weekend or go climbing in BFE Utah.
But I quit a job that I'd had for almost 10 years because of that fact of the work. After a while it's a really bullshit thing to do to folks.
It doesn't matter if I never get called- I still have to be available and that really is something people need to be compensated for.
The problem with being on-call isn't that you're working, the problem with being on-call is that you aren't not working.
You can't be on-call and go to a place you're not allowed to have your cell phone on. You can't be on-call and drink alcohol. (or other substances of various legalities) You can't be on-call and go camping in a wilderness location that may not have cell coverage. Depending on the circumstances, you may need to lug your work laptop around in your backpack.
You can't be on-call and go to a place you're not allowed to have your cell phone on. You can't be on-call and drink alcohol. (or other substances of various legalities) You can't be on-call and go camping in a wilderness location that may not have cell coverage. Depending on the circumstances, you may need to lug your work laptop around in your backpack.
On-call X% (0<X<100), and if you're called: The full 100% and then all the applicable bonuses for night shift and weekend etc.
It needs to be be 150% minimum, otherwise they're just paying the same for >40h.
To me the most important factor to tolerate on-call duty is that I must have 100% authority to fix the things that would ever make me be paged.
Under such an arrangement, I control my destiny (not 100%, but close enough). If I can make the code so robust and so easy to diagnose that I'm very likely to never ever be paged, I'm ok being on call.
But if there is some psychopath product manager making engineering decisions above engineering, forcing to increase tech debt constantly, guaranteeing constant outages and thus constant on-call pages then a big fat no. I won't be on call.
Under such an arrangement, I control my destiny (not 100%, but close enough). If I can make the code so robust and so easy to diagnose that I'm very likely to never ever be paged, I'm ok being on call.
But if there is some psychopath product manager making engineering decisions above engineering, forcing to increase tech debt constantly, guaranteeing constant outages and thus constant on-call pages then a big fat no. I won't be on call.
The whole, "How can I get people to do X when they don't want to do it for $Y?" is just a funny question. It is like asking, "How do I buy a sandwich when no one wants to sell me one for $1?"
Of course if everyone except a single employee IS willing to do X for $Y then the company many choose to replace them.
Of course if everyone except a single employee IS willing to do X for $Y then the company many choose to replace them.
Seriously, it's crazy that everyone doesn't already know the answer is, "Pay them enough to make it worth their time."
Or alternatively, just don't have on-call?
It's incredible how many teams have on-call for their internal service which is only used during business hours across 1-4 timezones. If your system has predictable usage and isn't global, there's very little reason to have on-call unless you have a client willing to pay $$$ or the loss of business would be catastrophic, in which case you can absolutely afford to compensate your devs for it.
It's incredible how many teams have on-call for their internal service which is only used during business hours across 1-4 timezones. If your system has predictable usage and isn't global, there's very little reason to have on-call unless you have a client willing to pay $$$ or the loss of business would be catastrophic, in which case you can absolutely afford to compensate your devs for it.
These are all very good points and well said, I agree!
They might even be able to do other things that would make it worth it. Like giving additional vacation for being on call, etc.
[deleted]
>"How do I buy a sandwich when no one wants to sell me one for $1?
Easy, you learn how to make your own sandwich. buy bulk and you amortize the cost to less than $1/sandwich
Of course, technical expertise isn't as easy as putting condiments between bread. Guess you gotta pay your labor if you're not willing to do it yourself.
Easy, you learn how to make your own sandwich. buy bulk and you amortize the cost to less than $1/sandwich
Of course, technical expertise isn't as easy as putting condiments between bread. Guess you gotta pay your labor if you're not willing to do it yourself.
This feels like a possible troll.
Also by the same person: I work from home as a software engineer and my job is happy with my performance. But I'm putting in little effort. Am I a bad person or employee? [1]
---
I used to be a Senior Software Engineer at another company but recently I took a lower level 2 position at another company remotely. I am very good at what I do. I can usually take a well groomed agile story in any tech stack and complete it way ahead of schedule. The company always praises me for this.
But the truth is they cannot keep me busy. I know I can help out else where but my output is already so very high, especially compared to everyone else.
So instead I just sit around all day playing video games while I wait for my next work assignment. This can sometimes be days or even a week. I don't go looking for it. I have so much free time on hands that I am seriously considering getting a 2nd full time software engineering job remotely and doing this all over again.
No one is the wiser.
Does this make a bad person or employee?
[1]: https://philosophy.stackexchange.com/questions/91282/i-work-...
Also by the same person: I work from home as a software engineer and my job is happy with my performance. But I'm putting in little effort. Am I a bad person or employee? [1]
---
I used to be a Senior Software Engineer at another company but recently I took a lower level 2 position at another company remotely. I am very good at what I do. I can usually take a well groomed agile story in any tech stack and complete it way ahead of schedule. The company always praises me for this.
But the truth is they cannot keep me busy. I know I can help out else where but my output is already so very high, especially compared to everyone else.
So instead I just sit around all day playing video games while I wait for my next work assignment. This can sometimes be days or even a week. I don't go looking for it. I have so much free time on hands that I am seriously considering getting a 2nd full time software engineering job remotely and doing this all over again.
No one is the wiser.
Does this make a bad person or employee?
[1]: https://philosophy.stackexchange.com/questions/91282/i-work-...
Well-noted.
Oh, I remember this. It made my eyes twitch a bit when I read that the manager wanted to put the dev on PIP, after changing the employment terms for fun, even though the dev asked about being on-call very specifically during interviews, according to the manager...
One other solution nobody has suggested.
Hire lots and lots of staff, and work in shifts. Have enough people on each shift that even when people use PTO or fall ill, someone is always working.
Nobody ever has to be on call. Instead everyone works normal hours, but someone is always available. You’ll still have to pay extra if you want people working on holidays, though.
Hire lots and lots of staff, and work in shifts. Have enough people on each shift that even when people use PTO or fall ill, someone is always working.
Nobody ever has to be on call. Instead everyone works normal hours, but someone is always available. You’ll still have to pay extra if you want people working on holidays, though.
A company I worked at started (inconsistently) adopting a "follow the sun" on-call approach once they were large enough to have enough engineering staff on more than one continent. There was certainly still some time that needed coverage that was outside anyone's work hours, but at least people weren't having entire days (including their sleep) ruined by on-call.
It's probably the worst solution for an employer. if they don't want to give 50% salary to keep current staff on call, they won't want to spend 2x to have staff on hand in staggering hours.
What about weekends?
Submitter but not OP. Just to be clear the only answer to this is a daily stipend for being on call PLUS a payout per incident (min. 1hr billing). And of course, it's opt-in.
Anything else is devaluing yourself to the poor decisions of others. "I will work harder!".
Anything else is devaluing yourself to the poor decisions of others. "I will work harder!".
No, the only solution in this case is full pay for each minute one is on-call. A (professional) firefighter is mostly on-call on his shift and is paid the same amount whether he's waiting or at an incident. Sure, he does stuff while waiting (like maintenance). But nobody said your It stuff couldn't do maintenance too.
If you really need on-call with 15min response time, design your business around it (and your finances!).
The whole blog could just stop at: > The CTO mandated [...]
If you really need on-call with 15min response time, design your business around it (and your finances!).
The whole blog could just stop at: > The CTO mandated [...]
I have a team of six people, I don't want two members working opposite hours for a week at a time. If anything that will produce more incidents...
The problem with every on-call article is that the best practice for a single team, a company with multiple teams, and a company with multiple continents are totally different.
The problem with every on-call article is that the best practice for a single team, a company with multiple teams, and a company with multiple continents are totally different.
> pay for each minute one is on-call
Isn't that just overtime? I'd much prefer to bring a laptop with me a few weekends a year than be expected to fill a full workday with my family around the house on a weekend.
Isn't that just overtime? I'd much prefer to bring a laptop with me a few weekends a year than be expected to fill a full workday with my family around the house on a weekend.
See it otherwise: It's not overtime, as a firefighter does not overtime 8h a day. It's your job and your job is to be on-call while writing code or vice-versa.
I think the only viable method for such a fast response time is that it's your work. You will have off on Friday but work on Saturday (or something like that).
If it's more like 1h response time sure, take your laptop with you to familys dinner.
I think the only viable method for such a fast response time is that it's your work. You will have off on Friday but work on Saturday (or something like that).
If it's more like 1h response time sure, take your laptop with you to familys dinner.
I don't really like the firefighter analogy to be honest. A Firefighter is on call but to compensate for their physical well-being they tend to only work 3-4 days a week (even if those 3 days are 24 hours spent in the station, potentially on the weekend). Moreover, fires are inevitable and frequent, despite being sporadic.
I think there is certainly merit to wondering what format a business needs based on client demands, but a mentally focused white collar job doesn't need to go as extreme as a firefighter who is literally burning out their body in the line of duty.
I think there is certainly merit to wondering what format a business needs based on client demands, but a mentally focused white collar job doesn't need to go as extreme as a firefighter who is literally burning out their body in the line of duty.
I partially agree with you and I wanted to use the analogy more for the part where he/she only really fights fires a part of the compensated time and is also paid for being "just waiting". I think taking the analogy to far won't work anyway (fires don't happen that frequent actually in less big cities, there's a lot of other stuff they do, compensation IMHO dies not relate to putting your life at risk at all, there's volunteers to which are even paid less, ...).
My main point is along the line if your customer wants these nines, they have to do (pay) something. If you as employer want it, same. You as employee are just at "the end of it".
My main point is along the line if your customer wants these nines, they have to do (pay) something. If you as employer want it, same. You as employee are just at "the end of it".
This only works if the incident rates are completely divorced from the work of the engineers. Otherwise it creates a perverse incentive to increase the number of incidents.
Yeah, I think just paying volunteers to be on call is fine. Adjusted for inflation, I'd expect this number to be around $2000/week. This is incredibly cheap compared to hiring 3 teams for follow-the-sun support, and gives the early-career folks on your team the opportunity to make a little bit of extra money; somewhat rare in the world of "engineering ladders" and whatnot where it is kind of a slog before you are rolling in cash.
HR and performance reviews should be able to handle that edge-case.
Where I work, the on-call deal is that it is opt-in, you're on one week and then have six weeks or so during which you will not be put back on call no matter what.
Base compensation which you receive regardless of whether there are any incidents during the week or not, plus hourly compensation (50-100% overtime + another 40% for the inconvenience)
Also, those who have opted in are quite flexible when it comes to covering for each other for a few hours - so you can still go to the movies or host a birthday party for a kid without fear of the phone suddenly buzzing.
Result? People are doing their best to get in on the roster, as the compensation feels worth it.
Try that.
Base compensation which you receive regardless of whether there are any incidents during the week or not, plus hourly compensation (50-100% overtime + another 40% for the inconvenience)
Also, those who have opted in are quite flexible when it comes to covering for each other for a few hours - so you can still go to the movies or host a birthday party for a kid without fear of the phone suddenly buzzing.
Result? People are doing their best to get in on the roster, as the compensation feels worth it.
Try that.
This is a pretty fair comment, but also not scratching the surface in some ways. Firstly, any decent boss will give you days off if some gnarly on-call stuff happens. Anywhere from "you can come in at 2pm and leave at 5pm" to "take Monday and Tuesday off, no one will track it". This is because they know it is a retention issue.
Secondly, there is a cultural divide sometimes. Generally, iOS programmers aren't on call, because they can't ship anything in less than a day or two. So, you get this phenomenon where they say "I'm an expert in Apple user interfaces" (a fine thing to know about!), but everyone resents them, because they are not on call.
Secondly, there is a cultural divide sometimes. Generally, iOS programmers aren't on call, because they can't ship anything in less than a day or two. So, you get this phenomenon where they say "I'm an expert in Apple user interfaces" (a fine thing to know about!), but everyone resents them, because they are not on call.
> Should we just PIP the engineer
No, fire them without cause like a grown up and take the hit to your unemployment insurance. Don't put them through some three-ring circus act masquerading as employee development. It's undignified and disrespectful. You and the employee have a disagreement over the job description -- this not performance related. Don't try and spin it as that.
It amazes me that these B-school pricks managed to enshrine at-will employment across the country, and they still won't just fucking fire people like adults. No, instead we got one of the worst MBA-isms visited on corporate America, the PIP. Fucking cowards, the lot of them.
No, fire them without cause like a grown up and take the hit to your unemployment insurance. Don't put them through some three-ring circus act masquerading as employee development. It's undignified and disrespectful. You and the employee have a disagreement over the job description -- this not performance related. Don't try and spin it as that.
It amazes me that these B-school pricks managed to enshrine at-will employment across the country, and they still won't just fucking fire people like adults. No, instead we got one of the worst MBA-isms visited on corporate America, the PIP. Fucking cowards, the lot of them.
What's the rationale for setting up the on-call schedule? Is the system that unstable?
> on call at least once a week, including the work day and evening, once a month, including the weekend of that week.
> I should also note there is a required 15 min response time to pages. The person MUST be able to be online and fixing the problem within 15 mins of the page.
I've worked a position where I had extra duties on the weekend, so I effectively worked 12+ days in a row many times. I've also been on an on-call schedule like this with a short response time. Both of these severely impact your personal life.
You have to realize that the two most valuable things you can give engineers is their own time back and money.
This is how I would probably try to handle it: To start, it would be opt-in and not mandatory, as this type of responsibility is difficult for those with small children or other family members, such as elderly, needing care. The 15 minute response time also means you should be being paid an hourly rate (maybe even 1.5x or 2x) while being on-call, whether there's an incident or not. This is because it severely limits what you can do -- you're basically chained to your laptop and an internet connection. Anyone responding to an on-call event after hours should also be given a half or full day off the next work day. Weekend on-call should at a minimum be given a day off the following week, preferable two.
> on call at least once a week, including the work day and evening, once a month, including the weekend of that week.
> I should also note there is a required 15 min response time to pages. The person MUST be able to be online and fixing the problem within 15 mins of the page.
I've worked a position where I had extra duties on the weekend, so I effectively worked 12+ days in a row many times. I've also been on an on-call schedule like this with a short response time. Both of these severely impact your personal life.
You have to realize that the two most valuable things you can give engineers is their own time back and money.
This is how I would probably try to handle it: To start, it would be opt-in and not mandatory, as this type of responsibility is difficult for those with small children or other family members, such as elderly, needing care. The 15 minute response time also means you should be being paid an hourly rate (maybe even 1.5x or 2x) while being on-call, whether there's an incident or not. This is because it severely limits what you can do -- you're basically chained to your laptop and an internet connection. Anyone responding to an on-call event after hours should also be given a half or full day off the next work day. Weekend on-call should at a minimum be given a day off the following week, preferable two.
> Is the system that unstable?
I'm generally anti-on-call, but c'mon, this is an unreasonable standard. No system is 100% perfect, never needing manual intervention to get things going again when something happens.
Any business that has a service that is supposed to be running 24/7 will likely need some form of on-call.
I'm generally anti-on-call, but c'mon, this is an unreasonable standard. No system is 100% perfect, never needing manual intervention to get things going again when something happens.
Any business that has a service that is supposed to be running 24/7 will likely need some form of on-call.
If you have a business that needs to operate 24/7 then you need your Operations team to hire accordingly and then on-call should normally be unnecessary. Operations should be able to fix an emergency that is unsafe, insecure or illegal. Anything else can wait until the next working day even if management might prefer it not to because that costs money.
If a business can't or won't make those arrangements (and pay the bill for them) then maybe its management team need to recognise that fully reliable 24/7 operation is not something they can guarantee and they should plan accordingly.
If a business can't or won't make those arrangements (and pay the bill for them) then maybe its management team need to recognise that fully reliable 24/7 operation is not something they can guarantee and they should plan accordingly.
> this is an unreasonable standard
My phrasing could have been more clear, I'm not saying you need a 100% perfect system and no on-call.
It's weird they're instituting on-call for everyone now, and I'm curious about the reason behind the policy change. I've heard of "on call for everyone" being used as both a response to programmers shipping unreliable code, and also as a preventative measure to encourage programmers to ship more reliable code.
My phrasing could have been more clear, I'm not saying you need a 100% perfect system and no on-call.
It's weird they're instituting on-call for everyone now, and I'm curious about the reason behind the policy change. I've heard of "on call for everyone" being used as both a response to programmers shipping unreliable code, and also as a preventative measure to encourage programmers to ship more reliable code.
In many jurisdictions suddenly "mandating" that an employee be on call every week/month would be a proposed change of employment contract. I.e. the employee would be fully within their right to refuse until the employer makes an acceptable offer, if at all.
Also, this "manager" sounds totally clueless...
Also, this "manager" sounds totally clueless...
In many jurisdictions, requiring uncompensated oncall would, by itself, possibly be "constructive dismissal".
PIPing without an actual performance problem is also constructive dismissal, and doing it because the employee is refusing a change is possibly retaliation.
Realistically, in the U.S. at least, there's very little risk that the employer will get sued, but it's still shitty and will eventually bite the employer if they get into a habit of this.
PIPing without an actual performance problem is also constructive dismissal, and doing it because the employee is refusing a change is possibly retaliation.
Realistically, in the U.S. at least, there's very little risk that the employer will get sued, but it's still shitty and will eventually bite the employer if they get into a habit of this.
The people you have will quit and the people you hire want to be compensated just as the others. So firing really isn't an option, unless you think you're overpaying your current staff and can get the same quality cheaper.
Over here the standard usually is 50% hourly pay for being on-call. On-call requires you to be 15-30 minutes away from whatever you can use to fix issues, usually work laptop + internet connection. Oh and you need to be sober (...enough to answer the phone while sounding sober =)
As soon as something happens, you jump to regular overtime pay so if it's on a weekend+Sunday, you're making around 200-300% pay. If it's just after others logged off, you're getting normal pay, maybe 50% extra depending on the company.
I know a lot of people who have paid off loans just by hogging all the on-call hours they can. It's not especially hard if the playbooks are well made and nothing goes horribly wrong.
As soon as something happens, you jump to regular overtime pay so if it's on a weekend+Sunday, you're making around 200-300% pay. If it's just after others logged off, you're getting normal pay, maybe 50% extra depending on the company.
I know a lot of people who have paid off loans just by hogging all the on-call hours they can. It's not especially hard if the playbooks are well made and nothing goes horribly wrong.
A lot of managers or CxO think they own the employees and when suddenly they change mindset, they expect it for free, there’s nothing more pity than a C level who’s more of business guy than technical and understands everything comes with a price, and yet wanted it free of charge, pay him for the extra services you require, each contract is a unique case, just because other engineers do or agreed to do so, doesn’t mean he should if the contract says otherwise.
From my own experience, I understand, being on call could be important for engineers, but CEO MUST understand, not all people could accept this.
So, in real life, need agile approach.
Some people just accept to be on call all time, you could consider them as your core team with appropriate benefits and value them, and respect them, really.
Second group of people could accept receive calls during their shift. This is also not bad. Value this.
And exists people, who are good workers, make great value, but could be on call very limited time or nearly no time. You should decide, with them individually, how you will work. It is really possible, some very valuable people will not be on call at all.
So, in real life, need agile approach.
Some people just accept to be on call all time, you could consider them as your core team with appropriate benefits and value them, and respect them, really.
Second group of people could accept receive calls during their shift. This is also not bad. Value this.
And exists people, who are good workers, make great value, but could be on call very limited time or nearly no time. You should decide, with them individually, how you will work. It is really possible, some very valuable people will not be on call at all.
I love how the manager’s immediate reaction is to go for constructive dismissal.
Good. If this goes through I hope the employee walks. They clearly want workers who can do those hours and they need to renegotiate that. be it with existing employees or new ones (who are likely going to cost more, plus time to train. Buf if a company wants to cut off its nose to spite the face, go for it).
The times I've been on-call as a sr backend dev, my compensation was one of
- $700 for any week that I was on call (whether or not there were any incidents, and more often than not there were none)
- paid my phone bill, roughly $100/mo (including data + amortized phone cost)
- free to take off the next day any time I'm paged off-hours.
There have been other times I was unofficially on-call (at small startups/companies) with no compensation, but I don't think there was enough scale/frequency for it to be a sticking point.
- $700 for any week that I was on call (whether or not there were any incidents, and more often than not there were none)
- paid my phone bill, roughly $100/mo (including data + amortized phone cost)
- free to take off the next day any time I'm paged off-hours.
There have been other times I was unofficially on-call (at small startups/companies) with no compensation, but I don't think there was enough scale/frequency for it to be a sticking point.
Incentives are important -- unlike other commentators, I agree with my management that paying extra for time spent actually working when on-call is not a great idea: it sets up a perverse incentive. We have a base hourly rate for on-call (with a half-hour response time) and time off in lieu for actual call-outs. Teams differ, but my team has separate rotas for in-hours and out-of-hours. This means that if you're up all night, you get time to sleep without needing someone to take on-call from you during the day.
An average week of out-of-hours involves precicely zero pages. This is also important: when we get paged for something, we take steps to avoid being paged for the same thing again. Not just because we value our sleep, but because management values our services being up and running so we're encouraged to fix stuff that's broken.
We did have a team saying that their services were noisier than other teams', and asking if they could be paid more as a consequence. The answer was to fix their service reliability, not to incentivise people to wait until out-of-hours to declare an incident so they trigger the bonus payment.
So long as I've got mobile coverage, I'm free to get on with my life. I carry my laptop if I'm more than 30 minutes from home, but I don't expect to need to use it.
An average week of out-of-hours involves precicely zero pages. This is also important: when we get paged for something, we take steps to avoid being paged for the same thing again. Not just because we value our sleep, but because management values our services being up and running so we're encouraged to fix stuff that's broken.
We did have a team saying that their services were noisier than other teams', and asking if they could be paid more as a consequence. The answer was to fix their service reliability, not to incentivise people to wait until out-of-hours to declare an incident so they trigger the bonus payment.
So long as I've got mobile coverage, I'm free to get on with my life. I carry my laptop if I'm more than 30 minutes from home, but I don't expect to need to use it.
>it sets up a perverse incentive.
how so. It's already the incentive to work. If you require more work and more of a employee's time, you pay more money. It's not that complicated.
In the same vein, there's expected business hours, so requiring a business to operate outside of those require more money. If they need that sort of timely response off hours, they pay more money too. So the incentive should be "try to call us during business hours or wait for a response".
how so. It's already the incentive to work. If you require more work and more of a employee's time, you pay more money. It's not that complicated.
In the same vein, there's expected business hours, so requiring a business to operate outside of those require more money. If they need that sort of timely response off hours, they pay more money too. So the incentive should be "try to call us during business hours or wait for a response".
I'm all in favour of paying someone to work! And every minute spent on-call should be compensated.
What I don't like is "if my service is flaky then I get more money", because I don't want my service to be flaky but I like getting money.
If you're paying me to be available, but giving them time in lieu for any time worked, then you're not getting more of my time for free. Conversely, if you pay me only when I'm called out (which I recognise isn't something you've suggested) then I get the hassle of being on call but I'm only paid if my service breaks.
What I don't like is "if my service is flaky then I get more money", because I don't want my service to be flaky but I like getting money.
If you're paying me to be available, but giving them time in lieu for any time worked, then you're not getting more of my time for free. Conversely, if you pay me only when I'm called out (which I recognise isn't something you've suggested) then I get the hassle of being on call but I'm only paid if my service breaks.
> So long as I've got mobile coverage, I'm free to get on with my life.
I don't doubt this, but surely you can agree many people lead lives which they would not be free to live in this arrangement? Putting children to bed. Driving somewhere. Cooking. Going to a bar...
I don't doubt this, but surely you can agree many people lead lives which they would not be free to live in this arrangement? Putting children to bed. Driving somewhere. Cooking. Going to a bar...
I do all of those things. And yes, I know it's frustrating not to be able to enjoy a drink but that's one of the things we're being paid to compensate for. The others are all possible while on-call, especially when I'm justified in not expecting to be called out.
I've also dropped off the on-call rota, and encouraged others to do the same, to care for very young children or family who need care that would be compromised by responding to a page. Or vice-versa, of course: if I've already been up half the night looking after a newborn then I'm not going to be much help. My contract doesn't oblige me to be on-call, I wouldn't do it if the compensation didn't make it worthwhile.
I've also dropped off the on-call rota, and encouraged others to do the same, to care for very young children or family who need care that would be compromised by responding to a page. Or vice-versa, of course: if I've already been up half the night looking after a newborn then I'm not going to be much help. My contract doesn't oblige me to be on-call, I wouldn't do it if the compensation didn't make it worthwhile.
I am amazed by how beautiful and clear the first answer is.
How nice, uncondemning it is, how it really gives the possibility of the OP is truly clueless that he's in the wrong.
How nice, uncondemning it is, how it really gives the possibility of the OP is truly clueless that he's in the wrong.
It really is underrated how effective calm direct communication can be when it's not coupled with attempts to injure with insults.
Recruit people into being on call by positively incentivising it so that it's their own choice, don't force it on them.
Throwaway so I can talk honestly.
I used to work on call in non-software, as a blood analyzer Field Service Engineer for medtech companies, like Siemens, Roche, Beckman-Coulter, Bio-Rad, etc. I spent 5 years doing this.
In that industry and job it's standard to work on call rotations that last for 7 days through the weekend and repeat every 3-5 weeks depending on team size. Because the job is hands-on and at customer sites, the minimum response time is usually a bit more than 15 minutes.
If anyone is considering accepting an on-call rotation, don't. It takes over your life. At first you may think it's fun to swoop in and be the hero fixit person, or maybe you at least tolerate it because of high base pay or something. Eventually you grow to hate the sound that accompanies the callout text or email. By the end, it triggered a visceral physical negative reaction I later learned was a minor anxiety attack.
The thing managers seeking to add on-call don't understand (those who haven't done it themselves), is that when you're on call you can't do anything. Not only can you not drink (minor annoyance), you can't even leave the house for normal stuff like a movie (medium), or attend events like a kids graduation or recital (major).
And before you say "just swap rotations around on your team," I worked with about 10-15 guys and you'd be surprised how hard it is to swap. Everyone makes plans for things all the time without realizing it would interfere with on-call, like going to the gym. At the start of the year when the on-call rotations would be released to us, there was a mad rush to submit vacation requests to get it locked in before illness or other things required swapping on-call around, much less having to take field requests from colleagues that you may (will) have to rely on when you in turn need a swap, so you can't just ignore every request.
This manager and CTO are hilariously unprepared and uninformed, bordering on pure dreaming. A 15 minute max response time is criminally unrealistic, you can't even go to the grocery store. And once a week seems low on paper until you realize that depending on team size, you can easily wind up being forced to give up a quarter of your weekends for the year (as I had to).
I haven't worked on-call for software, so maybe it's different, but I doubt it's really too different.
After 5 years of all that lost time, the amount I'd have to be paid to go back on-call is so astronomically high no company would ever pay it. Actually, now that I have a kid, there's probably no amount I could be paid if it means missing time with them.
I used to work on call in non-software, as a blood analyzer Field Service Engineer for medtech companies, like Siemens, Roche, Beckman-Coulter, Bio-Rad, etc. I spent 5 years doing this.
In that industry and job it's standard to work on call rotations that last for 7 days through the weekend and repeat every 3-5 weeks depending on team size. Because the job is hands-on and at customer sites, the minimum response time is usually a bit more than 15 minutes.
If anyone is considering accepting an on-call rotation, don't. It takes over your life. At first you may think it's fun to swoop in and be the hero fixit person, or maybe you at least tolerate it because of high base pay or something. Eventually you grow to hate the sound that accompanies the callout text or email. By the end, it triggered a visceral physical negative reaction I later learned was a minor anxiety attack.
The thing managers seeking to add on-call don't understand (those who haven't done it themselves), is that when you're on call you can't do anything. Not only can you not drink (minor annoyance), you can't even leave the house for normal stuff like a movie (medium), or attend events like a kids graduation or recital (major).
And before you say "just swap rotations around on your team," I worked with about 10-15 guys and you'd be surprised how hard it is to swap. Everyone makes plans for things all the time without realizing it would interfere with on-call, like going to the gym. At the start of the year when the on-call rotations would be released to us, there was a mad rush to submit vacation requests to get it locked in before illness or other things required swapping on-call around, much less having to take field requests from colleagues that you may (will) have to rely on when you in turn need a swap, so you can't just ignore every request.
This manager and CTO are hilariously unprepared and uninformed, bordering on pure dreaming. A 15 minute max response time is criminally unrealistic, you can't even go to the grocery store. And once a week seems low on paper until you realize that depending on team size, you can easily wind up being forced to give up a quarter of your weekends for the year (as I had to).
I haven't worked on-call for software, so maybe it's different, but I doubt it's really too different.
After 5 years of all that lost time, the amount I'd have to be paid to go back on-call is so astronomically high no company would ever pay it. Actually, now that I have a kid, there's probably no amount I could be paid if it means missing time with them.
It depends though. I totally see your point and when I was working in a company where we spread the oncall between all people it was a) usually no problem to swap a few hours (we did full weeks, so if you had one thing for 4h one night, usually someone would cover) and b) our response times weren't quite SO harsh - so we did some things, as long as you had cell coverage and a laptop nearby. Of course the cinema was out, or being on the road for hours, but going for groceries was no problem.
I still hated it and don't want to do it anymore because we had so many factors out of our control that made "false" alarms where the only thing to do was check if it was really a problem and then open a ticket upstream at a vendor...
I still hated it and don't want to do it anymore because we had so many factors out of our control that made "false" alarms where the only thing to do was check if it was really a problem and then open a ticket upstream at a vendor...
It seems like OP and their CTO would like to have reliable 24/7 operations but do not have the skill to do it by designing systems that won't fail catastrophically out of hours or the budget to do it by hiring dedicated staff to be available at all times to fix problems. So instead they are trying to abuse their existing staff.
The correct response to this IMHO is for the existing staff to collectively tell management where to go. In life you can't always get what you want if you don't have the money or make the effort it requires.
The correct response to this IMHO is for the existing staff to collectively tell management where to go. In life you can't always get what you want if you don't have the money or make the effort it requires.
Would be an easy problem to solve with a carrot instead of a stick. Just create a bonus for being the first person to respond to an out of hours group text, another bonus for being the person to respond to the most out of hours problems, and a year-end bonus for the whole team if no out of hours problem goes more than thirty minutes without a response from one of them. They'll organize the on call system themselves with no whining and you'll only spend money on actual problems since you're not paying anyone hourly to be on call. Everybody wins.
[deleted]
Put yourself into the oncall rotation for starters.
For a US exempt employee it would be fairly straightforward to push oncall expectations into a role. But in other countries with robust labor protections it might be more challenging. Ultimately the solution is money.
But doing it yourself goes a long way to convincing your team it’s necessary. I personally do this as a manager and I find it helps a lot. If you get paged a few times at 3am it really helps align your priorities with your team.
For a US exempt employee it would be fairly straightforward to push oncall expectations into a role. But in other countries with robust labor protections it might be more challenging. Ultimately the solution is money.
But doing it yourself goes a long way to convincing your team it’s necessary. I personally do this as a manager and I find it helps a lot. If you get paged a few times at 3am it really helps align your priorities with your team.
Another unspoken red flag is that this app is having so many bugs in production that it requires constant attention from its developers. How are all these bugs getting past QA? Is there a legit development process in place? Who is signing off on such buggy software?
This is what always gets me about these on-call discussions. It's like there is some kind of Stockholm syndrome thing going on where employees hired to write software are forced into working under completely different conditions to what software development should require and no-one ever stops to ask how badly you have screwed up the whole company culture for this issue to be relevant at all.
I recommend investing in real QA and dumping the move-fast-and-break-stuff attitude and probably 90% of the madness that gets bundled under the Agile and DevOps banners as an excuse for not planning or designing anything properly any more. Your Operations team who are running a 24/7 schedule (and hiring for it) anyway should be able to deal with genuine emergencies without requiring non-specialist developers to be on call routinely. The world has worked that way forever and still does outside the DevOps bubble - including numerous industries that operate 24/7 and rely on software to do so.
I recommend investing in real QA and dumping the move-fast-and-break-stuff attitude and probably 90% of the madness that gets bundled under the Agile and DevOps banners as an excuse for not planning or designing anything properly any more. Your Operations team who are running a 24/7 schedule (and hiring for it) anyway should be able to deal with genuine emergencies without requiring non-specialist developers to be on call routinely. The world has worked that way forever and still does outside the DevOps bubble - including numerous industries that operate 24/7 and rely on software to do so.
Assuming everyone can just "have an operations team running 24/7" out the gate is also a bubble.
Anyway, lots of jobs with variable demand (doctors, vets, supers, utility workers, etc.) work on a fixed / % of salary daily rate + time-and-a-half when activated. What I agree is bullshit is TFA though, if it's not in your contract, don't do it.
Anyway, lots of jobs with variable demand (doctors, vets, supers, utility workers, etc.) work on a fixed / % of salary daily rate + time-and-a-half when activated. What I agree is bullshit is TFA though, if it's not in your contract, don't do it.
But if you don't have 24/7 Operations coverage then you probably aren't really able to run fully reliably all the time anyway. And that is perfectly OK and normal for smaller and newer businesses. They just need to stop kidding themselves that they can operate at that level by exploiting their dev team.
Fuck you, hire an SRE team.
Current employer is trying to have me on call for free 24/7. Not happening.
Current employer is trying to have me on call for free 24/7. Not happening.
Easy. Pay them a decent salary for every hour they are on call. Simple as that.
You can't pay me a decent* salary for on-call. This is true of many people. One developer I worked with took medication at night which rendered him completely useless, and skipping the medication resulted in several very bad days. For me, it's simply a work/life balance thing, and I can find work that doesn't need it.
The "easy" answer is to put it in the job ad, and hire people who agree to it. Because forcing that requirement on an existing employee just isn't going to happen.
* by all means, pay me an indecent wage, enough that I can retire after a year!
The "easy" answer is to put it in the job ad, and hire people who agree to it. Because forcing that requirement on an existing employee just isn't going to happen.
* by all means, pay me an indecent wage, enough that I can retire after a year!
Years ago when I worked second-level support, I was the on-call guy for Thanksgiving. Our customers were specialty retailers, so I thought nothing of it. I figured they'd be closed. Turns out, we had an "Urban Clothing Store" as a customer, and on that day — what?! — someone couldn't ring up an 8-ball jacket? I was with my family, enjoying a cocktail when I got the call. I have seldom been more pissed off. In the end, from what I could remember, after wasting almost an hour, we tabled the whole incident.
I'm with you. On-call is never worth it. If a company needs 24/7 coverage, they should hire round the clock.
I'm with you. On-call is never worth it. If a company needs 24/7 coverage, they should hire round the clock.
Yup, agreed.
At the very least, make it opt-in, and keep raising the on-call pay until you have enough existing employees who will decide the trade off is worth it. At some point it maybe ends up being more economical to follow your suggestion to put out a job ad specifically for on-call people.
At the very least, make it opt-in, and keep raising the on-call pay until you have enough existing employees who will decide the trade off is worth it. At some point it maybe ends up being more economical to follow your suggestion to put out a job ad specifically for on-call people.
As another reply said, you can't afford me.
I did on call 24/7/365 for 23 years. I'll never do it again. My phone goes silent in the evening and doesn't come online again until I get up to come to work. That's never going to change. I'm not 20 anymore, I don't function on no sleep.
I did on call 24/7/365 for 23 years. I'll never do it again. My phone goes silent in the evening and doesn't come online again until I get up to come to work. That's never going to change. I'm not 20 anymore, I don't function on no sleep.
> I did on call 24/7/365 for 23 years.
I hope you made enough money to retire comfortably. That sounds like utter hell.
I hope you made enough money to retire comfortably. That sounds like utter hell.
Not even close. Military enlisted. Living the good life writing software 9 to 5 now though.
Pay more.
Reward them for issues fixed in off hours while on call. Make a game with rewards out of it.
Money won't answer this problem. Hire someone(s) to cover this is cheaper.
Pay them for the value they provide.
*laughs in Medicine
[deleted]
Pay more or let them go. That’s it. If the job suddenly requires on call, and someone refuses to comply then show them the door and hire someone else with the explicit expectation that there will be on-call.
“Show them the door” seems a bit callous. If the hiring manager didn’t think to talk about on call responsibilities, that’s their own mistake. Maybe that doesn’t change that they need someone on call and maybe they still need to find a new employee, but a job doesn’t own a person 24/7 unless that signed up for it. If a manager pulled that bait and switch shit on me I’d definitely never be recommending to my smart friends to work there.
Prepare yourself for a constructive dismissal settlement.
If you are begging your engineers to be on call, you have built the wrong engineering culture. Your core engineering team should feel ownership of the app, it should be their baby. What happens in this case is EVERYONE shows up when there is a site issue, and your best engineers burnt out. Instituting an on-call schedule is what a good engineering manager does to force engineers to NOT be on-call 24/7, and instead institute an equitable distribution of responsibility. I think all small eng teams should have on-call rotations, and it's the reason behind why I built https://heiioncall.com/.
> it should be their baby.
Made that mistake. Glad it only took one layoff to show that that's never true in Big Business. Never again. I am a worker providing my talent to work on a product I don't own. I will take responsibilities but never "ownership". "ownership" implies that I have a stake in the product, and the product goes with me no matter what. Which I have yet to experience (maybe one day).
Made that mistake. Glad it only took one layoff to show that that's never true in Big Business. Never again. I am a worker providing my talent to work on a product I don't own. I will take responsibilities but never "ownership". "ownership" implies that I have a stake in the product, and the product goes with me no matter what. Which I have yet to experience (maybe one day).
The standard in SV seems to be that on-call is required and uncompensated (some companies will dishonestly try to suggest that your salary is higher than it would be otherwise). This is gross, and we shouldn't stand for it. There are exceptions of course, but even companies I've heard of that compensate for on-call seem to fall far short of what should be expected and required.
On-call should be compensated at a base per-hour rate (outside of 9am-5pm), with an elevated rate for time spent handling an incident if the pager does go off, with a minimum payout of 1 hour of time. It should be opt-in. If not enough people want to do it to get you the coverage you want, raise the rates until your employees feel that the time is worth the money.
It's pretty messed up that OP here is throwing shade at an employee who is balking at this, wanting to put them on a PIP, plus a thinly-veiled suggestion that they're lying about having asked about on-call requirements during their interview process. Bottom line is that they're drastically changing the terms of employment without offering extra compensation, and while in many jurisdictions that's legal, it's nonetheless unethical and disgusting.