18-month $220k Powerwall solar energy installation nightmare(couriermail.com.au)
couriermail.com.au
18-month $220k Powerwall solar energy installation nightmare
https://www.couriermail.com.au/technology/innovation/im-sick-of-this-adelaide-dads-240000-18month-tesla-nightmare/news-story/3ee7702c7ec44bde25152585d267394e
41 comments
https://archive.ph/D8HM6
Summary: Tesla's Powerwall controller cannot handle 10 off grid batteries and sales should not of sold this system.
He is running beyond the scale of that Tesla's controller unit can really handle:
> Tesla soon worked out that the system was being overloaded due to having too many Powerwalls connected at once ... The gateway is being overloaded, reaching 100 per cent CPU and RAM usage. There’s too much info when they are all online.
He is doing something very uncommon among customers, and Tesla apparently doesn't even try to make it work:
> But while Tesla claims it can do off-grid 10-Powerwall setups, there are only a handful around the world – most are backed up by grid electricity, papering over any outages. ... “At the beginning there were actually only three 10-Powerwall installs in the world that were off-grid,”
They also apparently have poor support for off grid firmware updates, possibly related to the lack of CPU and memory to both keep the lights on and upgrade at once:
> “During these upgrade cycles, the system goes into this chaotic state that takes between four and eight hours, turning off and on 50 times,”
He is running beyond the scale of that Tesla's controller unit can really handle:
> Tesla soon worked out that the system was being overloaded due to having too many Powerwalls connected at once ... The gateway is being overloaded, reaching 100 per cent CPU and RAM usage. There’s too much info when they are all online.
He is doing something very uncommon among customers, and Tesla apparently doesn't even try to make it work:
> But while Tesla claims it can do off-grid 10-Powerwall setups, there are only a handful around the world – most are backed up by grid electricity, papering over any outages. ... “At the beginning there were actually only three 10-Powerwall installs in the world that were off-grid,”
They also apparently have poor support for off grid firmware updates, possibly related to the lack of CPU and memory to both keep the lights on and upgrade at once:
> “During these upgrade cycles, the system goes into this chaotic state that takes between four and eight hours, turning off and on 50 times,”
Somebody screwed up in the sales process then, either on the installer or vendor side. It's not acceptable to request a system in an unsupported configuration like this unless he was explicitly told beforehand "this isn't a supported configuration and we'll build it for you as long as you understand that".
In every company I have worked with, sales lies to get people to sign. It is a profession for naturally dishonest people.
A company I used to work for had an interesting solution to this.
There was an internal market, and all services used had to be paid for. Sales people could keep everything after paying the company a cut and all the costs of delivering what they promised the client.
So if they over promised to get a sale, it all came out of their personal profit. Seemed to work quite well.
There was an internal market, and all services used had to be paid for. Sales people could keep everything after paying the company a cut and all the costs of delivering what they promised the client.
So if they over promised to get a sale, it all came out of their personal profit. Seemed to work quite well.
How does that worked when delivery/implementations take months or years?
I'm not clear on the details. It was over 30 years ago and I wasn't a salesman. I was busy automating their warehouse systems, but I'd see the brand new Porsches pulling into the car park!
The entire company was structured as a set of internal markets though. Never seen anyone else do this.
The entire company was structured as a set of internal markets though. Never seen anyone else do this.
You’ve never worked for “that guy”? Because I have, several times. Some people get stupid when the checkbook comes out. In fact that’s a well known ploy by large effective companies. You can get misbehaving vendors to pay attention to you if you get out the checkbook or threaten to put it away.
> We were probably spending nearly $10,000 on electricity a year across the business and home,” he said. “We worked out we could do a large solar system which could deliver that 200-amp feed, be able to pay itself off and be off the grid.”
$220K to offset a $10k/yr electric bill is a 22 year payback period, which is quite a long time if you are using energy bills as a rationale.
The primary rationalization for storage when you have grid access already is grid backup, and the secondary rationalization is for energy price arbitrage. Backup is hard to value monetarily because it depends on how much not having electricity costs you. Arbitrage is easy to value, but doesn't pay back much unless you have extremely variable retail electric rates.
> “The most confusing and contentious point for me is that Tesla seems to express that the off-grid Powerwalls cannot guarantee 100 per cent uptime … [but] this was never expressed to me during deployment, and had I known this, I would not have embarked on this journey.”
For a $220k installation maybe they wouldn't have been incentivized to explain it. One mode of the grid behaves a bit like insurance, increasing reliability and masking variability. Going off grid when you have the option to stay on seems like an irrational choice.
$220K to offset a $10k/yr electric bill is a 22 year payback period, which is quite a long time if you are using energy bills as a rationale.
The primary rationalization for storage when you have grid access already is grid backup, and the secondary rationalization is for energy price arbitrage. Backup is hard to value monetarily because it depends on how much not having electricity costs you. Arbitrage is easy to value, but doesn't pay back much unless you have extremely variable retail electric rates.
> “The most confusing and contentious point for me is that Tesla seems to express that the off-grid Powerwalls cannot guarantee 100 per cent uptime … [but] this was never expressed to me during deployment, and had I known this, I would not have embarked on this journey.”
For a $220k installation maybe they wouldn't have been incentivized to explain it. One mode of the grid behaves a bit like insurance, increasing reliability and masking variability. Going off grid when you have the option to stay on seems like an irrational choice.
There is something to be said both financially and emotionally about getting your operating costs to be lower. And making a big capex expenditure in a year when you saw a windfall of money would flatten out your tax burden.
Of course here it’s not that simple because you’ll be replacing worn equipment at some point and you will have to spin proverbial plates to keep that working well.
I suspect this is about 3 parts financially sound and 2 parts Captain Ahab if you look at the full math.
Of course here it’s not that simple because you’ll be replacing worn equipment at some point and you will have to spin proverbial plates to keep that working well.
I suspect this is about 3 parts financially sound and 2 parts Captain Ahab if you look at the full math.
> There is something to be said both financially and emotionally about getting your operating costs to be lower.
I agree. I have a battery backup system too for similar reasons. However, I think that the operating cost savings start diminishing pretty quickly when you have to size a system to go completely off grid and simultaneously can't tolerate a higher outage rate.
That said, if you take the customers word, it sounds like Tesla Solar gave them the impression that they would have 100% off grid up time and didn't deliver on that due to their own inadequate equipment.
I agree. I have a battery backup system too for similar reasons. However, I think that the operating cost savings start diminishing pretty quickly when you have to size a system to go completely off grid and simultaneously can't tolerate a higher outage rate.
That said, if you take the customers word, it sounds like Tesla Solar gave them the impression that they would have 100% off grid up time and didn't deliver on that due to their own inadequate equipment.
But $150K of it was subsidized so it's only a 7 year payback period.
Not subsidized, loaned. FTA:
“Of that, $150,000 was from a South Australian government-backed solar loan, with another $70,000 in cash.”
There might be some subsidy in getting an attractive interest rate, but AFAICT, the article doesn’t say anything about that.
“Of that, $150,000 was from a South Australian government-backed solar loan, with another $70,000 in cash.”
There might be some subsidy in getting an attractive interest rate, but AFAICT, the article doesn’t say anything about that.
According to the article, the $150k was a loan, not subsidized. They have to pay it back.
This link doesn't work if you DNS block trackers. The URL redirects to https://tags.news.com.au which is a tracker. Do you have a direct link that works without a redirect?
The request there is for https://tags.news.com.au/prod/adblock/adblock.js which is blocked by uBo, the site still works fine for me.
> The gateway is being overloaded, reaching 100 per cent CPU and RAM usage. There’s too much info when they are all online.
Maybe there’s an n^2 algorithm somewhere, maybe the hardware is simply underspecced. Maintaining voltage stability in a system without mechanical inertia (like a generator would have) requires incredibly fast response times, and a system like that should have been verified to actually achieve those.
From the damage that was caused, it sounds like there are no “hardware” fuses built into this system either. That alone should be cause for regulators to take the product off the market.
What a joke.
Maybe there’s an n^2 algorithm somewhere, maybe the hardware is simply underspecced. Maintaining voltage stability in a system without mechanical inertia (like a generator would have) requires incredibly fast response times, and a system like that should have been verified to actually achieve those.
From the damage that was caused, it sounds like there are no “hardware” fuses built into this system either. That alone should be cause for regulators to take the product off the market.
What a joke.
> From the damage that was caused, it sounds like there are no “hardware” fuses built into this system either. That alone should be cause for regulators to take the product off the market.
No, it just sounds like inverter, to which there are two devices connected, one which suddenly draws a lot of current (like starting power tool, possibly the one pictured in the article) and an "electronic device" with switching power supply. When the large current draw happens, the inverter tries to compensate which results in a lot of power transmitted over higher frequency (several tens kilohertz rather than usual 50 Hz). That higher freq bypasses input cap and fries switching power supply.
Matthias Wandel did a video about his own problems with portable battery setup, which did the same: https://www.youtube.com/watch?v=c1QSn3i_IEo.
This inverter is just not suitable for powering different devices. ISTM those different "offgrid" technologies are just not ready for prod.
No, it just sounds like inverter, to which there are two devices connected, one which suddenly draws a lot of current (like starting power tool, possibly the one pictured in the article) and an "electronic device" with switching power supply. When the large current draw happens, the inverter tries to compensate which results in a lot of power transmitted over higher frequency (several tens kilohertz rather than usual 50 Hz). That higher freq bypasses input cap and fries switching power supply.
Matthias Wandel did a video about his own problems with portable battery setup, which did the same: https://www.youtube.com/watch?v=c1QSn3i_IEo.
This inverter is just not suitable for powering different devices. ISTM those different "offgrid" technologies are just not ready for prod.
It doesn’t have to be n² if one unit has to control the others. If a single unit requires base load + 10% per unit you could be in trouble at 7-8 units. That’s more than sufficient for most sales.
If anything maybe they should have a special power wall for large installations, with a beefier CPU and sturdier components.
It’s not uncommon in tech and even to an extent automotive for the premium model to stay one hardware revision ahead of the consumer unit and after a few recalls iron out the reliability problems, after the R&D is paid off on whales, bring it down market and introduce something new at the high end.
That’s both technologically sound and plays on baser behaviors of the well to do (namely, classism). If normal people have the same thing I have then it’s time to sell it so I can feel special again.
If anything maybe they should have a special power wall for large installations, with a beefier CPU and sturdier components.
It’s not uncommon in tech and even to an extent automotive for the premium model to stay one hardware revision ahead of the consumer unit and after a few recalls iron out the reliability problems, after the R&D is paid off on whales, bring it down market and introduce something new at the high end.
That’s both technologically sound and plays on baser behaviors of the well to do (namely, classism). If normal people have the same thing I have then it’s time to sell it so I can feel special again.
I've heard good stories of Tesla Powerwall, but I've heard enough horror stories to scare me away from that company's RE systems for good.
Browsing the "top" posts of r/TeslaSolar, you start to get an idea for how much of a pain in the ass is to deal with the company once things go pear shaped.
I'm not sure if I'm surprised or not that Tesla engineers didn't stress test their system by putting like, 50 Powerwalls on a string and seeing if the system could handle it.
Browsing the "top" posts of r/TeslaSolar, you start to get an idea for how much of a pain in the ass is to deal with the company once things go pear shaped.
I'm not sure if I'm surprised or not that Tesla engineers didn't stress test their system by putting like, 50 Powerwalls on a string and seeing if the system could handle it.
I find the whole layout perplexing, more like things went off into a marketing plan to show what can be done instead of doing something reliable.
If it can't handle 10 batteries and can't handle updates without backup from the grid, why did they make/sell this setup? Two controllers with ~5 batteries would make a situation where one is active/normal and the other is being the grid so the reliability would be the same as whatever they promise everyone else.. Similarly, he doesn't actually need to be off the grid, his connection just doesn't meet his peak usage so the level of chaos seems to be just to be able to say this is an off grid setup?
If it can't handle 10 batteries and can't handle updates without backup from the grid, why did they make/sell this setup? Two controllers with ~5 batteries would make a situation where one is active/normal and the other is being the grid so the reliability would be the same as whatever they promise everyone else.. Similarly, he doesn't actually need to be off the grid, his connection just doesn't meet his peak usage so the level of chaos seems to be just to be able to say this is an off grid setup?
Not bothering to troubleshoot customer problems seems to be a business strategy these days. One of my jobs is for a security firm that internally takes the attitude that if a small biz with a low ARR comes in with a critical problem, they are just to be strung along until they quit.
Does Tesla have a QA department, or is engineering expected to do this type of testing? I hear some companies got rid of QA to reduce costs.
Maybe this could have been simulated purely in software. Not sure.
Maybe this could have been simulated purely in software. Not sure.
I really miss my QA dept.
I think devs voted to get rid of them because it was often QA and Product ganging up on Dev, but if you’ve never experienced Dev and QA ganging up on Product, you’ve missed out.
Getting rid of QA meant that Product cannot be outvoted except by a walkout. It’s part of why I’m not as loud about DevOps becoming its own business unit. At least it’s 2 against 1 again.
I think devs voted to get rid of them because it was often QA and Product ganging up on Dev, but if you’ve never experienced Dev and QA ganging up on Product, you’ve missed out.
Getting rid of QA meant that Product cannot be outvoted except by a walkout. It’s part of why I’m not as loud about DevOps becoming its own business unit. At least it’s 2 against 1 again.
Question for people who have installed solar in a property that's used by other people: Is it easy to manage and perform basic troubleshooting?
A cousin proudly showed me her solar setup at a vacation home including the app showing her electricity savings and the various hardware components and all I could think of was the things that can go wrong at our own property when I am not around and my parents, kids, or siblings are using it.
After serving as my extended family's "IT support" for decades and dealing with a neverending stream of headaches relating to wifi, printers, Nest, unresponsive PCs, TV settings, car apps, and so on, I'm wary of installing a major system that requires someone to be basically on call to deal with minor problems or major outages as described in TFA.
A cousin proudly showed me her solar setup at a vacation home including the app showing her electricity savings and the various hardware components and all I could think of was the things that can go wrong at our own property when I am not around and my parents, kids, or siblings are using it.
After serving as my extended family's "IT support" for decades and dealing with a neverending stream of headaches relating to wifi, printers, Nest, unresponsive PCs, TV settings, car apps, and so on, I'm wary of installing a major system that requires someone to be basically on call to deal with minor problems or major outages as described in TFA.
For what it's worth, most solar installations aren't offgrid. They work alongside the regular grid so if the solar system (or a part of it) goes down, the house would still keep getting regular grid electricity.
If the OP's description of the situation is accurate, Tesla is at fault here. The company should never have sold a configuration (12 batteries + 1 controller) that no one has been able to make work despite trying again and again over 18 months. The company should apologize publicly right now, and do whatever it takes to fix this family's problems as soon as possible. I feel sorry for the family.
I have only one comment: He should’ve sued earlier. Inexcusable.
Also, the fact that the Powerwalls have processing overloads with 10 connected? Give me a break. A Nintendo Switch can coordinate with up to 11 other Switches in Mario Kart for a LAN party without internet.
Also, the fact that the Powerwalls have processing overloads with 10 connected? Give me a break. A Nintendo Switch can coordinate with up to 11 other Switches in Mario Kart for a LAN party without internet.
Possibly using the same json parser as GTA V?
"How I cut GTA Online loading times by 70%"
https://news.ycombinator.com/item?id=26296339
"How I cut GTA Online loading times by 70%"
https://news.ycombinator.com/item?id=26296339
well, we've seen elon's move-fast-and-break-lots-of-things attitude, so this isn't a surprise
wccrawford(3)
> The IT expert had the 10-battery system installed in early 2020 in order to power both his small business and family home from the property.
Nobody thought to implement this as two separate installs? Or the dad insisted it had to be done his way and is now mad that what he wanted didn’t work after they told him not to go that way?
They say doctors make the worst patients. I’m getting a hint of that here.
Nobody thought to implement this as two separate installs? Or the dad insisted it had to be done his way and is now mad that what he wanted didn’t work after they told him not to go that way?
They say doctors make the worst patients. I’m getting a hint of that here.
Tech has developed a really bad habit of shipping half-baked MVPs as fully fledged products with powerful marketing behind them, and that's an increasingly dangerous combination. I'm not sure where it started but I see it lot in software dev because that ethos is pretty much baked into iterative product development methodology. Don't build out something until a current or potential customer that's worth the time needs it.
After a while, it feels less like "iterative" product development and more like "highest bidder gets the feature" development.
After a while, it feels less like "iterative" product development and more like "highest bidder gets the feature" development.
> After a while, it feels less like "iterative" product development and more like "highest bidder gets the feature" development.
It does work that way. I work for a unicorn security firm that internally has the rule that a bug impacting a business that pays less than 10K a year cannot be fixed. They are to be strung along until they quit on us.
Whether we are actually working on the issue we claim to be working on entirely depends on your ARR. If you are a customer with less than 10K ARR, your bug is not getting any attention from engineering, just endless promises from support.
It does work that way. I work for a unicorn security firm that internally has the rule that a bug impacting a business that pays less than 10K a year cannot be fixed. They are to be strung along until they quit on us.
Whether we are actually working on the issue we claim to be working on entirely depends on your ARR. If you are a customer with less than 10K ARR, your bug is not getting any attention from engineering, just endless promises from support.