Former Tesla Programmer's anecdotes about problems(reddit.com)
reddit.com
Former Tesla Programmer's anecdotes about problems
https://www.reddit.com/r/EnoughMuskSpam/comments/99sbwa/former_tesla_programmers_anecdotes_about_problems/
10 comments
Note that the post is 2 years old. Much may have (and probably has) changed since then. I’m not sure why this is relevant to post now?
That's doubtful, companies get harder to manage at scale not easier.
And real long term issues like this are usually fixed through massive culture changes, leadership changes, followed by extremely heavy lifting by ICs.
Just look at Microsoft evolving from Windows to the Cloud, to Azure.
Two years, when you are dealing with a ton of software and hardware and a CEO who has somewhat of a "god" complex - it's unlikely that things have changed all too much.
More likely, if they decided to retool for the Model 3 and build from the ground up, yes, but that would add time delay, and Elon always feels like he's racing against an internal clock in his head so it's doubtful.
And real long term issues like this are usually fixed through massive culture changes, leadership changes, followed by extremely heavy lifting by ICs.
Just look at Microsoft evolving from Windows to the Cloud, to Azure.
Two years, when you are dealing with a ton of software and hardware and a CEO who has somewhat of a "god" complex - it's unlikely that things have changed all too much.
More likely, if they decided to retool for the Model 3 and build from the ground up, yes, but that would add time delay, and Elon always feels like he's racing against an internal clock in his head so it's doubtful.
[deleted]
Apart of reddit, I remember a site where all these horror stories were submitted (anonymously of course) and published.
Kinda like Wikileaks for horror tech stories. Anyone remember it?
Kinda like Wikileaks for horror tech stories. Anyone remember it?
Was it the source for what got reposted here? https://mobile.twitter.com/atomicthumbs/status/1032939617404...
[deleted]
Was very short-form but the mobile app Secret had a fair amount
Example of some of the stuff I've seen at various places:
- Team A self ddos Team B taking down a critical platform that stopped the entire company from processing orders.
- Bug lead to everyone's password being reset to QA value and restoring backups would have taken too long. Any customer who complained about access was told support had reset their password just now to test123 but in reality every customer's password had been reset to that.
- Another scenario where some QA values got ran on a production environment and some settings and configs had to be manually restored as it was a third party service it got applied against. Halted nearly whole company.
- Bug miscalculates how to charge users and basically drains most people's bank accounts.
- Accidental DNS rebinding exposing CDN traffic to another company and took them down.
- Dev mistake bypassing a endpoints on the CDN for caching and throttling rules. Nearly $1M mistake before anyone even noticed.
- Accidentally using another S3 bucket template for another S3 bucket and not noticing it had versioning on. Program that dumped it's state for restoring to this versioning bucket had +450TB of state versioned. It didn't care about the history of state just the present state for restoring. Noticed only by mistake when cleaning up somethings.
And many other stories, things happen, you learn, you improve and it is really a risk vs reward calculation on the business side.