bigmutant · HackerTrans

Pretty common attitude from folks who have never worked in one of the BigTech companies where Java rules (Amazon being a prime example). Since they never encounter Java in the "SF-style Startup" world, they assume that it must be dead. Meanwhile hundreds-of-thousands of Engineers deal with hundreds-of-millions (billions?) of lines of Java every day

bigmutant

9 mesi fadiscuss

DynamoDB is used *everywhere* in AMZN Retail, this is absolutely not surprising. Plus the vast majority of internal Services are using EC2 in the form of Apollo/ECS. So OP probably hit some parts of the site that are hosted in us-west-2. For all I know they started routing all requests for us-east-1 traffic to other DCs, figuring latency is a fine trade-off for availability

bigmutant

9 mesi fadiscuss

To clarify, most of CDO (Consumer Devices Other) does run on AWS in the sense that NAWS is the target state, MAWS is legacy and actively (slowly) being migrated off of. CDO (including Alexa) has been using DynamoDB/Lambda/Kinesis/SQS etc forever, its just the compute and kind-of network layers that are still MAWS. Even then, a large part of CDO has moved from Apollo to ECS/FarGate/whatever unholy Hex or DataPath thing they're pushing these days

Source: Ex-AMZN

bigmutant

anno scorsodiscuss

Sure, I did the same, BS/MS with a focus on Compilers/Programming Languages. It's been personally gratifying to understand programming "end-to-end" and to solve some tricky problems, but 99% of folks aren't going to hit those problems. There are tons of people interacting with Cloud Services every day that aren't aware of the basic issues like:

- Consistency models (can I really count on data being there? What do I have to do to make sure that stale reads/write conflicts don't occur?)

- Transactions (this has really fallen off, especially in larger companies outside of BI/Analytics)

- Causality (how can I handle write conflicts at the App Layer? Are there Data Structures ie CDTs that can help in certain cases?)

Even basic things like "use system time/monotonic clocks to measure elapsed time instead of wall-clock time" aren't well known, I've personally corrected dozens of CRs for this. Yes this can be built in to libs, AI agents etc but it never seems to actually be, and I see the same issues repeated over-and-over. So something is missing at the education layer

bigmutant

anno scorsodiscuss

Def agree. Most people will never touch an Abstract Syntax Tree or even Expression Trees. Almost everyone working in back-end will use Cloud Services, will make mistakes based on assumptions of what they provide

bigmutant

anno scorsodiscuss

The fundamental problems are communication lag and lack of information about why issues occur (encapsulated by the Byzantine Generals problem). I like to imagine trying to build a fault-tolerant, reliable system for the Solar System. Would the techniques we use today (retries, timeouts, etc) really be adequate given that lag is upwards of hours instead of milliseconds? But that's the crux of these systems, coordination (mostly) works because systems are close together (same board, at most same DC)

bigmutant

anno scorsodiscuss

Good resources for understanding Distributed Systems:

- MIT course with Robert Morris (of Morris Worm fame): https://www.youtube.com/watch?v=cQP8WApzIQQ&list=PLrw6a1wE39...

- Martin Kleppmann (author of DDIA): https://www.youtube.com/watch?v=UEAMfLPZZhE&list=PLeKd45zvjc...

If you can work through the above (and DDIA), you'll have a solid understanding of the issues in Distributed System, like Consensus, Causality, Split Brain, etc. You'll also gain a critical eye of Cloud Services and be able to articulate their drawbacks (ex: did you know that replication to DynamoDB Secondary Indexes is eventually consistent? What effects can that have on your applications?)

bigmutant

2 anni fadiscuss

Northrop Grumman had a lot of folks from Crenshaw/Hawthorne/Carson when I was there, due to a partnership program with the local Cal State (Long Beach). All of the security staff was from that area too. Good folks, would 100% work with them again.

On the other hand, I've seen exactly 1 guy at the FANG I work at. What's the difference? I think it's companies like Northrop realizing that folks from under-represented communities have great value and prioritize that instead of whatever the current HackerRank-based interview process selects for

comments