Stack Overflow runs on 9 web servers with (iirc) 48 logical cores (2 x 12-core Xeons) and 64GB RAM. Those servers are shared by a few apps (Talent/Job, Ads, Chat, Stack Exchange/Overflow itself) but the main app uses, on average, ~5% CPU. Those machines handle roughly 5000 requests/sec and were running .NET 5 as of September 2021 (when I moved on). That’s backed by 2 v. large SQL clusters (each consisting of a primary read/write, a secondary read-only in the primary DC and a secondary in the failover DC). Most traffic to a question page directly hits SQL - cache hit ratio tends to be low so caching in Redis for those hits tends to be not useful. As somebody mentioned below, being just a single network hop away yields really low latency (~0.016ms in this case) - that certainly helps being able to scale on little hardware - typically only 10
- 20 concurrent requests would be running in any instance at any one time because the overall request end-to-end would take < 10ms to run.
Back in full framework days we had to do a fair bit of optimisation to get great performance out of .NET, but as of .NET Core 3.1 the framework _just gets out the way_ - most memory dumps and profiling subsequent to that clearly pinpoint problem areas in your own app rather than being muddied by framework shennanigans.
Source: I used to work on the Platform Engineering team at Stack Overflow :)
^^ this! No need to install anything on my Apple devices. Long term plan is to have an instance of AirDrop Anywhere running perpetually on a Raspberry Pi and have a web interface accessible over the LAN for non-Apple devices to connect to when sharing files.
That said, this is all just a bit of fun so monetising it isn’t really on the cards anyway :)
This is not correct, you can receive files from non-Apple devices without any Apple keys. I’m still working on sending so haven’t uncovered the demons lurking there…
Well, that’s why I said it’s unlikely :)! Without purchasing additional hardware it’s not practical to be able to run AirDrop directly on non-Apple devices.
There’s no need to extract any Apple keys to be able to receive files, but the public root key appears to be needed able to send files to an Apple device.
OWL, OpenDrop and their latest project PrivateDrop (https://github.com/seemoo-lab/privatedrop) are linked heavily throughout the series - their reverse engineering of the protocol have been absolutely invaluable in building something that works, more or less sanely on non-Apple devices! Huge kudos to them!
It's in the remediations section, but maybe the wording isn't clear:
*> Hardening code paths that allow access into our dev tier. We cannot take our dev tier off of the internet because we have to be able to test integrations with third-party systems that send inbound webhooks, etc. Instead, we made sure that access can only be gained with access keys obtained by employees and that features such as impersonation only allow de-escalation—i.e. it only allows lower or equal privilege users to the currently authenticated user. We also removed functionality that allowed viewing emails, in particular account recovery emails.*
There was no "unauthenticated" access into dev - the access key here is what allows login at all to our dev environment, but the attacker was able to bypass that protection.
We’re spread over 9 servers but only due to ephemeral port and handle exhaustion issues. Each server is fronted by 4 HAProxy frontends that each handle ~18k connections.
Since Nick’s post we’ve moved from StackExchange.NetGain to the managed websocket implementation in .NET Core 3 using Kestrel and libuv. That sits at around 2.3GB RAM and 0.4% CPU. Memory could be better (it used to be < 1GB with NetGain) but improvements would likely come from tweaks to how we configure GC under .NET Core which we haven’t really investigated yet.
We could run on far fewer machines but we have 9 sitting there serving traffic for the sites themselves so no harm in spreading the load a little!
We use websockets for pushing things like comments, answers and question edits as they are added as well as for notifications like inbox messages and reputation updates.
Our websocket process currently has ~500k concurrent connections across 9 web servers.
Back in full framework days we had to do a fair bit of optimisation to get great performance out of .NET, but as of .NET Core 3.1 the framework _just gets out the way_ - most memory dumps and profiling subsequent to that clearly pinpoint problem areas in your own app rather than being muddied by framework shennanigans.
Source: I used to work on the Platform Engineering team at Stack Overflow :)