HackerTrans
TopNewTrendsCommentsPastAskShowJobs

ralph87

no profile record

Submissions

All positive returns to major market indices over 3 decades occurred overnight

overnightreturns.org
10 points·by ralph87·6 yıl önce·6 comments

comments

ralph87
·6 yıl önce·discuss
If you buy a matched OS + laptop in any other situation, why not also for Linux? Buy an XPS next time, or any other vendor (Lenovo?) with tested hardware, and just put an Ubuntu USB stick in.

"I bought the top result for 'wheel' on eBay and ruined my life trying to fit it to this aircraft engine, this is such a joke!" Shit has just worked for years if you want it to work.
ralph87
·6 yıl önce·discuss
The network stack integration was less than transparent too. Random places all over the system where e.g. a magical ioctl() just didn't work properly. Who really wants to spend their life reimplementing every last baroque detail of that stuff?
ralph87
·6 yıl önce·discuss
> The argument about Windows not having a single, central dentry cache doesn't hold water

This was more about cache coherency, and it's not an argument, it's a trait shared by every other Linux-over-X implementation (UML, coLinux). It is fundamental to solving a problem users want - seamless integration, i.e. no opaque ext4 blob. Why doubt the reasons given by Microsoft, when they match observations not just for WSL but every other system in the same class?
ralph87
·6 yıl önce·discuss
Contra opinion: the VM option reduced the service interface between Windows and Linux to a single kernel implementation and a few drivers, rather than every possible userspace program ever written. It's an amazing and obvious trade off. My inner architecture astronaut appreciates all the ideas in this post, but I've been trying to kill that guy for over a decade now. The bottom line is WSLv1 design SUCKED precisely because it tried to cross-breed 2 extremely complex semantically incompatible systems.

Example: Linux local filesystem performance mostly derives from the dentry cache. That cache keeps a parsed representation of whatever the filesystem knows appears on disk. The dentry cache is crucial to pretty much any filesystem system call that does not already involve an open file, and IIRC many that also do. Problem is, that same cache in WSL must be subverted because Linux is not the only thing that can mutate the NTFS filesystem - any Windows program could as well. This one fundamentally unfixable problem alone is probably 80% the reason WSL1 IO perf sucked - because the design absolutely required it.

Solutions are rip out a core piece of kernel functionality, and in the process basically taking over ownership and maintenance for ALL code anywhere in the kernel assuming the existence of said cache, engineer something that is somehow better, and support this in perpetuity, including any semantic mismatches that turn up much later that were never designed for

The idea of merged ps output where Windows binaries would show up in the Linux /proc. How would you even begin to implement that without confusing EVERY process management tool ever written that interfaced with /proc? What about /proc/.../environ? On UNIX that has no character set. On Windows it is Unicode.

A trillion problems like this made WSL1 a beautiful nightmare. Glad it was tried and from watching the tickets, that team bled heroically trying to make it all work, but ultimately, I'm also glad it's gone, because the replacement is infinitely easier to engineer, maintain, and use, and that team earns its quarterly bonus much more easily. Everyone wins, except the astronauts, but as experience teaches us they never do.
ralph87
·6 yıl önce·discuss
Do what exactly? This is essentially a performance optimization for folk bagholding oceans of vulnerable hardware
ralph87
·6 yıl önce·discuss
This method was talked about since the Spectre/Meltdown days. It has often taken Google much longer to release patches they've been sitting on, e.g. many original containers patches were like this.
ralph87
·6 yıl önce·discuss
Chrome is not open source, it's a derived proprietary superset of the open source Chromium. They are not the same thing in very material ways
ralph87
·6 yıl önce·discuss
We could argue this ad infinitum, K8s of course doesn't remove all proprietary elements from a solution but it is a huge step up. Speaking as an ex-Googler, it took over 10 years but I'm so happy the rest of the world finally has a standard like this, the world is a better place for it even though I at one point had to unlearn all my traditional sysadmin habits and immerse myself in an environment practising it successfully to finally understand.

Your original question was what is the point. These are the points. As for why not Docker, k8s network effects and strategy of its sponsors mean Docker is on a lifeline, everyone knows that.
ralph87
·6 yıl önce·discuss
Ever tried to host multiple apps on a single machine? Oh look, a custom Nginx config only one person understands. Oh look, some hacked up letsencrypt config only one person understands, etc etc.

> K8S on a single node does nothing for you network-wise

- Container IP auto-assignment

- Container security policy

- Container DNS management

- Ingress management ("custom Nginx config")

- "Environment that feels like a large network and doesn't change if moved to a large network"

What part of this is difficult to understand?
ralph87
·6 yıl önce·discuss
Kubernetes does a whole lot more than just restart jobs when a node fails, in fact of all its features this is one most people probably see in action the least.

Kubernetes centralizes all the traditional technical nonsense related to providing a robust environment to deploy applications to. I want Kubernetes even in a single node scenario because I want Kubernetes-like packaging, deployment and network services for any app I work on, as they are a net simplification of what was previously an ad-hoc world of init scripts, language-specific supervisors, logging, monitoring, etc etc., and rearchitecting an app deployment from scratch simply because its resource requirements increased.

If people want to continue trying to scale it down further, where is the harm in this? There are plenty of legitimate cases where it makes sense. There's no real limit to that work either, it's conceivable with the right implementation improvements (in k8s and the container runtime), it might eventually be possible to reuse the same deployment model even for extremely small devices.
ralph87
·6 yıl önce·discuss
Just for completeness, updating parent comment's Lambda estimates, not counting provisioned worker costs, and assuming no request takes more than 100 ms.

    Lambda requests: ((1.5e9 * 4) / 1e6) * .20     = $   1,200 
    Lambda CPU (1536 MB): 0.0000025000 * 1.5e9 * 4 = $  15,000
    API Gateway HTTP reqs:
        (count): 1.5e9 * 4 = (6 billion)
        (first 300m): 300 * 1.0                    = $     300
        (next 5700m): 5700 * 0.9                   = $   5,130

    LAMBDA MONTHLY TOTAL                           = $  21,630
    LAMBDA YEARLY TOTAL                            = $ 259,560
And for comparison:

     NLB (2x)
        (NLB hours 1 month):
           2 * 0.0225 * 24 * 30.45                 = $      33
        (NCLU hours):
           2 * (2280/50) * 0.006 * 24 * 30.45      = $     399

     NLB MONTHLY TOTAL                             = $     432
     NLB YEARLY TOTAL                              = $   5,184

     EC2 YEARLY
        (if  1 req/vCPU)                           = $ 253,926
        (if 15 reqs/vCPU)                          = $  67,704
        (if 30 reqs/vCPU)                          = $  33,852
Note the "1 req/vCPU" case would require requests to burn 250ms of pure CPU (i.e. not sleeping on IO) each -- which in an equivalent scenario would inflate the Lambda CPU usage by 3x due to the 100ms billing granularity, i.e. an extra $30,000/month.

That's an 87% reduction in operational costs in the ideal (and not uncommon!) case, and a minimum of a 59% reduction in the case of a web app from hell burning 250 ms CPU per request.
ralph87
·6 yıl önce·discuss
I stopped reading at "3000 cores"; there is a lot of money to be made mopping up disasters like that, it's clearly even something of a growth industry. We had one machine push 2,400 requests/sec average over election night, without even touching 30% capacity, costing around $600/mo including bandwidth. Its mirror in another region costs slightly more at $800/mo. As a side note, it's always the case with those folk that they invent new employees to top up their estimates, that wouldn't be required in the serverless world, yet in every serverless project I've ever seen, they absolutely still existed because they had to.

Price-perf ratio between Lambda and EC2 is obscene, even before accounting for Lambda's 100ms billing granularity, per-request fees, provisioned capacity or API Gateway. Assuming one request to a 1 vCPU, 1,792MB worker that lasted all month (impossible, I know), this comes to around $76, compared to (for example) a 1.7GB 1 vCPU m1.small at $32/mo or $17.50/mo partial upfront reserved.

Let's say we have a "50% partial-reserved" autoscaling group that never scales down, this gives us a $24.75/mo blended equivalent VM cost for a single $76 Lambda worker, or around 3x markup, rising to 6x if the ASG did scale down to 50% its size the entire month. That's totally fine if you're running an idle Lambda load where no billing occurs, but we're talking about the BBC, one of the largest sites in the world...

The BBC actually publish some stats for 2020, their peak month was 1.5e9 page views. Counting just the News home page, this translates to what looks like 4 dynamic requests, or 2,280 requests/sec.

Assuming those 4 dynamic requests took 250ms each and were pegging 100% VM CPU, that still only works out to 570 VMs, or $14,107/mo. Let's assume the app is not insane, and on average we expect 30 requests/sec per VM (probably switching out the m1.medium for a larger size taking proportionally increased load), now we're looking at something much more representative of a typical app deployment on EC2, $1,881/mo. on VM time. Multiply by 1.5x to account for a 100% idle backup ASG in another region and we have a final sane figure: $2,821/mo.

As an aside, I don't know anyone using 128mb workers for anything interactive not because of memory requirements, but because CPU timeslice scales with memory. For almost every load I've worked with, we ended up using 1,536mb slices as a good latency/cost tradeoff.
ralph87
·6 yıl önce·discuss
The emphasis here being on the word "should". I wouldn't buy just yet until some third party has published a sustained load/heat test. Apple have sucked at this for quite some years now, it's folly to touch a MacBook for anything remotely compute-intensive
ralph87
·6 yıl önce·discuss
The use case is benchmarks basically, and maybe faster cold boots. About the only place the typical heavy computer user is likely to notice major difference between SATA and NVME is probably running something like "find /", or some kind of full-disk search scenario.

Even in the latter, full text indexing which every platform has had for years now makes it much less likely that the full directory tree will even get walked, and differences even less likely to be noticed. As a side note, every desktop platform's full text search seems to suffer software performance problems that are largely independent of the underlying disk

Even in the full-tree enumeration case, since Spectre/Meltdown mitigations landed, system call overhead is so high now that even with a lightning fast disk, a large chunk of total time taken to walk the directory tree is lost basically twiddling the CPU mode securely. You can definitely still see the difference between SATA and NVMe, but you can also definitely measure the amount of time during the NVMe run that is spent in software -- incrementally faster NVMe will have quickly diminishing returns.

"What about databases!" This was my original interest in SSDs to begin with. It turns out, despite being a data monkey who loves large databases, since 2013 any time I've worked with a giant dataset like this, it is always in the form of large scans (usually from something like a CSV or XML file), where SSDs don't really have a mind-blowing advantage over magnetic (but of course they are still 5-10x faster a seq io, its just that data parsing and processing is typically the bottleneck now).
ralph87
·6 yıl önce·discuss
Small but important nitpick: since at least 2005 there is no "on the NYSE", the price you're likely seeing is the best price consolidated across all national market system venues, the primary listing exchange is mostly only an administrative entity in recent times. Actually some of the largest premarket trading is occurring on NASDAQ and a CBOE-owned exchange called BATS
ralph87
·6 yıl önce·discuss
The average web page can be as nasty as it likes, but if I can deliver apps that _feel_ awesome and optionally can deliver metrics that provide a commercial basis for that awesomeness to whoever is paying me, I don't really care much about the average web page at all. Insert some truism here about using the incompetence of others to justify one's own slovenly behaviour, and some other truism about standing out from the crowd by applying common sense.

"I built an average web application and it attracted an average number of users, it more or less worked if you visited it from mobile" would look fucking awful on anyone's resume.
ralph87
·6 yıl önce·discuss
Effectively this is arguing for the project's attention to detail being low, we covered this already
ralph87
·6 yıl önce·discuss
Marketed as a single codebase platform where the most ubiquitous deployment target has a user experience that tells your customers you hate them

4.39 MB in 50 requests is a showstopper on any mobile network, especially when alternative solutions do not have that problem. It is fair to assume some first time experiences will involve 7.5 seconds or more additional latency on 3G networks, double that for a poor signal areas.

It's still an instant pass
ralph87
·6 yıl önce·discuss
The gallery app looks to be authored using it:

- It needed to show a loading spinner on a 250 Mbit connection

- It hung the browser while 'booting' the page for a solid 3-5 seconds

- It downloaded 4.39 MB in 50 requests

- Opening web inspector in Firefox while reloading the page was sufficient to cause the boot process to hang indefinitely

- Page looks pretty, but at this point it barely matters

We could conclude either that the project's attention to important details is low, producing this experience, in which case what else might we discover once committed, or alternatively we could conclude that it is high, in which case, this is the best possible experience for any Uno app.

Instant pass
ralph87
·6 yıl önce·discuss
Hoping someone snarfed a git clone, revision history is waaay more interesting than the code itself