bmalehorn

letztes Jahrdiscuss

Would you mind sharing what AI note taking app do you use?

vor 6 Jahrendiscuss

Based on this article, it now uses Hypervisor.framework: https://www.quora.com/What-hypervisor-does-Docker-use

> In Linux, Docker uses kernel features. It does not use any hypervisor. > In Windows, Docker desktop uses Hyper-V. > In Mac, Docker desktop uses Apple Hypervisor.framework.

bmalehorn

vor 6 Jahrendiscuss

Interesting - would you mind sharing a link?

bmalehorn

vor 6 Jahrendiscuss

Cool, thanks for sharing. It's these kind of experiences that I was hoping to gather from making this post.

Did you notice at the end that you did NOT end up choosing ARM? You ended up going with x86_64 because that's what made more sense for your backend. That's part of my point - developers should choose their backend architecture based on the performance and pricing of their backend, not their development laptop. And if that decision is "we should keep using x86", then there will be a big performance hit in development.

bmalehorn

vor 6 Jahrendiscuss

Thanks, I didn't know about c6g, m6g, r6g. I've updated the post to remove this mention - I only counted a1 instances.

That still leaves storage optimized and GPU optimized instances missing. I'm guessing storage should be easy enough to add, but what about GPU?

From my novice experience with GPUs, they need finicky drivers that must be ported by the GPU manufacturers, so I figure it might take a while to get competitive ARM GPU instances.

bmalehorn

vor 6 Jahrendiscuss

Awesome, thanks for testing this out!

A 3x slowdown is not as bad as 6x, but it's still quite a bit. I also saw a slowdown of ~4x when I tried this experiment on a native Linux x86_64 running ARM - perhaps the Mac -> Linux virtualization slowed it down further.

5x may have been a bit alarmist, but regardless we should brace ourselves for a big performance hit on x86_64 virtualization.

bmalehorn

vor 6 Jahrendiscuss

I ran this on a Linux laptop - it looks like it's running qemu-user-static:

    root        9934  103  0.0 125444  6664 pts/0    Rl+  12:25   0:12 /usr/bin/qemu-aarch64-static /usr/bin/gzip

So it might be that Docker already runs a native x86_64 Linux, then uses qemu-static binary translation.

bmalehorn

vor 6 Jahrendiscuss

Author here.

> 1. If emulating aarch64 (arm64) on x86_64 is 6x slower (on your system, btw, it's not an universal constant), it doesn't mean emulating x86_64 on aarch64 will be 6x slower. It'd probably be worse, or at least that's my gut feeling.

Yup, performance benchmarks are inherently flawed and nobody knows anything right now without the hardware. However if ARM -> x86 emulation is anything like x86 -> ARM emulation, I would expect a really big performance loss.

> 2. Generic container images like the Ubuntu mentioned usually have aarch64 (arm64) support, so running the x86_64 image makes no sense for the presented use-case.

Ah actually I address this in the article, and even run an arm64 image. The short version is, it would be a lot of work to convert your whole backend infrastructure to ARM just because you got a new laptop.

> 3. You won't be able to use most software because they don't release ARM binaries ... and the example uses `wget` && `tar xf`, with no binary signature check. As someone who has been porting stuff from x86_64 to aarch64 for a couple of years, I admit I've seen this pattern frequently. The most obvious solution is to build from sources, which would have been better off on x86_64 too, instead of fetching a prebuilt (and unverified) binary from the internet. Maybe there are some CPU flags the compiler could notice and apply optimizations which are not included in the prebuilt binary.

Yes, if only everything were built from source! I'm not saying there's no solution, just that the solution would be a lot of work. If the library is obscure enough and the errors are strange enough, it might be so much work as to be impossible to the busy web developer.

My goal was to write a kind of hand-wavy article to get people talking about this problem.

bmalehorn

vor 6 Jahrendiscuss

Author here.

I'm glad somebody said something! Yes the gzip perf test is pretty silly, but illustrates a significant difference. /dev/urandom throughput on this setup was about 100 MB / s so it wasn't a bottleneck for this test - the bottlneck was gzip.

Feel free to come up with a performance test yourself! I personally want to know what an HTTP test would look like. You can run an ARM image by running:

    docker run -it arm64v8/ubuntu

Unfortunately, Rosetta 2 is not going to help here. Rosetta 2 translates x86 -> ARM, but only on Mac binaries. It does not translate Linux binaries, and cannot reach inside a Docker image.

bmalehorn

vor 6 Jahrendiscuss

Author here. That's a major point of the article - "are we screwed?" I'm not an expert on virtualization but I wanted to see some discussion on this topic, because it feels like we might be screwed and nobody is talking about. Anyway I was happy to see Docker worked, at least on a basic level.

bmalehorn

vor 7 Jahrendiscuss

The speaker mentions that Redis, MongoDB and background jobs were replaced by Erlang.

What does he mean by that exactly? Erlang provides some persistent state storage? Or is he just saying he used Erlang database drivers to access Redis / MongoDB?

bmalehorn

vor 7 Jahrendiscuss

> I would rephrase as companies whose express intent is to manipulate people's habits for their own gain. Entertainment is fine as long as it's done in relative good-faith (not gambling or manipulative microtransactions).

Wondering - is Netflix a good company? On one hand, they're a straightforward entertainment company, but on the other hand they optimize content and suggestions on maximizing viewing hours. Their goal is to change habits to watch more and more Netflix.

How about Apple? They sell hardware and software that people like and they try to respect user privacy. But they design their software to aggressively lock users into their own platform and prevent them from trying competitors.

> It can be about empowering people to manage and utilize their own information. It can be about empowering people to create new information.

This is almost exactly Google's mission statement, but most people here would put Google in the gray area.

My point is, almost no company's goal is to make peoples' lives worse. But all companies are fundamentally trying to manipulate people to buy more of their product. The question is if most customers feel good about the company after the transaction, if they feel like they were manipulated "too much".

bmalehorn

vor 7 Jahrendiscuss

It sounds like you're skeptical about companies that affect people's habits. So avoid these terms:

- finance

- advertising

- entertainment

I'd focus on companies that sell to other businesses. Changing the habits of businesses is usually considered a good thing.

And sell something less abstract. If you sell laptops, routers or office chairs, nobody's going to think you're a bad person. Just maybe a bit boring.

Software-only products are all about information. It's about gathering information about people (gray area) or understanding lots of information about people (gray area). So work for a company that does something in the physical world.

bmalehorn

vor 7 Jahrendiscuss

This is a nice article, with a comprehensive if long history of npm and node.

I agree with the author that there will one day be a reckoning with npm, inc - they'll shut down abruptly, or do something people are really unhappy with.

However, I'm afraid the package manager proposed at the end will have a lot of trouble gaining traction. It sounds like package maintainers will have to port their packages to the new package manager. Worse, developers won't be able to build, say, React until all 1000 of its indirect dependencies have been ported.

This will take a gargantuan effort to port over, all to avoid a vague future threat. I know it's "the right thing" to use this more decentralized system, but I can't see us getting to there from here with all the pain in between.

I think this is a more likely outcome:

1. npm, inc does a bad thing that makes people angry 2. A community-governed, charity-hosted registry appears. This registry includes everything in registry.npmjs.org, plus exclusive packages 3. users and authors switch to new community registry

In fact, we may already be poised for this coup. Did you know that 1/3 of all package installations go through registry.yarnpkg.com?

bmalehorn

vor 7 Jahrendiscuss

In a lot of organizations there are "bug teams", whose job is to detect bugs, fix some, and make fixing them easier. However, you probably don't call them "the bug team". Examples:

Site reliability. If devs ship a bug, the site reliability team will mitigate the damage and probably ship out a quick fix. They'll also make it easier for the original dev to provide a long-term fix.

Quality assurance. They find bugs, including more vague user experience bugs. In a sense they can fix bugs by delaying deploys and disabling buggy features.

Security. Part of the security team's job is to identify vulnerable packages and upgrade them, as well as find and fix vulnerable endpoints. Often this won't come back to the original developer at all unless they made a serious transgression.

So bug teams do exist, but usually with a spin on it. The conversation is not "we are separate; your mistakes are not your problem", but "let's work together to make your mistakes less costly".

bmalehorn

vor 7 Jahrendiscuss

I think the right approach here is to figure out how to use git in the company IDE. Then, have a presentation / workshop about how to use git in the IDE.

Personally I prefer the command line, but I imagine your coworkers will be a lot more receptive to using version control tools the way they're used to.

If you're concerned about command line complexity, you can try out gitless: https://gitless.com

It's a simplified command line interface to git, designed by looking at how users actually use git.

bmalehorn

vor 7 Jahrendiscuss

As a former Cisco employee, I can tell you why companies never want to open source their security-sensitive products:

Pros of open sourcing a product:

- fewer total number of vulnerabilities

Cons of open sourcing a product:

- more publicly-known vulnerabilities

- less effort required to find new vulnerabilities

The product might be more objectively secure, with more bug reports and more fixes.

But it will be less practically secure. There will be more known vulnerabilities, and many customers can't upgrade, leaving more total vulnerable customers. And worse, now anyone on the internet can try and find new vulnerabilities for $0, while before they'd need to buy a $1,000+ piece of hardware to even get a shot at the compiled code.

The real defense against this problem is security auditing. Security engineers try to hack the device while asking a bunch of questions about SSH connections and private keys. This is the technique most companies employ, often combined with bug bounties.

bmalehorn

vor 8 Jahrendiscuss

The article makes it sound like you have to choose between two options:

1. users see example.com in the address bar

2. hosting on www.example.com

However you can get the best of both worlds. If you go to amazon.com, you will see amazon.com in the address bar. Yet your browser makes requests only to www.amazon.com; the content is hosted on www.amazon.com.

Does anyone know how they pulled that off?

bmalehorn

vor 8 Jahrendiscuss

I have written exactly that kind of changelog. Let me explain why.

I write the changelog for a product. In fact I write two changelogs - one for engineers and one for customers.

Engineers:

- [New Feature] Added ability to ___

- Improved logging on ___

- Fixed a memory leak on ___

Users:

- [New Feature] Added ability to ___

Not every change can be explained in one sentence to users. Internal logging? Memory leaks? You're speaking a foreign language. Those entries must be thrown out when making the changelog for users.

Eventually, there comes a release when you have thrown everything out. What do you write? "Bug fixes & enhancements".

bmalehorn

vor 9 Jahrendiscuss

> Alowing Trim passthrough on SSDs without impacting vulnerability.

Unfortunately I don’t think that’s possible. If I scan your encrypted disk and see that 10% of blocks are zeroed out, I can assume that your disk is 90% full. So information has been leaked, which is arguably a vulnerability.

Submissions

ARM Mac: Why I'm Worried About Virtualization

comments