smspf · HackerTrans

4 года назадdiscuss

Nice read, I wonder how this was detected though. Did it trigger any alarms on the infected machine? Was a firewall or specialized traffic inspection involved?

smspf

4 года назадdiscuss

I couldn't finish reading the article.

The first iPhone made connectors and electronic components easier to get to? Sure, it indirectly helped, as any other new, recent electronic device did. But please don't try to rewrite history.

He couldn't make things work over the phone, although he invested large sums of money and dedicated hardware? Come on, I rigged an old analog (rotary) phone to do that using a relay when I was 14 or 15 years old. After all, in that case all he needed to do was to switch on the heating and nothing more.

The whole thing just reads like a marketing fluff piece.

smspf

5 лет назадdiscuss

This is nice and makes Red Hat look like an amazing OSS contributor, which I think they are. But truth be told, things were not that easy when it comes to RHEL and derivates on aarch64. As contractors working on getting stuff working on aarch64, I know we struggled quite a bit to get CentOS7 to get to the level Ubuntu was at the time on aarch64. And real progress (i.e. getting a fully working distro for our intents at the time) was only seen after money were thrown and a contract was signed.

The bottleneck however paid off imo - Red Hat insisted on doing things slow and steady, so once things started landing in the distro, they just worked. Unlike other distros which touted aarch64 support early, but struggled to fix issues along the way.

I remember working with OpenJDK and Opendaylight on aarch64. The crashes I've seen were terrifying. The JVM crashed so hard, our expert Java developer laughed 30 minutes straight looking at one trace. To this day I have no idea what was so funny, as he couldn't explain it in a simple manner and I lost interest.

Anyway, thank you Red Hat developers for all the work (minus the systemd "architects", but that's a different story).

smspf

6 лет назадdiscuss

By investing in these ports I was thinking about any contribution that enables/improves arm64 support. So yes, PRs for python itself definitely qualify. However, the python ecosystem is not only python, there are lots of python packages that still lack arm64 optimizations or native arm64 support. Most of them work, being python, but every once in a while you'd run into a package that does not just work on arm64 without some tweaking.

In my case, "we" referred to the working group of ARM contractors working a very specific higher level project (at the time we were doing some cloud/openstack/k8s stuff, which indirectly required some python packages). Note that the way contractors were assigned to projects at the time implied that sometimes you'd run into overlaps, e.g. some packages we needed had already been ported by a different team of contractors for a different project a while back. Sometimes multiple teams would hit the same missing port at the same time, most of the time we could sync and team up by escalating that through the right channels, but sometimes we'd just end up duplicating work items and end up with two different sets of patches/ports.

About wheels, someone else in my team handled that, so I'm not that familiar with the issue - it might have been a particular package that was lacking arm64 wheels or it might have been impractical to use a different repository only for arm64 (in general we'd try avoiding using separate resources based on architecture and we definitely did prefer the arch-agnostic approaches, e.g. debian repository URLs are arch-independent, dockerhub manifests allowed using the same container image names etc.). We were interested PyPI at the time (couple of years ago), maybe the situation has improved meanwhile, but we just accepted compilation at install time as a good enough solution.

smspf

6 лет назадdiscuss

To be honest, I don't blame Apple for this. I blame the ARM ecosystem which is very fragmented, each company working with ARM is contributing to the stuff they are interested in and that's it.

Lots of contractors and always shuffling/changing projects they work on.

smspf

6 лет назадdiscuss

A couple of years ago we reached out to the python community about wheels and arm64 - how it should be handled and whether they plan on embedding non-x86 blobs. We received the standard "we'll think about it and let you know". Now that Apple switched to arm64, all communities are suddenly interested in porting things to arm64.

And of course, Apple is not investing in these ports, at least as far as I know. They just rely on what other arm64 players did in the ecosystem before Apples rolled out M1; respectively lets developers figure out the remaining porting.

As much as I hate to say this, IBM does handle porting things to ppc64 right - you can find IBM contributed code and optimizations anywhere you look. For many packages, porting to arm64 was a matter of "does it have ppc64 support? if so, it can be reused for arm64" ...

Disclaimer - used to be a contractor porting stuff to arm64 for a couple of years.

smspf

6 лет назадdiscuss

In my experience, for tasks like everyday operation, kernel building etc.: Thunderx < ThunderX2 < eMAG (Ampere's platform before N1).

I wouldn't say it's the same order of magnitude for the 2 comparisons, but it's definitely noticeable.

smspf

6 лет назадdiscuss

>In an interesting turn of events, the investigation of the whole SolarWinds compromise led to the discovery of an additional malware that also affects the SolarWinds Orion product but has been determined to be likely unrelated to this compromise and used by a different threat actor.

Either that one was used to compromise the supply chain (in which case it makes little to no sense to keep it around and risk detection), or at least 2 different groups had the chance to target sensitive US infrastructure.

Funny how media coverage of this issue misses no chance of mentioning Russia and nobody else, not even possible suspects.

I wonder what happens if the attackers notice each other on the compromised system. Do they get along in exfiltrating data or do they fight quietly?

smspf

6 лет назадdiscuss

ThunderX was a huge disappointment. ThunderX2 (which one may think is the successor of ThunderX, but is actually a completely different system that Cavium obtained by acquiring a different company that was also working on ARMv8 hardware) was a (not so huge) disappointment. Cavium tried to copy-paste lots of coprocessors and offload things from the CPU, but the overall system was not that great.

Early AMD Softiron and Applied Micro boards (which had 8 cores unlike the ThunderX which had 48 or 96) were actually faster, which I always found interesting.

But Ampere's previous generation (before N1) is fast, much faster than the ThunderX2. Afaik, they built it on top of previous Applied Micro IP. So I'd expect N1 to be in a different league and not worth comparing to ThunderX2.

smspf

6 лет назадdiscuss

My bad, I didn't do proper research when writing that comment. Thank you for the correction.

And that's too bad, UEFI would have been easier to deal with imo.

smspf

6 лет назадdiscuss

I submitted arm64 patches to OSS ranging from the kernel to the most obscure userspace applications.

All my contributions required some support or at least confirmation from the hardware vendor that my assumptions were correct - e.g. I submitted a patch for a GICv3 errata on a specific chipset; I had to confirm with the vendor that my findings were correct - sure, the patch "worked", but was it doing the right thing or just hiding the real issue? (e.g. why did writing zeros to some magic register fixed the problem we observed? was it a hardware issue or a software issue in the kernel? without feedback from the vendor, such things are a lot harder to isolate and fix properly).

I agree about OSS thriving on major vendor hardware, it's just that Apple is special in this case and intentionally makes it hard for the OSS community to provide support for their hardware.

smspf

6 лет назадdiscuss

No. Because (in no particular order):

- it shouldn't be the community's (or a crowd-funded dev's) responsibility to provide software support for hardware produced by one of the largest companies out there (bonus: with zero hardware specs);

- Apple could make all this futile with a push of a button (SecureBoot can be disabled for now, but what guarantees are there this won't change?);

- other arm64 machines will be available soon enough, most if not all of them with publicly available specs;

- I do not own an M1 machine, nor do I plan on buying one;

From a technical perspective, it's doable. Looks like it has UEFI and can run Windows. But we know nothing about possible silicon errata and required driver changes (or at least I don't).

Anyway, I'm sure others would like to see this happening and would actually pay - hopefully the Twitter poll will reveal whether this is actually worth it.

Disclaimer: I ported things to arm64 for a couple of years as a contractor.

smspf

6 лет назадdiscuss

Afaik, UEFI (at least on ARM hardware I worked with) embeds the DeviceTree for non-ACPI boot usecases, so the user is no longer responsible for providing the proper DTB for that board, although the user can still override it if needed. Last time I worked with ARMv8, UEFI provided both ACPI tables and the embedded DTB, since ACPI support was undergoing a major rewrite in the kernel at that time.

smspf

6 лет назадdiscuss

This is sad and wrong at the same time. What if instead of postal services, Amazon required water or electricity to function. They would have payed just enough to get some level of priority (without making the water/electricity companies turn any profit, just keeping them barely alive), so when the grids get overloaded (e.g. during summer), you could run out of water/electricity because your neighbors ordered some useless stuff from Amazon. "We're sorry, you can't turn on your A/C today because your neighbor ordered the new UselessThing, which requires a lot of resources to be delivered. Maybe we'll let you use it tomorrow, unless they order the UselessThing 2.0."

smspf

6 лет назадdiscuss

I'm surprised it's only a 3x slowdown. But the single-thread performance of native execution (without emulation) is worse on aarch64, which was expected. Imo, a better benchmark would take into account the multithread performance with/without emulation.

smspf

6 лет назадdiscuss

Not sure about the socket used (it might be soldered down), but aarch64-based workstations are already available for the general public, e.g. [1].

[1] https://www.anandtech.com/show/15737/arm-development-for-the...

smspf

6 лет назадdiscuss

I agree on the performance loss. Just for kicks, I ran the same commands on some real aarch64 (32 cores, 3.0GHz, ARMv8.? - can't remember and already logged off the machine, but I can double check tomorrow). Without further context, numbers:

  someuser@some-aarch64-machine:~$ docker run arm64v8/ubuntu bash -c 'dd if=/dev/urandom bs=4k count=10k | gzip > /dev/null'
  10240+0 records in
  10240+0 records out
  41943040 bytes (42 MB, 40 MiB) copied, 2.18298 s, 19.2 MB/s
  someuser@some-aarch64-machine:~$ docker run amd64/ubuntu bash -c 'dd if=/dev/urandom bs=4k count=10k | gzip > /dev/null'
  warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
  warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
  warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
  10240+0 records in
  10240+0 records out
  41943040 bytes (42 MB, 40 MiB) copied, 6.72324 s, 6.2 MB/s

smspf

6 лет назадdiscuss

I've been using postgres, cassandra, redis and mysql/mariadb on aarch64. Only ran into issues with MySQL, which we rootcaused to some weird atomic locks not working as expected on the first generation of ARMv8(.0) a couple years back.

smspf

6 лет назадdiscuss

That's a fair point. Emulation implies other limitations too - code compiled on your machine might leverage only the CPU features emulated, which would lead to sub-optimal binaries, not to mention much slower builds.

smspf

6 лет назадdiscuss

So many wrong assumptions ...

1. If emulating aarch64 (arm64) on x86_64 is 6x slower (on your system, btw, it's not an universal constant), it doesn't mean emulating x86_64 on aarch64 will be 6x slower. It'd probably be worse, or at least that's my gut feeling.

2. Generic container images like the Ubuntu mentioned usually have aarch64 (arm64) support, so running the x86_64 image makes no sense for the presented use-case.

3. You won't be able to use most software because they don't release ARM binaries ... and the example uses `wget` && `tar xf`, with no binary signature check. As someone who has been porting stuff from x86_64 to aarch64 for a couple of years, I admit I've seen this pattern frequently. The most obvious solution is to build from sources, which would have been better off on x86_64 too, instead of fetching a prebuilt (and unverified) binary from the internet. Maybe there are some CPU flags the compiler could notice and apply optimizations which are not included in the prebuilt binary.

I'm not an Apple fan and I'm certainly not a fan of cross-architecture development either. I do agree with the general idea behind the article, however I find it a bit hand wavy.

comments