Apple Neural Engine Internal: From ML Algorithm to HW Registers(blackhat.com)
blackhat.com
Apple Neural Engine Internal: From ML Algorithm to HW Registers
https://www.blackhat.com/asia-21/briefings/schedule/index.html#apple-neural-engine-internal-from-ml-algorithm-to-hw-registers-22039
64 comments
Video : https://www.youtube.com/watch?v=1wvBDUnPNEo
Does anything actually use the ANE? I've had asitop on vanilla M1 running and have literally never seen anything use it.
This may be a dumb question, but would the space not be better used for more GPU cores? Then if an app actually does need to do neural net stuff, just use the GPU anyway? This way you wouldn't have the space the ANE uses idle 99.9% of the time.
This may be a dumb question, but would the space not be better used for more GPU cores? Then if an app actually does need to do neural net stuff, just use the GPU anyway? This way you wouldn't have the space the ANE uses idle 99.9% of the time.
> asitop
Neat little tool
> Does anything actually use the ANE?
I poked around for a few minutes to see if I could get anything. I saw a blip of usage on in Photos when adding a photo, probably for the new text recognition feature. I was able to see a lot of usage in Photo Booth, I guess for face detection for effects. There's definitely third-party apps making use of Core ML, but it's not something I know much about. On the iPhone devices, it's also used for Face ID (as the article mentions), which is probably much more important to Apple.
> This may be a dumb question, but would the space not be better used for more GPU cores? Then if an app actually does need to do neural net stuff, just use the GPU anyway?
The ANE is likely going to be more power-efficient than GPU cores loaded up for similar processing. Could they use the space for more GPU cores? Maybe, but they would have to work it into their memory architecture, support the additional power requirements, and presumably handle other engineering challenges like thermal design. Including the ANE also gives them something else to put in their marketing materials lol.
Neat little tool
> Does anything actually use the ANE?
I poked around for a few minutes to see if I could get anything. I saw a blip of usage on in Photos when adding a photo, probably for the new text recognition feature. I was able to see a lot of usage in Photo Booth, I guess for face detection for effects. There's definitely third-party apps making use of Core ML, but it's not something I know much about. On the iPhone devices, it's also used for Face ID (as the article mentions), which is probably much more important to Apple.
> This may be a dumb question, but would the space not be better used for more GPU cores? Then if an app actually does need to do neural net stuff, just use the GPU anyway?
The ANE is likely going to be more power-efficient than GPU cores loaded up for similar processing. Could they use the space for more GPU cores? Maybe, but they would have to work it into their memory architecture, support the additional power requirements, and presumably handle other engineering challenges like thermal design. Including the ANE also gives them something else to put in their marketing materials lol.
Dedicated neural net silicon is faster than GPU, at the cost of being less general purpose.
NVIDIA GPUs for example dedicate a modest fraction of their die to Tensor cores, at the cost of less general-purpose CUDA cores.
NVIDIA GPUs for example dedicate a modest fraction of their die to Tensor cores, at the cost of less general-purpose CUDA cores.
Thanks. I didn't realise how much faster. Probably you can do 10x the FLOPs in similar die space on dedicated silicion vs GPU. I thought it would be faster, but not an order of magnitude more.
ANE is used for the portrait video effect (blurred background) and other camera features.
Oof. Clicking this link got me blacklisted from my router. Had to spoof my MAC address to reconnect.
Sounds like you need DoH in your browser too?
What?
Overzealous "protection" filtering presumably (guessing from the Blackhat name). Happens on corporate VPNs too. We can't have nice things.
The router I'm connecting with is this Meraki thing I don't manage. It has some security features. Upon opening a link to blackhat.com, it put my MAC address on a blacklist.
That is so completely bass-ackwards it’s not even comprehensible. If the router knows you’re trying to connect to a malicious site, it should block your access to that site. Taking action against your device in this scenario doesn’t serve any purpose at all. I’m going to assume that this router is incredibly misconfigured, and that’s not a default behavior.
Routers are stupid (and the engineering around them is often incompetent), here's a random incidental example of how they used to make life hard for Chrome 10 years ago: https://bugs.chromium.org/p/chromium/issues/detail?id=12066#...
I think the logic is that if you're browsing blackhat.com then you are a malicious user, going there to download hacking tools or attacking the network.
Sounds like you should put the manufacturer on a blacklist.
> I just did byte flipping on the ANEProgram, and an iOS kernel OOB read was issued.
This does not give us a lot of confidence.
This does not give us a lot of confidence.
geohot reverse engineered ANE last year, for his tinygrad project, https://github.com/geohot/tinygrad
He streamed the many-hours efforts on Youtube, in case this is your thing:
https://m.youtube.com/watch?v=mwmke957ki4
Fascinating in a sense. All of this knowledge exists somewhere, but people are building their careers trying to understand what Apple built so that they can better understand what apple forgot.
retskrad(5)
Picking up am M1 Pro tomorrow I hope. Assuming it can play Factorio?
It doesn't support ARM64 and according the developer, probably never will but it runs very well on Rosetta.
Factorio has a Mac client[0] so the M1 Pro will surely play it just fine.
0: https://store.steampowered.com/app/427520/Factorio/
0: https://store.steampowered.com/app/427520/Factorio/