Apple Silicon: A little help from friends and co-processors(eclecticlight.co)

53 pointsby ingvevor 2 Jahren5 comments

Apple Silicon: A little help from friends and co-processors

https://eclecticlight.co/2024/03/01/apple-silicon-4-a-little-help-from-friends-and-co-processors/

5 comments

AhtiKvor 2 Jahren

Regarding the use of the Apple Neural Engine (ANE) for ML tasks: yes, it is closed source for direct access. However, reports indicate that it supports only forward propagation (without backward propagation) and offers a relatively limited set of supported precisions [1]. Should the API ever be opened, the ANE's primary use case would likely shift towards inference rather than training.

George Hotz made pretty good progress with the ANE access ca 3 years ago at one of his videos: https://www.youtube.com/watch?v=JAyw7OAcXDE

[1] Comments from MLX (Apple-contributed Metal-only array compute library): https://github.com/ml-explore/mlx/issues/18

neonsunsetvor 2 Jahren

NEON is strictly not a vector processor for all intents and purposes in Apple cores. Execution units (ports) capable of dispatching SIMD/FP instructions are part of the core itself alongside regular ALU, and Load/Store units.

https://dougallj.github.io/applecpu/firestorm.html

uticusvor 2 Jahren

> The Apple Neural Engine (ANE) is a separate unit in M-series chips, and isn’t directly accessible to third parties. As a result, knowledge of it is limited...Access to ANE computation is only available through CoreML and related features in macOS. Even there, developers are given limited options as to what does get executed in the ANE, and in practice it appears little used even when running Machine Learning tasks.

I had no idea ANE wasn't very accessible. Especially when the marketing materials [0] seem to tout is is available for use by devs, or when people are asking 'can I run LLM x on my Mac' [1]

[0] "An enhanced 16-core Neural Engine accelerates popular machine learning models. Image processing tools like DeNoise AI and Gigapixel AI from Topaz get even faster." https://www.apple.com/macbook-pro/

[1] https://news.ycombinator.com/item?id=38267869

saagarjhavor 2 Jahren

They are plenty accessible if you use the APIs Apple provides to talk to them, such as the one mentioned in the article: CoreML.

uticusvor 2 Jahren

Well, that's my point though, it is already pointed out in the section I quoted:

> Access to ANE computation is only available through CoreML and related features in macOS. Even there, developers are given limited options...

The article makes it sound like it's not 'plenty accessible.'