That is very interesting. Would you mind testing the same prompt with Claude Sonnet 3.5 and Opus? If not available to you, would you be willing to share the prompt/question? Thank you.
Has anyone seen an implementation of 'SpQR: A Sparse-Quantized Representation,' published in June 2023 by Tim Dettmers et al.? https://arxiv.org/abs/2306.03078
Regarding the use of the Apple Neural Engine (ANE) for ML tasks: yes, it is closed source for direct access. However, reports indicate that it supports only forward propagation (without backward propagation) and offers a relatively limited set of supported precisions [1]. Should the API ever be opened, the ANE's primary use case would likely shift towards inference rather than training.