That is very interesting. Would you mind testing the same prompt with Claude Sonnet 3.5 and Opus? If not available to you, would you be willing to share the prompt/question? Thank you.
Has anyone seen an implementation of 'SpQR: A Sparse-Quantized Representation,' published in June 2023 by Tim Dettmers et al.? https://arxiv.org/abs/2306.03078
Regarding the use of the Apple Neural Engine (ANE) for ML tasks: yes, it is closed source for direct access. However, reports indicate that it supports only forward propagation (without backward propagation) and offers a relatively limited set of supported precisions [1]. Should the API ever be opened, the ANE's primary use case would likely shift towards inference rather than training.
Not sure how many have discovered but ChatGPT works well with many languages, for example asking a question in estonian gives an adequate answer in estonian, just the grammar is a tiny bit more off.