Ask HN: Could Fable/Mythos be used to build Python's JIT?
Since Fable / Mythos was so capable, I've been wondering if Anthropic could build a JIT layer for python. Would it significantly reduce the cost of training and running LLMs.
2 comments
A JIT for Python is hard because it's a very dynamic language. You can redefine the functions on fly or monkey path a class.
Anyway, 99.99% of the hard calculation for LLM are done in libraries that are written in C. I once tried to rewrite a custom case of enisum in plain Python and it was very slow until I used gvectorize and other specialized stuff.
Anyway, 99.99% of the hard calculation for LLM are done in libraries that are written in C. I once tried to rewrite a custom case of enisum in plain Python and it was very slow until I used gvectorize and other specialized stuff.
They are bleeding money. Anything they do will cost more to the end user, not less.