NumPy has a whole dispatch mechanism to deal with the tradeoffs. The main problem is code bloat: how many microarchitectures are you going to support with dispatch at runtime?
PyPy dev here. Really cool project to convert RPython from python2 to rust. They have a long way to go, but seem to be making progress. Unfortunate name conflict with a typechrcker.
The problem is the million small paper cuts. The stdlib changes are not all in pure python, many have implications for compiled modules like _ssl. The interpreter changes, especially compatibility with small interpreter changes that are reflected in the dis module, also require work to figure out
CFFI is fast on PyPy. The JIT still cannot peer into the compiled C/C++ code, but it can generate efficient interface code since there is a dedicated _cffi_backend module built into PyPy. Originally that was the motivation for the PyPy developers to create CFFI.
- The pure python repl started off in PyPy, although a lot of work was done to make it ready for prime time by the COython core devs
- The lessons from HPy are slowly making their way into CPython, see https://github.com/py-ni
- There were many fruitful interactions in fixing subtle bugs in CPython that stemmed from testing the stdlib on an alternative implementation
Yes. The C API those libraries use is a good fit to CPython, a bad fit to PyPy. Hence CFFI and HPy. Actually, many if the lessons from HPy are making their way into CPython since their JIT and speedups face the same problems as PyPy. See https://github.com/py-ni
Third party libraries like SciPy scikit-learn, pandas, tensorflow and pytorch have been critical to python’s success. Since CPython is written in C and exposes a nice C API, those libraries can leverage it to quickly move from (slow) python to (fast) C/C++, hitting an optimum between speed of development and speed of runtime.
PyPy’s alternative, CFFI, was not attractive enough for the big players to adopt. And HPy, another alternative that would have played better with Cython and friends came too late in the game, by that time PyPy development had lost momentum.
PyPy core dev here. If anyone is interested in helping out, either financially or with coding, we can be reached various ways. See https://pypy.org/contact.html
> Imagine how Open Source Software could improve if a consortium of nations put their money and resources into commissioning bug fixes and enhancements, which would be of collective benefit.
This is the business model of Quansight Labs, whose employees help maintain much of the scientific python stack. Mostly tech companies, not governments, sponsoring the work
A bit if a generalisation. For a different view: currently almost all the core devs for NumPy and SciPy (those are the ones I know personally) have doctorates.