Python 3.14 Getting New Interpreter(docs.python.org)
docs.python.org
Python 3.14 Getting New Interpreter
https://docs.python.org/3.14/whatsnew/3.14.html#whatsnew314-tail-call
25 comments
While it's not implemented, Rust even has a keyword reserved [1] for musttail, "become": https://github.com/phi-go/rfcs/blob/guaranteed-tco/text/0000...
[1] https://doc.rust-lang.org/reference/keywords.html
[1] https://doc.rust-lang.org/reference/keywords.html
And like everything else in Rust, it will forever live in the "unstable" pile of features that are required to use the language without standing on your head (looking at you, pattern matching through boxes).
For us non-compiler nerds, can you comment on which compilers support musttail? I suppose latest clang has it. gcc: since you mentioned support was only added in July 2024, I suppose it's not in gcc 14 yet? How about Apple clang from Xcode 16? And MSVC?
I don't closely follow the release schedules for Clang, GCC, etc, so I don't have up-to-date info on that. Unfortunately I'm not aware of any support for musttail in MSVC.
Thanks for the article.
Funnily enough, the first half of the article made a lot more sense after the second half where i was wondering "wouldn't goto do the same?" And then i remembered it was mentioned in the first half...
Funnily enough, the first half of the article made a lot more sense after the second half where i was wondering "wouldn't goto do the same?" And then i remembered it was mentioned in the first half...
> Preliminary numbers on our machines suggest anywhere from -3% to 30% faster Python code, and a geometric mean of 9-15% faster on pyperformance depending on platform and architecture.
Nice!
Nice!
Is this against 3.12 and 3.13? I only ask because there was a noticeable drop in performance when I updated to those versions.
> The baseline is Python 3.14 built with Clang 19 without this new interpreter.
Possibly with your code but a lot of work was done to speedup python3.11+ especially to compensate for the single thread slowdown when they finally remove the GIL (3.13 has nogil but only if you explicitly enable it). By most measurements those versions are signficantly faster then earlier python3 releases.
I get the feeling that Python is finally reaching the critical mass needed to significantly improve performance, good times.
Kinda sad how the industry settled on Python. The performance is not the best and the integration with native languages is far my ideal.
Yesterday I tried to create a Zig library for Bun using bunffi and it was mindblowing how easy it was. You can get pretty much the best of both worlds without obscure bindings. But yeah, both projects are still fairly young.
Yesterday I tried to create a Zig library for Bun using bunffi and it was mindblowing how easy it was. You can get pretty much the best of both worlds without obscure bindings. But yeah, both projects are still fairly young.
I use Zig and C with Python regularily and I think the interoperability is excellent. It's one of the coolest part about Python in my opinion, but the ecosystem is a nightmare to navigate. It's easy to use C structs for more efficient "classes" with libraries like msgspec, but chances are you've never heard about msgspec because there are at least 10 other libraries which do something similar, and there is also __slots__.
Even your package and project manager isn't straight forward. With Javascript you're basically going to use either Deno or Bun, but with Python you're going to use what? I'd argue that UV was the only "real" option (come at me!) but like 90% of the Python programmers I work with tend to have never heard about it. Maybe it's just my part of the world, but when you enter a Python shop that does web-api's you're 95% sure to find Flask, FastAPI or ninja-django (might be django-ninja), all of which are ok. The internet is full of recomendations on these, but they are all so underperformant compared to Litestar.
Then you have the actual language design where it is ridiculously easy to shoot yourself in the foot. As an example there is a massive difference in how you can loop over collections with Lists and Generators. It's not that different (in concept) from C# where you have IEnumerable and IQuerable, but it sort of is different in the way that almost every C# developer will know the difference while many Python programmers will never have heard of generators.
Over all though, I think there is a reason Python is everywhere. It's very productive, and it's also very easy to deal with bottlenecks when you run into them. I think the philosophy of designing code to be perfect from the get go is flawed. Partly because you're likely not going to know where bottlenecks will show up, but mainly because 95% (I pulled this out of my ass, but I'd wager it's accurate) of all software will never succeed enough to have scaling issues. Stackoverflow didnt even get CDN's until something like 2018ish, and during it's early days it was just a couple of IIS and SQL servers. Python fits perfectly into this sort of YAGNI philosophy because almost everyone can get things working well enough, and then you can always use actual engineering to deal with performance once you need to. Sure, I tend to write performant and type safe Python from the get go, but in a world of too few software engineers, it's very handy to have non-SWE personal build software which is "good enough". Especially in non-SWE industries, and Python is, the, language every non-SWE programmer uses.
Even your package and project manager isn't straight forward. With Javascript you're basically going to use either Deno or Bun, but with Python you're going to use what? I'd argue that UV was the only "real" option (come at me!) but like 90% of the Python programmers I work with tend to have never heard about it. Maybe it's just my part of the world, but when you enter a Python shop that does web-api's you're 95% sure to find Flask, FastAPI or ninja-django (might be django-ninja), all of which are ok. The internet is full of recomendations on these, but they are all so underperformant compared to Litestar.
Then you have the actual language design where it is ridiculously easy to shoot yourself in the foot. As an example there is a massive difference in how you can loop over collections with Lists and Generators. It's not that different (in concept) from C# where you have IEnumerable and IQuerable, but it sort of is different in the way that almost every C# developer will know the difference while many Python programmers will never have heard of generators.
Over all though, I think there is a reason Python is everywhere. It's very productive, and it's also very easy to deal with bottlenecks when you run into them. I think the philosophy of designing code to be perfect from the get go is flawed. Partly because you're likely not going to know where bottlenecks will show up, but mainly because 95% (I pulled this out of my ass, but I'd wager it's accurate) of all software will never succeed enough to have scaling issues. Stackoverflow didnt even get CDN's until something like 2018ish, and during it's early days it was just a couple of IIS and SQL servers. Python fits perfectly into this sort of YAGNI philosophy because almost everyone can get things working well enough, and then you can always use actual engineering to deal with performance once you need to. Sure, I tend to write performant and type safe Python from the get go, but in a world of too few software engineers, it's very handy to have non-SWE personal build software which is "good enough". Especially in non-SWE industries, and Python is, the, language every non-SWE programmer uses.
Have you looked at pybind11? It's quite easy to use to integrate C/C++.
I'd recommend nanobind over pybind11 at this point. Similar api, from the same author, but under active development and support.
At this point it feels like there is nothing stopping Python's performance from approaching that of Julia over the next decade.
Except for the language semantics.
They didn’t add tail calls for the longest time because it blows the stack and makes it harder to debug when stepping through code. Maybe they should make a keyword in Python to explicitly enable tail call recursion when it’s needed.
> This is not to be confused with tail call optimization of Python functions, which is currently not implemented in CPython.
> This new interpreter type is an internal implementation detail of the CPython interpreter. It doesn’t change the visible behavior of Python programs at all. It can improve their performance, but doesn’t change anything else.
> This new interpreter type is an internal implementation detail of the CPython interpreter. It doesn’t change the visible behavior of Python programs at all. It can improve their performance, but doesn’t change anything else.
Pi-thon.
[deleted]
[deleted]
[deleted]
It uses a technique I published several years ago for writing fast interpreters with tail calls: https://blog.reverberate.org/2021/04/21/musttail-efficient-i...
There is also this tweet from the author: https://x.com/kenjin4096/status/1887935698906529903