> I do get the impression that multi-threaded Python code is full of hazards that the GIL covers up.
It was a design goal of free-threading that any race condition which can occur in pure python code without the GIL must have already been possible on the same code with the GIL. To achieve that, they added various locks which fix cpython primitive operations that were only threadsafe because of the GIL. That's why single-threaded execution is slower without the GIL.
However, it's possible there were races which were very unlikely with the GIL that are much more likely now, e.g. one requiring multiple context switches in evaluating a single statement like `a.b = c[a.b].d()`. With the GIL, that might require multiple operations in that assignment taking >5ms. If all those operations are fast might almost never come up, but without the GIL, a different thread could mutate a.b as often as it wants.
I haven't heard of this coming up in practice, but it's certainly something that could in principle happen. The amount of FUD about it in every discussion about python finally making progress on threading seems excessive though.
Akasa (https://akasa.com) | ML, backend, golang/typescript/python | REMOTE (Must be located in the US) or onsite in SSF
Akasa is a healthcare automation company, pioneering "Unified Automation" that seamlessly integrates machine learning, human domain experts and traditional software automation into healthcare workflows for revenue cycle management and other areas. We combine deep domain knowledge of healthcare revenue cycle management with innovative technology. We have contracts with some of the largest healthcare providers in the US and are growing quickly.
We're currently hiring for several roles in machine learning, security/infrastructure and our automation platform. We're also hiring in implementation engineering positions that can provide a good career path to level up on software engineering. https://akasa.com/careers/
It was a design goal of free-threading that any race condition which can occur in pure python code without the GIL must have already been possible on the same code with the GIL. To achieve that, they added various locks which fix cpython primitive operations that were only threadsafe because of the GIL. That's why single-threaded execution is slower without the GIL.
However, it's possible there were races which were very unlikely with the GIL that are much more likely now, e.g. one requiring multiple context switches in evaluating a single statement like `a.b = c[a.b].d()`. With the GIL, that might require multiple operations in that assignment taking >5ms. If all those operations are fast might almost never come up, but without the GIL, a different thread could mutate a.b as often as it wants.
I haven't heard of this coming up in practice, but it's certainly something that could in principle happen. The amount of FUD about it in every discussion about python finally making progress on threading seems excessive though.