notatallshaw

28 dagen geledendiscuss

> Their infrastructure (pip) getting worse and worse

Hi, I'm a pip maintainer, can you please report any issue you think is happening in recent versions of pip: https://github.com/pypa/pip/

I particularly work on performance and stability and think we've been making big strides over the last two years

> getting everyone to install private venv's for every app, leading to missing security updates, because updates just break everything.

venv's stop your OS tooling from breaking, Linux distro maintainers pushed Python packaging to not allow installing into the system packages by default, and they weren't wrong.

All these other problems are solved with proper use of a lock file (which are now standardized, and you can generate and install from them in the latest version of pip), and/or use a project tool (uv, poetry, etc.).

notatallshaw

2 maanden geledendiscuss

> Running pip install lightning is all that is needed to activate

FYI, pip added cooldowns in 26.1:

  * https://discuss.python.org/t/announcement-pip-26-1-release/107108
  * https://ichard26.github.io/blog/2026/04/whats-new-in-pip-26.1/

To use:

  * CLI: pip install --uploaded-prior-to=P1D ...
  * Env Var: PIP_UPLOADED_PRIOR_TO=P1D pip install ...
  * Config: pip config set global.uploaded-prior-to P1D

notatallshaw

3 maanden geledendiscuss

The note at the end of the post indicates the user asked Claude to review their own chat logs. It's impossible to tell if Claude used or built a a performance harness or just wrote those numbers based on vibes.

notatallshaw

4 maanden geledendiscuss

Pip maintainer here, to do this in pip (26.0+) now you have to manually calculate the date, e.g. --uploaded-prior-to="$(date -u -d '3 days ago' '+%Y-%m-%dT%H:%M:%SZ')"

In pip 26.1 (release scheduled for April 2026), it will support the day ISO-8601 duration format, which uv also supports, so you will be able to do --uploaded-prior-to=P3D, or via env vars or config files, as all pip options can be set in either.

notatallshaw

4 maanden geledendiscuss

The threat actor was sophisticated enough to spam GitHub issues with dozens of different accounts. I imagine they could completely overwhelm PyPI with unauthenticated reports.

notatallshaw

4 maanden geledendiscuss

> I had no idea how to contact anyone related to PyPI

https://pypi.org/security/:

> If you've identified a security issue with a project hosted on PyPI Login to your PyPI account, then visit the project's page on PyPI. At the bottom of the sidebar, click Report project as malware.

notatallshaw

4 maanden geledendiscuss

As a pip maintainer I don't think that's really true. The resolver in both pip and uv are fundamentally sequential and single threaded, you can't really queue up or split out jobs.

What uv does is parallelize the final download of packages after resolution, and batch pre-fetch metadata during resolution. I don't think these benefit from async, due to their batch nature classic multi-threaded download pools are probably the better solution, but I could be wrong!

Experiments have been done on the former in pip and didn't find much/any improvement in CPython, this may change in free threaded CPython. For the latter we currently don't have the information from the resolver to extract a range of possible metadata versions we could pre-range, I am working on this but it requires new APIs in packaging (the Python library) and changes to the resolver, and again we will need to benchmark to see if adding pre-fetching actually improves things.

notatallshaw

4 maanden geledendiscuss

pixi offloads PyPI ecosystem stuff to uv, but pixi is conda first. The team were actually the first to build a Rust based Python package resolver (rip), but after uv was released they migrated to uv's resolver (Python package resolvers are hard and a lot of work to build and must be tested against the whole ecosystem).

notatallshaw

4 maanden geledendiscuss

On a widely used open source project I maintain I've been seeing PRs in the last month that are a little off (look okayish but are trivial or trying to solve problems in weird ways), and then when I look at their account they started opening PRs within the last few weeks, and have opened hundreds of PRs spread over hundreds of repositories.

notatallshaw

4 maanden geledendiscuss

My understand is Astral's focus for ty has been on making a good experience for common issues, whereas they plan for very high compliance but difficult or rare edge cases aren't are prioritized.

Compliance suite numbers are biased towards edge cases and not the common path because that's where a lot of the tests need to be added.

My advise is to see how each type checker runs against your own codebase and if the output/performance is something you are happy with.

notatallshaw

5 maanden geledendiscuss

AI is somewhat helpful but I'm not interested in a company finding a way for me to pay to do my volunteer OSS work. GitHub Copilot offers a permanent free subscription for OSS maintainers.

I previously ignored a free offer when Claude reached out to me as an open source maintainer as it was a glorified free trial. I hope this one continues beyond the listed 6 months, I am not interested in a glorified free trial and if it requires entering credit card details I won't be signing up.

notatallshaw

6 maanden geledendiscuss

I found the diagrams on the Wikipedia pages help build an intuitive understanding of each step: https://en.wikipedia.org/wiki/Conflict-driven_clause_learnin...

Also, the pubgrub-rs guide I find has a gentle ramp in introducing complexity: https://pubgrub-rs-guide.pages.dev/internals/intro

notatallshaw

6 maanden geledendiscuss

I assume lock and dependency files are in the training data, so predicting version number tokens have high probabilities associated with them.

notatallshaw

6 maanden geledendiscuss

As a pip maintainer I primarily think about resolution and resolver performance. I got involved when pip introduced it's current resolver around late 2020, and got my first PR landed in mid 2021.

I recently became a packaging maintainer, from working on fixing edge case behavior around specifiers and prerelease versions.

When I did some recent profiling I noticed that A LOT of time was being spent in packaging, largely parsing version strings. I found a few places in pip and packaging that reduced the number of Version objects being created, Henry really ran with the idea of improving performance and made big improvements. I'm excited for this to be vendored in pip 26.0 coming out at the end of January.

If anyone is interested the next big improvement for pip is likely to implement a real CDCL (Conflict-Driven Clause Learning) resolver algorithm, like uv's use of pubgrub-rs. That said, I do this in my spare time so it may be a year or more before I make any real traction on implementing that.

notatallshaw

7 maanden geledendiscuss

> At one company I worked at, we had a system where each deploy got its own folder, and we'd update a symlink to point to the active one. It worked, but it was all manual, all custom, and all fragile.

The first time I saw this I thought it was one of the most elegant solutions I'd ever seen working in technology. Safe to deploy the files, atomic switch over per machine, and trivial to rollback.

It may have been manual, but I'd worked with a deployment processes that involved manually copying files to dozens of boxes and following 10 to 20 step process of manual commands on each box. Even when I first got to use automated deployment tooling in the company I worked at it was fragile, opaque and a configuration nightmare, built primarily for OS installation of new servers and being forced to work with applications.

notatallshaw

7 maanden geledendiscuss

We're getting there https://pip.pypa.io/en/stable/cli/pip_lock/ !

Pip has been a flag bearer for Python packaging standards for some time now, so that alternatives can implement standards rather than copy behavior. So first a lock file standard had to be agreed upon which finally happened this year: https://peps.python.org/pep-0751/

Now it's a matter of a maintainer, who are currently all volunteers donating their spare time, to fully implement support. Progress is happening but it is a little slow because of this.

notatallshaw

8 maanden geledendiscuss

Unfortunately that was posted 1 month before the Faster CPython project was disbanded by Microsoft, so I imagine things have slowed.

notatallshaw

8 maanden geledendiscuss

Python has about 40 keywords, I say I would regularly use about 30, and irregularly use about another 5. Hardly seems like a "junkyard".

Further, this lack of first class support for lazy importing has spawned multiple CPython forks that implement their own lazy importing or a modified version of the prior rejected PEP 690. Reducing the real world need for forks seems worth the price of one keyword.

notatallshaw

9 maanden geledendiscuss

> I had been hoping someone would introduce the non-virtualenv package management solution that every single other language has where there's a dependency list and version requirements (including of the language itself) in a manifest file (go.mod, package.json, etc) and everything happens in the context of that directory alone without shell shenanigans.

Isn't that exactly a pyproject.toml via the the uv add/sync/run interface? What is that missing that you need?

notatallshaw

9 maanden geledendiscuss

> You let people make the mistake and have the library throw an exception if they do that, not through type checking but just through something eventually calling a method that doesn't exist.

Exceptions or crashes would be annoying, but yes, are manageable, although try telling that to new users of the language that their code doesn't work because they didn't understand the transitive dependency tree of their install and it automatically vendored different versions of a library for different dependencies, and how did they not know that from some random exception occurring in a dependency.

But as I explain in my example, the real problem is that one version of the library reads the data in a different layout from the other, so instead you end of with subtle data errors. Now your code is working but your getting the wrong output, good luck debugging that.

comments