Pytest Tips and Tricks(pythontest.com)
pythontest.com
Pytest Tips and Tricks
https://pythontest.com/pytest-tips-tricks/
48 comments
I have a personal gripe against Pytest. It extremely implicit, does so many things behind the scenes that it makes learning the library difficult for new comers. I had a hard time. For example, fixtures are passed as arguments in function. You can't use IDE tools to find out where they are passed from. There are dozens of things that go completely against the philosophy of Python: Explicit is better than implicit. Pytest is full of magic, it irritates me. The kind of questions asked on Stack Overflow reflects my personal experience with libraries of this kind. So many people are confused.
I am using PyCharm and it perfectly integrates with pytest fixtures. You can navigate to the fixture definition with a click, and it clearly distinguishes fixtures and common arguments.
Also works in VSCode with little setup
The following link describes the set up used / needed by VS Code to integrate the framework with the IDE:
https://code.visualstudio.com/docs/python/testing
https://code.visualstudio.com/docs/python/testing
"with little setup" could be construed as "very little" or as the opposite; i.e., some amount of non-trivial setup.
Which meaning did you intend?
Which meaning did you intend?
I absolutely agree about fixtures-as-arguments thing. Ward does this a lot better, using default values for the fixture factory. There's a long issue on ideas to implement something like that as a pytest plugin (https://github.com/pytest-dev/pytest/issues/3834), but it seems the resulting plugin relies on something of a hack.
I have the exact opposite experience, writing tests are actually sort of enjoyable using pytest.
The trick might be not using an IDE at all, I consider python and most of its frameworks explicit enough to write with editors like vim. I never found a need to use an IDE for python even when the projects gets larger, because even then the unix tools seemed to work much better at whatever that IDE is doing. This compares sharply against languages in the JVM family.
The trick might be not using an IDE at all, I consider python and most of its frameworks explicit enough to write with editors like vim. I never found a need to use an IDE for python even when the projects gets larger, because even then the unix tools seemed to work much better at whatever that IDE is doing. This compares sharply against languages in the JVM family.
with or without IDEs, you cannot easily tell where fixtures come from. Is it from the same module? same package? a plugin? pytest itself? some non-test module that happened to be imported? You can't tell. And it's a global namespace where everything can override everything else.
The hierarchy is pretty clearly stated here and it seemed to be against what you're suggesting: https://docs.pytest.org/en/6.2.x/fixture.html#fixture-availa..., even though I don't think the hierarchy matters much if the fixtures are explicitly placed and named.
If you're instantiating the same fixture name across a bunch of different tests, using similar fixtures in different locations and not use import or a central or hierarchical location to define your fixtures after you have accumulated a number of similar items, I find it hard to blame the module for not being explicit.
If you're instantiating the same fixture name across a bunch of different tests, using similar fixtures in different locations and not use import or a central or hierarchical location to define your fixtures after you have accumulated a number of similar items, I find it hard to blame the module for not being explicit.
If for some reason a fixture is misbehaving, it's hard to find what package provides it because I don't explicitly import it.
I have to use `pytest --fixtures | less` and search for the fixture's name (or `pytest --fixtures | grep -C 50`)
> If you're instantiating the same fixture name across a bunch of different tests
It means I cannot name my fixture `database`, I have to call it `myplugin_database` instead to make sure it won't conflict with other plugins. And tests using this fixture have the long `myplugin_database` as argument.
Python already had namespaces as first-class citizens, it's a shame we have to go back to name prefixes.
I have to use `pytest --fixtures | less` and search for the fixture's name (or `pytest --fixtures | grep -C 50`)
> If you're instantiating the same fixture name across a bunch of different tests
It means I cannot name my fixture `database`, I have to call it `myplugin_database` instead to make sure it won't conflict with other plugins. And tests using this fixture have the long `myplugin_database` as argument.
Python already had namespaces as first-class citizens, it's a shame we have to go back to name prefixes.
I just ran into the fixture problem myself, maybe you can help me understand how I could find this myself.
I wanted to integrate pytest-benchmark[1]. Their docs mention fixtures a couple of times, but the examples only show that test functions receive a magic untyped parameter called "benchmark".
How would I make my usage more explicit? Can I somehow explicitly import the fixture, or at least type it?
[1] https://pytest-benchmark.readthedocs.io/en/latest/index.html
I wanted to integrate pytest-benchmark[1]. Their docs mention fixtures a couple of times, but the examples only show that test functions receive a magic untyped parameter called "benchmark".
How would I make my usage more explicit? Can I somehow explicitly import the fixture, or at least type it?
[1] https://pytest-benchmark.readthedocs.io/en/latest/index.html
I don't think you can do better than make pytest-benchmark a dependency
> I cannot name my fixture `database`, I have to call it `myplugin_database` instead
On the subject of implicit vs explicit, `myplugin_database` is unquestionably more explicit.
On the subject of implicit vs explicit, `myplugin_database` is unquestionably more explicit.
> I have the exact opposite experience, writing tests are actually sort of enjoyable using pytest.
Yes, likewise after you understand how it works. It's great. That's not what I am arguing about.
Pytest could have been designed more explicitly while retaining it's advantages. I am a huge fan of how I can just use "==" operators to test equality. The UX of pytest is great after you know how it works.
Yes, likewise after you understand how it works. It's great. That's not what I am arguing about.
Pytest could have been designed more explicitly while retaining it's advantages. I am a huge fan of how I can just use "==" operators to test equality. The UX of pytest is great after you know how it works.
Same feelings here. When I see pytest is like: "oh no...". Been using nose2/unittest for ages and I will continue to do so because is so damn simple.
> that it makes learning the library difficult for new comers
Not only for newcomers, I've used in the past and if I need to pick it up again I believe I'm going to spend some time refreshing a ton of things and probably get got caught by some caveat.
> fixtures are passed as arguments in function
Exactly questioned that myself so long ago. Implementing fixture functionality from scratch in Python (even supporting different formats) is only a few lines of code.
> that it makes learning the library difficult for new comers
Not only for newcomers, I've used in the past and if I need to pick it up again I believe I'm going to spend some time refreshing a ton of things and probably get got caught by some caveat.
> fixtures are passed as arguments in function
Exactly questioned that myself so long ago. Implementing fixture functionality from scratch in Python (even supporting different formats) is only a few lines of code.
I agree with you completely, but...
> There are dozens of things that go completely against the philosophy of Python: Explicit is better than implicit.
I have found that this is not just true of pytest, but also of the python language! :P
> There are dozens of things that go completely against the philosophy of Python: Explicit is better than implicit.
I have found that this is not just true of pytest, but also of the python language! :P
> I have found that this is not just true of pytest, but also of the python language! :P
Curious as to how? It is pretty explicit and simple when compared against the pack.
Curious as to how? It is pretty explicit and simple when compared against the pack.
It depends what's in the pack of comparables... But once you get used to an actually simple language like Clojure or Haskell (neither of which is perfect, but both of which are very very simple) it really is hard to unsee all the accidental complexity of most languages.
Hard disagree. Pytest is a great. Even with a really basic language support in neovim I have no trouble using it. For newcomers, it does have a learning curve but it's not that bad. There's a lot of "magic", but Python is full of magic.
Fixture names are globally unique, it makes it easy to find their definitions e.g., dumb-jump in Emacs works: one keystroke to go to the definition or if it is overridden then it shows available options.
My annoyance is that it's impossible to tell upfront which parameters are fixtures just by looking at the signature of a test function. I would've much preferred an explicit decorator.
My favorite pytest trick: Using jupytext notebooks as tests.
I always use a notebook for notes and experiments when building python packages. By using `jupytext`[0] I can store the notebook as a commented python-file, which I can run from a parametrized test that runs all notebooks in a given folder [1]. Adding a few `asserts`, this gives you reasonable test coverage right from the start.
[0]: https://jupytext.readthedocs.io/en/latest/ [1]: https://stackoverflow.com/a/56813896/212538
I always use a notebook for notes and experiments when building python packages. By using `jupytext`[0] I can store the notebook as a commented python-file, which I can run from a parametrized test that runs all notebooks in a given folder [1]. Adding a few `asserts`, this gives you reasonable test coverage right from the start.
[0]: https://jupytext.readthedocs.io/en/latest/ [1]: https://stackoverflow.com/a/56813896/212538
You can also use my article to learn a bit about testing with Pytest! https://stribny.name/blog/pytest/
A couple recent related podcasts on Pytest with co-authors:
- https://testandcode.com/195
- https://talkpython.fm/episodes/show/407/pytest-tips-and-tric...
- https://testandcode.com/195
- https://talkpython.fm/episodes/show/407/pytest-tips-and-tric...
Does anyone have tips on how to speed up test collection in pytest? In our django app (~5000 tests) it takes about 30s to run a single test with `-k testname` when the actual time spent in the test is on the order of a few hundred milliseconds.
It hangs forever on the `collecting ...` phase.
It hangs forever on the `collecting ...` phase.
Run the single test with "pytest tests/path/to/test_file.py::TestClass::test_method" instead of using pytest -k to parse all your test code looking for a method with the right name.
Alternatively, your IDE or editor probably already has an extension that allows you to right click - run single test. VS Code at least has some plugin that collects the tests in the background.
Alternatively, your IDE or editor probably already has an extension that allows you to right click - run single test. VS Code at least has some plugin that collects the tests in the background.
To add to this, the test ID pytest prints when a test has had a failure/warning is the same form you can call with pytest to run the test. For example, let's say pytest outputs this:
=============================== warnings summary ===============================
tests/parser/test_expressions.py::test_complex[+inf+1.23e7j-(inf+12300000j)]
pytest tests/parser/test_expressions.py::test_complex[+inf+1.23e7j-(inf+12300000j)]To add to that, pytest's parametrized IDs can be fully controlled in the test code [0], so your test can be something like:
pytest tests/parser/test_expressions.py::test_complex[special_case_7]
Also pytest has flags like —lf and —sw to rerun failed tests or tackle test suites with lots of failures.
Thanks, it is a lot better (11s instead of 30)
Times I've profiled large tests suites / codebases in the past, this has been entirely down to import overheads. Quite a difficult thing to fix ...
I'm hoping this PEP Proposal for lazy imports might solve things: https://peps.python.org/pep-0690/
Try profiling pytest with something like pyinstrument[1] and make sure you invoke it from python directly[2] with `--collect-only` as an argument. You'll essentially then just be profiling the import overheads before tests are ran.
[1] - https://pyinstrument.readthedocs.io/en/latest/guide.html#pro...
[2] - https://pyinstrument.readthedocs.io/en/latest/guide.html#pro...
I'm hoping this PEP Proposal for lazy imports might solve things: https://peps.python.org/pep-0690/
Try profiling pytest with something like pyinstrument[1] and make sure you invoke it from python directly[2] with `--collect-only` as an argument. You'll essentially then just be profiling the import overheads before tests are ran.
[1] - https://pyinstrument.readthedocs.io/en/latest/guide.html#pro...
[2] - https://pyinstrument.readthedocs.io/en/latest/guide.html#pro...
Thanks, pyinstrument is really nice. You're right it looks like imports are taking a lot of time...
With a Django project, what it is actually doing is spinning up a new test database at this point which will be the cause of your slowness. If you haven’t made database schema changes you can run it with the flag “—reuse-db”.
If you’ve got a large project that’s been going for a while, squashing migrations helps too. It can be a bit tricky to do this though.
If you’ve got a large project that’s been going for a while, squashing migrations helps too. It can be a bit tricky to do this though.
Thanks, I run it with --nomigrations so it's not the migrations. Reusing db shaves about 2 seconds (I'm down to ~9 seconds when running a single test and specifying the full path)
You can also consider using CodiumAI to create tests right inside your IDE.
https://www.codium.ai
https://plugins.jetbrains.com/plugin/21206-codiumai--meaning...
https://marketplace.visualstudio.com/items?itemName=Codium.c...
https://plugins.jetbrains.com/plugin/21206-codiumai--meaning...
https://marketplace.visualstudio.com/items?itemName=Codium.c...
Also, run your tests in watch mode using entr or pytest-watch [0] for faster feedback.
[0] https://pypi.org/project/pytest-watch/
[0] https://pypi.org/project/pytest-watch/
pytest-repeat for flaky tests!
Why does everyone want money for this kind of content. It's exhausting.
I mean, generally the ROI on stuff like this is pretty solid. Looks like the book is $36 new. From a quick Google, the average software engineer (who is the group most likely to read and benefit from this book) makes 90k per year. Gross, that's 45 dollars an hour. If I buy this book and it saves me one hour of time writing tests, writing better tests, debugging tests, or whatever, it will have paid for itself. That's obviously not counting reading time, but I believe the point stands. All the time saves after that first hour is gravy on top
Also, content takes time to make. If you want to charge for it I think that's completely fine. It's not like it must be our based.
Also, content takes time to make. If you want to charge for it I think that's completely fine. It's not like it must be our based.
"If I buy this book and it saves me one hour of time writing tests, writing better tests, debugging tests, or whatever, it will have paid for itself."
Only if you have unlimited potential work for that rate, and are actually billing for contracted work, not by the hour.
In most scenarios, you will have paid for the book yourself, spent some time reading it (hopefully billable hours) and there is a real chance increased knowledge actually leads to more work, as you now have a more in depth understanding of the topic.
To me it all reads like the typing speed vs productivity essays. Yes, you should not be hindered by hunt and peck style typing interfering with your train of thoughts. But never seen someone who writes actual loc or actual content (not casual e-mail replies) where typing speed drives productivity in any way.
Only if you have unlimited potential work for that rate, and are actually billing for contracted work, not by the hour.
In most scenarios, you will have paid for the book yourself, spent some time reading it (hopefully billable hours) and there is a real chance increased knowledge actually leads to more work, as you now have a more in depth understanding of the topic.
To me it all reads like the typing speed vs productivity essays. Yes, you should not be hindered by hunt and peck style typing interfering with your train of thoughts. But never seen someone who writes actual loc or actual content (not casual e-mail replies) where typing speed drives productivity in any way.
You're right that it doesn't translate literally to cash, but I've been able to cite increased velocity or point to systems I've designed at work with concepts I learned from similarly process books that allowed me to push for some solid compensation increases and I think the couple hundred dollars in books I purchase a year has allowed me to do that.
His book is great, highly recommend, especially if you like hands-on tutorial style. I am a pytest convert now after spending a decade using unittest.
yeah and there is a million other things we need to know. you want to us buy every book or course that comes along for a million different topics which you don't know if it will be worthwhile before the fact?
Its like saying why does an engineer want payment for work done.It's book which are notably hard to write, what your are saying is why is it not free?
Imo freeloaders are exhausting and you are free to search for content and release one for free.
Where is the author asking for money?
It looks like a free article to me?