Show HN: A New Python Linter(github.com)
github.com
Show HN: A New Python Linter
https://github.com/guilatrova/tryceratops
45 comments
Wow! Good catch! The idea was to be: "NO Try Catch" and somewhat recall "NO T ry C eratops", but now that you pointed out it doesn't make sense sense and it's not intuitive.
What do you think about "notry"?
What do you think about "notry"?
notry is better. But why not be more verbose and just have `no_try_catch`
Why not the standard `noqa`?
I'm not a native english speaker... but how in the world "notc" sounds like "nazi"?
I'll try to pronounce it like "no-ti-see", or "notick" or something like that
I'll try to pronounce it like "no-ti-see", or "notick" or something like that
I'm a native American English speaker and automatically read notc as "not-see" which is the same pronunciation as nazi
In my eyes, the options are "no tc" and "not c". "Not c" is pronounced exactly the same as the german loanword.
If I say it as "not C", then it does not sound much like "nazi", but if I say it quickly, then it could be mistaken for "nazi". I am not a native English speaker either.
I am finding this specific comment fascinating, as your two sibling comments--which I presume, like me, are native English speakers (I specifically am from the mid-wear, importantly near Chicago, with a slight bit of Southern, and now I have spent half my life in California)--believe "not C" to be pronounced the same as "nazi". FWIW, they aren't the same, even when said quickly; but I will claim the difference is entirely in the syllable stress: nah-TZSEE (the T in "not" is unvoiced due to collision with the tongue position of speaking the letter "C", even when said slowly, unless "enunciating") vs NAH-tzsee; but the underlying sounds are exactly the same... similar to how "content" has to very different meanings as each of an adjective ("i am content with that") and a noun ("i posted some content") and are pronounced very differently, and yet the same ;P.
Nobody seems to have touched on this so I'll go.
Since trying rust's ans zig's error system I have avoided using python exception and replaced them with [1]. Basically functions return Ok(value) or Err(msg). callers can then get the result with `a = fn(a,b).unroll()`. For Ok it will unwrap and for Err it will return early (with a decorator to catch the virtual exception). The only overhead is the decorator, apart from this the ergonomics are the same as rust.
It's so nice I hope it becomes a part of the language some day.
[1] https://pypi.org/project/result/
Since trying rust's ans zig's error system I have avoided using python exception and replaced them with [1]. Basically functions return Ok(value) or Err(msg). callers can then get the result with `a = fn(a,b).unroll()`. For Ok it will unwrap and for Err it will return early (with a decorator to catch the virtual exception). The only overhead is the decorator, apart from this the ergonomics are the same as rust.
It's so nice I hope it becomes a part of the language some day.
[1] https://pypi.org/project/result/
I have done (ok, result) returns in some python code. At some point though, I realized almost any line of python can raise an exception. I find that there is just too much friction trying to avoid the exception system. I just learned to embrace it.
Where exceptions are problematic for my code base is when unwinding the stack is not the best behavior. Sometimes, I want the following lines to execute and sometimes I want to retry. Exceptions make a default execution flow very easy, but the default is sometimes wrong, and the correct flow is not considered until undesirable behavior appears at runtime. An (ok, result) system seems like it might force one to think more carefully about the proper flow. But the price is always having to think about it, when stack unwinding is usually the right flow. Or one might be tempted to create a Haskell like monad that automatically skips execution of following expressions, leading to the same bugs as an exception system. Correct code is possible with either system. I’m undecided if one system is more likely to lead to correct code.
Where exceptions are problematic for my code base is when unwinding the stack is not the best behavior. Sometimes, I want the following lines to execute and sometimes I want to retry. Exceptions make a default execution flow very easy, but the default is sometimes wrong, and the correct flow is not considered until undesirable behavior appears at runtime. An (ok, result) system seems like it might force one to think more carefully about the proper flow. But the price is always having to think about it, when stack unwinding is usually the right flow. Or one might be tempted to create a Haskell like monad that automatically skips execution of following expressions, leading to the same bugs as an exception system. Correct code is possible with either system. I’m undecided if one system is more likely to lead to correct code.
That's an interesting experiment, but exceptions are pretty central to how Python works.
For example, `for` loops are implemented by throwing a `StopIteration` exception when they're finished.
For example, `for` loops are implemented by throwing a `StopIteration` exception when they're finished.
I'm fine with exception being under the hood. It's just in actual app code I find the non locality of the flow makes things very implicit.
I like explicit code.
I like explicit code.
That's nice, thanks for reminding me of this. I will use it for a project, as I, too, very much like the way Rust does error handling.
Shameless plug of Semgrep, a project I'm working on:
I just wrote this Semgrep rule in 45 seconds which replicates the TC201 rule from tryceratops: https://semgrep.dev/s/underyx:tc201
Writing this rule was definitely easier than writing a new linter package, but also easier than writing a custom pylint or flake8 plugin.
Message to OP: we are working on making Semgrep's rule registry (https://semgrep.dev/explore) a much more community-centric place. Our vision is that just like you could publish the GitHub repo of guilatrova/tryceratops, you could publish a set of Semgrep rules under your name and brand. Semgrep users could then find and use your lightweight "linter" via the registry. If you're interested in being one of the first authors to publish on the Semgrep Registry, please reach out at [email protected]!
I just wrote this Semgrep rule in 45 seconds which replicates the TC201 rule from tryceratops: https://semgrep.dev/s/underyx:tc201
Writing this rule was definitely easier than writing a new linter package, but also easier than writing a custom pylint or flake8 plugin.
Message to OP: we are working on making Semgrep's rule registry (https://semgrep.dev/explore) a much more community-centric place. Our vision is that just like you could publish the GitHub repo of guilatrova/tryceratops, you could publish a set of Semgrep rules under your name and brand. Semgrep users could then find and use your lightweight "linter" via the registry. If you're interested in being one of the first authors to publish on the Semgrep Registry, please reach out at [email protected]!
I would value knowing how to flag the _lack_ of usage of the variable within the catch body; I had this problem while trying to write a semgrep rule for Java, but if I were working with python, it'd make me just as mad
I'll show the Java version first, because it's more obvious, then the equivalent in Python to stay moderately on-topic
I'll show the Java version first, because it's more obvious, then the equivalent in Python to stay moderately on-topic
try {
((String)null).length();
} catch (Exception e) {
System.err.println("I swallow errors for fun!");
}
try:
dict()["kaboom"]
except KeyError as ke:
logger.error("it did not work")
If I understand the comment right, the solution is to first use pattern-inside to mark that you're interested in all try-except blocks, and then pattern-not-inside to remove the ones that correctly reraise the exception. Could you check this out? https://semgrep.dev/s/underyx:always-reraise
If this helped, I recommend joining https://r2c.dev/slack — we've got a #rules channel dedicated to solving problems like this :)
If this helped, I recommend joining https://r2c.dev/slack — we've got a #rules channel dedicated to solving problems like this :)
As I understood it, the GP is looking to flag the non-usage of the "ke" variable in the exception body.
Why did the approach mentioned not work?
1) recognize all exception bodies.
2) recognize all exception bodies that use the ke variable and mark them as okay.
Anything left should be an exception body that didn't use the ke variable, right ?
Anything left should be an exception body that didn't use the ke variable, right ?
It sounds correct (as long as it can distinguish between use and shadowing), but the GP talked about reraising so I wasn't sure if they meant the same thing.
Semgrep looks great, is it free/OSS? I don't know how powerful it is from looking at the front page, but if it can replace many of our tools, I'd love to use it as a pre-commit hook with pre-commit.com.
Totally, it's LGPL licensed! https://github.com/returntocorp/semgrep
Here's how to use it with pre-commit: https://semgrep.dev/docs/extensions/#pre-commit
Here's how to use it with pre-commit: https://semgrep.dev/docs/extensions/#pre-commit
Well that's awesome, I'm very excited about this now! Thank you for working on it!
Yay! I'm based out of Athens right now so (just a shot in the dark but) if you happen to be here also, drop by The Cube and I'll help you get set up with it :D
Very cool! I'd open a PR to implement it in our Python repo today if it were a flake8 plugin.
As a separate linter, I have to update the half-a-dozen places my current linters (flake8 and mypy) get applied (Makefile, .vscode/settings.json, shared vimrc, cloudbuild.yaml, etc.)
If it were a flake8 plugin, I can just pip install and select the TC codes in my .flake8 configuration.
As a separate linter, I have to update the half-a-dozen places my current linters (flake8 and mypy) get applied (Makefile, .vscode/settings.json, shared vimrc, cloudbuild.yaml, etc.)
If it were a flake8 plugin, I can just pip install and select the TC codes in my .flake8 configuration.
I'm so glad to hear that. It should be very simple to implement a plugin. Once I'm able to, I'll create it and let you know! :)
I second this.
it looks like this linter is _specifically_ for try/except linting. do you anticipate users will use this alongside another linter? or do you plan to grow into a full linter?
I would love to see this as a flake8 extension. An own tool with own formatting, workflow, etc.? I'm not sure...
[deleted]
Today I use black (formatting), isort (import sort), and flake8 (general linting) for all my projects. I imagine people combining tryceratops power with other linters as well since the focus is very narrow.
I'll create a flake8 plugin soon since some people are interested in it.
I'll create a flake8 plugin soon since some people are interested in it.
I want a linter where I can exclude specific things in some kind of configuration. Having linter hash tags in files is:
- Ugly and makes for less readable code.
- Doesn't apply the same exception in similar situations. If I made an exception to a rule in one place, I'd like it to apply everywhere I have similar contexts.
- Gives a binary good / bad. A linter ought to flag questionable choices to look at -- not just bad choices. At that point, I should be able to say "oops, let me fix that," or "I meant to do that" without leaving garbage around.
I'd like this stuff in a linter config/data file, ideally, as semantically as possible. If I tell a spell checker that yes, in fact, my company name is a misspelling of a common word, I don't need to tell it twice. This is not true for linters.
I'd also like all the linty stuff configured in one place, with one syntax. Not pylint + pycodestyle.
Examples of use-cases:
- A lot of the times when I'm building an API, I have unused arguments in interim versions of the code. That's okay.
- I occasionally break Python naming conventions, especially when either planning a code refactor, or when trying to comply with non-Python naming conventions
- I generally dislike lines > 80 characters, but often, that's the right way to do things, especially if the alternative is breaking up a long token like a URL.
And so on. I'd like all of these caught, and then I'd like to give the linter a pat on the back, say "Thank you, but here it's okay," and move on with my life.
There's actually a lot more stuff which could be put into a linter once you assume that being wrong even 80% of the time still makes for a helpful tool. If a linter flags give things and I change one as a result, that's well worth the little bit of hassle. If a linter keeps flagging those things forever, it's no longer worth the hassle.
- Ugly and makes for less readable code.
- Doesn't apply the same exception in similar situations. If I made an exception to a rule in one place, I'd like it to apply everywhere I have similar contexts.
- Gives a binary good / bad. A linter ought to flag questionable choices to look at -- not just bad choices. At that point, I should be able to say "oops, let me fix that," or "I meant to do that" without leaving garbage around.
I'd like this stuff in a linter config/data file, ideally, as semantically as possible. If I tell a spell checker that yes, in fact, my company name is a misspelling of a common word, I don't need to tell it twice. This is not true for linters.
I'd also like all the linty stuff configured in one place, with one syntax. Not pylint + pycodestyle.
Examples of use-cases:
- A lot of the times when I'm building an API, I have unused arguments in interim versions of the code. That's okay.
- I occasionally break Python naming conventions, especially when either planning a code refactor, or when trying to comply with non-Python naming conventions
- I generally dislike lines > 80 characters, but often, that's the right way to do things, especially if the alternative is breaking up a long token like a URL.
And so on. I'd like all of these caught, and then I'd like to give the linter a pat on the back, say "Thank you, but here it's okay," and move on with my life.
There's actually a lot more stuff which could be put into a linter once you assume that being wrong even 80% of the time still makes for a helpful tool. If a linter flags give things and I change one as a result, that's well worth the little bit of hassle. If a linter keeps flagging those things forever, it's no longer worth the hassle.
flake8 is the most common linter front end and allows for exclusions/inclusions in configuration files.
https://flake8.pycqa.org/en/latest/user/configuration.html
https://flake8.pycqa.org/en/latest/user/configuration.html
The animation shows replacing 'raise ex' with 'raise' but just leaves the now unused ex in 'catch Exception as ex', and in one of the comments here there's a similar unused name ke in 'Keyerror as ke'. That hurts my C++ eyes a bit, but not having studied a lot of other Python code I wonder: is this a common thing to do, or are these just 2 'bad' examples?
It's 2 bad examples, this linter is focused on try/except blocks. For more general linting I would suggest flake8 :)
I'm planning on writing a plugin for it.
I'm planning on writing a plugin for it.
I'd love to see a violation category for error handling (ie. making sure that all exceptions are handled appropriately).
After using Golang for an extended period of time, coming back to Python has been frustrating at times because it is not as straight forward to handle errors/exceptions (especially when using external libraries that aren't well documented).
After using Golang for an extended period of time, coming back to Python has been frustrating at times because it is not as straight forward to handle errors/exceptions (especially when using external libraries that aren't well documented).
It's somewhat confusing that there is already a `triceratops` python module: https://pypi.org/project/triceratops/
What is an advantage of this over just writing new rules to something like Pylint?
Why not just add a rule to pylint or flake8?
Great timing. I was just looking into pylint
But also it would be pronounced like "nazi"? Not exactly the association you might be looking for.