Using Enumerated Types in Python(johnlekberg.com)
johnlekberg.com
Using Enumerated Types in Python
https://johnlekberg.com/blog/2020-06-06-enum.html
42 comments
Enums are best when combined with a language that does pattern matching and that can warn you if you forgot a value in your case statement. Python lacks this, so there is no spectacular benefit of using enums over just a list of strings.
Agreed on pattern matching, though mypy can help a lot here with exhaustive checks at the very least. They do work on enums and discriminated unions. They should document it:
https://github.com/python/mypy/issues/6366
https://github.com/python/mypy/issues/5818
https://github.com/python/mypy/issues/6366
https://github.com/python/mypy/issues/5818
The article is incredibly misleading. Python Enums are way more powerful than what is showcased there.
First, rudimentary/basic enums are as simple as instance variables on a class and allow super-helpful IDE "auto-completion" that doesn't necessitate the use of magic-strings to at least "set" the enum. I've been using this ridiculously simple trick since 2.5 ten years ago. Magic-strings are bad, and even worse when you have a disparate code-base in a dynamically typed language. Example:
First, rudimentary/basic enums are as simple as instance variables on a class and allow super-helpful IDE "auto-completion" that doesn't necessitate the use of magic-strings to at least "set" the enum. I've been using this ridiculously simple trick since 2.5 ten years ago. Magic-strings are bad, and even worse when you have a disparate code-base in a dynamically typed language. Example:
class MyEnum(object):
Pass = 1
Fail = 2
my_value = MyEnum. <-- at this point, the IDE gives you code-completion.
class MyEnum(Enum):
Pass = 1
Fail = 2
def my_func(did_they_pass: MyEnum):
if did_they_pass.Something: # IDE / type-checker will go nuts here.
print("Something")
So, you don't use open("w"), do you ? You use the non magic open(os.O_WRONLY) of course.
Because magic strings are bad.
Because magic strings are bad.
I will now - thanks for the tip! :)
And, moreover, this will inspire me to ask "How many other traditional magic numbers/strings/etc have I gotten used to that might be defined more readably?" strcmp, I'm looking at you!
I know your post was sarcastic/snarky but it was genuinely helpful
And, moreover, this will inspire me to ask "How many other traditional magic numbers/strings/etc have I gotten used to that might be defined more readably?" strcmp, I'm looking at you!
I know your post was sarcastic/snarky but it was genuinely helpful
f.tell(offset, os.SEEK_CUR) over f.tell(offset, 1)
https://docs.python.org/3/library/os.html#os.SEEK_CUR
I wish the docs on f.tell at least mentioned the option
https://docs.python.org/3/tutorial/inputoutput.html#methods-...
https://docs.python.org/3/library/os.html#os.SEEK_CUR
I wish the docs on f.tell at least mentioned the option
https://docs.python.org/3/tutorial/inputoutput.html#methods-...
Except it wasn't, os.O_WRONLY is for os.open() (equivalent of open syscall, takes numeric flags), not the built-in open() (equivalent of C fopen, takes "w" and similar strings).
Python 3.8 has typing.Literal for this.
That's a fair point, and I'll be sure to keep an eye out for that type of magic-string usage going forward. Though most "magic string" usages are not nearly as official as the one you mention and usually occur in business-level code written by ordinary developers that may not be disciplined or experienced. Those spots most certainly need all the assistance they can get instead of relying on magic-strings, hence my comment of magic strings being bad.
You would have to use os.open
I would also have to pass the path as a first param. This was not meant to be correct, just teasy.
>Second, real Enums added as a language-feature along with type-hinting are almost 100% on-par with statically typed languages
This is conclusive proof that you're not actually aware of the capabilities of enums and sum types in "statically typed languages". Where's pattern matching? Exhaustivity checking? Sum types? GADTs?
This is conclusive proof that you're not actually aware of the capabilities of enums and sum types in "statically typed languages". Where's pattern matching? Exhaustivity checking? Sum types? GADTs?
C and C++ don't have some features in that list like pattern matching and there is no question that they represent one of the most popular statically typed languages in history.
Honestly, over my nearly 20 year period of programming experience (half of which was not spent on python), I can't recall ever having encountered those terms/concepts. But on the plus side, I guess I have now and I'll be sure to read up on them. Maybe I know the concepts but haven't linked them to the terms.
Python 4... It's just going to be Haskell.
>real Enums added as a language-feature along with type-hinting are almost 100% on-par with statically typed languages
sorry i don't understand this. i'm not trying to be pedantic but: statically typed (and type-checked) language will fail to compile ... not just warn you. that's not a 1% difference that's the entire difference (because lots of people will miss or flat out ignore ide hints).
sorry i don't understand this. i'm not trying to be pedantic but: statically typed (and type-checked) language will fail to compile ... not just warn you. that's not a 1% difference that's the entire difference (because lots of people will miss or flat out ignore ide hints).
> Python lacks this, so there is no spectacular benefit of using enums over just a list of strings.
Enums represent a discrete set of constant values. They are extremely helpful to communicate to developers what to expect from a component. Lists of strings do not come near to offer the same functionality.
Enums represent a discrete set of constant values. They are extremely helpful to communicate to developers what to expect from a component. Lists of strings do not come near to offer the same functionality.
Agreed. I’ve currently switched from Swift to Python and keep reaching for Swifts expressive enums and awesome pattern matching.
I disagree. A list of strings requires manual validation whether a value is actually in the list; enums have that built in. Additionally, enums (at least in Python) can have behaviour associated with each value, which can be useful in many contexts.
There are also good reasons why Python does not have switch/case statements, which are generally an anti-pattern; they work fine for a quick fix, but what they do is better solved with proper polymorphism.
There are also good reasons why Python does not have switch/case statements, which are generally an anti-pattern; they work fine for a quick fix, but what they do is better solved with proper polymorphism.
There's a benefit in the cardinality during type checking when you use an external type checker. Strings have infinite possibilities an enum type is strictly defined.
Also I'm sure with reflection you can actually implement a simple enum pattern matching function on an enum that is exhaustive and throws a runtime error immediately if every possible enum case is not covered.
Also I'm sure with reflection you can actually implement a simple enum pattern matching function on an enum that is exhaustive and throws a runtime error immediately if every possible enum case is not covered.
I'm not following this thread at all. What "exhaustive search" are you guys referring to? In a switch/case block?
Python doesn't have that, unless I missed it while living under a rock.
I.e. if you're working with python Enums, you type-check for that Enum, and then the IDE does it all for you. If you want to "check" for a specific enum, then you cast to it and the runtime Enum implementation does that checking for you by throwing an error if you try create that Enum with an invalid string value. And if you've got a raw string, before it goes too-far into your code base, you cast it into the specific Enum type, and when serializing you serialize it to the string value. Easy-peasy.
I.e. if you're working with python Enums, you type-check for that Enum, and then the IDE does it all for you. If you want to "check" for a specific enum, then you cast to it and the runtime Enum implementation does that checking for you by throwing an error if you try create that Enum with an invalid string value. And if you've got a raw string, before it goes too-far into your code base, you cast it into the specific Enum type, and when serializing you serialize it to the string value. Easy-peasy.
AFAIK, they're talking about languages like Rust, or C/C++ with `-Wswitch` (also implied by `-Werror`, which will warn you if you haven't handled all enum cases in a match/switch statement.
as someone else in this thread pointed out [0], it's possible with MyPy when using if/elif/else, or if you did it with a dictionary lookup, you could write a very simple test that checks that the set of enum values is equal to the dictionary key members. I've done this before, it works well [1].
[0] https://news.ycombinator.com/item?id=23441023 [1] https://github.com/aws-cloudformation/cloudformation-cli/blo...
as someone else in this thread pointed out [0], it's possible with MyPy when using if/elif/else, or if you did it with a dictionary lookup, you could write a very simple test that checks that the set of enum values is equal to the dictionary key members. I've done this before, it works well [1].
[0] https://news.ycombinator.com/item?id=23441023 [1] https://github.com/aws-cloudformation/cloudformation-cli/blo...
In some typed/compiled languages, a substantial benefit of using enums is that it provides some way to automatically determine if you've handled every eventuality. That's super not a part of normal Python as you've mentioned.
The second part of crimsonalucard1's post was referring to somehow hacking in that type of functionality into Python.
I think you could somehow do that with nested context managers - something like:
The second part of crimsonalucard1's post was referring to somehow hacking in that type of functionality into Python.
I think you could somehow do that with nested context managers - something like:
with EnumChecker(e) as checker:
with checker(enum_type.A):
# do thing
with checker(enum_type.B):
# do thing
...
imagine an enum called bool with two values: True and False.
So imagine where someone writes this code:
The power of this basically allows you to encode safety of if else logic into a type and pass that safety throughout your program. It's pretty incredible, so much so that whenever this feature is part of a language I have stopped using if else statements almost completely.
Keep in mind the power of pattern matching extends beyond just exhaustive enums. An int is a form of enum as well:
switch(value: bool):
case True:
// do something
case False:
// do something else
end
So imagine where someone writes this code:
switch(value):
case True:
// do something
end
The power of this basically allows you to encode safety of if else logic into a type and pass that safety throughout your program. It's pretty incredible, so much so that whenever this feature is part of a language I have stopped using if else statements almost completely.
Keep in mind the power of pattern matching extends beyond just exhaustive enums. An int is a form of enum as well:
switch(value: int):
case 0:
// do something
case >0:
// do something else
end
It always feels a little clumsy when these class generators require the name of the identifier to which they are assigned.
Clown = circus.Actor(“Clown”)
Clown = circus.Actor()
circus.Actor(“Clown”)
bozo = Clown()
> Why is it needed, instead of just creating an anonymous class?
I think the question should be: why do you want an anonymous class? If it's sufficiently ad-hoc that it doesn't warrant a name, you would typically use a dict.
> Or allowing the class factory to manipulate the current namespace?
I think that would just be considered too surprising and non-standard behaviour (other Python functions don't get to modify the scope they're called from).
I think the question should be: why do you want an anonymous class? If it's sufficiently ad-hoc that it doesn't warrant a name, you would typically use a dict.
> Or allowing the class factory to manipulate the current namespace?
I think that would just be considered too surprising and non-standard behaviour (other Python functions don't get to modify the scope they're called from).
Named tuples, the example I called out, are a good improvement over dicts for adhoc structs that don’t require full class definitions.
But you’re right. It’s hardly cumbersome to create classes that simply exist to contain data in a sightly more structured way than a dict.
But you’re right. It’s hardly cumbersome to create classes that simply exist to contain data in a sightly more structured way than a dict.
> Named tuples, the example I called out, are a good improvement over dicts for adhoc structs that don’t require full class definitions.
Doesn't a lot of that improvement come from the fact that they have a name, though? I guess you could argue for an "UnnamedTuple" which only enforces tuple size and member names. But if you're able to specify the member names, it feels like the thing being represented is concrete enough that you should be able to come up with a name for it, and that having a name would be beneficial.
Doesn't a lot of that improvement come from the fact that they have a name, though? I guess you could argue for an "UnnamedTuple" which only enforces tuple size and member names. But if you're able to specify the member names, it feels like the thing being represented is concrete enough that you should be able to come up with a name for it, and that having a name would be beneficial.
Recently I started to use enumeration classes instead of enums ( where it fits)
https://docs.microsoft.com/en-us/dotnet/architecture/microse...
If you develop in c#, I would definitely recommend trying it out
A better example can be found here actually: https://lostechies.com/jimmybogard/2008/08/12/enumeration-cl...
https://docs.microsoft.com/en-us/dotnet/architecture/microse...
If you develop in c#, I would definitely recommend trying it out
A better example can be found here actually: https://lostechies.com/jimmybogard/2008/08/12/enumeration-cl...
What's the basic idea here? That instead of ints or strings, your enum values themselves are instances of custom classes with whatever behavior you want?
If I would describe the use-case:
If you use switches with enums for functionality, this is a better way ( personal opinion). Since Enums quickly need more than 1 switch and this could wrap that behaviour
Your description is also pretty good
If you use switches with enums for functionality, this is a better way ( personal opinion). Since Enums quickly need more than 1 switch and this could wrap that behaviour
Your description is also pretty good
Have constantly struggled over how to deal with this problem. The biggest need for enums has been function arguments which can be one of a few values.
In the end, the Literal class from the typing library seems like the simplest, least wordy solution. Pycharm and mypy capture any mistakes you make which seem more than sufficient for my use cases.
In the end, the Literal class from the typing library seems like the simplest, least wordy solution. Pycharm and mypy capture any mistakes you make which seem more than sufficient for my use cases.
Why not use Enum anyway, which works fine with MyPy? I don't mind the verbosity, it's a one-time commitment to what, 10 extra lines of code?
Issue is I'll now need two imports everytime I call this method, which becomes mildly annoying. I was doing this, but a lot of times the Literal type seems more than sufficient. It also seems to play real nice with pydantic so it's an added win!
I'm not sure what you mean by "two imports".
If you define your Enum subclass in the same file, you use one import for enum.Enu. If you eschew enums and use Literal, you use one import for typing.Literal.
If you define your Enum subclass in the same file, you use one import for enum.Enu. If you eschew enums and use Literal, you use one import for typing.Literal.
Prefer using named tuples over enums in Python.
Named tuples and enums do entirely different things (loosely-speaking: the former are product types, the latter are sum types).
You also probably want dataclasses [1] instead of named tuples.
[1] https://docs.python.org/3/library/dataclasses.html
You also probably want dataclasses [1] instead of named tuples.
[1] https://docs.python.org/3/library/dataclasses.html
Or data bearing enums, which bring some of the best of all of those worlds, with (value declaration, not usage) exhaustiveness enforced at compile time:
https://github.com/klaviyo/matrix_enum
My colleague and I started making use of that pattern last year (before open sourcing the package) and found it very useful.
https://github.com/klaviyo/matrix_enum
My colleague and I started making use of that pattern last year (before open sourcing the package) and found it very useful.
They do different things but in the wild are used in the same way. Data classes are not immutable so it would not be ideal for what I used named tuples for.
Data classes can be made immutable using ‘frozen=True’. It sucks that it’s not default though.
Can you give an example of how you use enums and named tuples in the same way? That just doesn’t seem to make a lot of sense to me, but maybe I’m missing something.
Can you give an example of how you use enums and named tuples in the same way? That just doesn’t seem to make a lot of sense to me, but maybe I’m missing something.