Proposing a struct syntax for Python(snarky.ca)
snarky.ca
Proposing a struct syntax for Python
https://snarky.ca/proposing-a-struct-syntax/
50 comments
This is the reasoning from the article for why its suggestion is better then dataclasses:
> - Typing is optional (there are still folks who don't want to lean into that and it simply isn't always necessary)
> - Class construction should be faster (which intuitively you would think shouldn't matter, but I have talked to folks where this is an actual concern, especially when startup time is critical)
> - Syntax typically leads to better tooling support since there's no ambiguity
> - Easier to teach than (data)classes, so can act as a stepping stone towards classes
> - Better semantics than dataclasses have by default (at least in my opinion )
> - Typing is optional (there are still folks who don't want to lean into that and it simply isn't always necessary)
> - Class construction should be faster (which intuitively you would think shouldn't matter, but I have talked to folks where this is an actual concern, especially when startup time is critical)
> - Syntax typically leads to better tooling support since there's no ambiguity
> - Easier to teach than (data)classes, so can act as a stepping stone towards classes
> - Better semantics than dataclasses have by default (at least in my opinion )
> - Typing is optional (there are still folks who don't want to lean into that and it simply isn't always necessary)
Someone who really wants to avoid type annotations at all costs can just use ": object" everywhere
Someone who really wants to avoid type annotations at all costs can just use ": object" everywhere
I believe it should be ": Any".
Hey, you're leaning into it too much!
Who are people who are against typing in this day and age?
Who are people who are against typing in this day and age?
> Who are people who are against typing in this day and age?
People who don't find the benefits of typing are worth the overhead?
Python is, after all, a dynamically-typed language, so it is, surely, not too surprising that those who use it might include those who want a ... dynamic language ?
If it's too much to take the joys of Java are freely available.
People who don't find the benefits of typing are worth the overhead?
Python is, after all, a dynamically-typed language, so it is, surely, not too surprising that those who use it might include those who want a ... dynamic language ?
If it's too much to take the joys of Java are freely available.
> are worth the overhead?
What overhead, specifically?
I've found it reduces overhead. Rather than using English to describe the types, in the comments/docstrings, I can just type the terse type hint in the function definition, and leave the rest to the documentation renderer. If you don't document anyways, sure.
What overhead, specifically?
I've found it reduces overhead. Rather than using English to describe the types, in the comments/docstrings, I can just type the terse type hint in the function definition, and leave the rest to the documentation renderer. If you don't document anyways, sure.
I use python daily and dont use typing. The reason being that I choose python for it readability. And I find typed python to be less readable than non-typed python. If I wanted a typed language, I wouldnt want python.
> there are still folks who don't want to lean into that
Honestly why would you cater to people who want to write worse code? Should Python have a mode with implicit type coercion too because some people can't be bothered to type `int()`?
> Class construction should be faster.
It's Python. You accepted your code would be extremely slow going in. Fiddling with the syntax isn't going to make it fast.
Honestly why would you cater to people who want to write worse code? Should Python have a mode with implicit type coercion too because some people can't be bothered to type `int()`?
> Class construction should be faster.
It's Python. You accepted your code would be extremely slow going in. Fiddling with the syntax isn't going to make it fast.
Yeah, honestly I think if startup time is a real concern, python is simply the wrong choice. Slow startup is largely inherent in its design.
Not that it's not worth spending some effort to optimize, of course. But the lower bound is quite high even if you're very careful.
Not that it's not worth spending some effort to optimize, of course. But the lower bound is quite high even if you're very careful.
Doesn't support algebraic data types, though.
FWIW I think this article mentioning them at all is sort of weird because the article doesn't propose them either, lmao
FWIW I think this article mentioning them at all is sort of weird because the article doesn't propose them either, lmao
Which Python 3.10 and static type checkers much of the benefits of algebraic data types is already available [1].
[1]: https://stackoverflow.com/questions/16258553/how-can-i-defin...
[1]: https://stackoverflow.com/questions/16258553/how-can-i-defin...
Right, the article talks about "type Point = Point1D | Point2D | Point3D" syntax at the end... which would work just as well with dataclasses as with structs!
It already does too. Just remove the `type` bit and you have an alias for a union, which has been supported since typing was added (though the `|` syntax was only added in 3.10). And AFAIK the type checkers do handle completeness on unions.
The typed equivalent of a sum type in python is a union (https://docs.python.org/3/library/typing.html#typing.Union). Works with pretty much every type you can think of, but has the usual union limitation that you can't have two variants of the same type (e.g. int | int | float).
Then again, you can just create newtype wrappers, and that's essentially the same thing as different constructors for a sum type.
Then again, you can just create newtype wrappers, and that's essentially the same thing as different constructors for a sum type.
You'd need to create newtype wrappers anyway in order to tell them apart, I'd assume. Otherwise it wouldn't really be an algebraic data type as much as a... union type.
Hence “equivalent”. Of your variants all have trivial and non-overlapping payloads you can just use them as-is, or erlang-style tagged tuples, in most cases it’ll do fine.
Do you mean Python's lack of a labeled union type?
Pydantic does though.
class A(BaseModel):
status: Literal["success"]
class B(BaseModel):
status: Literal["error"]
class C(BaseModel, ABC):
__root__: Union[A, B] = Field(discriminator="status")
aorb = C.parse_obj(blah…).__root__
if aorb.status == "error":
# aorb will type check as B.You are a friend and a scholar.
I was looking for a solution in pydantic that is wildly relevant. Thanks, I didn’t realize that was implemented/existed.
My only objection is using ABC in your MRO. Haha, just a little bit overloaded with abstract base classes and using A, B, and C.
I got it eventually! Cool.
I was looking for a solution in pydantic that is wildly relevant. Thanks, I didn’t realize that was implemented/existed.
My only objection is using ABC in your MRO. Haha, just a little bit overloaded with abstract base classes and using A, B, and C.
I got it eventually! Cool.
Ya know what, on reading that that does seem confusing. My brain had long since internalized ABC as a single token I didn't even notice.
I believe this is a good example of type narrowing :)
It doesn't seem that far off though, it does have a union type and with pattern matching you can achieve pretty much the same thing (sure you may need a few extra classes if the types aren't distinct already but that seems like a small difference).
I don't know if type checkers can figure out if a pattern match statement is exhaustive yet though, that would be interesting.
I don't know if type checkers can figure out if a pattern match statement is exhaustive yet though, that would be interesting.
Yeah. Been spoiled by Rust.
I was surprised at how dataclasses are not the first thing people are taught when it comes to "classes" in python. My two personal "you should probably learn how these work" things for beginners are pprint and dataclass. Goes a long long way to helping look at things.
It's because the traditional teaching focus is OOP and not data-driven design.
The part that blows my mind, on this, is that I see it in the "data science" community. An area that seems custom made for leaning on smaller or non-existent inheritance trees.
Dataclasses wind up cumbersome over time. They're nice when the defaults are what you want. After a handful of default overrides, I'd rather have not used dataclasses.
In the office job, I'd push on why the defaults didn't work out. In particular, I would be worried that I was specifically aiming to make future life a bit harder by not fitting with the defaults of dataclass.
To that end, I'm curious what defaults you didn't use?
To that end, I'm curious what defaults you didn't use?
I'm on board with wanting an alternative to dataclasses that's faster, doesn't require types, and is less noisy (having to put @dataclass on its own line before every class is annoying when defining many small dataclasses).
But not having methods would produce APIs that feel really non-idiomatic. Essentially all Python types have methods, even immutable 'pure data' types. str has methods (though some things like `len` and regex search, which are methods in other languages, are functions in Python). int has methods (though they're rarely used). Immutable standard library classes – say, datetime or Path – have lots and lots of methods.
And inheritance… well, I could live without it but I've made good use of inheritance with dataclasses.
And as others have mentioned, the syntax is awkward.
I think my ideal solution would basically be dataclasses, but implemented in C and with more syntax sugar:
It might be interesting if these structs acted as true value types, so that e.g. just assigning an instance from one variable to another would make a copy of all the fields. That way you could mutate the object without worrying about sharing. But arguably, having both value types and reference types would overcomplicate the mental model.
But not having methods would produce APIs that feel really non-idiomatic. Essentially all Python types have methods, even immutable 'pure data' types. str has methods (though some things like `len` and regex search, which are methods in other languages, are functions in Python). int has methods (though they're rarely used). Immutable standard library classes – say, datetime or Path – have lots and lots of methods.
And inheritance… well, I could live without it but I've made good use of inheritance with dataclasses.
And as others have mentioned, the syntax is awkward.
I think my ideal solution would basically be dataclasses, but implemented in C and with more syntax sugar:
# Instead of "@dataclass\nclass Foo:"
struct Foo:
# Allow untyped fields:
x
# Make this do what you'd expect, instead of
# requiring "field(default_factory=list)":
y = []
# Of course, types are still allowed:
z: int = 0
It might be interesting if these structs acted as true value types, so that e.g. just assigning an instance from one variable to another would make a copy of all the fields. That way you could mutate the object without worrying about sharing. But arguably, having both value types and reference types would overcomplicate the mental model.
How does this avoid name collisions with the built-in standard library module "struct"? Don't you think that's a pretty important consideration?
With the new parser (https://docs.python.org/3/whatsnew/3.9.html#new-parser), Python can have context-dependent keywords (like `match`).
Type creep can be a serious concern for dynamic language ecosystems. The Javascript/Typescript schism is a tragic case in point. It appears that young developers are indoctrinated into the belief that strong types are an invincible hammer and can't see the drawbacks that lack of flexibility can introduce. Clojure began with defstruct, and Rich Hickey regrets its addition to the language now. While python certainly is a kitchen sink language, and the addition of structs on top is probably sustainable, but I am of the belief that such suggestions need more counter-arguments lest we lose another mainstream dynamic language to naive good intentions.
What's unsaid here, is that such construct could be optimized way more for speed than data classes because:
- it's immutable
- doesn't have methods so no overloaded operator or dynamic attribute/index accesss
- shape is known and is set in stone
That's why Mojo has them, and why they are the bread and butter in Rust.
It's one of the step in the direction of a faster Python.
- it's immutable
- doesn't have methods so no overloaded operator or dynamic attribute/index accesss
- shape is known and is set in stone
That's why Mojo has them, and why they are the bread and butter in Rust.
It's one of the step in the direction of a faster Python.
> I want syntax that will keep people from using collections.namedtuple() just for its attribute API and ignoring/regretting its index API.
The author is proposing a worse dataclass, because they don't want to ignore positional indexing on namedtuple?
The author is proposing a worse dataclass, because they don't want to ignore positional indexing on namedtuple?
Here's a long-running question I have; please tell me if I'm confused / missing something obvious.
Say I want to model in Python some collection of key-value pairs. It's a singleton -- I don't need a class or namedtuple that can be instantiated. So for example, I just want something like:
Say I want to model in Python some collection of key-value pairs. It's a singleton -- I don't need a class or namedtuple that can be instantiated. So for example, I just want something like:
SingletonDataContainer Config:
root_directory: str = "/tmp"
should_frobnicate: bool = False
Just put them in a module as variables. Import the module named and you have them as “dotted attributes”. Python doesn’t need all this OOP stuff like structs when we have dataclasses and attrs, modules and dicts, that can represent draconian C structs.
As well as modules
being natural singletons you can also make your class an ABC to stop instantiation.
A module variable set to a SimpleNamespace in the typing package is also an option.
A module variable set to a SimpleNamespace in the typing package is also an option.
Thanks. I wonder what it would be like if Python had syntax for modules instead of them being tied to separate files. I would prefer being able to create this container in the same file as other code.
Say no more! Python? Rules? Unheard of.
from importlib.machinery import ModuleSpec
import importlib.util
import sys
spec = ModuleSpec("mymodule", loader=None)
module = importlib.util.module_from_spec(spec)
module.__dict__.update({"a": 42})
sys.modules["mymodule"] = module
import mymodule
print(mymodule.a) # -> 42
import sys
from types import SimpleNamespace
sys.modules["mymodule"] = SimpleNamespace(b=37)
import mymodule
print(mymodule.b) # -> 37
# ^^^ This is literally just calling __getattr__ on the module object. Do with that what you will.
Thanks! This was a great answer.
> Python? Rules? Unheard of.
OK, since it's Christmas, I'd like to be able to write
> Python? Rules? Unheard of.
OK, since it's Christmas, I'd like to be able to write
Module MyModule
a = 42
b = 37I think I can do that.
import sys
class DataModule:
def __init_subclass__(cls):
sys.modules[cls.__name__.lower()] = cls
class MyModle(DataModule):
a = 42
b = 37
import mymodule
mymodule.a == 42
def datamodule(cls):
sys.modules[cls.__name.__.lower()] = cls
@datamodule
class MyModule:
a = 42
import mymodule
mymodule.a = 42So basically you want something like how typescript does, where you can do this:
const config = {
rootDirectory: "/tmp",
shouldFrobnicate: false,
}
from dataclasses import dataclass
def _create_config():
@dataclass(kw_only=True)
class Config:
root_directory: str = "/tmp"
should_frobnicate: bool = False
return Config()
config = _create_config()
@iife
def config():
@dataclass(kw_only=True)
class Config:
root_directory: str = "/tmp"
should_frobnicate: bool = False
return Config()
Could use a class, but access the value without ever instantiating it? Not sure if this fits the bill for what you want, or if the fact that it could theoretically be instantiated renders the approach non-suitable.
class Config:
root_directory: str = "/tmp"
should_frobnicate: bool = False
if __name__=="__main__":
print(f"{Config.root_directory=}\n{Config.should_frobnicate=}")Will you ever worry about thread safety for these values? If not, then as others have mentioned, a `config` module seems like what you want.
i want something like this for lots of reasons, but the headliner is structual typing.
It should be possible to get simple attribute access while not creating a new nominal type and not going outside the Python stdlib.
I'm constantly using NamedTuples when what i really want is an anonymous struct.
It should be possible to get simple attribute access while not creating a new nominal type and not going outside the Python stdlib.
I'm constantly using NamedTuples when what i really want is an anonymous struct.
Oh no, not more unnecessary syntax
To add this to python I feel like you would have it has an external package and or write a pep about it and judge by how prevalent it becomes due to the fact that it says it will replace other features…
Don’t steal Mojo’s mojo
I dislike this general mode of defining an ADT by first having to create multiple separate nominal types and then combining them using another free-floating statement (e.g. 3 dataclasses and then a Union in Python). An ADT is conceptually one thought, so its definition should syntactically be able to be one statement too.
https://docs.python.org/3/library/dataclasses.html#module-da...
..and the syntax is way better, handling more members, with room to document, handling methods, and not conflicting with Mojo or stdlib