Parse, Don't Validate (2019)(lexi-lambda.github.io)
lexi-lambda.github.io
Parse, Don't Validate (2019)
https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/
282 comments
I've found this to be simply a matter of experience, not tooling. As the years go by I find the majority of my code just working right - never touched anything like pydantic or validation boilerplate for my own code, besides having to write unit tests as an afterthought at work to keep the coverage metric up.
No this was like over a week, and 100% due to the tooling. Pydantic, pycharm, black, mypy, and flake8. Pretty much went from "type hints here and there" to "what happens if I try writing python as if it were (95%) statically typed." I'd been testing well before this point but it's not the same as writing test.
The development process is totally different when you write structured types first and then write your logic. 10/10 would recommend.
Usual caveat: this is what makes sense to me and my brain. Your experience may be different based on neurotype.
The development process is totally different when you write structured types first and then write your logic. 10/10 would recommend.
Usual caveat: this is what makes sense to me and my brain. Your experience may be different based on neurotype.
The development process is totally different when you write structured types first and then write your logic. 10/10 would recommend.
Unless you were writing very small throwaway scripts, in what world where you writing your logic first and thinking about your data structures later?
Unless you were writing very small throwaway scripts, in what world where you writing your logic first and thinking about your data structures later?
Even having type hints on basic functions like def foo(bar: str) in a throwaway-script helps because it gives me reliable autocompletion on bar. I might be getting old or toggle between so many different languages nowadays that even basic stuff i don't want to remember, like whether it is bar.lower(), bar.lowerCase(), bar.toLower(), bar.toLocaleLowerCase().
Defining a data structure up front doesn't require a lot of boilerplate as Java incorrectly have taught all of us. Writing a statically typed typing.NamedTuple or @dataclass is literally a one-liner.
Defining a data structure up front doesn't require a lot of boilerplate as Java incorrectly have taught all of us. Writing a statically typed typing.NamedTuple or @dataclass is literally a one-liner.
The world of data science, ML and computer vision research. It's very academic-heavy, which has two effects. There's an insulation between commercial software dev and it, which results in a lot of NIH, a lot of reinforcement of bad habits, and a lag in propagation of best practices. Second, and related to this, there is a tendency to just piecemeal hack towards the solution, rather than architect the system from the ground up.
It's not zero consideration of data structures, it's mostly a focus on the main data type (arrays and data frames) and not really thinking about typed records, data models and such. The majority of types are float, str, dict, np.ndarray, pd.DataFrame. No dataclasses, minimal classes, and when classes are used, it's Java101 style "all the bad parts of OOP" programming. Sadly, I've spent years in this space before learning better.
It's not zero consideration of data structures, it's mostly a focus on the main data type (arrays and data frames) and not really thinking about typed records, data models and such. The majority of types are float, str, dict, np.ndarray, pd.DataFrame. No dataclasses, minimal classes, and when classes are used, it's Java101 style "all the bad parts of OOP" programming. Sadly, I've spent years in this space before learning better.
Tests aren't to make sure your code works when you write it, it is to make sure it doesn't break when you make changes down the line.
How do you know your code works when you write it if you don't test it?
Sorry, should have said “aren’t JUST to make sure your code works when you write it”
I was specifically responding to the commenter I replied to, who said they didn’t need tests because their code just worked the first time after he wrote it.
I was specifically responding to the commenter I replied to, who said they didn’t need tests because their code just worked the first time after he wrote it.
You don't. But you only need to test it once (manually), then commit it.
You write automated tests so that you can keep running the tests later such that the behaviour is maintained through refactor and non-breaking changes.
You write automated tests so that you can keep running the tests later such that the behaviour is maintained through refactor and non-breaking changes.
you run it. look at the results or output. like the stdout, or a file it changed, or in a REPL or debugger. depends on situation
Yeah I‘m not sure that‘s how software engineering should work.
Tests should prove a desired behaviour. Sometimes it‘s not possible to fully run code until late in some staging, just because there are a lot of dependencies and conplexity. That‘s what tests are for (on various lebels of abstraction).
Tests should prove a desired behaviour. Sometimes it‘s not possible to fully run code until late in some staging, just because there are a lot of dependencies and conplexity. That‘s what tests are for (on various lebels of abstraction).
I think it depends on the task. Some code we write is so simple and only used a few times that you don’t need tests.
Sounds laborious to manually check edge cases each time you change that code or its dependencies. I'd rather just write a test.
Man, for a dev with as much experience as you’re claiming to have, this comment ain’t a great look.
I’d argue that the more experience you get the more you write code for other people which involves adding lots of tooling, tests, etc. Even if the code works the first time, a more senior dev will make sure others have a “pit of success” they can fall into. This involves a lot more than just some “unit tests as an afterthought to keep the coverage up.”
I’d argue that the more experience you get the more you write code for other people which involves adding lots of tooling, tests, etc. Even if the code works the first time, a more senior dev will make sure others have a “pit of success” they can fall into. This involves a lot more than just some “unit tests as an afterthought to keep the coverage up.”
Adding lots, no. I agree with the grandparent.
Keeping the code simple, finding the right abstractions, untangling coupling, gets the most bang for the buck. See the “beyond pep8” talk for a enlightened perspective.
That said, lightweight testing and tools like pyflakes to prevent egregious errors helps an experienced dev write very productively. Typing helps the most with large, venerable projects with numerous devs of differing experience levels.
Keeping the code simple, finding the right abstractions, untangling coupling, gets the most bang for the buck. See the “beyond pep8” talk for a enlightened perspective.
That said, lightweight testing and tools like pyflakes to prevent egregious errors helps an experienced dev write very productively. Typing helps the most with large, venerable projects with numerous devs of differing experience levels.
> Typing helps the most with large, venerable projects
I disagree. I've started using types from the ground up and it helps almost equally at every stage of the game. Also I aggressively rely on autocomplete for methods. It's faster this way than usual "dynamic" or "pythonic" python.
Part of it might be exactly because writing my datatypes first helps me think about the right abstractions.
The big win with python is maybe 2-10% of functions, I just want to punt and use a dict. But I have shifted >80% of what used to be dicts to Models/dataclasses and it's so much faster to write and easier to debug.
I disagree. I've started using types from the ground up and it helps almost equally at every stage of the game. Also I aggressively rely on autocomplete for methods. It's faster this way than usual "dynamic" or "pythonic" python.
Part of it might be exactly because writing my datatypes first helps me think about the right abstractions.
The big win with python is maybe 2-10% of functions, I just want to punt and use a dict. But I have shifted >80% of what used to be dicts to Models/dataclasses and it's so much faster to write and easier to debug.
I don’t need to aggressively rely on tools, they are merely in the background. Perhaps what the earlier post about experience was thinking.
Also, what makes you think I’m not aware of datatypes? Currently working eight hours a day on Django models.
Also, what makes you think I’m not aware of datatypes? Currently working eight hours a day on Django models.
Typing is just another guardrail, it's not a substitute for finding the right abstractions and keeping things simple.
I agree but guardrails are pretty awesome. And if people were saying "don't use guardrails, just drive properly", I'd ask why they think guardrails and driving properly are mutually exclusive.
Exactly. To be clear I’m very pro-type systems.
Agreed. It's like saying "oh well I just fly the airplane really carefully". A lot of codebases eclipse the point where one person can understand the whole system. Testing, static analysis and tooling are what allows us to keep the plane flying.
Agreed with the end of your post. However, the top post approaches religious dogma. I argue against that even if one has some good points.
> However, the top post approaches religious dogma.
I don't deny it. Join the cult of Static Python. We have cookies! And lower stress levels!
I usually wrap that spiel with my caveat "this depends greatly on your neurotype, style, environment, and other things." I have ADHD and my brain struggles with keeping bits of state in memory, so having to remember the type of every variable without my IDE tracking it for me is a huge performance drain.
However, I would contend even if your neurotype supported that mental workflow... it isn't actually better. Humans on average can handle 7 +/- 2 "pieces" of information in focus. Why spend any of your precious half-dozen pieces of salient consciousness on something a machine is really good at doing?
I don't deny it. Join the cult of Static Python. We have cookies! And lower stress levels!
I usually wrap that spiel with my caveat "this depends greatly on your neurotype, style, environment, and other things." I have ADHD and my brain struggles with keeping bits of state in memory, so having to remember the type of every variable without my IDE tracking it for me is a huge performance drain.
However, I would contend even if your neurotype supported that mental workflow... it isn't actually better. Humans on average can handle 7 +/- 2 "pieces" of information in focus. Why spend any of your precious half-dozen pieces of salient consciousness on something a machine is really good at doing?
Because it adds a lot of work that many projects don't need. Read wpietri's post on how candidates who write static get half as much accomplished in an interview.
Also since the tools are immature and bolted on afterward in Python, I think it's even a bit worse than it would be in something decent like C#.
Also since the tools are immature and bolted on afterward in Python, I think it's even a bit worse than it would be in something decent like C#.
[deleted]
It's an immediate tell when someone makes statements like the one you're replying to.
It immediately tells me that they've never worked on large software projects, and if they have they haven't worked on ones that lasted more than a few months.
I apologize to folks reading this for my rather aggressive tone but I've been writing software for a long time in numerous languages, and people with the unit tests as an afterthought attitude are typically rather arrogant in fool hardy.
The most recent incarnation I've encountered is the hotshot data scientist who did okay in a few Kaggle competitions using Jupyter notebooks, and thinks they can just write software the way they did for the competitions with no test of any kind.
I had one of these on my team recently and naturally I had to do 95% of the work to turn anything he produced into a remotely decent product. I couldn't even get the guy to use nbdev, which would have allowed him to use Jupyter to write tested, documented, maintainable code.
It immediately tells me that they've never worked on large software projects, and if they have they haven't worked on ones that lasted more than a few months.
I apologize to folks reading this for my rather aggressive tone but I've been writing software for a long time in numerous languages, and people with the unit tests as an afterthought attitude are typically rather arrogant in fool hardy.
The most recent incarnation I've encountered is the hotshot data scientist who did okay in a few Kaggle competitions using Jupyter notebooks, and thinks they can just write software the way they did for the competitions with no test of any kind.
I had one of these on my team recently and naturally I had to do 95% of the work to turn anything he produced into a remotely decent product. I couldn't even get the guy to use nbdev, which would have allowed him to use Jupyter to write tested, documented, maintainable code.
I've worked on large scale projects for a long time. A large portion of the kind of code I've written is impractical or impossible to actually "unit test" e.g. Unity3D components or frontend JS that interacts with a million things. When something weird is going on I'll have to dig in with console logs and breakpoints.
On certain backend code where I am able to do unit tests, they do catch the occasional edge case logic error but not at a rate that makes me concerned about only checking them in some time after the original code, which I'll have already tested myself in real use as I went along.
On certain backend code where I am able to do unit tests, they do catch the occasional edge case logic error but not at a rate that makes me concerned about only checking them in some time after the original code, which I'll have already tested myself in real use as I went along.
> A large portion of the kind of code I've written is impractical or impossible to actually "unit test" e.g. Unity3D components or frontend JS that interacts with a million things.
Opinion: This is actually a symptom of what is (imho) a pervasive problem lodged deep in the collective consciousness of software dev: OOP with fine-grained objects. I blame (early) Java in large part for exacerbating this mentality. Encapsulation of state with mutator methods in particular. It sprays state all over the application, encourages mutation in place over immutability, coupling, validating-not-parsing, and makes it nigh-well impossible to write good tests.
It's really hard to write objects that enforce all invariants under every mutation. And when you have state strewn everywhere, it's impossible to test every nook and cranny. The combinatorial space explodes.
Objects are helpful for encapsulating state when they are course-grained, mutations are atomic, coupling occurs in one place, state changes are auditable, and the entire state can be replayed/set at once, to enable mock tests and subsystem integration tests. AKA, things like databases, reactors, and persistent data structures.
Opinion: This is actually a symptom of what is (imho) a pervasive problem lodged deep in the collective consciousness of software dev: OOP with fine-grained objects. I blame (early) Java in large part for exacerbating this mentality. Encapsulation of state with mutator methods in particular. It sprays state all over the application, encourages mutation in place over immutability, coupling, validating-not-parsing, and makes it nigh-well impossible to write good tests.
It's really hard to write objects that enforce all invariants under every mutation. And when you have state strewn everywhere, it's impossible to test every nook and cranny. The combinatorial space explodes.
Objects are helpful for encapsulating state when they are course-grained, mutations are atomic, coupling occurs in one place, state changes are auditable, and the entire state can be replayed/set at once, to enable mock tests and subsystem integration tests. AKA, things like databases, reactors, and persistent data structures.
You got paid to do the work presumably. You might also be able to push back on it. Coding standards should be a thing just about anywhere competent.
In short, there are choices besides, “I alone have to do all the hard work.”
In short, there are choices besides, “I alone have to do all the hard work.”
I quit the company and the team as a result of the bosses refusing to make their pet data scientists write remotely professional code.
I was more experienced with predictive algorithms and deep learning than any of the data scientists at the company but because they were brought in from an acquisition of a company that had an undeserved reputation due to a loose affiliation with MIT, they were treated like magicians while the rest of us were treated like blacksmiths.
I had the choice and I made the choice to leave. And of course I raised hell with the bosses about them not writing remotely production quality code that required extensive refactoring.
And yes I was paid to do the work but the work occupied time that I could have spent working on the other projects I had that were more commercially successful but less sexy to Silicon Valley VCs who look at valuations based on other companies' newest hottest product.
I was more experienced with predictive algorithms and deep learning than any of the data scientists at the company but because they were brought in from an acquisition of a company that had an undeserved reputation due to a loose affiliation with MIT, they were treated like magicians while the rest of us were treated like blacksmiths.
I had the choice and I made the choice to leave. And of course I raised hell with the bosses about them not writing remotely production quality code that required extensive refactoring.
And yes I was paid to do the work but the work occupied time that I could have spent working on the other projects I had that were more commercially successful but less sexy to Silicon Valley VCs who look at valuations based on other companies' newest hottest product.
I've worked with plenty of coders who talk about how awesome their code is even though they just write unit test as an afterthought. They also talk about how they don't need validation and everything is just awesome.
I hated working with those coders because they weren't really very good and their code was always the worst to maintain. They are the equivalent of a carpenter who brags about how quickly they can bang nails but can't build a stable structure to save their life.
I hated working with those coders because they weren't really very good and their code was always the worst to maintain. They are the equivalent of a carpenter who brags about how quickly they can bang nails but can't build a stable structure to save their life.
I agree. I'm often baffled by some developers who seem to think dynamic typing is a minefield that inevitably goes wrong all the time. I note these are almost always Javascript programmers, though. In practice, experience developers in dynamic languages like Python, Lisp etc. rarely make such errors. The number of bugs we deal with that would have been caught early by static typing are vanishingly small.
The best argument I've heard for doing type annotation is for documentation purposes to help future devs. But I don't completely buy this either. I touch new codebases all the time and I rarely spend much time thinking about what types will be passed. I can only assume it comes with experience.
Type annotation actually ends up taking a hell of a long time to do and is of questionable benefit if some of the codebase is not annotated. People spend sometimes hours just trying to get the type checker to say OK for code that actually works just fine!
The best argument I've heard for doing type annotation is for documentation purposes to help future devs. But I don't completely buy this either. I touch new codebases all the time and I rarely spend much time thinking about what types will be passed. I can only assume it comes with experience.
Type annotation actually ends up taking a hell of a long time to do and is of questionable benefit if some of the codebase is not annotated. People spend sometimes hours just trying to get the type checker to say OK for code that actually works just fine!
It's okay if you're working on a blog site, less so if you're working on an air-planes autopilot.
JPL sent Lisp to space https://flownet.com/gat/jpl-lisp.html
Sure and I know people who write Python that goes into space too, but it doesn't mean it'd the norm or even a good idea
Sounded familiar. Worked correctly the first time it was run, sans types.
https://www.python.org/about/success/esr/
https://www.python.org/about/success/esr/
Curious, but how does pydantic compare to marshmallow?
I'm currently using marshmallow in a project, specifically using the functionality that builds parsers from dataclasses.
I was curious what the differences were.
I'm currently using marshmallow in a project, specifically using the functionality that builds parsers from dataclasses.
I was curious what the differences were.
My company soured on Marshmallow a while back due to performance. Maybe it’s gotten a lot better, but it has a bad reputation here. Most people seem really happy after we started using Pydantic. Take all that with a grain of salt since I’m just parroting hearsay :)
Personal opinion: pydantic crushes Marshmallow. Not even a fair fight. Pydantic is more performant, has better mypy/linter integration, and more powerful data model. We had a project where we pre-emptively used marshmallow to marshall/validate data. Had to remove that and solely use it at the ORM layer because of performance (and it still struggles).
I haven't used pydantic's ORM integration, but I don't hesitate to use pydantic models everywhere as business logic classes unless I need ludicrous speed.
That's all opinion, but I'd definitely give pydantic a swing.
I haven't used pydantic's ORM integration, but I don't hesitate to use pydantic models everywhere as business logic classes unless I need ludicrous speed.
That's all opinion, but I'd definitely give pydantic a swing.
Marshmallow feels like a more flexible project, but it needs more work for a similar effect IMO.
My cons are: - Uses some dsl to define types - Doesn’t marshal to model objects by default, but from DICT to DICT
My pro is: - much more configurable and powerful
My cons are: - Uses some dsl to define types - Doesn’t marshal to model objects by default, but from DICT to DICT
My pro is: - much more configurable and powerful
I've also found Pydantic to be a valuable library to use.
However, it does have a strongly opinionated approach to casting that can sometimes yield non-obvious results. This behavior is documented and I would suggest new potential adopters of the library to explore this casting/coerce feature in the context of your product/app requirements.
For the most part, it's not an huge issue, but I've run into a few surprising cases. For example, sys.maxint, 0, '-7', 'inf', float('-inf') are all valid datetime formats.
- https://pydantic-docs.helpmanual.io/usage/models/#data-conve... - https://gist.github.com/mpkocher/30569c53dc3552bc5ad73e09b48...
However, it does have a strongly opinionated approach to casting that can sometimes yield non-obvious results. This behavior is documented and I would suggest new potential adopters of the library to explore this casting/coerce feature in the context of your product/app requirements.
For the most part, it's not an huge issue, but I've run into a few surprising cases. For example, sys.maxint, 0, '-7', 'inf', float('-inf') are all valid datetime formats.
- https://pydantic-docs.helpmanual.io/usage/models/#data-conve... - https://gist.github.com/mpkocher/30569c53dc3552bc5ad73e09b48...
Just started working on a new SaaS startup and using FastAPI & Pydantic. The development experience has been great.
Can't you do this already with Python type annotations? I am a fan of typing in general (not just for data model as this seems to be) and using types everywhere saves a lot of debugging hassle and even allows for catching some bugs with static analysis.
There is one thing concerns me about type annotation. With annotation Python code looks like Java code, but with it I don't have advantages of Java (like speed). Besides I see things like `x: typing.Union[str, typing.Any]` - maybe better no annotation.
> x: typing.Union[str, typing.Any]
That's absolutely a valid and useful annotation. It tells me, and autocomplete, that "x" is probably a str, more likely than not, but I need to be aware that it might not be.
That's absolutely a valid and useful annotation. It tells me, and autocomplete, that "x" is probably a str, more likely than not, but I need to be aware that it might not be.
We've had a similar experience using pydantic. We integrated it quite tightly with a django project and it's been awesome.
Where did you find it valuable to wire in to Django?
We effectively use them as serializers, and also as a way to allow users to interact with the models in a controlled manner.
In our app, django essentially sits between a compute cluster and a front end. The pydantic objects are used to define the work to be done on the compute cluster.
In our app, django essentially sits between a compute cluster and a front end. The pydantic objects are used to define the work to be done on the compute cluster.
This is a great post. I come back to it frequently.
There's beautiful clarity in the articulation, and the essence is easy to grasp yet powerful. It reminds me a bit of Scott Wlaschin's Railway Oriented Programming (ROP) [0]. As a technique, ROP nicely complements "parse don't validate". As an explanation, it's similarly simple yet wonderfully effective.
I've a real admiration for people who can explain and present things so clearly. With ROP, for example, the reader learns the basics of monads without even realising it.
[0]: https://fsharpforfunandprofit.com/rop/
There's beautiful clarity in the articulation, and the essence is easy to grasp yet powerful. It reminds me a bit of Scott Wlaschin's Railway Oriented Programming (ROP) [0]. As a technique, ROP nicely complements "parse don't validate". As an explanation, it's similarly simple yet wonderfully effective.
I've a real admiration for people who can explain and present things so clearly. With ROP, for example, the reader learns the basics of monads without even realising it.
[0]: https://fsharpforfunandprofit.com/rop/
I agree. It's a very well-written post. I am not a Haskell person, but it was quite clear to me.
I feel that we don't put enough value, these days, on the ability to write clear, articulate exposition. Also, I believe that many people are not willing to read articles, books, or papers, of any meaningful length.
Everything needs to be boiled down to <10 min. read time, or <18 min. TED talks.
I feel that we don't put enough value, these days, on the ability to write clear, articulate exposition. Also, I believe that many people are not willing to read articles, books, or papers, of any meaningful length.
Everything needs to be boiled down to <10 min. read time, or <18 min. TED talks.
The way you somewhat randomly mention the value of clear and articulate exposition makes me assume you just had to wade through a 300-page specification document for a government contract regarding pencil sharpeners or something similar. If that's the case, you have my sympathy.
Anyways, I definitely agree.
Anyways, I definitely agree.
Not recently (thank the Gods), but I used to work for a defense contractor, and I have dealt with many specification documents (like the Bluetooth spec).
Those are the extras. This is the post:
https://fsharpforfunandprofit.com/posts/recipe-part2/
https://fsharpforfunandprofit.com/posts/recipe-part2/
This principle can be applied to dynamic languages as well if you have some mechanism such as type hinting, pre-conditions etc. that are checked by a linter during development, even if it isn't, you can still use it at runtime with sufficient error handling.
The essential point of this blog post is to avoid "shotgun parsing", where parsing/validating is done just from a procedural standpoint, where it matters when exactly it happens. In the paper "Out of the Tar Pit" it is asserted that this leads to "accidental complexity" (AKA "pain and anxiety"), which is something every programmer has experienced before, possibly many times.
I've become a fan of declarative schema to (json-schema/OpenApi, clojure spec etc.) to express this kind of thing. Usually this is used at the boundaries of an application (configuration, web requests etc.) but there are many more applications for this within the flow of data transformations. If you apply the "parse don't validate" principle you turn schema-validated (sic!) data into a new thing. Whether that is a "ValidatedData" type or meta data, a pre-condition or runtime check says more about the environment you program rather than the principle in discussion. The benefit however is clear: Your code asserts that it requires parsed/validated data where it is needed, instead of when it should happen.
The essential point of this blog post is to avoid "shotgun parsing", where parsing/validating is done just from a procedural standpoint, where it matters when exactly it happens. In the paper "Out of the Tar Pit" it is asserted that this leads to "accidental complexity" (AKA "pain and anxiety"), which is something every programmer has experienced before, possibly many times.
I've become a fan of declarative schema to (json-schema/OpenApi, clojure spec etc.) to express this kind of thing. Usually this is used at the boundaries of an application (configuration, web requests etc.) but there are many more applications for this within the flow of data transformations. If you apply the "parse don't validate" principle you turn schema-validated (sic!) data into a new thing. Whether that is a "ValidatedData" type or meta data, a pre-condition or runtime check says more about the environment you program rather than the principle in discussion. The benefit however is clear: Your code asserts that it requires parsed/validated data where it is needed, instead of when it should happen.
I think the article goes a little further than what you describe -- it would have you use a strong type that cannot represent illegal values.
There's a follow-up article by the same author (that I unfortunately can't find), in which she explains this point.
As an example, returning a NonZero newtype over Int is not as type safe as using an ADT that lacks a zero value altogether. Using a NonEmpty newtype over List is not as type safe as using the NonEmpty ADT that has an element as part of its structure.
Basically newtype still has use, but it is not as airtight as a well-designed ADT.
There's a follow-up article by the same author (that I unfortunately can't find), in which she explains this point.
As an example, returning a NonZero newtype over Int is not as type safe as using an ADT that lacks a zero value altogether. Using a NonEmpty newtype over List is not as type safe as using the NonEmpty ADT that has an element as part of its structure.
Basically newtype still has use, but it is not as airtight as a well-designed ADT.
> There's a follow-up article by the same author (that I unfortunately can't find), in which she explains this point.
I think this is it: https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...
I think this is it: https://lexi-lambda.github.io/blog/2020/11/01/names-are-not-...
> This principle can be applied to dynamic languages as well if you have some mechanism such as type hinting, pre-conditions etc. that are checked by a linter during development, even if it isn't, you can still use it at runtime with sufficient error handling.
Every time I read something like this my mind translates it to "after building an ad-hoc compiler you can do all the things a compiler can do. Just not as well, but you can do it." -- Same with "I don't need a compiler, my tests stop all this kind of bugs"
Every time I read something like this my mind translates it to "after building an ad-hoc compiler you can do all the things a compiler can do. Just not as well, but you can do it." -- Same with "I don't need a compiler, my tests stop all this kind of bugs"
I know of the advantages of static typing and very much appreciate them. My point was more about how the concept in the article may be translated to other types of tooling.
A related mantra is to "Make impossible states impossible" https://www.youtube.com/watch?v=IcgmSRJHu_8
I think the original formulation is "make illegal states unrepresentable" from Yaron Minsky: https://blog.janestreet.com/effective-ml/ and https://blog.janestreet.com/effective-ml-revisited/ (or maybe there are older sources than 2010?)
I think making this about the type checker is a bit of a red herring. There is nothing in this – otherwise excellent – advice that can’t be applied to a dynamically typed language like Ruby. It’s the same insight that leads OOP folks to warn against the Primitive Obsession code smell (http://wiki.c2.com/?PrimitiveObsession). It’s also the insight that leads to the Hexagonal Architecture ( https://en.wikipedia.org/wiki/Hexagonal_architecture_(softwa... ).
I agree but I also think it’s on the right path. This seems partially like a “why Haskell” in disguise to me.
I’ve run across DSLs that have three or more layers of parsing and validation. Embedding different languages within each other (eg: JSON snippets within your own DSL) definitely leads to the issues the article talks about.
Also, growing your own parser without understanding standard lexer/parser basics seems far more common than it ought to be. I’m not talking brilliant design, rather the extremely naive one-character-at-a-time-in-a-really-complex-loop variety of design.
The better level of bad is, “I know what lexers/parsers are, now I’ll write something to basically implement a type-checking parser with the lexed+parsed tree as input.”
This article is basically stating, “Why not just get your parser to do it all for you in one swell foop?” When I have refactored code to follow this kind of design, I have never regretted the outcome.
I’ve run across DSLs that have three or more layers of parsing and validation. Embedding different languages within each other (eg: JSON snippets within your own DSL) definitely leads to the issues the article talks about.
Also, growing your own parser without understanding standard lexer/parser basics seems far more common than it ought to be. I’m not talking brilliant design, rather the extremely naive one-character-at-a-time-in-a-really-complex-loop variety of design.
The better level of bad is, “I know what lexers/parsers are, now I’ll write something to basically implement a type-checking parser with the lexed+parsed tree as input.”
This article is basically stating, “Why not just get your parser to do it all for you in one swell foop?” When I have refactored code to follow this kind of design, I have never regretted the outcome.
I don't think types are a red herring here. Because if you follow this advice, then just using logic and your source code, you can prove what data is valid. And, since types and (constructive) logic are so strongly related, then the types are, in some sense "there" even if you don't see them. To put it another way, it's nice if your computer can make the proofs but if it can't, does that make the theorems any less true?
The advantage of the type checker is that it automatically check types for you. If you have only one function that produces a parsedArray and multiple functions that accept a parsedArray, you can be sure where they come from.
Yes, that’s literally the advantage of a static type checker. I don’t dispute that. I’m just saying that the advice in the article is just as applicable in a dynamic language and confers the same benefits. True, you’re not protected against accidentally using one type where another was expected, but that’s not really the point as I see it. The point is to use better types. Types you can lean on instead instead of nervously tiptoe around.
What would you rather have: an object of value potentially outside your domain and an expensive boolean function saying if it's ok that you need to apply everywhere just to be sure, or a method of producing values that you know are always within your domain which you have to apply just once and no expensive boolean function?
In typescript parsing/asserting types with combinators works very well merging runtime with static type system [0], it has to be used at i/o boundary, then it enters static type system guarantee and no assertions are necessary, makes very nice codebase.
[0] https://github.com/appliedblockchain/assert-combinators
[0] https://github.com/appliedblockchain/assert-combinators
I wish it had actual proper examples. I've no idea how to use that.
from a look at the readme, you combine those `$.TYPE` things to build a validation function that checks if its argument matches some pattern (and throws an exception if it doesn't).
https://github.com/appliedblockchain/assert-combinators/blob...
https://github.com/appliedblockchain/assert-combinators/blob...
import * as $ from '@appliedblockchain/assert-combinators'
const validateFooBar = (
$.object({
foo: $.string,
bar: $.boolean
})
)
// probably roughly equivalent to
/*
const validateFooBar = (x) => {
console.assert(
typeof x === 'object' &&
typeof x.foo === 'string' &&
typeof x.bar === 'boolean'
)
return x
}
*/
const test1 = { foo: "abc", bar: false }
const test2 = { foo: 0, quux: true }
const { foo, bar } = validateFooBar(test1) // ok
const oops = validateFooBar(test2) // throws error
https://github.com/appliedblockchain/assert-combinators/blob...
https://github.com/appliedblockchain/assert-combinators/blob...
Yes, the difference between console.assert and assert-combinators is that assert combinators return well typed ts result (and are more terse/minimal).
Check out https://github.com/gcanti/io-ts/blob/master/index.md instead. I find it more composable and you can define a codec and get a native type from it so you are only defining things once.
Assert combinators are composable, light, terse (very little verbosity), types can be defined in single place, instead of type/interface definition you can use return type of assert function.
They don’t go into deep category theory, you won’t find monads and friends, they are first level, straightforward any typescript developer can pick up in minutes - this is by design. It stops at combinators in typescript to solve very specific problem and nothing more. Haskell in ts is not the goal of this npm.
They don’t go into deep category theory, you won’t find monads and friends, they are first level, straightforward any typescript developer can pick up in minutes - this is by design. It stops at combinators in typescript to solve very specific problem and nothing more. Haskell in ts is not the goal of this npm.
Thank you, you are right. I’ll add examples and ping here.
Software Engineers: Parse, don't validate.
Mathematicians: Parsing is validation
https://gallais.github.io/pdf/draft_sigbovik21.pdf
Mathematicians: Parsing is validation
https://gallais.github.io/pdf/draft_sigbovik21.pdf
To everyone in this subthread: sigbovik is a conference published every 1st of April.
This paper is an April's fool joke. I didn't think people could take that one seriously. I guess it's a good April's fool then. :)
This paper is an April's fool joke. I didn't think people could take that one seriously. I guess it's a good April's fool then. :)
The conference is indeed a spoof, but in so far as what Mathematicians call a "proof" - the paper contains one. Agda is a proof assistant in the spirit of the Calculus of Constructions ( https://en.wikipedia.org/wiki/Calculus_of_constructions ).
So is the joke on Computer Scientists or Mathematicians? You decide ;)
Beware of bugs in the above code; I have only proved it correct, not tried it --Donald Knuth
So is the joke on Computer Scientists or Mathematicians? You decide ;)
Beware of bugs in the above code; I have only proved it correct, not tried it --Donald Knuth
Sigbovik's jokes are of the kind where the premise is completely bonkers. The rest of the development is made with the utmost rigor to highlight said bonkersitude, Reductio ad absurdum.
Yeah, but that is precisely how inductive types work.
"Bonkers" premises. Iterate, iterate, iterate. "Bonkers" conclusions. GIGO.
And yet the result is reified, exists and speaks for itself. So what is so "absurd" and "bonkers" about a result that is right before your eyes?
https://en.wikipedia.org/wiki/Reification_(computer_science)
"Bonkers" premises. Iterate, iterate, iterate. "Bonkers" conclusions. GIGO.
And yet the result is reified, exists and speaks for itself. So what is so "absurd" and "bonkers" about a result that is right before your eyes?
https://en.wikipedia.org/wiki/Reification_(computer_science)
The wide adoption of Flask as Python's backend development framework of choice makes it quite clear that software developers have a hard time picking up April fool's jokes.
Or how great April fool's ideas can actually be if they turn out to be real
Software engineers like efficiently running software.
Mathematicians like beautiful definitions.
Scientists like non-trivial discoveries.
This paper.... uh.... what exactly is it good for?
I suppose it could be kind of nice as some kind of undergraduate paper writing project kind of thing but it looks too professional for that.... I am kind of at a loss why this was written. Maybe it is some strange kind of satire....
This paper.... uh.... what exactly is it good for?
I suppose it could be kind of nice as some kind of undergraduate paper writing project kind of thing but it looks too professional for that.... I am kind of at a loss why this was written. Maybe it is some strange kind of satire....
It's written for sigbovik 2021 [1][2], which is very much a joke conference. Other papers this year were "Lowestcase and Uppestcase letters: Advances in Derp Learning" and "On the dire importance of mru caches for human survival (against Skynet)".
[1]: http://www.sigbovik.org/ [2]: http://www.sigbovik.org/2021/proceedings.pdf
[1]: http://www.sigbovik.org/ [2]: http://www.sigbovik.org/2021/proceedings.pdf
SIGBOVIK is a parody of computer science conferences. It's a running joke hosted on April Fools Day every year at CMU - and apparently a quite convincing one. ;-)
Source: I attended SIGBOVIK a few times in grad school.
Source: I attended SIGBOVIK a few times in grad school.
This paper is good for parsing/validating your source code (from the view-point of your compiler/interpreter).
Code is data after all.
Code is data after all.
The point being, the converse of ‘parsing is validation’ is not true.
Then you have some formally inexpressible/impredicative notion of "validation" in mind. For posterity (lifting from the depths of the threads):
General case: Validating random data as input into some program.
Particular case: Validating random source code (data) as input into some compiler (program).
Do compilers parse or validate?
> "the converse of ‘parsing is validation’ is not true."
If that were the case then you should be able to give an example of a compiler validating random source code (data) but not parsing it.
What determines the validity of random input is precisely a compiler's ability to parse it.
General case: Validating random data as input into some program.
Particular case: Validating random source code (data) as input into some compiler (program).
Do compilers parse or validate?
> "the converse of ‘parsing is validation’ is not true."
If that were the case then you should be able to give an example of a compiler validating random source code (data) but not parsing it.
What determines the validity of random input is precisely a compiler's ability to parse it.
The word "is" implies an isomorphism.
If you see it differently you are implicitly assuming a non-formalist perspective on what "validation" means. Tell us about it.
If you see it differently you are implicitly assuming a non-formalist perspective on what "validation" means. Tell us about it.
‘A square is a rectangle’ means squares are isomorphic to rectangles?
You are tripping up over polymorphism. "Is" means many things - which meaning you infer is precisely parsing!
"A square is a rectangle" means "A square is a TYPE of rectangle" (at least, that is what I am parsing it as).
"Parsing is Validation" means Parsing is isomorphic to Validation.
How do I know? Because that is how I want you to parse it.
"A square is a rectangle" means "A square is a TYPE of rectangle" (at least, that is what I am parsing it as).
"Parsing is Validation" means Parsing is isomorphic to Validation.
How do I know? Because that is how I want you to parse it.
> 'When I use a word,' Humpty Dumpty said in rather a scornful tone, 'it means just what I choose it to mean — neither more nor less.'
+∞
parse verb. resolve (a sentence) into its component parts and describe their syntactic roles.
In computer science what we do is precisely syntax analysis. Determining the meaning of operators.
Mathematicians have the exact same problem with respect to the equality operator.
https://ncatlab.org/nlab/show/equality#DifferentKinds
parse verb. resolve (a sentence) into its component parts and describe their syntactic roles.
In computer science what we do is precisely syntax analysis. Determining the meaning of operators.
Mathematicians have the exact same problem with respect to the equality operator.
https://ncatlab.org/nlab/show/equality#DifferentKinds
> "A square is a rectangle" means "A square is a TYPE of rectangle" (at least, that is what I am parsing it as).
In that case your former statement that 'The word "is" implies an isomorphism' seems to be wrong.
In that case your former statement that 'The word "is" implies an isomorphism' seems to be wrong.
It may be wrong in your model/interpretation of my words, but it's not wrong in my interpretation of my words.
In what interpretation is it consistent for 'A square is a rectangle" means "A square is a TYPE of rectangle"' and 'The word "is" implies an isomorphism' to be simultaneously true? No matter how I cut it, the latter seems to prevent the former to me.
> The word "is" implies an isomorphism.
Are you talking about a bijective mapping or are you saying it's a synonym for identical?
Because the former doesn't make any sense here and the latter is not true.
Red is a color does not imply that all colors are red.
Are you talking about a bijective mapping or are you saying it's a synonym for identical?
Because the former doesn't make any sense here and the latter is not true.
Red is a color does not imply that all colors are red.
I am talking about the polymorphic use of the verb "is" during the process of formalization.
"Red is a color" can be formalized as "Red is a type of color" or "Red is member of set Colors".
You can't formalize "Color is red" because it doesn't mean anything.
When I say "Parsing is validation" I am using the verb "is" to mean an isomorphism.
"Red is a color" can be formalized as "Red is a type of color" or "Red is member of set Colors".
You can't formalize "Color is red" because it doesn't mean anything.
When I say "Parsing is validation" I am using the verb "is" to mean an isomorphism.
Judging by all the other disagreeing comments, your in some sort of idiosyncratic context that only you understand.
Good luck with that.
Good luck with that.
The word “is” is also often used informally to mean “is a kind of”.
"A kind of" is precisely its formal use from the PoV of a type theorist.
Two things are the same type of thing if they share all of their extensional properties.
That is what it means for two things to be identical/equal.
Two things are the same type of thing if they share all of their extensional properties.
That is what it means for two things to be identical/equal.
But what I am saying is that parsing is a kind of validation. But all validation is not parsing.
For example let's say that I have written an HTTP API that accepts application/x-www-form-urlencoded data to one of its endpoints. Let's say `POST /users`, and this is where the client-side application posts data to.
Now I can implement this in many ways. I can for example define
And of course I could go further and define types for the name and the phone number but in this case lets say that I have decided that strings are the proper representation in this case.
If the fields of my structs were directly accessible
The point is that here I am doing some rudimentary validation on the phone number, requiring it to be in Norwegian format. But I am enforcing this in the implementation of the handler for the HTTP API endpoint, rather than in the data type itself.
Whereas if I was instead doing
And in this manner, even though my validation is still rudimentary, and a type purist might find the type insufficiently constrained, I have indeed in my own book gone from just validation to actual parsing. Because I have made it so that the construction of the type enforces the constraints on the data.
For example let's say that I have written an HTTP API that accepts application/x-www-form-urlencoded data to one of its endpoints. Let's say `POST /users`, and this is where the client-side application posts data to.
Now I can implement this in many ways. I can for example define
pub struct Person {
name: String,
phone_number: String,
}
And of course I could go further and define types for the name and the phone number but in this case lets say that I have decided that strings are the proper representation in this case.
If the fields of my structs were directly accessible
pub struct Person {
pub name: String,
pub phone_number: String,
}
// ...
let name = post_data.name;
let phone_number = post_data.phone_number;
let norwegian_phone_number_format = Regex::new(r"^(\+47|0047)?\d{8}$").unwrap();
// ...
The point is that here I am doing some rudimentary validation on the phone number, requiring it to be in Norwegian format. But I am enforcing this in the implementation of the handler for the HTTP API endpoint, rather than in the data type itself.
Whereas if I was instead doing
pub struct Person {
name: String,
phone_number: String,
}
impl Person {
pub fn try_new (name: String, phone_number: String) -> std::result::Result<Self, PersonDataError> {
// ...
let norwegian_phone_number_format = Regex::new(r"^(\+47|0047)?\d{8}$").unwrap();
// ...
}
}
And in this manner, even though my validation is still rudimentary, and a type purist might find the type insufficiently constrained, I have indeed in my own book gone from just validation to actual parsing. Because I have made it so that the construction of the type enforces the constraints on the data.
You are over-complicating this into obscurity.
General case: Validating random data as input into some program.
Particular case: Validating random source code (data) as input into some compiler (program).
Do compilers parse or validate?
"parsing is validation, but validation is not parsing" if that were true then you should be able to give an example of a compiler doing some sort of validation on the random source code (data) that is not parsing.
The very thing which determines the validity of random source code is the compiler's ability to parse it.
General case: Validating random data as input into some program.
Particular case: Validating random source code (data) as input into some compiler (program).
Do compilers parse or validate?
"parsing is validation, but validation is not parsing" if that were true then you should be able to give an example of a compiler doing some sort of validation on the random source code (data) that is not parsing.
The very thing which determines the validity of random source code is the compiler's ability to parse it.
But I’m not talking about compilers here
Why not?
Compilers are computable functions.
If “Parsing is validation, but validation is not parsing” is true then it is also true about compilers.
Compilers are computable functions.
If “Parsing is validation, but validation is not parsing” is true then it is also true about compilers.
> If “Parsing is validation, but validation is not parsing” is true then it is also true about compilers.
This is a false statement, considering that the cases where something is validation but not parsing may very well lie in the complement of the set of compilers to the set of all computable functions. The converse statement that if this were true about the smaller set (of compilers), it would also be true about the larger set (of all programs) would on the other hand be correct.
This is a false statement, considering that the cases where something is validation but not parsing may very well lie in the complement of the set of compilers to the set of all computable functions. The converse statement that if this were true about the smaller set (of compilers), it would also be true about the larger set (of all programs) would on the other hand be correct.
[deleted]
[deleted]
That's how the validation tool for Python Maat works. By creating a completely new dictionary.
https://github.com/Attumm/Maat
For js / typescript I like: https://github.com/paperhive/fefe
basically ist’s just functions that take a value of one type and return a other one
basically ist’s just functions that take a value of one type and return a other one
There is also joi, zod, myzod just to name a few.
I personally use myzod as its fast it parsing, zero dependancies and you can infre types from your schemas.
I personally use myzod as its fast it parsing, zero dependancies and you can infre types from your schemas.
Don’t forget https://github.com/gcanti/io-ts
How does Maat compare with pydantic? https://github.com/samuelcolvin/pydantic
Pydantic is using classes and typehinting. The new dataclasses style. Currently Maat doens't have a parser for dataclasses, it could come in the future. Pydantic works great with typehinting.
Maat was created before dataclasses existed. For validation Maat offers the same. But it also allows for some really neat features such as validation on encrypted data. https://github.com/Attumm/Maat/blob/main/tests/test_validati...
Since validation is written as dictionaries its possible to store the validations in caching db such as Redis.
And since its simple its easy to extend for anyone use case. And there are no other dependencies.
Benchmarks of pydantic has Maat around twice as Pydantic.
Maat was created before dataclasses existed. For validation Maat offers the same. But it also allows for some really neat features such as validation on encrypted data. https://github.com/Attumm/Maat/blob/main/tests/test_validati...
Since validation is written as dictionaries its possible to store the validations in caching db such as Redis.
And since its simple its easy to extend for anyone use case. And there are no other dependencies.
Benchmarks of pydantic has Maat around twice as Pydantic.
Unable to change my comment.
Benchmarks of pydantic has Maat around twice the speed of Pydantic
Benchmarks of pydantic has Maat around twice the speed of Pydantic
If only for conciseness, readability and speed, I'd take Pydantic over any day. Being able to express 80% of type checking using Python native type hints + dataclasses is just so intuitive!
And it's getting some wide adoption, for instance FastAPI which uses it for request validations.
And it's getting some wide adoption, for instance FastAPI which uses it for request validations.
Engineering is about tradeoffs, even though both projects do validation.
The points you made are all very valid points.
At my employer we use both projects. If the data is very nested, or really large Maat is used.
The points you made are all very valid points.
At my employer we use both projects. If the data is very nested, or really large Maat is used.
Mmmh interesting requirement! Indeed, defining very nested structure with Pydantic is one of its weaknesses.
And of course I agree 100% about tradoffs in engineering. However I usually advise against using 2 dependencies doing mainly the same thing if possible within the same project.
Anyway, good catch, thanks for enlightening me!
And of course I agree 100% about tradoffs in engineering. However I usually advise against using 2 dependencies doing mainly the same thing if possible within the same project.
Anyway, good catch, thanks for enlightening me!
Could someone rewrite the examples in TypeScript?
Some points really elude me because Haskell uses many symbols and is very dense.
Some points really elude me because Haskell uses many symbols and is very dense.
From the Twitter link:
> IME, people in dynamic languages almost never program this way, though—they prefer to use validation and some form of shotgun parsing. My guess as to why? Writing that kind of code in dynamically-typed languages is often a lot more boilerplate than it is in statically-typed ones!
I feel that once you've got experience working in (usually functional) programming languages with strong static type checking, flakey dynamic code that relies on runtime checks and just being careful to avoid runtime errors makes your skin crawl, and you'll intuitively gravitate towards designs that takes advantage of strong static type checks.
When all you know is dynamic languages, the design guidance you get from strong static type checking is lost so there's more bad design paths you can go down. Patching up flakey code with ad-hoc runtime checks and debugging runtime errors becomes the norm because you just don't know any better and the type system isn't going to teach you.
More general advice would be "prefer strong static type checking over runtime checks" as it makes a lot of design and robustness problems go away.
Even if you can't use e.g. Haskell or OCaml in your daily work, a few weeks or just of few days of trying to learn them will open your eyes and make you a better coder elsewhere. Map/filter/reduce, immutable data structures, non-nullable types etc. have been in other languages for over 30 years before these ideas became more mainstream best practices for example (I'm still waiting for pattern matching + algebraic data types).
It's weird how long it's taking for people to rediscover why strong static types were a good idea.
> IME, people in dynamic languages almost never program this way, though—they prefer to use validation and some form of shotgun parsing. My guess as to why? Writing that kind of code in dynamically-typed languages is often a lot more boilerplate than it is in statically-typed ones!
I feel that once you've got experience working in (usually functional) programming languages with strong static type checking, flakey dynamic code that relies on runtime checks and just being careful to avoid runtime errors makes your skin crawl, and you'll intuitively gravitate towards designs that takes advantage of strong static type checks.
When all you know is dynamic languages, the design guidance you get from strong static type checking is lost so there's more bad design paths you can go down. Patching up flakey code with ad-hoc runtime checks and debugging runtime errors becomes the norm because you just don't know any better and the type system isn't going to teach you.
More general advice would be "prefer strong static type checking over runtime checks" as it makes a lot of design and robustness problems go away.
Even if you can't use e.g. Haskell or OCaml in your daily work, a few weeks or just of few days of trying to learn them will open your eyes and make you a better coder elsewhere. Map/filter/reduce, immutable data structures, non-nullable types etc. have been in other languages for over 30 years before these ideas became more mainstream best practices for example (I'm still waiting for pattern matching + algebraic data types).
It's weird how long it's taking for people to rediscover why strong static types were a good idea.
I have the same feeling after spending a few years with Haskell, StandardML, Agda, Idris, Coq, etc.
One trick I've found very useful is to realise that Maybe (AKA Option) can be though of as "a list with at most one element". Dynamic languages usually have some notion of list/array, which we can use as if it were a Maybe/Option type; e.g. we can follow a 'parse, don't validate' approach by wrapping a "parsed" result in a list, and returning an empty list otherwise. This allows us to use their existing 'map', 'filter', etc. too ;)
(This is explored in more detail, including links to logic programming, in https://link.springer.com/chapter/10.1007%2F3-540-15975-4_33 )
If we want to keep track of useful error messages, I've found Scala's "Try" type to be useful ('Try[T]' is isomorphic to 'Either Throwable T'). Annoyingly, built-in sum type; the closest thing is usually a tagged pair like '[true, myFoo]'/'[false, myException]', which is pretty naff.
One trick I've found very useful is to realise that Maybe (AKA Option) can be though of as "a list with at most one element". Dynamic languages usually have some notion of list/array, which we can use as if it were a Maybe/Option type; e.g. we can follow a 'parse, don't validate' approach by wrapping a "parsed" result in a list, and returning an empty list otherwise. This allows us to use their existing 'map', 'filter', etc. too ;)
(This is explored in more detail, including links to logic programming, in https://link.springer.com/chapter/10.1007%2F3-540-15975-4_33 )
If we want to keep track of useful error messages, I've found Scala's "Try" type to be useful ('Try[T]' is isomorphic to 'Either Throwable T'). Annoyingly, built-in sum type; the closest thing is usually a tagged pair like '[true, myFoo]'/'[false, myException]', which is pretty naff.
> a list with at most one element
I've found scala or even LINQ to really hammer down this point, even to those who aren't into FP very much. Doing that map/flatmap makes it click for just about anyone
I've found scala or even LINQ to really hammer down this point, even to those who aren't into FP very much. Doing that map/flatmap makes it click for just about anyone
Ya, having 'foreach( null )' be a no-op is my #1 language feature request.
Not being a big fan of method chaining, a null saavy foreach would probably eliminate most of my null checks, need for Optional.
Not being a big fan of method chaining, a null saavy foreach would probably eliminate most of my null checks, need for Optional.
For what its worth: People don't use dynamic language because they don't know better or never used a static language. To better understand what dynamic languages bring to the table, here are some disadvantages of static types to consider:
Static types are awesome for local reasoning, but they are not that helpful in the context of the larger system (this already starts at the database, see idempotency mismatch).
Code with static types is sometimes larger and more complex than the problem its trying to solve
They tightly couple data to a type system, which (can) introduce incidental complexity >(I'm still waiting for pattern matching + algebraic data types) This is a good example, if you pattern match to a specific structure (e.g. position of fields in your algebraic data type), you tightly coupled your program to this particular structure. If the structure change, you may have to change all the code which pattern matches this structure.
Static types are awesome for local reasoning, but they are not that helpful in the context of the larger system (this already starts at the database, see idempotency mismatch).
Code with static types is sometimes larger and more complex than the problem its trying to solve
They tightly couple data to a type system, which (can) introduce incidental complexity >(I'm still waiting for pattern matching + algebraic data types) This is a good example, if you pattern match to a specific structure (e.g. position of fields in your algebraic data type), you tightly coupled your program to this particular structure. If the structure change, you may have to change all the code which pattern matches this structure.
The example with pattern matching doesn't have anything to do with static types. You'll have exactly the same problem if you pattern match against positional arguments in Python:
In both Python and statically typed languages you can avoid this by matching against field names rather than positions, or using some other interface to access data. This is an important design aspect to consider when writing code, but does not have anything to do with dynamic programming. The only difference static typing makes is that when you do change the type in a way that breaks existing patterns, you can know statically rather than needing failing tests or runtime errors.
The same is true for the rest of the things you've mentioned: none are specific to static typing! My experience with a lot of Haskell, Python, JavaScript and other languages is that Haskell code for the same task tends to be shorter and simpler, albeit by relying on a set of higher-level abstractions you have to learn. I don't think much of that would change for a hypothetical dynamically typed variant of Haskell either!
[1]: https://www.python.org/dev/peps/pep-0636/#matching-sequences
match event.get():
case Click((x, y)):
handle_click_at(x, y)
In both Python and statically typed languages you can avoid this by matching against field names rather than positions, or using some other interface to access data. This is an important design aspect to consider when writing code, but does not have anything to do with dynamic programming. The only difference static typing makes is that when you do change the type in a way that breaks existing patterns, you can know statically rather than needing failing tests or runtime errors.
The same is true for the rest of the things you've mentioned: none are specific to static typing! My experience with a lot of Haskell, Python, JavaScript and other languages is that Haskell code for the same task tends to be shorter and simpler, albeit by relying on a set of higher-level abstractions you have to learn. I don't think much of that would change for a hypothetical dynamically typed variant of Haskell either!
[1]: https://www.python.org/dev/peps/pep-0636/#matching-sequences
You're absolutely right. I guess I mentioned pattern matching in particular because of the cited sentence from OP "I'm still waiting for pattern matching + algebraic data types".
> The same is true for the rest of the things you've mentioned: none are specific to static typing!
Sure, I could be wrong here. I frequently am. But could you point out why do you think that?
> The same is true for the rest of the things you've mentioned: none are specific to static typing!
Sure, I could be wrong here. I frequently am. But could you point out why do you think that?
Seems like static typing is neither necessary nor sufficient to cause the particular problems you mentioned.
Static types can absolutely help with more than local reasoning. One of the main reasons I like static types is that they give me a place in the code to reflect aspects of the architecture that otherwise go implicit. Databases are a great example: in both static and dynamic languages, the behavior of your code is going to depend on the database schemas you're using, and static types give you a mechanism to declare and manage this dependency explicitly. In a general sense, the key in architecture is how a system is split up into parts and what interfaces the parts have, and a static type system is a way to make those interfaces first-class citizens of a language subject to abstraction and automation.
Code with static types might be larger than the problem it's trying to solve, but so might code with dynamic types—that's more a function of the sort of abstractions and complexity that go into the code. I've written some code to do very similar things in Haskell and Python, and my experience is that the Haskell code tends to be noticeably more compact than the Python code, even though I make liberal use of types and write out all the top-level type signatures. While some of this comes down to other differences between the languages, part of Haskell's expressiveness absolutely comes down to static types—static types make it easier to write very general code against high-level abstractions (eg monads) that would need to be narrower in dynamic languages (eg promises).
And sure, you can couple code and data in static languages, but at least the types will help you when it comes time to change. I've worked with dynamically typed programs where functions rely on some mixture of nested tuples, hashmaps, lists... etc, and it's hard to understand exactly how concepts are represented in code, much less change that representation. If you represent some central notion in your code as some nested primitive type (a hashmap of tuples of ...) and you want to change that representation, you'll still have to update the places it's used in the code, but without a static type system you won't get a list of those places, and you won't have assurance that updated everywhere that mattered. I'm not sure I'm explaining this pattern well, but I've worked in JavaScript codebases where making breaking changes to core data components was basically impossible.
Point being, all the problems you mention come up in both static an dynamic languages. There might be a case that they're more common in one or the other, but it's not obvious either way, and it's going to depend a lot on which specific static type system you're thinking of.
Static types can absolutely help with more than local reasoning. One of the main reasons I like static types is that they give me a place in the code to reflect aspects of the architecture that otherwise go implicit. Databases are a great example: in both static and dynamic languages, the behavior of your code is going to depend on the database schemas you're using, and static types give you a mechanism to declare and manage this dependency explicitly. In a general sense, the key in architecture is how a system is split up into parts and what interfaces the parts have, and a static type system is a way to make those interfaces first-class citizens of a language subject to abstraction and automation.
Code with static types might be larger than the problem it's trying to solve, but so might code with dynamic types—that's more a function of the sort of abstractions and complexity that go into the code. I've written some code to do very similar things in Haskell and Python, and my experience is that the Haskell code tends to be noticeably more compact than the Python code, even though I make liberal use of types and write out all the top-level type signatures. While some of this comes down to other differences between the languages, part of Haskell's expressiveness absolutely comes down to static types—static types make it easier to write very general code against high-level abstractions (eg monads) that would need to be narrower in dynamic languages (eg promises).
And sure, you can couple code and data in static languages, but at least the types will help you when it comes time to change. I've worked with dynamically typed programs where functions rely on some mixture of nested tuples, hashmaps, lists... etc, and it's hard to understand exactly how concepts are represented in code, much less change that representation. If you represent some central notion in your code as some nested primitive type (a hashmap of tuples of ...) and you want to change that representation, you'll still have to update the places it's used in the code, but without a static type system you won't get a list of those places, and you won't have assurance that updated everywhere that mattered. I'm not sure I'm explaining this pattern well, but I've worked in JavaScript codebases where making breaking changes to core data components was basically impossible.
Point being, all the problems you mention come up in both static an dynamic languages. There might be a case that they're more common in one or the other, but it's not obvious either way, and it's going to depend a lot on which specific static type system you're thinking of.
This argument is common, but I've never understood how a dynamically typed language is supposed to avoid coupling algorithms to data structures.
When using a data structure, I know what set of fields I expect it to have. In TypeScript, I can ask the compiler to check that my function's callers always provide data that meets my expectations. In JavaScript, I can check for these expectations at runtime or just let my function have undefined behavior.
Either way, if my function's assumptions about the data's shape don't turn out to be correct, it will break, whether or not I use a dynamic language.
It seems that most of the people who make this argument against static typing are actually arguing against violations of the Robustness Principle[0]: "be conservative in what you send, be liberal in what you accept".
A statically typed function that is as generous as possible should be no more brittle against outside change than an equally-generous dynamically typed function. The main difference is that the statically typed function is explicit about what inputs it has well-defined behavior for.
[0] https://en.wikipedia.org/wiki/Robustness_principle
When using a data structure, I know what set of fields I expect it to have. In TypeScript, I can ask the compiler to check that my function's callers always provide data that meets my expectations. In JavaScript, I can check for these expectations at runtime or just let my function have undefined behavior.
Either way, if my function's assumptions about the data's shape don't turn out to be correct, it will break, whether or not I use a dynamic language.
It seems that most of the people who make this argument against static typing are actually arguing against violations of the Robustness Principle[0]: "be conservative in what you send, be liberal in what you accept".
A statically typed function that is as generous as possible should be no more brittle against outside change than an equally-generous dynamically typed function. The main difference is that the statically typed function is explicit about what inputs it has well-defined behavior for.
[0] https://en.wikipedia.org/wiki/Robustness_principle
If you are doing things with what is basically strings, as you find a lot of user input from web form inputs is, the advantages in having a lot of different strict types of Strings isnt huge.
In these scenarios, using just basic types gets you a long way, because there is often very little business logic - route here, render that template, set these attributes.
Even a lot of JSON or XML parsing, you throw it into a parser and take what you need; if an unrelated field isnt what you expected, just move on with things rather than stop everything because the library author forgot about extension or openness possibilities (ie: an xs:any in a schema).
This attitude comes from the assumption not that types are unhelpful; just the chances we've modelled every outcome into our view of the world and gotten that right is unlikely.
When you get to complex systems with state changes to data and strict, well defined policies, rules engines, etc? Thats where dynamic languages often start adding all of that validation layer, to assert for right now you should act more like a type system and it's important - it probably has financial or security or other risks.
Even a lot of JSON or XML parsing, you throw it into a parser and take what you need; if an unrelated field isnt what you expected, just move on with things rather than stop everything because the library author forgot about extension or openness possibilities (ie: an xs:any in a schema).
This attitude comes from the assumption not that types are unhelpful; just the chances we've modelled every outcome into our view of the world and gotten that right is unlikely.
When you get to complex systems with state changes to data and strict, well defined policies, rules engines, etc? Thats where dynamic languages often start adding all of that validation layer, to assert for right now you should act more like a type system and it's important - it probably has financial or security or other risks.
In my own anecdotal experience, I think it comes down to what the product you are working on needs, and how your team works.
Recently, I spent 3 years on Scala then switched jobs and spent 3 years in Ruby.
It was a shock to go back to dynamic languages, but after 3 months, I honestly couldn't tell which felt more productive or led to more stable high quality product.
In Ruby, we had all the issues people point out about dynamic languages, but the product didn't lean heavily on complex data structures or algorithms. We embraced complexity and failure and get good processes, designs and practices to deal with this.
In Scala, we had more rigour, but I also know I spent a lot of time on type design. Once things were sorted there was a lot of confidence in it, but generally, it took a lot longer to get there.
For certain systems that is absolutely worth it, for others (and in my case) it did feel like the evolution of the product meant this effort never really paid off.
Recently, I spent 3 years on Scala then switched jobs and spent 3 years in Ruby.
It was a shock to go back to dynamic languages, but after 3 months, I honestly couldn't tell which felt more productive or led to more stable high quality product.
In Ruby, we had all the issues people point out about dynamic languages, but the product didn't lean heavily on complex data structures or algorithms. We embraced complexity and failure and get good processes, designs and practices to deal with this.
In Scala, we had more rigour, but I also know I spent a lot of time on type design. Once things were sorted there was a lot of confidence in it, but generally, it took a lot longer to get there.
For certain systems that is absolutely worth it, for others (and in my case) it did feel like the evolution of the product meant this effort never really paid off.
When you said "idempotency mismatch" you were meaning impedance mismatch, right?
Strange if so because it's the "Object-relational impedance mismatch" not the "Static type-relational impedance mismatch".
https://en.wikipedia.org/wiki/Object%E2%80%93relational_impe...
https://en.wikipedia.org/wiki/Object%E2%80%93relational_impe...
Your are right! Thank you for correcting me.
Plenty of people know multiple statically typed and dynamic languages, and multiple functional, imperative, and other languages; and use dynamic languages for some things but not other things. The set of people using dynamic languages isn't just "those that haven't had their eyes opened yet to what static languages can do". Different languages and paradigms make different things easier.
I do believe that, for long lasting, larger projects, static typing tends to make the code easier to maintain as time goes on. But not every project is like that. In fact, not every project uses a single language. Some use statically typed languages for some parts, and dynamically typed for others (this is common in web dev).
I do believe that, for long lasting, larger projects, static typing tends to make the code easier to maintain as time goes on. But not every project is like that. In fact, not every project uses a single language. Some use statically typed languages for some parts, and dynamically typed for others (this is common in web dev).
For sure. I do most of my work in situations of high volatility of domain and requirements and relatively high risk. (E.g., startups, projects in new areas.)
Static typing really appeals to me on a personal level. I enjoy the process of analysis it requires. I love the notion of eliminating whole classes of bugs. It feels way more tidy. I took Odersky's Scala class for fun and loved it.
But in practice, they're just a bad match for projects where the defining characteristic is unstable ground. They force artificial clarity when the reality is murky. And they impose costs that pay off in the long run, which only matters if the project has a long run. If I'm building something where we don't know where we're going, I'll reach for something like Python or Ruby to start.
This has been brought home to me by doing interviews recently. I have a sample problem that we pair on for an hour or so; there are 4 user stories. It involves a back end and a web front end. People can use any tools they want. My goal isn't to get particular things done; it's to see them at their best.
After doing a couple dozen, I'm seeing a pattern: developers using static tooling (e.g., Java, TypeScript) get circa half as much done as people using dynamic tooling (Python, plain JS). In the time when people in static contexts are still defining interfaces and types, people using dynamic tools are putting useful things on the page. Making a change in the static code often requires multiple tweaks in situations where it's one change in the dynamic code. It makes the extra costs of static tooling really obvious.
That doesn't harm the static-language interviewees, I should underline. The goal is to see how they work. But it was interesting to see that it wasn't just me feeling the extra costs. And those costs are only worth paying when they create payoffs down the road.
Static typing really appeals to me on a personal level. I enjoy the process of analysis it requires. I love the notion of eliminating whole classes of bugs. It feels way more tidy. I took Odersky's Scala class for fun and loved it.
But in practice, they're just a bad match for projects where the defining characteristic is unstable ground. They force artificial clarity when the reality is murky. And they impose costs that pay off in the long run, which only matters if the project has a long run. If I'm building something where we don't know where we're going, I'll reach for something like Python or Ruby to start.
This has been brought home to me by doing interviews recently. I have a sample problem that we pair on for an hour or so; there are 4 user stories. It involves a back end and a web front end. People can use any tools they want. My goal isn't to get particular things done; it's to see them at their best.
After doing a couple dozen, I'm seeing a pattern: developers using static tooling (e.g., Java, TypeScript) get circa half as much done as people using dynamic tooling (Python, plain JS). In the time when people in static contexts are still defining interfaces and types, people using dynamic tools are putting useful things on the page. Making a change in the static code often requires multiple tweaks in situations where it's one change in the dynamic code. It makes the extra costs of static tooling really obvious.
That doesn't harm the static-language interviewees, I should underline. The goal is to see how they work. But it was interesting to see that it wasn't just me feeling the extra costs. And those costs are only worth paying when they create payoffs down the road.
Great comment. There are no silver bullets. I am Team static typing, but recognize how heavy of a burden would be to start a purely exploratory development in Rust or Java. It just "cuts your wings" in the name of correctness... well some times it is useful to have the ability to start with a technically incorrect implementation that anyways only fails in a corner case that is not your main point of research.
On the other hand, as the initial code grows and grows, the cost of moving it all to a saner language grows too... discouraging a rewrite. So we end up with very complex production software that started as dynamic and is still dynamic.
On the other hand, as the initial code grows and grows, the cost of moving it all to a saner language grows too... discouraging a rewrite. So we end up with very complex production software that started as dynamic and is still dynamic.
[deleted]
Perhaps the Goldilocks mixture will be languages that allow type annotations but don't require it (e.g. Typescript, Elixir, Racket, and I think this is how Python's works).
Yeah, I've been using Python's gradual typing for a while. It's not perfect, but I'm excited for the possibilities. But the real test is to see what it's like on a large, long-lived project, so I'm keeping any open mind. I figure if that doesn't work fully, it'll still be a nice step toward things that can be pulled out as isolated services.
> But in practice, they're just a bad match for projects where the defining characteristic is unstable ground. They force artificial clarity when the reality is murky.
The rebuttal to this is provided by another post from Alexis King: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...
> This story sounds compelling, but it isn’t true. The flaw is in the premise: static types are not about “classifying the world” or pinning down the structure of every value in a system. The reality is that static type systems allow specifying exactly how much a component needs to know about the structure of its inputs, and conversely, how much it doesn’t.
Static typing helps you to safely model the parts of the system that you actually do know about, while allowing you to leave the unknown parts loosely defined. And when business requirements change, the compiler helps you to make those changes in a safe, guided way. This is why people often report the experience of doing large refactorings driven by compiler error messages, then running and finding that the change works correctly on the first try. A confidence few report in dynamically-typed languages.
> After doing a couple dozen, I'm seeing a pattern: developers using static tooling (e.g., Java, TypeScript) get circa half as much done as people using dynamic tooling (Python, plain JS).
I hope you understand and accept the fact that interview code challenges are not very representative of real-world software engineering. On-the-spot interview code is throwaway; real production code is not.
The rebuttal to this is provided by another post from Alexis King: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...
> This story sounds compelling, but it isn’t true. The flaw is in the premise: static types are not about “classifying the world” or pinning down the structure of every value in a system. The reality is that static type systems allow specifying exactly how much a component needs to know about the structure of its inputs, and conversely, how much it doesn’t.
Static typing helps you to safely model the parts of the system that you actually do know about, while allowing you to leave the unknown parts loosely defined. And when business requirements change, the compiler helps you to make those changes in a safe, guided way. This is why people often report the experience of doing large refactorings driven by compiler error messages, then running and finding that the change works correctly on the first try. A confidence few report in dynamically-typed languages.
> After doing a couple dozen, I'm seeing a pattern: developers using static tooling (e.g., Java, TypeScript) get circa half as much done as people using dynamic tooling (Python, plain JS).
I hope you understand and accept the fact that interview code challenges are not very representative of real-world software engineering. On-the-spot interview code is throwaway; real production code is not.
In this case, the interview problem was a simplified version of a thing the team gets asked to do. So I believe it was as representative as possible for an hour's coding.
I disagree that "real production code" is not throwaway. In situations of high requirements volatility, a great deal of production code ends up getting thrown away, because either a) it's intended to be a short-lived way to learn about the domain, or b) it's built on an assumption that turns out to be incorrect. And until the project hits economic sustainability, a lot of assumptions necessarily get made. It's only once the project is economically sustainable that we have some confidence that the code will be around for the long haul.
If your point is that most people are using statically typed languages wrong for high-volatility situations, I'll take your word for it. There were definitely times Typescript users seemed to be unnecessarily constraining things, for example. But I'm not sold that the more elaborate approach of being thoughtfully and explicitly untyped for a project's early stages for the sake of prototyping in a statically typed language would yield much practical gain.
I disagree that "real production code" is not throwaway. In situations of high requirements volatility, a great deal of production code ends up getting thrown away, because either a) it's intended to be a short-lived way to learn about the domain, or b) it's built on an assumption that turns out to be incorrect. And until the project hits economic sustainability, a lot of assumptions necessarily get made. It's only once the project is economically sustainable that we have some confidence that the code will be around for the long haul.
If your point is that most people are using statically typed languages wrong for high-volatility situations, I'll take your word for it. There were definitely times Typescript users seemed to be unnecessarily constraining things, for example. But I'm not sold that the more elaborate approach of being thoughtfully and explicitly untyped for a project's early stages for the sake of prototyping in a statically typed language would yield much practical gain.
> In this case, the interview problem was a simplified version of a thing the team gets asked to do. So I believe it was as representative as possible for an hour's coding.
That's exactly my point–code written under pressure during an hour-long code challenge will never be representative of code written during normal day-to-day coding. Even if the problem domain is similar to a real-world problem.
> I disagree that "real production code" is not throwaway. In situations of high requirements volatility, a great deal of production code ends up getting thrown away
But it's not written to be thrown away an hour later. That's not really the kind of 'production code' that any reasonable person would be talking about in this thread. If you're talking about exploratory data analysis, data science or whatever, sure, you can consider it 'production code', but it's not the product of the craft of software engineering, it's more about running quick experiments.
> I'm not sold that the more elaborate approach of being thoughtfully and explicitly untyped for a project's early stages for the sake of prototyping in a statically typed language would yield much practical gain.
There are many different kinds of projects. And people who work with good statically typed languages can prototype quite well using types to drive their domain modelling. It's a different way of working. You should read some of what Scott Wlaschin has written to get an idea.
That's exactly my point–code written under pressure during an hour-long code challenge will never be representative of code written during normal day-to-day coding. Even if the problem domain is similar to a real-world problem.
> I disagree that "real production code" is not throwaway. In situations of high requirements volatility, a great deal of production code ends up getting thrown away
But it's not written to be thrown away an hour later. That's not really the kind of 'production code' that any reasonable person would be talking about in this thread. If you're talking about exploratory data analysis, data science or whatever, sure, you can consider it 'production code', but it's not the product of the craft of software engineering, it's more about running quick experiments.
> I'm not sold that the more elaborate approach of being thoughtfully and explicitly untyped for a project's early stages for the sake of prototyping in a statically typed language would yield much practical gain.
There are many different kinds of projects. And people who work with good statically typed languages can prototype quite well using types to drive their domain modelling. It's a different way of working. You should read some of what Scott Wlaschin has written to get an idea.
> but it's not the product of the craft of software engineering
Sure? This seems pretty no-true-Scotsman to me. If you want to define some the things I need to do as not real software, feel free. But they're still things my team has to get done. To me there's a spectrum of domain stability on which the code rests. It's frequently not possible to know in advance which bits will be long lived and which won't be. So whether or not one of htose bits is the true "craft of software engineering" in somebody's eyes has very little interest to me. Especially when it's an anonymous rando.
> people who work with good statically typed languages can prototype quite well using types to drive their domain modelling.
I can certainly believe that's possible. Hopefully some of them will interview with me so I can see how they work. I'm just reporting the data I have.
> that's not really the kind of 'production code' that any reasonable person
In startup contexts I will quite often try an experiment that is quick to build. A few hours is not at all uncommon. That you consider me unreasonable is a sign I can better spend my time elsewhere.
Sure? This seems pretty no-true-Scotsman to me. If you want to define some the things I need to do as not real software, feel free. But they're still things my team has to get done. To me there's a spectrum of domain stability on which the code rests. It's frequently not possible to know in advance which bits will be long lived and which won't be. So whether or not one of htose bits is the true "craft of software engineering" in somebody's eyes has very little interest to me. Especially when it's an anonymous rando.
> people who work with good statically typed languages can prototype quite well using types to drive their domain modelling.
I can certainly believe that's possible. Hopefully some of them will interview with me so I can see how they work. I'm just reporting the data I have.
> that's not really the kind of 'production code' that any reasonable person
In startup contexts I will quite often try an experiment that is quick to build. A few hours is not at all uncommon. That you consider me unreasonable is a sign I can better spend my time elsewhere.
I have had the exact same experience. There's lots of utility in statically typed languages. They're great if your problem space is well defined. With respect to type checking, it's like a jig in wood or metal working. You trade flexibility for correctness.
When the problem space is less well defined the type-related boiler plate adds a lot of friction. It's not impossible to overcome that friction but it slows down progress. When you're under a tight deadline development velocity is often more valuable than absolute correctness or even overall runtime efficiency.
An delivered product that works is usually more valuable than an undelivered product that's more "correct" or efficient. A development project is just a cost (for various values of cost) until it ships.
When the problem space is less well defined the type-related boiler plate adds a lot of friction. It's not impossible to overcome that friction but it slows down progress. When you're under a tight deadline development velocity is often more valuable than absolute correctness or even overall runtime efficiency.
An delivered product that works is usually more valuable than an undelivered product that's more "correct" or efficient. A development project is just a cost (for various values of cost) until it ships.
Definitely. And for me the early stages of a product are often about buying information. "Users say they want X, so let's ship X and see." Key to exploring a product space is tight feedback loops between having an idea and seeing what people really do. It's only once I have enough active users (especially active paying users) to justify the project that I have some confidence about what "long term" really means for the code base.
"type-related boiler plate"
That phrase makes me sad. Mainstream languages have a lot of scope for improvement in their type systems.
That phrase makes me sad. Mainstream languages have a lot of scope for improvement in their type systems.
This is the best comment on the subject and should be at the top rather the current dogmatic ones.
I'm one of those people. But the reasons I use dynamic languages is in spite of them being dynamic. I choose Python because it has insane library support, I don't like Python as a language though. I would instantly choose Haskell if it had even half the available libraries. But I can't justify having to write everything myself and take 10x as long.
> The set of people using dynamic languages isn't just "those that haven't had their eyes opened yet to what static languages can do".
This was more aimed at people who are new to the idea of parsing over validating. In a strong statically typed language, the type system would naturally guide you to use this approach so if this isn't natural to you then time in other languages would probably be worthwhile.
This was more aimed at people who are new to the idea of parsing over validating. In a strong statically typed language, the type system would naturally guide you to use this approach so if this isn't natural to you then time in other languages would probably be worthwhile.
My misunderstanding then. Your comment came across (to me) as saying that people using dynamic language just don't know any better; that, if they would just learn a static language, they would suddenly understand the error of their ways. That's the thought that I was responding to.
> It's weird how long it's taking for people to rediscover why strong static types were a good idea.
I don't think it's weird. Most of those languages were not popular in industry for various reasons, and the ones that were (especially in say, the 90s) did not have particularly capable static type systems. The boilerplate/benefit ratio was all off.
The way I describe this dichotomy personally is, I would rather use Ruby than Java 1.5. I would rather use Rust than Ruby. (Java 1.5 is the last version of Java I have significant development experience in, and they've made the type system much more capable since those days.)
I don't think it's weird. Most of those languages were not popular in industry for various reasons, and the ones that were (especially in say, the 90s) did not have particularly capable static type systems. The boilerplate/benefit ratio was all off.
The way I describe this dichotomy personally is, I would rather use Ruby than Java 1.5. I would rather use Rust than Ruby. (Java 1.5 is the last version of Java I have significant development experience in, and they've made the type system much more capable since those days.)
Yeah, I remember I used to get frustrated when I had to read code that used map() or even .forEach() extensively, thinking a simple, imperative for loop would suffice. I slowly came to realize that a for loop gives you too much power. It's a hammer. It holds the place of a bug you just haven't written yet. Now I'm the one writing JavaScript like it's Haskell. Although Haskell could learn a thing or two from TypeScript about row polymorphism.
On the other end I'm endlessly tired of 'too simple' foreach/map iterators. They're OK until you want to do something like different execution on first and/or last element... Give me a way to implement a 'join' pattern over the foreach iterators, or less terse iterators (with 'some' positional information). I think I'm just ranting about the for-of iterator in Ada...
Ada provides a kind of iterator called a Cursor which could be used to build up a package of functions similar to the various C++ standard library algorithms. I believe this has actually already been done. Cursors can also be converted back to positional information if it makes sense (like with a Vector).
I like the cursor, because it abstracts indexes. Problem is, if I want the element, I still have to go fetch it (calling Element on the Cursor) and then I have to save it in a local variable and then I need to add a declare block for the constant standing for the result of Element and then the thing makes 5 lines instead of two and is not much more readable. Maybe I can just use Element() (especially since AdaCore seems to want to generalize dotted notation) repeatedly and have it inlined, but my past experience says 'expensive'...
And sadly Cursors haven't been generalized to all associative iterable structures in Ada (arrays for example).
And sadly Cursors haven't been generalized to all associative iterable structures in Ada (arrays for example).
I quite like the “enumerate” pattern. When indexes matter, instead of `for x in v` you would write, `for (i, x) in enumerate(v)`, then the language only needs one type of for loop as both cases use the same enumerator interface.
Yes I was thinking of something like that. Only I wish I could also know whether I is first and/or last without calling into the iterated structure. I know this looks like corner case syntactic sugar but it comes up a lot, e.g. when serializing to JSON. I guess I should write my own iterators but I want them everywhere...
Is using "row polymorphism" the same as using a "structutal type system"?
I never heard about the former.
I never heard about the former.
Not really, and correct me if I'm wrong but afaik TS doesn't actually do row polymorphism so much as structural subtyping - although the difference between the two is pretty small and you can get pretty close to row polymorphism with structural subtyping + generics.
But even if these were the same thing and we want to be a bit pendantic since this is HN after all, structural type systems often support some kind of subtyping or row polymorphism, but it's not a strict requirement for a type system to be "structural". You could have a structural type system that doesn't allow
But even if these were the same thing and we want to be a bit pendantic since this is HN after all, structural type systems often support some kind of subtyping or row polymorphism, but it's not a strict requirement for a type system to be "structural". You could have a structural type system that doesn't allow
{ a: int, b: int }
{ a: int }
> How practical such a type system would be... I don't know. Flow type checker for JavaScript makes a distinction between "exact" types, i.e. object must have exactly the properties listed and no more, and "inexact" types where such subtyping is allowed.
TypeScript doesn't have this check (https://github.com/Microsoft/TypeScript/issues/12936) and I've found it can be really error prone when you're wanting to return a copy of an object with some fields updated. Spot the bug in this example: https://www.typescriptlang.org/play?#code/C4TwDgpgBAKlC8UDeU...
TypeScript doesn't have this check (https://github.com/Microsoft/TypeScript/issues/12936) and I've found it can be really error prone when you're wanting to return a copy of an object with some fields updated. Spot the bug in this example: https://www.typescriptlang.org/play?#code/C4TwDgpgBAKlC8UDeU...
FWIW, you can achieve row polymorphism in TypeScript, although it's not super intuitive.
function rowPolymorphic<R extends { a: number }>(record: R): R & { a: string } {
return {
...record,
a: record.a.toString(),
}
}
const rec = rowPolymorphic({
a: 123,
b: "string",
})
console.log(rec.b)This matches my personal experience, for sure. I started out writing Python (pre-type-annotations) and JS, with lots of “raw” dicts/objects. For the past ~7 years though, I’ve written mostly Scala, but also a decent amount of TypeScript, Go and Java, and it’s completely transformed how I code, dramatically for the better.
Now, even in the rare case where I write some Python, JS or PHP, I write it in a very statically typed style, immediately parsing input into well-thought-out domain classes. And for backend services, I almost always go with 3 layers of models:
1) Data Transfer Objects. Map directly to the wire format, e.g. JSON or Protobuf. Generally auto-generated from API specs, e.g. using Open API Generator or protoc. A good API spec + code gen handles most input validation well
2) Domain Objects. Hand written, purely internal to the backend service, faithfully represent the domain. The domain layer of my code works exclusively with these. Sometimes there’s a little more validation when transforming a DTO into a domain model
3) Data Access Objects. Basically a representation of DB tables. Generally auto-generated from DB schemas, e.g. using libs like Prisma for TS or SQLBoiler for Go
Can’t imagine going back to the “everything is a dictionary” style for any decent sized project, it becomes such a mess so quickly. This style is a little more work up front, when you first write the code, but WAYYYYYY easier to maintain over time, fewer bugs and easier to modify quickly and confidently, with no nasty coupling of your domain models to either DB or wire format concerns. And code gen for the DTO and DAO layers makes it barely more up-front work.
Now, even in the rare case where I write some Python, JS or PHP, I write it in a very statically typed style, immediately parsing input into well-thought-out domain classes. And for backend services, I almost always go with 3 layers of models:
1) Data Transfer Objects. Map directly to the wire format, e.g. JSON or Protobuf. Generally auto-generated from API specs, e.g. using Open API Generator or protoc. A good API spec + code gen handles most input validation well
2) Domain Objects. Hand written, purely internal to the backend service, faithfully represent the domain. The domain layer of my code works exclusively with these. Sometimes there’s a little more validation when transforming a DTO into a domain model
3) Data Access Objects. Basically a representation of DB tables. Generally auto-generated from DB schemas, e.g. using libs like Prisma for TS or SQLBoiler for Go
Can’t imagine going back to the “everything is a dictionary” style for any decent sized project, it becomes such a mess so quickly. This style is a little more work up front, when you first write the code, but WAYYYYYY easier to maintain over time, fewer bugs and easier to modify quickly and confidently, with no nasty coupling of your domain models to either DB or wire format concerns. And code gen for the DTO and DAO layers makes it barely more up-front work.
> It's weird how long it's taking for people to rediscover why strong static types were a good idea.
Its weird how long its taken for languages with static typing and type systems designed for correctness (and designed well for that end) rather than princupally for convenience of compilation to be available that are generally usable (considering licensing model, features, ecosystem, etc.)
Its weird how long its taken for languages with static typing and type systems designed for correctness (and designed well for that end) rather than princupally for convenience of compilation to be available that are generally usable (considering licensing model, features, ecosystem, etc.)
"It's weird how long it's taking for people to rediscover why strong static types were a good idea."
It sounds like you're projecting your limited experience eg you've dismissed all Ms dev. Many people have been using strong static for decades. The benefits have never been out of sight. Many use dynamic, but many have always used strong static.
> > IME, people in dynamic languages almost never program this way, though—they prefer to use validation
I wonder how many people the author met.
I wonder how many people the author met.
Every programming paradigm is a good idea if the respective trade-offs are acceptable to you.
For example, one good reason why strong static types are a bad idea... they prevent you from implementing dynamic dispatch.
Routers. You can't have routers.
For example, one good reason why strong static types are a bad idea... they prevent you from implementing dynamic dispatch.
Routers. You can't have routers.
Are you sure you know what dynamic dispatch is? Java has dynamic dispatch, and it is a statically typed language. In Java, it's often called "runtime polymorphism".
https://www.geeksforgeeks.org/dynamic-method-dispatch-runtim...
And using it doesn't give up any of Java's type safety guarantees. The arguments and return type of the method you call (which will be invoked with dynamic dispatch) are type checked.
https://www.geeksforgeeks.org/dynamic-method-dispatch-runtim...
And using it doesn't give up any of Java's type safety guarantees. The arguments and return type of the method you call (which will be invoked with dynamic dispatch) are type checked.
In English there seems to be the eternal confusion between what things are and what we call them.
When a router does lookups from a static table it's "static routing".
When Java does lookups from a static table it's "dynamic dispatch".
The same type of computation is being characterised as both "static" and "dynamic".
When a router does lookups from a dynamic table it's "dynamic routing" - there is no equivalent in Java because making the dispatch table reflexive/mutable is precisely what violates type-safety!
When a router does lookups from a static table it's "static routing".
When Java does lookups from a static table it's "dynamic dispatch".
The same type of computation is being characterised as both "static" and "dynamic".
When a router does lookups from a dynamic table it's "dynamic routing" - there is no equivalent in Java because making the dispatch table reflexive/mutable is precisely what violates type-safety!
I don't think this analogy quite holds together. A router doing lookups from a table is implementing a static routing strategy from a control plane perspective; it's using statically configured values instead of using dynamic information about the network topology gleaned using a routing protocol like BGP. But an implementation of that strategy in terms of table lookups is dynamic—it's walking a data structure to retrieve values which were specified in the runtime configuration, not at compile time.
The reason that "dynamic dispatch" in Java, etc. is called that is that the instance of the dispatch table to use is chosen dynamically, rather than being fixed at the callsite. It's true that Java doesn't let the shape of the dispatch table change at runtime, but that's not what dynamic vs static refers to conventionally in this context. The ability to dynamically add and remove methods from a class is something which you typically only get in dynamically typed languages but dynamic dispatch and dynamic typing are not the same thing.
In particular, while a full-featured routing information base implementation will usually use some form of dynamic dispatch to customize the behavior of routes originated through different protocols, it's very uncommon for an implementation to rely on dynamic typing which adds or mutates the methods associated with different entities. That's simply a different kind of tool used for different purposes. It's something which can be helpful in object-relational mapping, for instance, because you can create methods based on a dynamic database schema. The RIB is not going to have a schema like that which changes at runtime.
The reason that "dynamic dispatch" in Java, etc. is called that is that the instance of the dispatch table to use is chosen dynamically, rather than being fixed at the callsite. It's true that Java doesn't let the shape of the dispatch table change at runtime, but that's not what dynamic vs static refers to conventionally in this context. The ability to dynamically add and remove methods from a class is something which you typically only get in dynamically typed languages but dynamic dispatch and dynamic typing are not the same thing.
In particular, while a full-featured routing information base implementation will usually use some form of dynamic dispatch to customize the behavior of routes originated through different protocols, it's very uncommon for an implementation to rely on dynamic typing which adds or mutates the methods associated with different entities. That's simply a different kind of tool used for different purposes. It's something which can be helpful in object-relational mapping, for instance, because you can create methods based on a dynamic database schema. The RIB is not going to have a schema like that which changes at runtime.
>But an implementation of that strategy in terms of table lookups is dynamic—it's walking a data structure to retrieve values which were specified in the runtime configuration, not at compile time.
That's precisely the point. You can specify part of the routing table at compile/configure time - the rest gets generated at runtime.
The data/control plane distinction is conceptual. It doesn't hold in memory when the router is handling its own network traffic - it has a single routing table/world-view.
My own routing table is shared by the data plane AND control plane.
At some point you will receive an external data (routing update) which requires runtime validation, you will do reflection and update your own routing table (ring 0 address space) based on external events.
>The RIB is not going to have a schema like that which changes at runtime.
The schema need not change. The entries/relations between objects changing is sufficient to violate type-safety.
Route add( *str1, *str2) to Number.add().
That's precisely the point. You can specify part of the routing table at compile/configure time - the rest gets generated at runtime.
The data/control plane distinction is conceptual. It doesn't hold in memory when the router is handling its own network traffic - it has a single routing table/world-view.
My own routing table is shared by the data plane AND control plane.
At some point you will receive an external data (routing update) which requires runtime validation, you will do reflection and update your own routing table (ring 0 address space) based on external events.
>The RIB is not going to have a schema like that which changes at runtime.
The schema need not change. The entries/relations between objects changing is sufficient to violate type-safety.
Route add( *str1, *str2) to Number.add().
I mean, there's ample evidence on this thread to suggest we're not going to reach a productive conclusion here but I guess I'll keep biting.
It's not clear to me if you are suggesting that a dynamic routing table with different kinds of routes cannot be implemented in a type-safe manner in a statically typed language, or if you're working with an analogy where a routing table is like a dynamic programming language at runtime, in that a static set of entities and relations are known ahead of time and those are modified by runtime input. If it's the latter I'm not really sure how the analogy works—if programming languages are to routers as types are to routing entries, what in a router is analogous to a value of a given type?
I can speak more to the former possibility; here's a rough sketch of how one could implement a routing table using the tools available in a statically typed environment (and in a type-safe way). One way to do it (I believe the common way, and certainly the only one I've seen in commercial router implementations) is to treat statically populated and dynamically learned routes more or less uniformly in the data structures used to perform data-plane lookups. Each such route entry has the same fields and gets inserted into a data structure with a predefined shape. Where special behavior is needed for routes of different kinds, that behavior can be implemented by using dynamic dispatch in the sense it's usually used in C++, Java, Rust, etc. to call a method associated with a route entry, or using other techniques common to statically compiled languages—there is a fixed set of such operations defined up front. Adding and removing entries from the routing table at runtime does not typically implicate type safety because the types used to describe the table describe all of its possible valid states. For instance, the type for a node in a radix trie might describe how it can either be leaf node or contain subnodes, etc.
> The schema need not change. The entries/relations between objects changing is sufficient to violate type-safety. > > Route add( str1, str2) to Number.add().
It's obviously not always true that entries or relations changing will validate type safety; any non-trivial system will let you perform some kinds of data manipulation at runtime. Conventional static type systems will allow some kinds of mutations (like changing around pointers in a radix trie to insert a new node) but will not have the flexibility to support some others (like changing the shape of a dispatch table at runtime).
One kind of call pattern which is incompatible with statically compiled dynamic dispatch is where the types of parameters change along with the base type which owns the dispatch table; I think this is what your add() example is getting at—you need the type of the second parameter to match the first, which you can't validate without runtime checks if you don't know what concrete implementations will be in use at runtime. In the case of a routing table I don't think this kind of polymorphism is needed though; I can't think of an instance where an operation would fundamentally require a fixed relation in the concrete types of different routes. For instance, when routes overlap you can derive a priority value for each one to decide which one to use, rather than directly implementing some kind of function whichIsBetter(a, b) which relies on knowing what concrete route kinds a and b are.
It's not clear to me if you are suggesting that a dynamic routing table with different kinds of routes cannot be implemented in a type-safe manner in a statically typed language, or if you're working with an analogy where a routing table is like a dynamic programming language at runtime, in that a static set of entities and relations are known ahead of time and those are modified by runtime input. If it's the latter I'm not really sure how the analogy works—if programming languages are to routers as types are to routing entries, what in a router is analogous to a value of a given type?
I can speak more to the former possibility; here's a rough sketch of how one could implement a routing table using the tools available in a statically typed environment (and in a type-safe way). One way to do it (I believe the common way, and certainly the only one I've seen in commercial router implementations) is to treat statically populated and dynamically learned routes more or less uniformly in the data structures used to perform data-plane lookups. Each such route entry has the same fields and gets inserted into a data structure with a predefined shape. Where special behavior is needed for routes of different kinds, that behavior can be implemented by using dynamic dispatch in the sense it's usually used in C++, Java, Rust, etc. to call a method associated with a route entry, or using other techniques common to statically compiled languages—there is a fixed set of such operations defined up front. Adding and removing entries from the routing table at runtime does not typically implicate type safety because the types used to describe the table describe all of its possible valid states. For instance, the type for a node in a radix trie might describe how it can either be leaf node or contain subnodes, etc.
> The schema need not change. The entries/relations between objects changing is sufficient to violate type-safety. > > Route add( str1, str2) to Number.add().
It's obviously not always true that entries or relations changing will validate type safety; any non-trivial system will let you perform some kinds of data manipulation at runtime. Conventional static type systems will allow some kinds of mutations (like changing around pointers in a radix trie to insert a new node) but will not have the flexibility to support some others (like changing the shape of a dispatch table at runtime).
One kind of call pattern which is incompatible with statically compiled dynamic dispatch is where the types of parameters change along with the base type which owns the dispatch table; I think this is what your add() example is getting at—you need the type of the second parameter to match the first, which you can't validate without runtime checks if you don't know what concrete implementations will be in use at runtime. In the case of a routing table I don't think this kind of polymorphism is needed though; I can't think of an instance where an operation would fundamentally require a fixed relation in the concrete types of different routes. For instance, when routes overlap you can derive a priority value for each one to decide which one to use, rather than directly implementing some kind of function whichIsBetter(a, b) which relies on knowing what concrete route kinds a and b are.
>Where special behavior is needed for routes of different kinds, that behavior can be implemented by using dynamic dispatch in the sense it's usually used.
How? A routing function is precisely a M:N mapping with untyped input.
It's just some data somebody sent you! Route it.
Unless you can infer more about the meaning of those bits the only strategies possible are static(M), round-robin(M) or random(M).
How? A routing function is precisely a M:N mapping with untyped input.
It's just some data somebody sent you! Route it.
Unless you can infer more about the meaning of those bits the only strategies possible are static(M), round-robin(M) or random(M).
The conclusion was my very first post really. Understand the limits of your tools; and your own limits - then make acceptable trade offs.
Programming is about explaining desires to computers. It is a complex, error-prone activity and we depend on our automated tooling to save us from ourselves.
Obviously, in theory you can build the exact same software in Assembly as you can with Java. But not in practice because the programming language is a human-computer interface and a symbiosis with useful feedback loops emerges at higher levels of abstraction.
The design choices of your language impose certain limits/discipline on your expressive power.
Some design patterns become easy to express; where others become difficult to express and your paradigm’s features become obstacles.
At which point it is your choice to switch off the guard rails.
Like the chap somewhere down below telling me he will use Rust “unsafe” so he can have direct control of his memory. Yes! You can! You can also read/write to /dev/mem directly!
Translated in English “I will turn off the compiler’s safety checks because it is getting in my way”. There are many ways to shoot yourself in the foot with this level of power.
He is agreeing with me, but this is HN and opposition must be maintained. Dung must be flung. For reasons.
Sure, for certain kinds of protocols this kind of polymorphism is not always needed. But I am merely pointing out the threshold where the type-safe paradigm begins to falter. The corner cases where the pros don’t outweigh the cons. Where the “compile time” and “runtime” distinction becomes a hindrance rather than a useful separation of concerns.
You can go further. Software Defined Networks. Application router.
In general, contexts in which it would be useful to know the shape of your data at runtime!
As a spontaneous aside… if your entire information-processing system is well-typed (top to bottom) - congratulations. You are well on your way to understanding Category Theory.
The entire data-schema of your stack is what Mathemaricians call a (small) “Category”.
Programming is about explaining desires to computers. It is a complex, error-prone activity and we depend on our automated tooling to save us from ourselves.
Obviously, in theory you can build the exact same software in Assembly as you can with Java. But not in practice because the programming language is a human-computer interface and a symbiosis with useful feedback loops emerges at higher levels of abstraction.
The design choices of your language impose certain limits/discipline on your expressive power.
Some design patterns become easy to express; where others become difficult to express and your paradigm’s features become obstacles.
At which point it is your choice to switch off the guard rails.
Like the chap somewhere down below telling me he will use Rust “unsafe” so he can have direct control of his memory. Yes! You can! You can also read/write to /dev/mem directly!
Translated in English “I will turn off the compiler’s safety checks because it is getting in my way”. There are many ways to shoot yourself in the foot with this level of power.
He is agreeing with me, but this is HN and opposition must be maintained. Dung must be flung. For reasons.
Sure, for certain kinds of protocols this kind of polymorphism is not always needed. But I am merely pointing out the threshold where the type-safe paradigm begins to falter. The corner cases where the pros don’t outweigh the cons. Where the “compile time” and “runtime” distinction becomes a hindrance rather than a useful separation of concerns.
You can go further. Software Defined Networks. Application router.
In general, contexts in which it would be useful to know the shape of your data at runtime!
As a spontaneous aside… if your entire information-processing system is well-typed (top to bottom) - congratulations. You are well on your way to understanding Category Theory.
The entire data-schema of your stack is what Mathemaricians call a (small) “Category”.
> I can't think of an instance where an operation would fundamentally require a fixed relation in the concrete types of different routes. For instance, when routes overlap you can derive a priority value for each one to decide which one to use, rather than directly implementing some kind of function whichIsBetter(a, b) which relies on knowing what concrete route kinds a and b are.
Any routing logic which makes routing decisions based on information nested arbitrarily deep into your packets.
Is this an IP packet containing data; or an IP packet containing an HTTP GET request to www.google.com?
You have to do arbitrarily deep type inference in real time.
The the data has no semantic content until you infer some! Is just a bitstream!
Any routing logic which makes routing decisions based on information nested arbitrarily deep into your packets.
Is this an IP packet containing data; or an IP packet containing an HTTP GET request to www.google.com?
You have to do arbitrarily deep type inference in real time.
The the data has no semantic content until you infer some! Is just a bitstream!
>If it's the latter I'm not really sure how the analogy works—if programming languages are to routers as types are to routing entries, what in a router is analogous to a value of a given type?
It's not an analogy. Programming languages and routers are particular instances of computable functions. "dispatching" and "routing" are just another example of us using different English words to describe the exact same computation: M:N mapping function.
Whether the input is mapped to an IP address or a memory address - boring implementation details.
Nothing in a router is analogous to a value of a type because there is no such thing as "types" at runtime unless you infer them! Types exist only at compile time. Types are semantic annotations of data. You are helping the compiler help you by telling it what you know about the data you are handling.
This blob encodes a Number. That blob encodes a String.
If you don't want any help from your compiler, you don't have to tell it the types of anything - just manipulate the data directly!
That's precisely what an Assembly language do. Everything is untyped.
It's not an analogy. Programming languages and routers are particular instances of computable functions. "dispatching" and "routing" are just another example of us using different English words to describe the exact same computation: M:N mapping function.
Whether the input is mapped to an IP address or a memory address - boring implementation details.
Nothing in a router is analogous to a value of a type because there is no such thing as "types" at runtime unless you infer them! Types exist only at compile time. Types are semantic annotations of data. You are helping the compiler help you by telling it what you know about the data you are handling.
This blob encodes a Number. That blob encodes a String.
If you don't want any help from your compiler, you don't have to tell it the types of anything - just manipulate the data directly!
That's precisely what an Assembly language do. Everything is untyped.
Are you sure you know when type-checking and when dynamic dispatching happens?
Compile time is not runtime.
The Java compiler is not the JVM.
The compiler does type checking. The JVM does the dynamic dispatching.
Neither does both.
Compile time is not runtime.
The Java compiler is not the JVM.
The compiler does type checking. The JVM does the dynamic dispatching.
Neither does both.
All those statements are correct. The people downvoting you know that too. I don't think anyone has figured out what point you're trying to make, though. Could you spell it out in more detail?
Consider addition. The compiler does type checking, and the JVM actually adds the numbers. Nonetheless, the addition is type checked, and does not represent a weakness of static type checking. How is dynamic dispatch different than this?
Consider addition. The compiler does type checking, and the JVM actually adds the numbers. Nonetheless, the addition is type checked, and does not represent a weakness of static type checking. How is dynamic dispatch different than this?
The point is trivial.
You can’t have both static type safety AND dynamic dispatch at the same time and in the same context about the same data.
Choose one. Give up the other. Make a conscious trade off.
The language that you are using is making such trade offs for you - they are implicit in the language design. Best you know what they are because they are meaningful in principle and in practice.
You can’t have both static type safety AND dynamic dispatch at the same time and in the same context about the same data.
Choose one. Give up the other. Make a conscious trade off.
The language that you are using is making such trade offs for you - they are implicit in the language design. Best you know what they are because they are meaningful in principle and in practice.
Java has both static type safety AND dynamic dispatch at the same time and in the same context about the same data.
No, it doesn’t.
The input-data to the compiler can’t be handled by the JVM and vice versa.
The JVM handles byte code as input. The compiler handles source code as input.
That is two different functions with two different data domains.
They literally have different types!
Which one of the two functions is the thing you call “Java”?
The input-data to the compiler can’t be handled by the JVM and vice versa.
The JVM handles byte code as input. The compiler handles source code as input.
That is two different functions with two different data domains.
They literally have different types!
Which one of the two functions is the thing you call “Java”?
It's pretty clear at this point you don't understand what dynamic dispatch means.
I don't think it's worthwhile for anyone else to argue with you further. C++ and Java both support dynamic dispatch although you deny it.
You've taken up almost a full page of HN arguing with everyone trying to explain it to you. People have pointed you to wikipedia showing you that you're wrong. [1]
ISOCPP of which Bjarne is a director [2] says that C++ supports dynamic dispatch. [3]
And you continue to attempt to argue that everyone on HN, Wikipedia and the creator of the C++ language are all wrong and don't know what dynamic dispatch is.
Your continued insistence is both wrong and a negative impact at this point on hn. Please stop arguing something that numerous people have taken lots of patience and charity in trying every way possible to explain to you and what is clearly factually wrong.
If you're going to reply, please explain why an organization that Bjarne Stroustrup is a director of believes that C++ supports dynamic dispatch.
1. https://en.wikipedia.org/wiki/Dynamic_dispatch#C++_implement...
2 https://isocpp.org/about
3. https://isocpp.org/wiki/faq/big-picture#why-use-oo
I don't think it's worthwhile for anyone else to argue with you further. C++ and Java both support dynamic dispatch although you deny it.
You've taken up almost a full page of HN arguing with everyone trying to explain it to you. People have pointed you to wikipedia showing you that you're wrong. [1]
ISOCPP of which Bjarne is a director [2] says that C++ supports dynamic dispatch. [3]
And you continue to attempt to argue that everyone on HN, Wikipedia and the creator of the C++ language are all wrong and don't know what dynamic dispatch is.
Your continued insistence is both wrong and a negative impact at this point on hn. Please stop arguing something that numerous people have taken lots of patience and charity in trying every way possible to explain to you and what is clearly factually wrong.
If you're going to reply, please explain why an organization that Bjarne Stroustrup is a director of believes that C++ supports dynamic dispatch.
1. https://en.wikipedia.org/wiki/Dynamic_dispatch#C++_implement...
2 https://isocpp.org/about
3. https://isocpp.org/wiki/faq/big-picture#why-use-oo
> It's pretty clear at this point you don't understand what dynamic dispatch means.
Terms in computing are so overloaded that these days I try[0] and never correct anyone on how they use a term. Instead I ask them to define it, and debate off of that definition.
So instead of downvoting this guy for using different terminology - we can ask him what he means and just have a discussion.
[0] alright I don't always succeed but it's an ideal to strive for
Terms in computing are so overloaded that these days I try[0] and never correct anyone on how they use a term. Instead I ask them to define it, and debate off of that definition.
So instead of downvoting this guy for using different terminology - we can ask him what he means and just have a discussion.
[0] alright I don't always succeed but it's an ideal to strive for
Dynamic dispatch is not terribly overloaded. It's dispatching based on run-time information instead of just compile-time information.
The problem in this discussion is that ukj has come to the belief (but communicated it poorly) that dynamic dispatch is somehow incompatible with static typing. And for some reason this also matters.
Static typing does not preclude dynamic dispatch, and despite being pointed to several mainstream languages that have both features, ukj decided to ignore reality or the common understanding of the phrase "dynamic dispatch" and produced this grotesque example of trying to communicate with an individual who is, apparently, just a troll. Feeding the troll, ukj, is probably the dumbest thing I did today, but I'll blame that on the insomnia reducing my ability to detect trolls.
The problem in this discussion is that ukj has come to the belief (but communicated it poorly) that dynamic dispatch is somehow incompatible with static typing. And for some reason this also matters.
Static typing does not preclude dynamic dispatch, and despite being pointed to several mainstream languages that have both features, ukj decided to ignore reality or the common understanding of the phrase "dynamic dispatch" and produced this grotesque example of trying to communicate with an individual who is, apparently, just a troll. Feeding the troll, ukj, is probably the dumbest thing I did today, but I'll blame that on the insomnia reducing my ability to detect trolls.
>Dynamic dispatch is not terribly overloaded. It's dispatching based on run-time information instead of just compile-time information.
According to the above definition C++ does not have dynamic dispatch!
The vtable in C++ is generated at compile time and used at runtime. It's immutable at runtime. That means it is NOT dynamic!
https://en.wikipedia.org/wiki/Virtual_method_table#Compariso...
"Virtual method tables also only work if dispatching is constrained to a known set of methods, so they can be placed in a simple array built at compile time."
Like, I don't care how you use words, but you (and everyone) are using "dynamic" to speak about a statically-behaving system!
According to the above definition C++ does not have dynamic dispatch!
The vtable in C++ is generated at compile time and used at runtime. It's immutable at runtime. That means it is NOT dynamic!
https://en.wikipedia.org/wiki/Virtual_method_table#Compariso...
"Virtual method tables also only work if dispatching is constrained to a known set of methods, so they can be placed in a simple array built at compile time."
Like, I don't care how you use words, but you (and everyone) are using "dynamic" to speak about a statically-behaving system!
This is a commendable approach.
Computation is a general, abstract and incredibly useful idea disconnected from any particular model of computation (programming language).
Different languages are just different models of computation and have different (desirable, or undesirable) semantic properties independent from their (trivial) syntactic properties.
It's this sort of angry dogmatism which prevents people from talking about programming language design.
Not for a second do they pause to think their own understanding may be limited.
Computation is a general, abstract and incredibly useful idea disconnected from any particular model of computation (programming language).
Different languages are just different models of computation and have different (desirable, or undesirable) semantic properties independent from their (trivial) syntactic properties.
It's this sort of angry dogmatism which prevents people from talking about programming language design.
Not for a second do they pause to think their own understanding may be limited.
> It's this sort of angry dogmatism which prevents people from talking about programming language design.
The dogmatism demonstrated today was in your comments, ukj. Your inability to understand that your non-standard use of terms makes it impossible for others to communicate with you in any effective way made this a remarkable farce of a conversation or debate.
The dogmatism demonstrated today was in your comments, ukj. Your inability to understand that your non-standard use of terms makes it impossible for others to communicate with you in any effective way made this a remarkable farce of a conversation or debate.
>If you're going to reply, please explain why an organization that Bjarne Stroustrup is a director of believes that C++ supports dynamic dispatch.
Because the meaning of "dynamic" is ambigous!
Since you are pointing me to wikipedia I'll point you right back...
https://en.wikipedia.org/wiki/Virtual_method_table#Compariso...
"Virtual method tables also only work if dispatching is constrained to a known set of methods, so they can be placed in a simple array built at compile time."
If the vtable is generated at compile and is constrained to a known set of methods then that array is immutable! Calling that "dynamic" is an obvious misnomer!
You are neither charitable nor patient. You are committing the bandwagon fallacy as we speak. You and the other 120 (and counting) angry downvoters ;)
I am using the word "dynamic" to actually mean dynamic! I am not going to define it. Use your judgment. Dynamic is NOT static. I am not asking you to "educate me", or to tell me I am right; or wrong. I am asking you to understand the sort of programming language design I have in mind!
Either you understand; or you don't.
Because the meaning of "dynamic" is ambigous!
Since you are pointing me to wikipedia I'll point you right back...
https://en.wikipedia.org/wiki/Virtual_method_table#Compariso...
"Virtual method tables also only work if dispatching is constrained to a known set of methods, so they can be placed in a simple array built at compile time."
If the vtable is generated at compile and is constrained to a known set of methods then that array is immutable! Calling that "dynamic" is an obvious misnomer!
You are neither charitable nor patient. You are committing the bandwagon fallacy as we speak. You and the other 120 (and counting) angry downvoters ;)
I am using the word "dynamic" to actually mean dynamic! I am not going to define it. Use your judgment. Dynamic is NOT static. I am not asking you to "educate me", or to tell me I am right; or wrong. I am asking you to understand the sort of programming language design I have in mind!
Either you understand; or you don't.
A type system is sound when:
for all expressions e:
if e type checks with type t, then one of the following holds:
- e evaluates to a value v of type t; or
- e does not halt; or
- e hits an "unavoidable error"
like division by 0 or null deref
(what counts as "unavoidable" varies from lang to lang)
Yes this is precisely what I am talking about.
Different implementations of eval() a.k.a different programming languages have different semantics.
Which implicit eval() implementation does the above hold?
What type system do you have in mind such that the above holds for ALL expressions. Type-checking itself is not always decidable.
Different implementations of eval() a.k.a different programming languages have different semantics.
Which implicit eval() implementation does the above hold?
What type system do you have in mind such that the above holds for ALL expressions. Type-checking itself is not always decidable.
Huh? Could you give a specific example? Because e.g. C++ and Rust definitely have dynamic dispatch through their vtable mechanisms.
Do you understand the difference between compile time and runtime?
Neither C++ nor Rust give you static type safety AND dynamic dispatch because all of the safety checks for C++ and Rust happen at compile time. Not runtime.
Neither C++ nor Rust give you static type safety AND dynamic dispatch because all of the safety checks for C++ and Rust happen at compile time. Not runtime.
This is sort of a perplexing perspective to me. It seems tantamount to saying “you can’t predict whether a value will be a string or a number AND have static type safety because the value only exists at runtime, and static type safety only happens at compile-time.” Yes, obviously static typechecking happens at compile-time, but type systems are carefully designed so that the compile-time reasoning says something useful about what actually occurs at runtime—that is, after all, the whole point!
Focusing exclusively on what happens at compile-time is to miss the whole reason static type systems are useful in the first place: they allow compile-time reasoning about runtime behavior. Just as we can use a static type system to make predictions about programs that pass around first-class functions via static dispatch, we can also use them to make predictions about programs that use vtables or other constructions to perform dynamic dispatch. (Note that the difference between those two things isn’t even particularly well-defined; a call to a first-class function passed as an argument is a form of unknown call, and it is arguably a form of dynamic dispatch.)
Lots of statically typed languages provide dynamic dispatch. In fact, essentially all mainstream ones do: C++, Java, C#, Rust, TypeScript, even modern Fortran. None of these implementations require sacrificing static type safety in any way; rather, type systems are designed to ensure such dispatch sites are well-formed in other ways, without restricting their dynamic nature. And this is entirely in line with the OP, as there is no tension whatsoever between the techniques it describes and dynamic dispatch.
Focusing exclusively on what happens at compile-time is to miss the whole reason static type systems are useful in the first place: they allow compile-time reasoning about runtime behavior. Just as we can use a static type system to make predictions about programs that pass around first-class functions via static dispatch, we can also use them to make predictions about programs that use vtables or other constructions to perform dynamic dispatch. (Note that the difference between those two things isn’t even particularly well-defined; a call to a first-class function passed as an argument is a form of unknown call, and it is arguably a form of dynamic dispatch.)
Lots of statically typed languages provide dynamic dispatch. In fact, essentially all mainstream ones do: C++, Java, C#, Rust, TypeScript, even modern Fortran. None of these implementations require sacrificing static type safety in any way; rather, type systems are designed to ensure such dispatch sites are well-formed in other ways, without restricting their dynamic nature. And this is entirely in line with the OP, as there is no tension whatsoever between the techniques it describes and dynamic dispatch.
You must be strawmanning my position to make this comment.
Obviously static type systems are useful. I don't even think my point is contrary to anything you are saying. This is not being said as way of undermining any particular paradigm because computation is universal - the models of computation on the other hand (programming languages) are not all “the same”. There are qualitative differences.
Every single programming paradigm is a self-imposed restriction of some sort. It is precisely this restriction that we deem useful because they prevent us from shooting off our feet with shotguns. And we also prevent ourselves from being able to express certain patterns (of course we can deliberately/explicitly turn off the self-imposed restriction! ).
Like the restriction you are posing on your self is explicit in "type systems are carefully designed so that the compile-time reasoning says something useful about what actually occurs at runtime"
If you could completely determine everything that happens at runtime you wouldn't need exception/error handling!
All software would be 100% deterministic.
And it isn't.
I can say nothing of the structure of random bitstreams from unknown sources. I only know what I EXPECT them to be. Not what they actually are.
In this context parsing untrusted data IS runtime type-checking.
Obviously static type systems are useful. I don't even think my point is contrary to anything you are saying. This is not being said as way of undermining any particular paradigm because computation is universal - the models of computation on the other hand (programming languages) are not all “the same”. There are qualitative differences.
Every single programming paradigm is a self-imposed restriction of some sort. It is precisely this restriction that we deem useful because they prevent us from shooting off our feet with shotguns. And we also prevent ourselves from being able to express certain patterns (of course we can deliberately/explicitly turn off the self-imposed restriction! ).
Like the restriction you are posing on your self is explicit in "type systems are carefully designed so that the compile-time reasoning says something useful about what actually occurs at runtime"
If you could completely determine everything that happens at runtime you wouldn't need exception/error handling!
All software would be 100% deterministic.
And it isn't.
I can say nothing of the structure of random bitstreams from unknown sources. I only know what I EXPECT them to be. Not what they actually are.
In this context parsing untrusted data IS runtime type-checking.
I think you might need to define what you mean by dynamic dispatch, because it is very clearly something totally different than how the term is commonly understood.
Deciding which implementation of a function handles any given piece of data at runtime.
Trivially, because you don’t have this knowledge (and therefore you can’t encode it into your type system) at compile time.
Trivially, because you don’t have this knowledge (and therefore you can’t encode it into your type system) at compile time.
Aha! I think I have debugged your thinking. Wow you made that hard by arguing so much.
Apparently you do know what dynamic dispatch is, you're just wrong that it can't be type checked.
In Java, say you have an interface called `Foo` with a method `String foo()`, and two classes A and B that implement that method. Then you can write this code (apologies if the syntax isn't quite right, it's been a while since I've written Java):
Apparently you do know what dynamic dispatch is, you're just wrong that it can't be type checked.
In Java, say you have an interface called `Foo` with a method `String foo()`, and two classes A and B that implement that method. Then you can write this code (apologies if the syntax isn't quite right, it's been a while since I've written Java):
Foo foo = null;
if (random_boolean()) {
foo = new A();
} else {
foo = new B();
}
// This uses dynamic dispatch
System.out.println(foo.foo())
So, there is nothing dynamic about that dispatch.
Because the implementation details of Foo are actually know at compile time. Which is why you are able to type-check it.
You have literally declared all allowed (but not all possible) implementations of Foo.
What happens when Foo() is a remote/network call?
Because the implementation details of Foo are actually know at compile time. Which is why you are able to type-check it.
You have literally declared all allowed (but not all possible) implementations of Foo.
What happens when Foo() is a remote/network call?
That is not what dynamic dispatch means! It is an extremely well established term, with a very clear meaning, and that is not what it means.
I thought you were just mistaken about something, but no, instead you've redefined a well understood term without telling anyone, then aggressively refused to clarify what you meant by it and argued for hours with people, while saying they were all wrong when they used the well established term to mean its well established meaning.
The thing you're talking about is an interesting concept, but it's not called dynamic dispatch, and you will confuse everyone you talk to if you call it that. I don't know if there's a term for it.
I thought you were just mistaken about something, but no, instead you've redefined a well understood term without telling anyone, then aggressively refused to clarify what you meant by it and argued for hours with people, while saying they were all wrong when they used the well established term to mean its well established meaning.
The thing you're talking about is an interesting concept, but it's not called dynamic dispatch, and you will confuse everyone you talk to if you call it that. I don't know if there's a term for it.
It turns out "dynamic dispatch" is not as well established or as clear as you insist. It means at least one of five things:
0. Dynamic dispatch (as you are using it) 1. Double dispatch 2. Multiple dispatch 3. Predicate dispatch 4. All of the above collectively.
What you thought was a confusing re-definition on my part might have been an ignorance of alternative uses on yours ;)
Java doesn't do 1, 2 or 3.
0. Dynamic dispatch (as you are using it) 1. Double dispatch 2. Multiple dispatch 3. Predicate dispatch 4. All of the above collectively.
What you thought was a confusing re-definition on my part might have been an ignorance of alternative uses on yours ;)
Java doesn't do 1, 2 or 3.
“Well established” doesn’t mean anything.
According to who?
Computer scientists talk about “well formed” not “well established”.
Those are categorical definitions.
According to who?
Computer scientists talk about “well formed” not “well established”.
Those are categorical definitions.
> According to who?
Wikipedia, every textbook you can find, the top dozen search results for "dynamic dispatch", me who has a PhD in computer science plus all the other CS PhD people I know, everyone in my office who knows the term (who are industry people, not academia people), every blog post I have ever read that uses the term, and all the other HN commenters except you. I'm really not exaggerating; a lot of CS terms have disputed meanings but not this one.
EDIT: Sorry all for engaging the troll. I thought there might have been some legitimate confusion. My bad.
Wikipedia, every textbook you can find, the top dozen search results for "dynamic dispatch", me who has a PhD in computer science plus all the other CS PhD people I know, everyone in my office who knows the term (who are industry people, not academia people), every blog post I have ever read that uses the term, and all the other HN commenters except you. I'm really not exaggerating; a lot of CS terms have disputed meanings but not this one.
EDIT: Sorry all for engaging the troll. I thought there might have been some legitimate confusion. My bad.
There is a legitimate confusion indeed and it seems to be on your part!
What I am talking about when I say "dynamic dispatch" is the sort of dispatching done by R, LISP and Julia (and not by C++ or Java). Now, we can bicker about labels and you can insist that it's actually called "multiple dispatch" and not "dynamic dispatch", but you can't bicker about the semantic fact that "multiple dispatch" is actually more dynamic than "dynamic dispatch".
And this semantic point would've been trivial to unpack if you weren't try to win an argument.
Indeed, sorry for engaging the trolls. 20; or 30 of you. Lost count.
What I am talking about when I say "dynamic dispatch" is the sort of dispatching done by R, LISP and Julia (and not by C++ or Java). Now, we can bicker about labels and you can insist that it's actually called "multiple dispatch" and not "dynamic dispatch", but you can't bicker about the semantic fact that "multiple dispatch" is actually more dynamic than "dynamic dispatch".
And this semantic point would've been trivial to unpack if you weren't try to win an argument.
Indeed, sorry for engaging the trolls. 20; or 30 of you. Lost count.
So which textbook contains the meaning of "meaning"?
Oh, that's recursive! Which is Computer Science's domain of expertise, not the public domain.
We are talking about formal semantics here. What do programs (and computer languages are themselves programs) mean?
Point 0 of Wadler's law.
https://en.wikipedia.org/wiki/Semantics_(computer_science)
If you can type-check it at compile time then it is NOT dynamic dispatch. It's a contextual confusion.
Oh, that's recursive! Which is Computer Science's domain of expertise, not the public domain.
We are talking about formal semantics here. What do programs (and computer languages are themselves programs) mean?
Point 0 of Wadler's law.
https://en.wikipedia.org/wiki/Semantics_(computer_science)
If you can type-check it at compile time then it is NOT dynamic dispatch. It's a contextual confusion.
so you are using a different definition of dynamic dispatch than the rest of the software industry.
I am using a conception (NOT a definition) that is actually dynamic.
If you can type-check your dispatcher at compile time then there is nothing dynamic about it.
Decidable (ahead of time) means your function is fully determined. Something that is fully determined is not dynamic.
It is the conception computer scientists use.
If you can type-check your dispatcher at compile time then there is nothing dynamic about it.
Decidable (ahead of time) means your function is fully determined. Something that is fully determined is not dynamic.
It is the conception computer scientists use.
Dynamic languages do it at runtime too, JUST LIKE rust and C++ do. What's the difference?
C++ and Rust let you have compile-time safety, until you choose to give it up and have runtime checks instead. Dynamic languages only allow the latter. Static languages let you choose, dynamic languages chose the latter for you in all cases. Both can have dynamic dispatch.
Besides, static languages can have compile-time type safe dynamic dispatch, if you constrain the dispatch to compile-time-known types (eg std::variant). You only lose that if you want fully unconstrained dynamism, in which case you defer type checking to runtime. Which is what dynamic languages always have.
So both C++ and Rust DO have dynamic dispatch and the programmer gets to choose what level of the dynamism/safety trade off they want. And yes, these features ARE first class features of the languages.
C++ and Rust let you have compile-time safety, until you choose to give it up and have runtime checks instead. Dynamic languages only allow the latter. Static languages let you choose, dynamic languages chose the latter for you in all cases. Both can have dynamic dispatch.
Besides, static languages can have compile-time type safe dynamic dispatch, if you constrain the dispatch to compile-time-known types (eg std::variant). You only lose that if you want fully unconstrained dynamism, in which case you defer type checking to runtime. Which is what dynamic languages always have.
So both C++ and Rust DO have dynamic dispatch and the programmer gets to choose what level of the dynamism/safety trade off they want. And yes, these features ARE first class features of the languages.
>until you choose to give it up
PRECISELY
You have to give up the safety to get the feature.
So you "want type-safety". Until you don't.
>static languages can have compile-time type safe dynamic dispatch
"Compile-time dynamic dispatch" is an oxymoron. Dynamic dispatch happens at runtime.
PRECISELY
You have to give up the safety to get the feature.
So you "want type-safety". Until you don't.
>static languages can have compile-time type safe dynamic dispatch
"Compile-time dynamic dispatch" is an oxymoron. Dynamic dispatch happens at runtime.
[deleted]
> Neither C++ nor Rust have dynamic dispatch
You appear to be using some other definition of dynamic dispatch than the rest of the software industry...
You appear to be using some other definition of dynamic dispatch than the rest of the software industry...
You appear to be conflating compilers with runtimes.
Dynamic dispatch happens at runtime.
C++ and Rust are compile-time tools, not runtimes.
Dynamic dispatch happens at runtime.
C++ and Rust are compile-time tools, not runtimes.
You are wrong. C++ supports dynamic dispatch.
Please read about it on Wikipedia
https://en.m.wikipedia.org/wiki/Dynamic_dispatch
And for the future to not litter HN with comments like these, next time 10 different people in thread are all explaining to you why you're mistaken, take a moment to try to listen and think through what they're saying instead of just digging deeper.
Having an open mind to learning something new, not just arguing a point is a great approach to life.
Please read about it on Wikipedia
https://en.m.wikipedia.org/wiki/Dynamic_dispatch
And for the future to not litter HN with comments like these, next time 10 different people in thread are all explaining to you why you're mistaken, take a moment to try to listen and think through what they're saying instead of just digging deeper.
Having an open mind to learning something new, not just arguing a point is a great approach to life.
ukj(1)
Sure you can. You just need the right amount of indirection and abstraction. I think almost every language has some escape hatch which lets you implement dynamic dispatch.
This is a trivial and obvious implication of Turing completeness. Why do you even bother making the point?
With the right amount of indirection/abstraction you can implement everything in Assembly.
But you don't. Because you like all the heavy lifting the language does for you.
First Class citizens is what we are actually interested in when we talk about programming language paradigm-choices.
https://en.wikipedia.org/wiki/First-class_citizen
With the right amount of indirection/abstraction you can implement everything in Assembly.
But you don't. Because you like all the heavy lifting the language does for you.
First Class citizens is what we are actually interested in when we talk about programming language paradigm-choices.
https://en.wikipedia.org/wiki/First-class_citizen
I mean, GP said "you can't have routers" and maybe I'm being dense by interpreting that as "never or almost never," but even with a generous "too hard to be practical," I still don't think it's correct.
And I explicitly said "escape hatch" meaning language feature. You don't need that much indirection to get routers in Haskell, Rust, Go, C, C++... like I fail to see how implementing routers are a barrier in strict type system languages.
Is it easier in python or js? Sure. can't? hardly.
E: here's some vtable dispatch (unless that doesn't count as "dynamic dispatch") in Rust. Looks really straightforward.
https://doc.rust-lang.org/1.8.0/book/trait-objects.html
And I explicitly said "escape hatch" meaning language feature. You don't need that much indirection to get routers in Haskell, Rust, Go, C, C++... like I fail to see how implementing routers are a barrier in strict type system languages.
Is it easier in python or js? Sure. can't? hardly.
E: here's some vtable dispatch (unless that doesn't count as "dynamic dispatch") in Rust. Looks really straightforward.
https://doc.rust-lang.org/1.8.0/book/trait-objects.html
What C++, Haskell, Rust, Go etc. call "dynamic dispatch" is what a router calls "static routing". The defining characteristic is that the lookup table is pre-determined and immutable at runtime.
What routers call "dynamic routing" is having the lookup table mutable at runtime.
There can be no equivalent to that in type-safe languages because when you mutate the dispatch table you lose type-safety.
What routers call "dynamic routing" is having the lookup table mutable at runtime.
There can be no equivalent to that in type-safe languages because when you mutate the dispatch table you lose type-safety.
> What routers call "dynamic routing" is having the lookup table mutable at runtime.
> There can be no equivalent to that in type-safe languages because when you mutate the dispatch table you lose type-safety.
That's exactly what I mean by escape hatches. Rust has unsafe. You can write a WTF FastInverseSqrt in Rust. You shouldn't 99% of the time. But you can.
Rust also has Box, Rc, Arc, and other tools. I'm not fluent enough in rust to know how to, but I'm quite confident and will eat my hat if you can't accomplish what is effectively a mutable vtable dispatch in Rust.
But also...why? Yeeting a function call into the void without any type knowledge seems way more bug-prone than interface/trait-based dispatch, to what benefit? Save developer time? I've spent countless dev-days of my life I won't get back debugging exactly this kind of dynamic-dispatched, json-dsl, frankly loosey-goosey untyped bullshit. There's so many better ways.
> There can be no equivalent to that in type-safe languages because when you mutate the dispatch table you lose type-safety.
That's exactly what I mean by escape hatches. Rust has unsafe. You can write a WTF FastInverseSqrt in Rust. You shouldn't 99% of the time. But you can.
Rust also has Box, Rc, Arc, and other tools. I'm not fluent enough in rust to know how to, but I'm quite confident and will eat my hat if you can't accomplish what is effectively a mutable vtable dispatch in Rust.
But also...why? Yeeting a function call into the void without any type knowledge seems way more bug-prone than interface/trait-based dispatch, to what benefit? Save developer time? I've spent countless dev-days of my life I won't get back debugging exactly this kind of dynamic-dispatched, json-dsl, frankly loosey-goosey untyped bullshit. There's so many better ways.
ukj(1)
Did you mean something other than "dynamic dispatch", or what do you mean by "first class support"?
No offense, but your claim sounds like you're confused to me, but maybe I am the one confused. AFAIK I do dynamic dispatch all the time in strongly typed languages. Can you show an example that a strongly typed language can't accomplish?
No offense, but your claim sounds like you're confused to me, but maybe I am the one confused. AFAIK I do dynamic dispatch all the time in strongly typed languages. Can you show an example that a strongly typed language can't accomplish?
>I do dynamic dispatch all the time in strongly typed languages.
I believe semantics is getting in our way of communicating.
You don't do dynamic dispatch ___ALL___ the time. You only do it at runtime. And you only do static type safety at compile time. Those are different times.
You can't have both of those features at the __SAME TME__, therefore you can't have both features ALL the time. They are mutually exclusive.
I believe semantics is getting in our way of communicating.
You don't do dynamic dispatch ___ALL___ the time. You only do it at runtime. And you only do static type safety at compile time. Those are different times.
You can't have both of those features at the __SAME TME__, therefore you can't have both features ALL the time. They are mutually exclusive.
"All the time" is an english expression, please, look it up before going all-caps Python name mangle convention on me.
Yes, of course dynamic dispatch is a runtime phenomenom, that's the dynamic part of it. But there's nothing stopping the code that performs dynamic dispatch from being strongly typed. Strong types are instructions used to prove that the code holds certain properties, they are a separate program from the final binary the compiler gives you. Do you also think that code that gets unit tested can't perform dynamic dispatch?
If your point is that "types don't exist at runtime anyway" (reflection aside) then you don't understand what the purpose of a type system is, nor what strongly typed code means.
Yes, of course dynamic dispatch is a runtime phenomenom, that's the dynamic part of it. But there's nothing stopping the code that performs dynamic dispatch from being strongly typed. Strong types are instructions used to prove that the code holds certain properties, they are a separate program from the final binary the compiler gives you. Do you also think that code that gets unit tested can't perform dynamic dispatch?
If your point is that "types don't exist at runtime anyway" (reflection aside) then you don't understand what the purpose of a type system is, nor what strongly typed code means.
> "All the time" is an english expression, please, look it up before going all-caps Python name mangle convention on me.
I know reaction comments are discouraged on HN, but this had me in stitches. Top-tier gourmet dig right there.
I know reaction comments are discouraged on HN, but this had me in stitches. Top-tier gourmet dig right there.
There is something stopping you from running the proof against your implementation!
Proof-checking happens only at compile time.
The implementation that you want to prove things about is only available at runtime!
Non-availability (incompleteness) of information is what is preventing you…
Proof-checking happens only at compile time.
The implementation that you want to prove things about is only available at runtime!
Non-availability (incompleteness) of information is what is preventing you…
Dynamic dispatch is literally a runtime feature of a language. That's why it's called "dynamic". GP can absolutely use dynamic dispatch "all the time" in the sense that they use it regularly in their program, perhaps in every or nearly every program they write.
Your statement is verging on the nonsensical, like saying to someone "You don't use integers all the time, sometimes you use strings, they're different things." Well, duh?
EDIT: Also, it's discouraged on this site in general to use all caps for emphasis. *phrase* produces an italicized/emphasized form with phrase.
Your statement is verging on the nonsensical, like saying to someone "You don't use integers all the time, sometimes you use strings, they're different things." Well, duh?
EDIT: Also, it's discouraged on this site in general to use all caps for emphasis. *phrase* produces an italicized/emphasized form with phrase.
I am using a perfectly sensible notion of "dynamic" (e.g NOT static) when I am talking about "dynamic dispatch".
Registering pointers to new implementations at runtime (adding entries to the dispatch table) Unregistering pointers to old implementations at runtime (removing entries from the dispatch table).
If your dispatch table is immutable (e.g static!), there's nothing dynamic about your dispatch!
Registering pointers to new implementations at runtime (adding entries to the dispatch table) Unregistering pointers to old implementations at runtime (removing entries from the dispatch table).
If your dispatch table is immutable (e.g static!), there's nothing dynamic about your dispatch!
> Why do you even bother making the point?
Maybe because you said:
> they prevent you from implementing dynamic dispatch.
and
> Routers. You can't have routers.
Which just isn't true. You can implement dynamic dispatch and you can have routers, but they come at a cost (either of complex code or of giving up compile-time type safety, but in a dynamic language you don't have the latter anyway, so with a static language you can at least choose when you're willing to pay the price).
> First Class citizens is what we are actually interested in when we talk about programming language paradigm-choices.
But that's not what you said in your other comment. You just said you can't have these things, not they're not first class citizens. Besides, some static languages do have first class support for more dynamic features. C++ has tools like std::variant and std::any in its standard library for times you want some more dynamism and are willing to pay the tradeoffs. In Java you have Object. In other static languages, you have other built-in tools.
Maybe because you said:
> they prevent you from implementing dynamic dispatch.
and
> Routers. You can't have routers.
Which just isn't true. You can implement dynamic dispatch and you can have routers, but they come at a cost (either of complex code or of giving up compile-time type safety, but in a dynamic language you don't have the latter anyway, so with a static language you can at least choose when you're willing to pay the price).
> First Class citizens is what we are actually interested in when we talk about programming language paradigm-choices.
But that's not what you said in your other comment. You just said you can't have these things, not they're not first class citizens. Besides, some static languages do have first class support for more dynamic features. C++ has tools like std::variant and std::any in its standard library for times you want some more dynamism and are willing to pay the tradeoffs. In Java you have Object. In other static languages, you have other built-in tools.
[deleted]
Everything comes at a cost of something in computation!
That is what “trade offs” means.
You can have any feature in any language once you undermine the default constraints of your language. You can implement Scala in Brainfuck. Turing completeness guarantees it!
But this is not the sort of discourse we care about in practice.
https://en.wikipedia.org/wiki/Brainfuck
That is what “trade offs” means.
You can have any feature in any language once you undermine the default constraints of your language. You can implement Scala in Brainfuck. Turing completeness guarantees it!
But this is not the sort of discourse we care about in practice.
https://en.wikipedia.org/wiki/Brainfuck
Yes, and? How is that relevant here?
You said "you can't", kortex said "you can" and then you moved the goal posts to "you can because of turing completeness, but its bad, Why do you even bother making the point?" to which I replied "because its a valid response to you're `you can't`" now you moved them again to "everything comes at a cost" (which... I also said?).
Of course everything comes at a cost and yes, that's what "trade off" means. Dynamic languages come at a cost too (type checking is deferred to run time). So, this time, let me ask you: Why do you even bother making the point?
You said "you can't", kortex said "you can" and then you moved the goal posts to "you can because of turing completeness, but its bad, Why do you even bother making the point?" to which I replied "because its a valid response to you're `you can't`" now you moved them again to "everything comes at a cost" (which... I also said?).
Of course everything comes at a cost and yes, that's what "trade off" means. Dynamic languages come at a cost too (type checking is deferred to run time). So, this time, let me ask you: Why do you even bother making the point?
Tractability vs possibility.
You don't grok the difference.
You can implement EVERYTHING in Brainfuck. Tractability is the reason you don't.
The goalposts are exactly where I set them. With my first comment.
"Every programming paradigm is a good idea if the respective trade-offs are acceptable to you."
You don't grok the difference.
You can implement EVERYTHING in Brainfuck. Tractability is the reason you don't.
The goalposts are exactly where I set them. With my first comment.
"Every programming paradigm is a good idea if the respective trade-offs are acceptable to you."
Its perfectly tractable though. Just because you don't understand it or don't think it is, doesn't make it true.
> "Every programming paradigm is a good idea if the respective trade-offs are acceptable to you."
That's not what we are responding to. Nobody here is arguing over this statement. We are responding to you assertion that static typed compile-time checked languages _prevent_ you from having dynamic dispatch and that you _can't have_ routers because of that. Neither of which are true.
Dynamic languages prevent you from having compile time checks. Does that make them bad? Static languages give you compile time safety, but if you're willing to forego that [1], then you can get the EXACT SAME behavior as dynamic languages give you.
You literally said:
Just because a hammer can't screw in screws doesn't mean its a bad idea, it just means that you can't use it for all use cases. This is the same. You can use static types and for the few cases where you need runtime dynamism, then you use that. That doesn't make the static types in the rest of your code bad. It just gives you additional tools that dynamic types alone don't have.
[1] to various degrees, its not all or nothing like you seem to be implying, there are levels of middle ground, like std::variant which maintains safety but you need to enumerate all possible types, or std::any which is fully dynamic but you give up compile time checks
> "Every programming paradigm is a good idea if the respective trade-offs are acceptable to you."
That's not what we are responding to. Nobody here is arguing over this statement. We are responding to you assertion that static typed compile-time checked languages _prevent_ you from having dynamic dispatch and that you _can't have_ routers because of that. Neither of which are true.
Dynamic languages prevent you from having compile time checks. Does that make them bad? Static languages give you compile time safety, but if you're willing to forego that [1], then you can get the EXACT SAME behavior as dynamic languages give you.
You literally said:
For example, one good reason why strong static types are a bad idea... they prevent you from implementing dynamic dispatch.
Routers. You can't have routers.
Just because a hammer can't screw in screws doesn't mean its a bad idea, it just means that you can't use it for all use cases. This is the same. You can use static types and for the few cases where you need runtime dynamism, then you use that. That doesn't make the static types in the rest of your code bad. It just gives you additional tools that dynamic types alone don't have.
[1] to various degrees, its not all or nothing like you seem to be implying, there are levels of middle ground, like std::variant which maintains safety but you need to enumerate all possible types, or std::any which is fully dynamic but you give up compile time checks
[deleted]
ukj(1)
You can't actually implement everything in Brainfuck. You can implement something which is performing an equivalent computation in an abstract, mathematical sense. But there's no way to write Firefox or Windows or Fortnite in Brainfuck. Turing completeness means you can evaluate any computable function of type N -> N (and the many things isomorphic to that), it doesn't give you anything else.
Just as a Turing machine requires an infinitely sized tape to compute any computable function, so too would brainfuck require an indirect sized tape (or whatever it’s called in BF) to compute any computable function. Since memory is finite, neither of these properties are actually available on real hardware.
I am interested in computation. Period. Not any particular model of computation (programming language); and not merely computation with functions from N->N.
Quoting from "http://math.andrej.com/2006/03/27/sometimes-all-functions-ar..."
"The lesson is for those “experts” who “know” that all reasonable models of computation are equivalent to Turing machines. This is true if one looks just at functions from N to N. However, at higher types, questions of representation become important, and it does matter which model of computation is used."
Quoting from "http://math.andrej.com/2006/03/27/sometimes-all-functions-ar..."
"The lesson is for those “experts” who “know” that all reasonable models of computation are equivalent to Turing machines. This is true if one looks just at functions from N to N. However, at higher types, questions of representation become important, and it does matter which model of computation is used."
[deleted]
When I think of validation I think of receiving a data file and checking that all rows and columns are correct and generating a report about all the problems.
Does my thing have a different name? Where can I read up on how to do that best?
Does my thing have a different name? Where can I read up on how to do that best?
You can use the now widely adopted Great Expections[0] library, which fits exactly this use-case for data validation!
[0] https://greatexpectations.io
[0] https://greatexpectations.io
Thanks for the link. It looks really nice.
I see they’ve raised a lot of money. Does anyone know what their revenue model is?
I see they’ve raised a lot of money. Does anyone know what their revenue model is?
I think that is in fact validating in the sense that the article means it.
Here's validating a CSV in Python (which I'm using because it's a language that's, well, less excited about types than the author's choice of Haskell, to show that the principle still applies):
The insight is that the work of reporting errors in the data is exactly the same as the work of getting non-erroneous data into a usable form. If a row of data doesn't have an error, that means it's usable; if you can't turn it into a directly usable format, that necessarily means it has some sort of error.
So what you want is a function that takes the data and does both of these at the same time, because it's actually just a single task.
In a language like Haskell or Rust, there's a built-in type for "either a result or an error", and the convention is to pass errors back as data. In a language like Python, there isn't a similar concept and the convention is to pass errors as exceptions. Since you want to accumulate all the errors, I'd probably just put them into a separate list:
And if your file format changes, you make the change in one spot; you've kept the code DRY.
Here's validating a CSV in Python (which I'm using because it's a language that's, well, less excited about types than the author's choice of Haskell, to show that the principle still applies):
def validate_data(filename):
reader = csv.DictReader(open(filename))
for row in reader:
try:
date = datetime.datetime.fromisoformat(row["date"])
except ValueError:
print("ERROR: Invalid date", row)
if date < datetime.datetime(2021, 1, 1):
print("ERROR: Last year's data", row))
# etc.
return errors
def actually_work_with_data(filename):
reader = csv.DictReader(open(filename))
for row in reader:
try:
date = datetime.datetime.fromisoformat(row["date"])
except ValueError:
raise Exception("Wait, didn't you validate this already???")
# etc.
The insight is that the work of reporting errors in the data is exactly the same as the work of getting non-erroneous data into a usable form. If a row of data doesn't have an error, that means it's usable; if you can't turn it into a directly usable format, that necessarily means it has some sort of error.
So what you want is a function that takes the data and does both of these at the same time, because it's actually just a single task.
In a language like Haskell or Rust, there's a built-in type for "either a result or an error", and the convention is to pass errors back as data. In a language like Python, there isn't a similar concept and the convention is to pass errors as exceptions. Since you want to accumulate all the errors, I'd probably just put them into a separate list:
@attr.s # or @dataclasses.dataclass, whichever
class Order:
name: str
date: datetime.datetime
...
def parse(filename):
data = []
errors = []
reader = csv.DictReader(open(filename))
for row in reader:
try:
date = datetime.datetime.fromisoformat(row["date"])
except ValueError:
errors.append(("Invalid date", row))
continue
if date < datetime.datetime(2021, 1, 1):
errors.append(("Last year's data", row))
continue
# etc.
data.append(Order(name=row["name"], date=date, ...))
return data, errors
And if your file format changes, you make the change in one spot; you've kept the code DRY.
I thought of input validation for web forms.
Similar thing I guess. In Haskell you can create a type that you know is a validated email address but you still need a validation function from String -> Maybe Email to actually validate it at runtime
I think for the usecase GP gives it'd be even better to have a function `String -> Either (LineNumber,String,[Problem]) Email`, so that you can report back which of the lines had problems and what kind of problems. For web form validation you can skip the line number but it'd still be useful to keep the list of problems, so that you can report back to the user what about their input did not conform to expectations.
That's just a parser though. Like described in the post, parsers sometimes can fail but importantly they always pass along the result if they succeed. Validation functions on the other hand only validate that said data is valid.
The argument is that if you need to interact with or operate on some data you shouldn't be designing functions to validate the data but rather to render it into a useful output with well defined behaviour.
The argument is that if you need to interact with or operate on some data you shouldn't be designing functions to validate the data but rather to render it into a useful output with well defined behaviour.
Data validation?
Wonderful piece, it has really opened my eyes on foreign data ingestion.
“Parse, don’t [just] validate”.
Say I have a string that’s supposed to represent an integer. To me, “Validate” means using a regex to ensure it contains only digits (raising an error if it doesn’t) but then continuing to work with it as a string. “Parse” means using “atoi” to obtain an integer value (but what if the string’s malformed?) and then working with that.
I first thought this article was recommending doing the latter instead of the former, but the actual recommendation (and I believe best practice) is to do both.
Say I have a string that’s supposed to represent an integer. To me, “Validate” means using a regex to ensure it contains only digits (raising an error if it doesn’t) but then continuing to work with it as a string. “Parse” means using “atoi” to obtain an integer value (but what if the string’s malformed?) and then working with that.
I first thought this article was recommending doing the latter instead of the former, but the actual recommendation (and I believe best practice) is to do both.
You seem to suggest that it's possible to parse without validating, which I'm not sure I follow. Surely validation is just one of the phases or steps of parsing?
Functions like `atoi` parse strings into integers, but will happily accept “ 10blah” and return 10. In my experience it’s best to validate that the string is well-formed (e.g. contains only digits) before passing it to one of those functions.
The point is that validation is (or should/can be) a byproduct of parsing. I.e., you shouldn't "do both", rather the validation should be encompassed by the parsing, as much as it makes sense.
This still sounds like validation but with extra steps. (or less?)
The post is saying:
- don't drop the info gathered from checks while validating, but keep track of it
- if you do this, you'll effectively be parsing
- parsing is more powerful that validating
"Extra steps" would be keeping track of info gathered from checks.
- don't drop the info gathered from checks while validating, but keep track of it
- if you do this, you'll effectively be parsing
- parsing is more powerful that validating
"Extra steps" would be keeping track of info gathered from checks.
Right. My takeaway was "verify and validate once, then put it in a specially marked datastructure, or if your language allow it make the typesystem guarantee some conditions of the data, then work with that from there". Where does parsing come in the picture?
Well, ain't that it? If you validate a string so that it contains some angle bracket tags at the beginning and the end, ensure that both the tag values contain the same string (except for one extra marker in the end tag), and store the tag name and the string within the tags in a purpose-made data structure, you can call it whatever you want. Some will call that a rudimentary XML parser.
You understand it. It's using the word "parse" metaphorically, to mean "validate, then put it in a specially marked data structure". For example, `parse_int :: string -> maybe int` is a parsing function, and it "validates that the string is an integer, then puts it in a specially marked data structure called int". However, the post uses the word "parse" not only for true parsing functions (that convert text into a data structure), but also for conversions from data structure to data structure.
I also find this a confusing use of the word "parse", and it's not explained in the post, and I think "parse, don't validate" is a poor slogan as a result. The traditional slogan is "make illegal states unrepresentable", though that's a bit narrower of a concept.
I also find this a confusing use of the word "parse", and it's not explained in the post, and I think "parse, don't validate" is a poor slogan as a result. The traditional slogan is "make illegal states unrepresentable", though that's a bit narrower of a concept.
I don't think this is the same as "make illegal states unrepresentable"; it's a corollary (or the converse maybe):
"Make assertions of legal states representable"
"Make assertions of legal states representable"
They're sort of the same. If you have str and safestr, and safestr is known to confiorm to some invariant, the illegal state of a, say str where validate hasn't been called, isn't representable.
> verify and validate once, then put it in a specially marked datastructure
Not to steal vvillena's thunder, but that's pretty much the dictionary definition of "parsing"
> analyze (a string or text) into logical syntactic components, typically in order to test conformability to a logical grammar.
Parsing is taking some collection of symbols, and emitting some other structure that obeys certain rules. Those symbols need not be text, they can be any abstract "thing". A symbol could be a full-blown data structure - you can parse a List into a NotEmptyList, where there's some associated grammar with the NEL that's a stricter version of the List grammar.
Not to steal vvillena's thunder, but that's pretty much the dictionary definition of "parsing"
> analyze (a string or text) into logical syntactic components, typically in order to test conformability to a logical grammar.
Parsing is taking some collection of symbols, and emitting some other structure that obeys certain rules. Those symbols need not be text, they can be any abstract "thing". A symbol could be a full-blown data structure - you can parse a List into a NotEmptyList, where there's some associated grammar with the NEL that's a stricter version of the List grammar.
Haha, "Those symbols need not be text", you say, right after quoting a definition that says they need to be "a string or text"!
There's a field of study called "parsing", which studies "parsers". Hundreds of papers. Very well defined problem: turning a list of symbols into a tree shaped parse tree (or data structure). The defining aspect of parsing, that makes it difficult and an interesting thing to study, is that you're starting with a list and ending with a tree. If you're converting tree to tree (that is, a typical data structure to a typical data structure), all the problems vanish (or change drastically) and all the parsing techniques are inapplicable.
I'm kind of annoyed that people are starting to use the word "parse" metaphorically. Bit by bit, precise words turn fuzzy. Alas, it will be a lost battle.
There's a field of study called "parsing", which studies "parsers". Hundreds of papers. Very well defined problem: turning a list of symbols into a tree shaped parse tree (or data structure). The defining aspect of parsing, that makes it difficult and an interesting thing to study, is that you're starting with a list and ending with a tree. If you're converting tree to tree (that is, a typical data structure to a typical data structure), all the problems vanish (or change drastically) and all the parsing techniques are inapplicable.
I'm kind of annoyed that people are starting to use the word "parse" metaphorically. Bit by bit, precise words turn fuzzy. Alas, it will be a lost battle.
Sure, but the list of symbols can be an arbitrary collection where the symbols are 0 and 1.
Voila, now your 'string' is 'binary data' not 'text'.
Parsing binary data is my bread and butter, so I might be biased but: it works fine.
Anything which comes over the wire is a string, anything which comes out of store is a string. If you're using something like protobufs, that's great, because having to marshal/serialize/parse along every process boundary is expensive and probably unnecessary.
But at some point, and anywhere on the 'surface' of the system, data has to be un-flattened into a shape. That's parsing.
Voila, now your 'string' is 'binary data' not 'text'.
Parsing binary data is my bread and butter, so I might be biased but: it works fine.
Anything which comes over the wire is a string, anything which comes out of store is a string. If you're using something like protobufs, that's great, because having to marshal/serialize/parse along every process boundary is expensive and probably unnecessary.
But at some point, and anywhere on the 'surface' of the system, data has to be un-flattened into a shape. That's parsing.
> Haha, "Those symbols need not be text", you say, right after quoting a definition that says they need to be "a string or text"!
Ha yeah nice catch, that's why I added that in there. In this case the dictionary is slightly wrong.
> The defining aspect of parsing, that makes it difficult and an interesting thing to study, is that you're starting with a list and ending with a tree.
Ah, I didn't know that! Great bit to learn.
In that case, I will say that the "increase a data structure's rules" is a bit ambiguous.
I think my statement is still correct in that "a symbol could be a data structure," right? Like you could take a list of dicts and emit a tree of dicts.
But wait, a list is a kind of tree, or rather, there is a parse tree of recursive head/tail branches. So I think you could still argue List->NotEmptyList is a Parse because NEL requires a nonzero "head" and zero or one NEL as "tail."
Ha yeah nice catch, that's why I added that in there. In this case the dictionary is slightly wrong.
> The defining aspect of parsing, that makes it difficult and an interesting thing to study, is that you're starting with a list and ending with a tree.
Ah, I didn't know that! Great bit to learn.
In that case, I will say that the "increase a data structure's rules" is a bit ambiguous.
I think my statement is still correct in that "a symbol could be a data structure," right? Like you could take a list of dicts and emit a tree of dicts.
But wait, a list is a kind of tree, or rather, there is a parse tree of recursive head/tail branches. So I think you could still argue List->NotEmptyList is a Parse because NEL requires a nonzero "head" and zero or one NEL as "tail."
> I think my statement is still correct in that "a symbol could be a data structure," right? Like you could take a list of dicts and emit a tree of dicts.
Yeah I guess. Text combinator libraries like Haskell's Parsec and Rust's Nom are typically parametric over the type of "characters". Realistically I don't think I've ever seen anyone use one of those libraries for an input that wasn't text-like, though; do you have a use case in mind?
> But wait, a list is a kind of tree, or rather, there is a parse tree of recursive head/tail branches.
Yes, so you can run into parsing problems when working with trees, if you work really hard at it. But if you do the correct action is "reconsider your life choices" and not "use parsing theory".
Yeah I guess. Text combinator libraries like Haskell's Parsec and Rust's Nom are typically parametric over the type of "characters". Realistically I don't think I've ever seen anyone use one of those libraries for an input that wasn't text-like, though; do you have a use case in mind?
> But wait, a list is a kind of tree, or rather, there is a parse tree of recursive head/tail branches.
Yes, so you can run into parsing problems when working with trees, if you work really hard at it. But if you do the correct action is "reconsider your life choices" and not "use parsing theory".
Validation is checking if something looks like a data structure. Parsing is smashing data into a data structure, and failing out if you can't do it.
At the end of parsing, you have a structure with a type. After validation, you may or may not have a structure with that type, depending on how you chose to validate.
But I think the big win is, parsers are usually much easier to compose (since they themselves are structured functions) and so if you start with the type first, you often get the "validation" behavior aspect of parsing for "free" (usually from a library). Maybe title should have been "Parse, don't manually validate."
But if your type doesn't catch all your invariants, yeah it does feel kinda just like validation.
At the end of parsing, you have a structure with a type. After validation, you may or may not have a structure with that type, depending on how you chose to validate.
But I think the big win is, parsers are usually much easier to compose (since they themselves are structured functions) and so if you start with the type first, you often get the "validation" behavior aspect of parsing for "free" (usually from a library). Maybe title should have been "Parse, don't manually validate."
But if your type doesn't catch all your invariants, yeah it does feel kinda just like validation.
It's more like putting the "extra steps" at the right place in the code.
Previous discussion https://news.ycombinator.com/item?id=21476261
I couldn't agree with this post more.
I found myself replacing the configuration parsing code in a C++ project that was littered with exactly the validation issues described, and converted it to that which the author advocates. The result was a vastly more readable and maintainable codebase, and it was faster and less buggy to boot.
Another nice advantage is that the types are providing free/self- documentation, too.
I found myself replacing the configuration parsing code in a C++ project that was littered with exactly the validation issues described, and converted it to that which the author advocates. The result was a vastly more readable and maintainable codebase, and it was faster and less buggy to boot.
Another nice advantage is that the types are providing free/self- documentation, too.
Related discussion from last month which links to this article in the repo description:
https://news.ycombinator.com/item?id=27166162
I'd call it: pattern match, don't validate.
Gotta go and program more Elixir...
Gotta go and program more Elixir...
a tiny advice: can the blog author do some separation between his next and previous article? it's really hard to read unless you hover your mouse over one of them otherwise. link to what i'm talking about:
https://ibb.co/kc0HKy9
https://ibb.co/kc0HKy9
discussed alot a month ago
https://news.ycombinator.com/item?id=27166162
https://news.ycombinator.com/item?id=27166162
[0] https://pydantic-docs.helpmanual.io/