Functional Programming in Python(github.com)
github.com
Functional Programming in Python
https://github.com/dry-python/returns
63 comments
Python 3 threw away pattern matching in function heads; used to be able to deconstruct tuples. I had just discovered that feature in Python 2 when I realized 2 was about to go EOL, and upon porting my code I discovered the change.
A sad day for this Erlang enthusiast.
A sad day for this Erlang enthusiast.
Union[Duck, OtherDuck] is another nice trick you can use in case of ducks.
This is neat. It reminds me of an silly project [0] I made a while back to implement do-notation in python. In OP's project you still end up with code that's basically this:
You just need to use my @with_do_notation decorator, which just completely rewrites your function using the ast library wherever it finds a block of `with do(SomeClass) as variable:`. I was even able to write ergonomically nice parser combinators [1] that would actually work pretty well if python had tail call optimization.
You shouldn't use it but it was great fun and opened my eyes to the ways you can abuse python if you really wanted to. Using decorators to introspect and completely rewrite functions is a fun exercise.
[0] https://github.com/imh/python_do_notation
[1] https://github.com/imh/python_do_notation/blob/master/parser...
y = (Maybe(just=x) if x > 0 else Maybe()).bind(lambda a:
Maybe(just=x*a) .bind(lambda b:
Maybe.mreturn(a+b)))
with do(Maybe) as y:
a = Maybe(just=x) if x > 0 else Maybe()
b = Maybe(just=x*a)
mreturn(a+b)
You just need to use my @with_do_notation decorator, which just completely rewrites your function using the ast library wherever it finds a block of `with do(SomeClass) as variable:`. I was even able to write ergonomically nice parser combinators [1] that would actually work pretty well if python had tail call optimization.
You shouldn't use it but it was great fun and opened my eyes to the ways you can abuse python if you really wanted to. Using decorators to introspect and completely rewrite functions is a fun exercise.
[0] https://github.com/imh/python_do_notation
[1] https://github.com/imh/python_do_notation/blob/master/parser...
Your example isn't using "y", so you should also be able to drop the " as y" part (unless the AST hack requires it for some reason).
Edit: Oh, nevermind - the "y" is how you access it after the with statement. Weird for python, but it makes sense given the rewritten version. I was thinking your version would have a non-rewritten "result = something" within the with block in a real use, but that's not what it's doing.
Edit: Oh, nevermind - the "y" is how you access it after the with statement. Weird for python, but it makes sense given the rewritten version. I was thinking your version would have a non-rewritten "result = something" within the with block in a real use, but that's not what it's doing.
Heh, yup, it's part of the hackiness of it all. "with do(MyMonadClass) as my_variable: <some code>" equates to haskell's "myVariable = do <some code>". I can't remember why the MyMonadClass part was necessary. Maybe for the mreturn.
Cool hack – not because I want monads in Python particularly, but because I sometimes think a ‘where’ syntax would be nice
Python + Functional = http://coconut-lang.org/
If I didn't already do elixir professionally I'd definitely be learning coconut, it looks super cool.
As a language purist this one is surprisingly well put together. Its impl suffers from legacy (py2 support in general, predating builtin dataclasses, staying in the shallow end of type annotations) but it really does what it does better enough than its host language to be worth giving attention to. Check it out.
> destructuring assignment
Sign me up plz.
Sign me up plz.
HN discussion on coconut: https://news.ycombinator.com/item?id=18815125
Ooo! That looks awesome!
(Something about Haskell...)
(Something about Haskell...)
I like strongly typed and functional programming. But I would rather use a "pythonic" approach to solve problems in python.
OFC there might be some cases where for example one of your dependency is python, and you still want to be able to write correct code. But in general, If you want types and functional constructs, IMO you should use something else.
If there was one thing I'd change with python to make it more 'functional' it would be having lambdas that weren't expression-only. I know it's not a functional feature because it's for stateful programming, but the lack of it disallows some nice stuff.
lambda use in Python is discouraged in general. If you need something beyond an expression, PEP-8 encourages you to use a def in that case instead.
The issue with that is the lack of lexical scope. Lambdas have lexical scoping, def functions do not. To “fake” lexical scope, you have to use the nonlocal or global statements, which are gross and self-defeating.
Not sure I understand, defs have lexical scoping too, if you just read variables (or am I getting rusty?).
It's when you assign that you need P3's global etc. annotations, and you can't assign in lambdas.
x = 3
def prt():
print(x)
prt()
It's when you assign that you need P3's global etc. annotations, and you can't assign in lambdas.
I agree with you that having more than a single expression in lambdas would be super nice. That being said, I don't think it's necessarily stateful to allow more than a single expression! If you don't reassign any variables, then it's equivalent to a single expression. For instance, something like this (using made up syntax for extended lambdas) isn't stateful:
lambda i:
square = i * i
if i < 0:
return -1 * square
else:
return square
\i ->
let square = i * i in
if x < 0 then -1 * x else x
lambda i:
square = i * i
if i < 0:
square *= -1
return squareI'm sure there are cases where multiple lines would be nice, but I don't think this is one. This seems easier to read to me:
lambda i: -i * i if i < 0 else i * i
Yeah, this is a trivial example. I picked it specifically to show that multi-line lambdas don't have to be stateful, not as an example of a lambda that couldn't be written in one line.
And there is math.copysign
very quick reply, and I've done this in other langs that allow procedures-done-as-lambdas, create your own control flow mechanisms with 'reasonable' syntax, something like
Python can do this another way (with 'with' objects or something, and it's quite clean and neat, but that's quite recent IIRC), this simpler IMO. Have used this, it's called the loan pattern, in scala, vb.net and C#, and prob others.
The arbitrary restriction on lambdas blocks composability, which disallows useful tricks like this.
with_file("c:/blah/data.csv", lambda fl:
# do stuff with file variable fl
# other statements
# ...
)
Python can do this another way (with 'with' objects or something, and it's quite clean and neat, but that's quite recent IIRC), this simpler IMO. Have used this, it's called the loan pattern, in scala, vb.net and C#, and prob others.
The arbitrary restriction on lambdas blocks composability, which disallows useful tricks like this.
Hi, author of `returns` here! Thanks a lot for sharing this link.
I would love to highlight several features of `0.14` release we are working on right now:
1. Typed `partial` and `curry` functions: https://returns.readthedocs.io/en/latest/pages/curry.html
2. `Future` and `FutureResult` containers for working with async Python! https://returns.readthedocs.io/en/latest/pages/future.html
3. Typed functional pipelines with very good type inference: https://returns.readthedocs.io/en/latest/pages/pipeline.html
It is going to be released soon, stay tuned!
I would love to highlight several features of `0.14` release we are working on right now:
1. Typed `partial` and `curry` functions: https://returns.readthedocs.io/en/latest/pages/curry.html
2. `Future` and `FutureResult` containers for working with async Python! https://returns.readthedocs.io/en/latest/pages/future.html
3. Typed functional pipelines with very good type inference: https://returns.readthedocs.io/en/latest/pages/pipeline.html
It is going to be released soon, stay tuned!
The project cites this blog post[0] on the "anti-pattern" that is Python exceptions, and I honestly couldn't be turned off on this project anymore after reading it if this is the inspiration behind it.
The examples in the README and this blog post just give me huge "nope" vibes. Obviously Python could learn a lot more from functional programming, but this is the wrong way to go about a lot of it.
0. https://sobolevn.me/2019/02/python-exceptions-considered-an-...
The examples in the README and this blog post just give me huge "nope" vibes. Obviously Python could learn a lot more from functional programming, but this is the wrong way to go about a lot of it.
0. https://sobolevn.me/2019/02/python-exceptions-considered-an-...
It would be helpful if you could provide specific things in the blog post that you don't agree with. I skimmed through it and it seemed reasonable.
The example cited where they don't know what to return in their `divide` function if a division by zero occurs is nonsense.
The answer is simple: let the `DivsionByZero` exception raise! You don't have to return anything!
Better yet, you should have input cleansing/data validation before it gets to that point. The alternative presented in the blog post is absurd over-engineering.
The answer is simple: let the `DivsionByZero` exception raise! You don't have to return anything!
Better yet, you should have input cleansing/data validation before it gets to that point. The alternative presented in the blog post is absurd over-engineering.
How about writting modern and proper Python first? Not to mention designing a decent API?
Let's examine the README example for a minute:
- one should probably never reach this part of the code if there is no user. But I'll indulge the author.
- don't put can_buy_stuff in an else close, what's the point?
- using type hints for no reason, but not other modern facilities like the walrus operator?
- do we really want users without a balance? Let's indulge this, but it seems a bad design.
- what's with all those unecessary conditional blocks?
- credit_amount should never be None. It's a Balance object, put a sane default value. But ok, indulging again.
So, you get down to:
If we have a well designed API, and we use a function (more testable, and hey, are we doing FP or not ?), then:
You could even that as a method or raise NoBalance depending of your case.
Bottom line, if you really feel that strongly about None, don't jump on the bazooka to kill this fly, go to https://discuss.python.org and advocate for PEP 505 (None-aware operators): https://www.python.org/dev/peps/pep-0505/
It's been deferred since 2015.
That doesn't mean we should not experiment with other paradigms in Python, and I do think this lib is an interesting experiment, but I don't find it conclusive.
Let's examine the README example for a minute:
user: Optional[User]
if user is not None:
balance = user.get_balance()
if balance is not None:
balance_credit = balance.credit_amount()
if balance_credit is not None and balance_credit > 0:
can_buy_stuff = True
else:
can_buy_stuff = False
- one should probably never reach this part of the code if there is no user. But I'll indulge the author.
- don't put can_buy_stuff in an else close, what's the point?
- using type hints for no reason, but not other modern facilities like the walrus operator?
- do we really want users without a balance? Let's indulge this, but it seems a bad design.
- what's with all those unecessary conditional blocks?
- credit_amount should never be None. It's a Balance object, put a sane default value. But ok, indulging again.
So, you get down to:
user: Optional[User]
can_buy_stuff = False
if user and (balance := user.get_balance()):
can_buy_stuff = (balance.credit_amount() or 0) > 0
can_buy_stuff: Maybe[bool] = Maybe.from_value(user).map(
lambda real_user: real_user.get_balance(),
).map(
lambda balance: balance.credit_amount(),
).map(
lambda balance_credit: balance_credit > 0,
)
If we have a well designed API, and we use a function (more testable, and hey, are we doing FP or not ?), then:
def can_buy_stuff(user: User):
if (balance := user.get_balance()):
return balance.credit_amount() > 0
return False
You could even that as a method or raise NoBalance depending of your case.
Bottom line, if you really feel that strongly about None, don't jump on the bazooka to kill this fly, go to https://discuss.python.org and advocate for PEP 505 (None-aware operators): https://www.python.org/dev/peps/pep-0505/
It's been deferred since 2015.
That doesn't mean we should not experiment with other paradigms in Python, and I do think this lib is an interesting experiment, but I don't find it conclusive.
I'd like to know what happens to can_buy_stuff and the rest of the code when user is Not None and balance_credit is -1.
user can't buy stuff and one unfortunate developer will hunt the bug that charged someone with negative credit.
can_buy_stuff is then False.
For 'Maybe' an 'Result' specifically, I feel that Python builtins + mypy offer superior experience: easier, safer, and less dependencies with a similar level of verbosity.
For Maybe example: I think this 'functional' style with fmaps in Python is problematic, because lambdas can't be multiline. If you have sevearal lines of logic, you'd need an auxiliary helper def, and at this point it becomes as unreadable.
For Results: I think returning Union[Exception, Value] (where Value is the 'desired' type) and then using isinstance(result, Exception) is much cleaner.
- it can be statically checked with mypy to ensure the same level of type safety as Rust would have
- minimal performance impact
- no extra wrapping and unwrapping in the code. You can completely ignore mypy and error handling, until you're happy, then you harden your program by making sure it complies to mypy.
- no extra dependencies, third party code dealing with your library doesn't have to deal with your wrappers! If they don't check for error type, when Exception is encountered, the program will most likely terminate with AttributeError, which is a desirable behaviour in such situation.
- it's much easier to compose: propagating error with a decorator is neat, until you have some more sophisticated logic, e.g. for error aggregation
- the only downside is that you end up with occasional `if isinstance(result, Exception)...`.
I reviewed results library specifically here [0] and elaborate on different error handling techniques in Python, including the approach I described.
[0] https://beepb00p.xyz/mypy-error-handling.html#container
For Maybe example: I think this 'functional' style with fmaps in Python is problematic, because lambdas can't be multiline. If you have sevearal lines of logic, you'd need an auxiliary helper def, and at this point it becomes as unreadable.
For Results: I think returning Union[Exception, Value] (where Value is the 'desired' type) and then using isinstance(result, Exception) is much cleaner.
- it can be statically checked with mypy to ensure the same level of type safety as Rust would have
- minimal performance impact
- no extra wrapping and unwrapping in the code. You can completely ignore mypy and error handling, until you're happy, then you harden your program by making sure it complies to mypy.
- no extra dependencies, third party code dealing with your library doesn't have to deal with your wrappers! If they don't check for error type, when Exception is encountered, the program will most likely terminate with AttributeError, which is a desirable behaviour in such situation.
- it's much easier to compose: propagating error with a decorator is neat, until you have some more sophisticated logic, e.g. for error aggregation
- the only downside is that you end up with occasional `if isinstance(result, Exception)...`.
I reviewed results library specifically here [0] and elaborate on different error handling techniques in Python, including the approach I described.
[0] https://beepb00p.xyz/mypy-error-handling.html#container
Lambdas can be multiline/multistep, but it's just as ugly as you'd expect. Maybe the most "pure" functional way to do it is to have multiple binds/applies/maps, so instead of something like `x=1; y=2; x + y` it's something like `1.apply(lambda x: 2.apply(lambda y: x + y)).` Still unreadable, but for different reasons. It's more about the grossness of using `xEmitter.map(lambda x: stuff with x in scope)` to bind x as a variable than it is about the inflexibility of lambdas.
I'm a big fan of functional programming but this documentation is kind of funny, unintentionally. I realize there is no better way to do it in Python.
And that’s how your initial refactored code will look like:
And that’s how your initial refactored code will look like:
user: Optional[User]
can_buy_stuff: Maybe[bool] = Maybe.from_value(user).map( # type hint is not required
lambda real_user: real_user.get_balance(),
).map(
lambda balance: balance.credit_amount(),
).map(
lambda balance_credit: balance_credit > 0,
)
I'm also a FP fanboy, and this concept doesn't quite convince me. I'm not sure a Maybe type really helps that much in a dynamically typed language. All the safety you get from using Maybes is worth Nothing (pun intended) when credit_amount could still return let's say Just a string, or raises an exception.
If this chain of calls supposedly handles Nones at any level, then .map is not the right method. It should be .flatMap (or .bind). Unless this is a magic .map that handles either X or Maybe[X] and even exceptions (just like JS promises). This flexibility may be convenient and I can appreciate its pythonicness, but it's not really in the spirit of typed FP.
If this chain of calls supposedly handles Nones at any level, then .map is not the right method. It should be .flatMap (or .bind). Unless this is a magic .map that handles either X or Maybe[X] and even exceptions (just like JS promises). This flexibility may be convenient and I can appreciate its pythonicness, but it's not really in the spirit of typed FP.
Why the lambdas at all in these examples...
(lambda real_user: real_user.get_balance())
User.get_balance
It's a bad example. You can of course do more with lambdas in python. Maybe "lambda real_user: real_user.get_balance()+1" or "lambda some_junk, real_user: real_user.get_balance()" would have been better.
Cool stuff but what's with the logo? The branding is a bit too Affliction-esque for something usually associated with elegance...
Functional programming in Python:
Step 1: try to integrate more functional methods in your current Python programming due to their usefulness.
Step 2: get increasingly frustrated, and then give up, because Guido and the Python devs seemingly hate functional programming and keep hobbling it.
Step 1: try to integrate more functional methods in your current Python programming due to their usefulness.
Step 2: get increasingly frustrated, and then give up, because Guido and the Python devs seemingly hate functional programming and keep hobbling it.
A part of me is happy to see these types of tools introduced to python, but another part makes me wonder why people continue to use python when they see the value of things like `Option`/`Maybe`'s, `Result`'s, instead of just using a language with the features included from the start.
In my experience, the initial ease and speed of development when using python doesn't nearly outweigh the medium to long-term costs of maintaining it and developing the codebase further - at least for codebases that are more than a simple tool or something like a django app. Writing things like go, rust, scala, java etc. isn't that much more difficult or slower, but it does require more up front planning and understanding of your problem domain.
In my experience, the initial ease and speed of development when using python doesn't nearly outweigh the medium to long-term costs of maintaining it and developing the codebase further - at least for codebases that are more than a simple tool or something like a django app. Writing things like go, rust, scala, java etc. isn't that much more difficult or slower, but it does require more up front planning and understanding of your problem domain.
I don't have the skills to pull it off completely, or maintain it, but I've often toyed with the idea of a language/DSL that superficially looks and behaves like Python, but transpires down into Rust or something.
Mostly just to see what a language that 'feels like' Python would be like with the addition of things like proper Option/Result's and a couple of other features (stronger typing?).
Mostly just to see what a language that 'feels like' Python would be like with the addition of things like proper Option/Result's and a couple of other features (stronger typing?).
The `Maybe` container seems to be a close analog of `Option` in Rust
https://doc.rust-lang.org/rust-by-example/std/option.html
https://doc.rust-lang.org/rust-by-example/std/option.html
Is this surprising? Of course it is
It's fairly common across languages: https://en.wikipedia.org/wiki/Option_type
Python is all about the magic. Lambda expressions are a bit of a black box for me in Python, it would be nice if that was made more transparent, or perhaps a different function entirely
It's hard to strike a balance for this.
For many FP programmers, the lambdas are not nearly magic enough.
In Swift, you can do something like this :
So you see the position Python is in: it has to balance a bit of magic for the power user, and yet not too much for the casual user.
It's a very delicate exercice, and it receives a lot of critics for it.
An F# or Lisp dev comming to Python will complain that it's not expressive enough.
A geographer comming to Python will struggle reading advanced code.
Yet we have to catters to all of them, give the huge Python popularity and it's goal to be "the second best language for everything".
I think Guido did a very decent job at it, although he gets a lot of heat for it. People don't like to hear "no" when they ask for a poney.
And the lambda expression is one of the most controversial decisions. Beginners have a hard time with it, but professional coders may snap when they hear it's limited to one line.
For many FP programmers, the lambdas are not nearly magic enough.
In Swift, you can do something like this :
reversedNames = names.sorted(by: { $0 > $1 } )
So you see the position Python is in: it has to balance a bit of magic for the power user, and yet not too much for the casual user.
It's a very delicate exercice, and it receives a lot of critics for it.
An F# or Lisp dev comming to Python will complain that it's not expressive enough.
A geographer comming to Python will struggle reading advanced code.
Yet we have to catters to all of them, give the huge Python popularity and it's goal to be "the second best language for everything".
I think Guido did a very decent job at it, although he gets a lot of heat for it. People don't like to hear "no" when they ask for a poney.
And the lambda expression is one of the most controversial decisions. Beginners have a hard time with it, but professional coders may snap when they hear it's limited to one line.
can't say i'm a fan of the decorator to implement functional concepts. jut feels dirty. type hints in python are just as meh. feels like it's not taking advantage of pythons duck typing.
a version of try and either, with a decent do notation taking advantage of for comprehension... https://github.com/papaver/pyfnz
a version of try and either, with a decent do notation taking advantage of for comprehension... https://github.com/papaver/pyfnz
I think if that some of these concepts should just be included with the 3.9 Annotated Type
Annotated[IO[str], unpure]
Why would you want to use the I/O monad in Python?
A big reason to use IO as a value (which IMO is a better name than IO monad), is the same in all languages: reasoning about immutable values is easier than side effects. If we can take complex IO operations and use composition tools exactly the same as other immutable values, it’s very nice.
Of course, in my experience, this is so foreign to people who haven’t worked with it for a time it is very difficult to sell in small reply.
Of course, in my experience, this is so foreign to people who haven’t worked with it for a time it is very difficult to sell in small reply.
We have coroutines for that: it delegates any blocking behavior to outside of your code.
Introducing a new paradigm while this one just got in would be overkill IMO.
Introducing a new paradigm while this one just got in would be overkill IMO.
That's why a lot of functional language research is moving towards effect handlers, which could be explained in a nutshell as "coroutines, but for anything"
this looks like it will slow down the code immensely
> But, having null checks here and there makes your code unreadable.
if user is not None:
balance = user.get_balance()
if balance is not None:
balance_credit = balance.credit_amount()
if balance_credit is not None and balance_credit > 0:
can_buy_stuff = True
else:
can_buy_stuff = False
user: Optional[User]
can_buy_stuff: Maybe[bool] = Maybe.from_value(user).map( # type hint is not required
lambda real_user: real_user.get_balance(),
).map(
lambda balance: balance.credit_amount(),
).map(
lambda balance_credit: balance_credit > 0,
)
Much better, isn't it?
Yeah, I'd much rather read this:
def can_buy_stuff(user):
if user is None:
return False
balance = user.get_balance()
if balance is None:
return False
balance_credit = balance.credit_amount()
if balance_credit is None:
return False
return balance_credit > 0
def can_buy_stuff(user):
return (user?.get_balance()?.credit_amount() ?? 0) > 0writing `lambda` must get really old
I look at it as a reminder that there may be a more Pythonic way of doing whatever it is you are using all the lambdas for.
Nice premise. But it makes the Python code look super complicated.
It overloads map() with a lambda function, to compose function pipelining.
Then, it introduces flow() as the new pipelining tool. But you have to use bind() on the last function call to return the value.
Interesting concepts, but unless Python incorporates this as a standard feature, then this will remain a fringe idea. And it will add significant load to the development and maintenance process of Python programs.
Admittedly, I like Elixir’s pipe forward concept |>
That makes it super simple to do functional composition, and it will automatically bind and return the last value.
Then getting the user profile, can be like so:
It overloads map() with a lambda function, to compose function pipelining.
Then, it introduces flow() as the new pipelining tool. But you have to use bind() on the last function call to return the value.
Interesting concepts, but unless Python incorporates this as a standard feature, then this will remain a fringe idea. And it will add significant load to the development and maintenance process of Python programs.
Admittedly, I like Elixir’s pipe forward concept |>
That makes it super simple to do functional composition, and it will automatically bind and return the last value.
Then getting the user profile, can be like so:
user_profile =
userid |> get_request |> parse_json
Cool library, though.