PyO3: Python Rust binding(github.com)
github.com
PyO3: Python Rust binding
https://github.com/PyO3/pyo3
15 comments
Very cool! Good binding libraries between C++ and Python have been super useful for my projects in the past, and I'm glad to see something similar developed for Rust.
For anyone reading the comments first, this is a fork of rust-cpython. The motivation for the fork is linked from the Readme: https://github.com/PyO3/pyo3/issues/55
For anyone reading the comments first, this is a fork of rust-cpython. The motivation for the fork is linked from the Readme: https://github.com/PyO3/pyo3/issues/55
It looks like you forked because rust-cpython was abandoned. I see that Dan Grunwald pushed a commit in today after no activity since May. He probably just went on summer vacation.
rust-cpython will most likely stay abandoned; I rarely have the time (or interest) to work on it. The commit today was me just finally finishing up a change I started in February...
What happened with the fork: in May @fafhrd91 contacted me about joining the project. When I had time to consider his request (on the following weekend), he had already forked the project and deleted all his original pull requests. He even deleted his bug report about that segfault he keeps mentioning when comparing PyO3 to rust-cpython.
On the technical side, the API changes in PyO3 are interesting. It's kind of a return to my original design (with `&'a PyObject<'p>`), but he makes it work with just a single lifetime by using a "release pool" that temporarily holds ownership over the python references. This does make the API significantly nicer to use, at the cost of leaking any Python objects until the GIL is released / until the extension module returns to the interpreter. For the typical extension module usecase, this doesn't hurt much. But a long-running loop that allocates a temporary Python object each iteration might run out of memory due to this design.
What happened with the fork: in May @fafhrd91 contacted me about joining the project. When I had time to consider his request (on the following weekend), he had already forked the project and deleted all his original pull requests. He even deleted his bug report about that segfault he keeps mentioning when comparing PyO3 to rust-cpython.
On the technical side, the API changes in PyO3 are interesting. It's kind of a return to my original design (with `&'a PyObject<'p>`), but he makes it work with just a single lifetime by using a "release pool" that temporarily holds ownership over the python references. This does make the API significantly nicer to use, at the cost of leaking any Python objects until the GIL is released / until the extension module returns to the interpreter. For the typical extension module usecase, this doesn't hurt much. But a long-running loop that allocates a temporary Python object each iteration might run out of memory due to this design.
sorry about not waiting longer. I didn't want to fork.
as for leaking refs, it maybe be fixed by introducing nested "Pool" objects. objective-c runtime has similar concepts and same problem with long running loops.
as for leaking refs, it maybe be fixed by introducing nested "Pool" objects. objective-c runtime has similar concepts and same problem with long running loops.
> He even deleted his bug report about that segfault he keeps mentioning when comparing PyO3 to rust-cpython.
That is just sleazy :/.
That is just sleazy :/.
A part that bothers me about the PyO3 design is this: does it mean you can't keep Python objects around with the GIL released? e.g. looking at this comment [https://github.com/PyO3/pyo3/issues/55#issuecomment-31728794...] they compare
In PyO3 (second snippet) that does not seem to be the case, the object itself has a lifetime dependency on holding the GIL. So you can't e.g. release the GIL to do native processing if you'll still need the native objects afterwards.
impl PyList {
fn new(py: Python) -> PyList {...}
fn get_item(&self, py: Python, index: isize) -> PyObject {...}
}
impl PyList {
fn new(py: Python) -> &PyList {...}
fn get_item(&self, index: isize) -> &PyObject {...}
}
In PyO3 (second snippet) that does not seem to be the case, the object itself has a lifetime dependency on holding the GIL. So you can't e.g. release the GIL to do native processing if you'll still need the native objects afterwards.
you have two options: PyObject and Py<PyList>. Both has Send + Sync and all pyo3 types defines `std::convert::From<T>` for both.
So this is for running Rust code from Python... Just curious, is there an existing way to do the opposite and run Python code from Rust?
this binding can be used both ways. check first example in readme. and https://github.com/PyO3/pyo3/blob/master/src/python.rs#L203
pybind11 is a similar library, but for interfacing with C++ from Python:
https://github.com/pybind/pybind11
https://github.com/pybind/pybind11
https://github.com/PyO3/tokio