Ask HN: Why are a lot of AI projects written in Python?
11 comments
If you're talking about the industry, machine learning and data science projects are inherently experimental processes that rely on setting up quick feedback loops to analyze your data and see what works.
The ideal setup for that is the interpreted shell/compiled core approach that languages like Python have championed forever. Competitors to Python would be the likes of R, Lua, Ruby, Perl, Javascript; not C, C++ or Rust.
Why Python in particular? The answer is that it has an excellent scientific stack with Numpy, Scipy and Pandas.
R is very competitive for statistical work.
Julia is an interesting, if unproven, alternative.
Once you're done with your model, you write it in C (or whatever) for production, but that's a more traditional software engineering project.
In academia, the incentives are such that most people don't really care about code. Their careers depend on them publishing papers, so that's where they're going to focus their efforts. Code is a way to prove a point for publication, not a product that needs to generate value.
> a lot of things are popularly rewritten in Rust or Zig
Beware of the HN bubble. Rust and especially Zig are tiny, fringe players in the industry, regardless of their technological merits.
The ideal setup for that is the interpreted shell/compiled core approach that languages like Python have championed forever. Competitors to Python would be the likes of R, Lua, Ruby, Perl, Javascript; not C, C++ or Rust.
Why Python in particular? The answer is that it has an excellent scientific stack with Numpy, Scipy and Pandas.
R is very competitive for statistical work.
Julia is an interesting, if unproven, alternative.
Once you're done with your model, you write it in C (or whatever) for production, but that's a more traditional software engineering project.
In academia, the incentives are such that most people don't really care about code. Their careers depend on them publishing papers, so that's where they're going to focus their efforts. Code is a way to prove a point for publication, not a product that needs to generate value.
> a lot of things are popularly rewritten in Rust or Zig
Beware of the HN bubble. Rust and especially Zig are tiny, fringe players in the industry, regardless of their technological merits.
> Once you're done with your model, you write it in C (or whatever) for production, but that's a more traditional software engineering project.
This doesn't always need to happen IRL, most large ML systems are running TensorFlow/PyTorch (a few run C++ without the Python layer).
This doesn't always need to happen IRL, most large ML systems are running TensorFlow/PyTorch (a few run C++ without the Python layer).
The best libraries are in Python.
> Curious that since a lot of things are popularly rewritten in Rust or Zig, why has this not happened on these types of projects?
1. It's a lot of work and you need to good hardware support. There are lots of people putting in a lot of work into PyTorch, TensorFlow, JAX etc.
2. Some things have been rewritten, mostly as hobby projects.
3. Even if you rewrote most of this sutff, it would be likely be more difficult to use. Writing Rust is a completely different experience from writing Python + PyTorch/TensorFlow.
4. Zig is a thing only on HN and in similar mini-bubbles. Hardly anyone has even heard of Zig, let alone considering to rewrite anything in it.
> Curious that since a lot of things are popularly rewritten in Rust or Zig, why has this not happened on these types of projects?
1. It's a lot of work and you need to good hardware support. There are lots of people putting in a lot of work into PyTorch, TensorFlow, JAX etc.
2. Some things have been rewritten, mostly as hobby projects.
3. Even if you rewrote most of this sutff, it would be likely be more difficult to use. Writing Rust is a completely different experience from writing Python + PyTorch/TensorFlow.
4. Zig is a thing only on HN and in similar mini-bubbles. Hardly anyone has even heard of Zig, let alone considering to rewrite anything in it.
> Curious that since a lot of things are popularly rewritten in Rust or Zig
Are you referring to personal or professional codebases rewritten in Zig? Personally I haven't seen the Zig language used professionally in 1 company. And my impression from the last discussion about Zig:
https://news.ycombinator.com/item?id=33331549
was that the compiler was still too unstable to be used for real work.
Are you referring to personal or professional codebases rewritten in Zig? Personally I haven't seen the Zig language used professionally in 1 company. And my impression from the last discussion about Zig:
https://news.ycombinator.com/item?id=33331549
was that the compiler was still too unstable to be used for real work.
It's username-404. Replying from here because I can't remember my password.
> https://bun.sh/
From their GitHub: "Bun is experimental software".
> https://tigerbeetle.com/
From their GitHub: "TigerBeetle is not yet production-ready".
> https://www.youtube.com/watch?v=SCj2J3HcEfc (Uber)
From the linked blog post "Uber does not have any plans to use zig-the-language yet."
:(
I was referring to production-ready examples.
> https://bun.sh/
From their GitHub: "Bun is experimental software".
> https://tigerbeetle.com/
From their GitHub: "TigerBeetle is not yet production-ready".
> https://www.youtube.com/watch?v=SCj2J3HcEfc (Uber)
From the linked blog post "Uber does not have any plans to use zig-the-language yet."
:(
I was referring to production-ready examples.
Yeah you forgot your password for sure, not like your reply from that account got flagged already... right?
https://ibb.co/5LcjFgD ;^)
https://ibb.co/5LcjFgD ;^)
that's a regulation sized oof if I've ever seen one.
Because it is very easy to write interop in python with other langs, e.g. c/cpp in which most of the underlying implementations are written
Because Python has numpy/scipy, at least every AI project I have observed has these libraries.
Curious that since a lot of things are popularly rewritten in Rust or Zig, why has this not happened on these types of projects?