Candle: Torch Replacement in Rust(github.com)
github.com
Candle: Torch Replacement in Rust
https://github.com/huggingface/candle
191 comments
It may just be for different people, not sure. I'm a Rust fanatic, i use it for everything, including web dev. We even have a shop full of similar Rust folks.
Personally it's my most productive language, and this is after years in Python, NodeJS, and ~5 years Go.
Which isn't to attempt to invalidate your experience. But i do think:
> writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
Is quite a leap. A language you describe not speaking natively is, unsurprisingly, difficult for you. Which is a far cry to say it's always that way.
You of course shouldn't be in a rush to rewrite anything unless you see the value in it. I make no claims of that value, either. Rewrite It In Rust (RIIR) is a meme, imo a destructive meme. But i do take issue with labeling any language broadly.
Most languages are pretty good when you're very fluent in them, and while they do have tradeoffs, i don't think Rust's tradeoffs are as you put them.
Personally it's my most productive language, and this is after years in Python, NodeJS, and ~5 years Go.
Which isn't to attempt to invalidate your experience. But i do think:
> writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
Is quite a leap. A language you describe not speaking natively is, unsurprisingly, difficult for you. Which is a far cry to say it's always that way.
You of course shouldn't be in a rush to rewrite anything unless you see the value in it. I make no claims of that value, either. Rewrite It In Rust (RIIR) is a meme, imo a destructive meme. But i do take issue with labeling any language broadly.
Most languages are pretty good when you're very fluent in them, and while they do have tradeoffs, i don't think Rust's tradeoffs are as you put them.
Rust for web dev instead of Go? I want to believe. What’s the killer app?
> Rust for web dev instead of Go? I want to believe. What’s the killer app?
No killer app here. We just prefer Rust itself. Go is very capable and a great choice.
For me personally i found Go to spread logic out far too much. Some features of Rust, Iterators for example, massively simplified how i think about data flow. That was my #1 issue with Go. This has side root-causes in Go of course, like no generics to allow for Iterators, etc - but these days Go has generics so it's likely improving. Features around Enums are also huge for me.
I left Go ages ago and it's improved a lot sense then. I still prefer Rust, though.
As i said no judgement on any lang, including Go. This is merely my experience and preferences.
No killer app here. We just prefer Rust itself. Go is very capable and a great choice.
For me personally i found Go to spread logic out far too much. Some features of Rust, Iterators for example, massively simplified how i think about data flow. That was my #1 issue with Go. This has side root-causes in Go of course, like no generics to allow for Iterators, etc - but these days Go has generics so it's likely improving. Features around Enums are also huge for me.
I left Go ages ago and it's improved a lot sense then. I still prefer Rust, though.
As i said no judgement on any lang, including Go. This is merely my experience and preferences.
I would say for me, the killer app would be just about anything you might reach for Electron with, but want a much smaller install footprint. Tauri in and of itself is pretty cool, kind of a lighter alternative to electron using native platform browser engine.
In general, I would probably reach for Rust if I needed to do anything wither Web-Assembly (wasm) as the path to working is much nicer than other languages imo. Along those lines, Yew is pretty nice as a rust-like framework.
I've also played with Rocket, and found it very nice and effective for api driven web-apps. Haven't done any templating engines in/with Rocket, however.
In general, I would probably reach for Rust if I needed to do anything wither Web-Assembly (wasm) as the path to working is much nicer than other languages imo. Along those lines, Yew is pretty nice as a rust-like framework.
I've also played with Rocket, and found it very nice and effective for api driven web-apps. Haven't done any templating engines in/with Rocket, however.
Tauri is indeed very cool, and also very small (the installation package is typically under 5MB). I used it to build a wrapper application for the ChatGPT official website, extending the site with many interesting features through script injection. However, the compatibility of the system webview across different platforms is a frustrating matter. https://github.com/lencx/ChatGPT
Tauri might be cool in theory, but Hello World consumers TWO POINT EIGHT GIGABYTES of hard disk on my machine.
Arguing "Rust has no dev speed penalty" in that context is just... no. You are damaging your own cause there, because all advocacy becomes laughable.
Arguing "Rust has no dev speed penalty" in that context is just... no. You are damaging your own cause there, because all advocacy becomes laughable.
> but Hello World consumers TWO POINT EIGHT GIGABYTES of hard disk on my machine.
While yes, that's really bad, it's just temporary build files. The actual executable should be fairly tiny.
While yes, that's really bad, it's just temporary build files. The actual executable should be fairly tiny.
So, you built the application, created the installer... and your install on another machine is 2.8GB? I find that hard to believe.
From another response, their Windows installer is under 7mb, and much of that is MSI boilerplate.
https://github.com/lencx/ChatGPT/releases/tag/v1.1.0
From another response, their Windows installer is under 7mb, and much of that is MSI boilerplate.
https://github.com/lencx/ChatGPT/releases/tag/v1.1.0
On the backend it's probably axum and tokio
https://github.com/tokio-rs/axum https://tokio.rs/tokio/tutorial
https://github.com/tokio-rs/axum https://tokio.rs/tokio/tutorial
Dioxus liveview or if you don't want websockets, dioxus fullstack, you can use warp or salvo but axum is good.
Rust embed is also great, I cross compile everything into a single dynamically linked binary and scp to my vps, deployment is a breeze.
Rust embed is also great, I cross compile everything into a single dynamically linked binary and scp to my vps, deployment is a breeze.
Yew is pretty well loved, but I haven't had a chance to do much beyond toy projects with it.
https://yew.rs/
https://github.com/yewstack/yew
https://yew.rs/
https://github.com/yewstack/yew
There are front end Rust frameworks that work via wasm.
I use Tide with great success
So if you take a Python program and you add the mental effort of having to deal with static types, lifetimes, the borrow checker and performance considerations, and this added mental effort actually allows you to write the program faster?
That is incredible. It's like being able to carve- and install a door from a slab of wood faster than installing a prefabricated door.
That is incredible. It's like being able to carve- and install a door from a slab of wood faster than installing a prefabricated door.
Ongoing maintenance and feature development on top of an existing code base is easily 10x the amount of time and effort of starting a new project greenfield. Even modest improvements in long-term readability and maintainability have disproportionately beneficial effects, even if the initial cost is somewhat high.
In my personal experience, once you're proficient in Rust, the initial cost is not somewhat high because those improvements in your ability to reason about your code base start snowballing quickly. And the long-term effects are enormous. I have literally picked up a project I hadn't worked on in years, made sweeping changes to the internals, and had every test pass in a pretty comprehensive suite the very first time it compiled.
In my personal experience, once you're proficient in Rust, the initial cost is not somewhat high because those improvements in your ability to reason about your code base start snowballing quickly. And the long-term effects are enormous. I have literally picked up a project I hadn't worked on in years, made sweeping changes to the internals, and had every test pass in a pretty comprehensive suite the very first time it compiled.
This line of reasoning is utterly alien to me. If your types add mental overhead, you’re doing something wrong. You should be able to rely on the typechecker to check invariants for you, decreasing mental overhead. You have to think about types anyway, especially in an untyped language, where you can’t rely on the typechecker.
The leap from thinking about your program with inheritance to thinking about your program with traits is large and cannot be overstated.
It's one of the reasons why Python developers don't like working in Ruby, for example.
It's one of the reasons why Python developers don't like working in Ruby, for example.
Having types lets you get better editor completion and such; I'd say Rust pushes you into bottom-up design, which I find significantly more productive than top-down too; doing bottom-up design is much trickier in a dynamic language, though, because changes to one of your "bottom layers" break upper layers in a way that the compiler/interpreter can't see
Depends, maybe time to first run is lower in python, but time to a robust/quality solution could be a very different story.
I code faster in types languages. I find typed code easier to read and understand. A lot of writing new code is just reading and understanding existing code.
So the claim doesn’t sound so crazy to me. For me at least, types remove mental overhead, they don’t add to it. I can’t count the times I’ve been reading through untyped (or inaccurately typed) python code and have had to jump a dozen levels deep just trying to figure out if a function can or cannot return null.
So the claim doesn’t sound so crazy to me. For me at least, types remove mental overhead, they don’t add to it. I can’t count the times I’ve been reading through untyped (or inaccurately typed) python code and have had to jump a dozen levels deep just trying to figure out if a function can or cannot return null.
I'd like to make the jump to FT Rust myself, just don't have enough hands on... and as I get older, motivation to work on more stuff on the side, as well as time, becomes more limited.
[deleted]
Yeah, but the goal of this isn’t “use this instead of Python to hack up a model”
…it’s:
> And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.
You know, those hugging face guys kindaaaaa do know what they’re talking about.
Like, really.
Python is a pain in the ass in production environments.
…it’s:
> And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.
You know, those hugging face guys kindaaaaa do know what they’re talking about.
Like, really.
Python is a pain in the ass in production environments.
> Python is a pain in the ass in production environments.
I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature. With those in place Python becomes great in production.
Yes it'll require your organization to take some serious CI culture decisions and also enforce them. The thing is, however, while you can get a long way without doing that with some programming languages, you're frankly always going to want them by the time you reach enterprise size. So it's just so much easier if you get it done right away, and you will, if you're serious about using Python.
I'd argue that if it's a pain to work in your production environments then the issue is not the technology but the process.
I really dislike blanket statements like that one. Because any programming language can be a pain in production environments. One of the disadvantages of Python is also one of it's advantages. Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature. With those in place Python becomes great in production.
Yes it'll require your organization to take some serious CI culture decisions and also enforce them. The thing is, however, while you can get a long way without doing that with some programming languages, you're frankly always going to want them by the time you reach enterprise size. So it's just so much easier if you get it done right away, and you will, if you're serious about using Python.
I'd argue that if it's a pain to work in your production environments then the issue is not the technology but the process.
My experience is that python is a mess in production. Even with a full CI, tests, mypy for static typing, pinned dependencies recursively, bazel with custom rules to build those dependencies (because many python dep have undeclared deps). And of course abysmal performances. And I wouldn't dare letting juniors touch python.
Rust is far from perfect. But in my opinion formed by experience, Rust is the best imperative language available right now.
Note that I have both written software in Python and built a whole devxp stack that includes Python. I wrote the first version of Docker in Python for what is worth, and at my last job; over many years; I built and maintained the polyglot dev experience for 150 devs.
Rust is far from perfect. But in my opinion formed by experience, Rust is the best imperative language available right now.
Note that I have both written software in Python and built a whole devxp stack that includes Python. I wrote the first version of Docker in Python for what is worth, and at my last job; over many years; I built and maintained the polyglot dev experience for 150 devs.
> Rust is far from perfect. But in my opinion formed by experience, Rust is the best imperative language available right now.
But what about along with the ecosystem/libraries. Would you say in total Java might be better (assuming webdev)? Especially with the Loom project where you don't have to do "async" syntax.
But what about along with the ecosystem/libraries. Would you say in total Java might be better (assuming webdev)? Especially with the Loom project where you don't have to do "async" syntax.
I understand "ecosystem/libraries" as code available for free. That a quick search online would convince most of us that it is trustworthy enough. That we hope won't require us to ever have to read, modify, and re-design the code. And if we are lucky has a state of the art abstraction for the domain it solves. So that we can get going with our task at hand as quickly and efficiently as possible.
Under this light, Rust has now quite a large ecosystem of pretty good quality I would say.
I have only written small thread safe system level libraries in Java. But I sure noticed that javac did not have a borrow checker ;)
For webdev I had no problem with Rust async or not. But my definition of webdev is reduced to a web server that calls out to a database or another service.
Under this light, Rust has now quite a large ecosystem of pretty good quality I would say.
I have only written small thread safe system level libraries in Java. But I sure noticed that javac did not have a borrow checker ;)
For webdev I had no problem with Rust async or not. But my definition of webdev is reduced to a web server that calls out to a database or another service.
Saying "any programming language can be a pain in production environments" is technically true but irrelevant and misleading.
It implies that all programming languages are equally painful in production, but that obviously isn't the case.
Python is definitely among the worst. Easily the worst among popular languages. It's closely followed by C++. Go and Rust are faaar nicer. Totally different league.
It implies that all programming languages are equally painful in production, but that obviously isn't the case.
Python is definitely among the worst. Easily the worst among popular languages. It's closely followed by C++. Go and Rust are faaar nicer. Totally different league.
> Python is definitely among the worst.
I don't think that is true at all.
Django would frankly be an excellent choice for most web apps and if you're into microservice architecture then frameworks like pantherpy are almost as great as Java's Quarkus. That being said, we do try to limit our Python use to be exclusively for the BI/AI/ML part of our business and that's solely because that's the language those people work with in my area of the world.
This is mainly for pragmatic reasons. We use Typescript for most things because it's nice to have a language that's shared between the front and backend of things when you work in the small teams that are non-tech Enterprise. Then when we need efficiency we use C (and Rust more and more because it protects us from ourselves), when we do embedded it's C and when we need something like Odata or whatever .NET(y) it's C#.
I think Go is nice to write and terrible to work with. Not so much because of the language but because you're going to be writing basically all your libraries from scratch, or alternatively, putting massive maintenance efforts into the ones you consume. Which to be fair, isn't that different from using Typescript, but as long as the frontend of enterprise apps is as it is you're going to be working with Typescript whether you like it or not.
I don't think that is true at all.
Django would frankly be an excellent choice for most web apps and if you're into microservice architecture then frameworks like pantherpy are almost as great as Java's Quarkus. That being said, we do try to limit our Python use to be exclusively for the BI/AI/ML part of our business and that's solely because that's the language those people work with in my area of the world.
This is mainly for pragmatic reasons. We use Typescript for most things because it's nice to have a language that's shared between the front and backend of things when you work in the small teams that are non-tech Enterprise. Then when we need efficiency we use C (and Rust more and more because it protects us from ourselves), when we do embedded it's C and when we need something like Odata or whatever .NET(y) it's C#.
I think Go is nice to write and terrible to work with. Not so much because of the language but because you're going to be writing basically all your libraries from scratch, or alternatively, putting massive maintenance efforts into the ones you consume. Which to be fair, isn't that different from using Typescript, but as long as the frontend of enterprise apps is as it is you're going to be working with Typescript whether you like it or not.
> Because it's much easier to end up with a terrible production environment when you use "chaos" languages like Python or JavaScript, the tooling to avoid the pain points is so mature.
Could you describe what you mean, as in what that tooling is? Having done some funky deployments in Python before, I've never been very impressed by the ecosystem and tooling.
Could you describe what you mean, as in what that tooling is? Having done some funky deployments in Python before, I've never been very impressed by the ecosystem and tooling.
>Because any programming language can be a pain in production environments.
Except one with memory safety and static builds. Like Rust can offer.
Except one with memory safety and static builds. Like Rust can offer.
I wish my bugs were mostly preventable via static typing and memory safety checks. For most ML and data work algorithm design and logic errors are the primary source of bugs. Type errors are usually the trivial things you fix in minutes.
The topic wasn't bugs though - and nobody mentioned static typing.
The grandparent said "Python is a pain in the ass in production environments".
And the parent said: "any programming language can be a pain in production environments".
My point was how Rust is better for backend services for fewer problems in production, as it wont have memory bugs (which have long plagued backend services and have resulted in many exploits), and it also offers static builds (so deployment can be dead easy, and wont be having the kind of environment issues some languages have).
I understood the grandparent to mean something about how Python has issues with dependencies, and isolation, and paths, and updates, and so on. And the parent to mean that you can have deployment issues with any language.
And to obviously neither to refer to design and logic errors, since the former wouldn't be singling out Python, and the latter would be saying a tautology. And neither would make sense to talk about those being an issue in production specifically.
The grandparent said "Python is a pain in the ass in production environments".
And the parent said: "any programming language can be a pain in production environments".
My point was how Rust is better for backend services for fewer problems in production, as it wont have memory bugs (which have long plagued backend services and have resulted in many exploits), and it also offers static builds (so deployment can be dead easy, and wont be having the kind of environment issues some languages have).
I understood the grandparent to mean something about how Python has issues with dependencies, and isolation, and paths, and updates, and so on. And the parent to mean that you can have deployment issues with any language.
And to obviously neither to refer to design and logic errors, since the former wouldn't be singling out Python, and the latter would be saying a tautology. And neither would make sense to talk about those being an issue in production specifically.
You’re right, I misread static builds as typing. Python’s dependency management system sure does suck, although I think the problem could be solved without static builds. Overall they are a nice feature, though.
I think the memory management element is a little dubious, though. Torch’s memory management, for example, is fairly safe, and it internally has its own allocator. A Rust version (I think) that implemented the same would have to rely on unsafe blocks, and would likely have many of the same potential pitfalls as Torch. Time spent satisfying Rust’s borrow checker is likely lost productivity compared to just relying on Python’s reference counting and garbage collection. That productivity could be better spent on making the product better with new features and fixes of algorithmic mistakes.
I think the memory management element is a little dubious, though. Torch’s memory management, for example, is fairly safe, and it internally has its own allocator. A Rust version (I think) that implemented the same would have to rely on unsafe blocks, and would likely have many of the same potential pitfalls as Torch. Time spent satisfying Rust’s borrow checker is likely lost productivity compared to just relying on Python’s reference counting and garbage collection. That productivity could be better spent on making the product better with new features and fixes of algorithmic mistakes.
Pytorch has multiple production paths for deployment models without the python interpreter already, like torchscript and libtorch, or more annoying paths like onnx export and onnx runtime. So there is no need to have rust to solve this problem.
Those are great technologies for making the model inference independent of Python. However, there's always a good deal of pre and post processing, or other business logic, that needs to go around model inference. This sort of thing needs to be done in a general programming language. It's in these situations where Python is often used (because the supporting model code is often written by the same person, and that code is probably going to be Python because your model training and eval code is most likely going to be Python too).
This is where having a non-Python PL, such as Rust, can make a huge difference for both ease of production deployment/maintenance as well as having truely efficient production inference.
This is where having a non-Python PL, such as Rust, can make a huge difference for both ease of production deployment/maintenance as well as having truely efficient production inference.
I've found that there is usually very little code to be shared within training and inference when most of the pre and post processing is pulled into the model itself with custom operations. Relatively few people are aware that you can write training code in C++ today with torch to share a common language with inference and training, but very few take advantage of this in practice.
This comment seems abnormally toxic for what's being discussed.
> Yeah, but the goal of this isn’t “use this instead of Python to hack up a model”
This was never said.
> You know, those hugging face guys kindaaaaa do know what they’re talking about.
This was never challenged.
> Python is a pain in the ass in production environments.
This is "I don't like python", but phrased adversarially.
Language flamewars suck. Some people don't like certain languages in certain contexts. I see Rust's syntax (having written a small to medium amount of Rust) and feel pretty confident I wouldn't task my data science team with writing or maintaining Rust codebases. We can squabble over "But eventually you'll get it", but ultimately that's more of a priorities comment than a language feature.
> Yeah, but the goal of this isn’t “use this instead of Python to hack up a model”
This was never said.
> You know, those hugging face guys kindaaaaa do know what they’re talking about.
This was never challenged.
> Python is a pain in the ass in production environments.
This is "I don't like python", but phrased adversarially.
Language flamewars suck. Some people don't like certain languages in certain contexts. I see Rust's syntax (having written a small to medium amount of Rust) and feel pretty confident I wouldn't task my data science team with writing or maintaining Rust codebases. We can squabble over "But eventually you'll get it", but ultimately that's more of a priorities comment than a language feature.
> This is "I don't like python", but phrased adversarially.
But it is an incredible pain in the ass in production environments.
Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e.
One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
Recently we had mypy stop checking one of our internal packages because of a Dockerfile refactoring. The package could still be found at runtime but the type information just wasn't found by mypy. We didn't notice it until we started getting errors at runtime, which made the fix a lot more difficult to diagnose.
There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.
For a language which states "There should be one -- and preferably only one -- obvious way to do it" as part of its guiding principles, that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".
But it is an incredible pain in the ass in production environments.
Environment drift is something that _always_ seems to happen, no matter how diligent people are about using venvs or dependency managers or w/e.
One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
Recently we had mypy stop checking one of our internal packages because of a Dockerfile refactoring. The package could still be found at runtime but the type information just wasn't found by mypy. We didn't notice it until we started getting errors at runtime, which made the fix a lot more difficult to diagnose.
There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.
For a language which states "There should be one -- and preferably only one -- obvious way to do it" as part of its guiding principles, that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".
> But it is an incredible pain in the ass in production environments.
"I find it hard to maintain Python in production environments", but worded adversarially.
> One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
This is literally just a misconfigured set of tools then.
> There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.
This is true of all languages and frameworks. There are people _in this comment section_ insisting those that have had negative Rust experiences must simply be doing it wrong. This is more a comment on our industry than any individual technology.
> For a language which states "There should be one -- and preferably only one -- obvious way to do it"
Yes, this has always been a bold faced lie from the Python community. Marketing speak is marketing speak.
> that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".
It's as much a pain in the ass as literally every piece of technology, as the above statement is true for every technology and framework that exists.
I acknowledge that you don't like it. That's great! But again, language flamewars suck, and suggesting that something is bad because you've individually had a bad experience is pretty silly. Lots of people successfully deploy and maintain python applications of various sizes without referring to the process as "a pain in the ass".
"I find it hard to maintain Python in production environments", but worded adversarially.
> One of favorites was when we did a dependency upgrade (something that required a change to Click) we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
This is literally just a misconfigured set of tools then.
> There's always someone saying "you're using it wrong" or "you're doing something worng", even if you follow the guides and howtos to get started.
This is true of all languages and frameworks. There are people _in this comment section_ insisting those that have had negative Rust experiences must simply be doing it wrong. This is more a comment on our industry than any individual technology.
> For a language which states "There should be one -- and preferably only one -- obvious way to do it"
Yes, this has always been a bold faced lie from the Python community. Marketing speak is marketing speak.
> that there is no single one way to use it right in production -- let alone an obvious one -- that makes it, simply, a "pain in the ass".
It's as much a pain in the ass as literally every piece of technology, as the above statement is true for every technology and framework that exists.
I acknowledge that you don't like it. That's great! But again, language flamewars suck, and suggesting that something is bad because you've individually had a bad experience is pretty silly. Lots of people successfully deploy and maintain python applications of various sizes without referring to the process as "a pain in the ass".
>we had an upgrade of Black forced on us in order to satisfy Poetry which then reformatted all of our Python code.
Unlikely. If you are using pre-commit to run black then the version being used lives outside your venv and hence not subject to dependencies you are using. Poetry should also be outside your venv.
Unlikely. If you are using pre-commit to run black then the version being used lives outside your venv and hence not subject to dependencies you are using. Poetry should also be outside your venv.
It's not "unlikely"; it's something that happened.
And, again, if something that was configured wrong this goes back to how managing Python is a pain in the ass. I should be able to dedicate my brain power to the problems I need to solve, not what things need to be in my venv and what things need to be outside of it.
And, again, if something that was configured wrong this goes back to how managing Python is a pain in the ass. I should be able to dedicate my brain power to the problems I need to solve, not what things need to be in my venv and what things need to be outside of it.
I agree that managing python development environments is a pain in the ass. I'm completely with you there.
But I'm curious what the nature of a black upgrade being forced on you. Like a manager or stakeholder had an edict stating everyone must upgrade to 23.x?
But I'm curious what the nature of a black upgrade being forced on you. Like a manager or stakeholder had an edict stating everyone must upgrade to 23.x?
> There's always someone saying "you're using it wrong"
I feel like your reply is demonstrating this comment perfectly. You could be perfectly correct, but their point still stands that _they_ had a horrible experience that shouldn't have been so difficult. If there's a _right_ way to do things then it should be blindingly obvious and these kinds of experiences would dry up.
I feel like your reply is demonstrating this comment perfectly. You could be perfectly correct, but their point still stands that _they_ had a horrible experience that shouldn't have been so difficult. If there's a _right_ way to do things then it should be blindingly obvious and these kinds of experiences would dry up.
A person had a bad experience. There's a decent chance that bad experience is more that person's fault than it is any technology's. But also:
> If there's a _right_ way to do things then it should be blindingly obvious and these kinds of experiences would dry up.
This has never happened for any technology ever. People misconfigure things all the time. Even hardened technologies get used incorrectly very frequently. It's just the reality of our industry. Which is why I say, all of this is just a proxy for "I didn't like my experience with Python", which is fine! But also in no way condemning of the technology.
> If there's a _right_ way to do things then it should be blindingly obvious and these kinds of experiences would dry up.
This has never happened for any technology ever. People misconfigure things all the time. Even hardened technologies get used incorrectly very frequently. It's just the reality of our industry. Which is why I say, all of this is just a proxy for "I didn't like my experience with Python", which is fine! But also in no way condemning of the technology.
I'm not getting a sense of toxicity from this discussion. Sure, people are disagreeing and have distinct positions but the manner of discussion seems constructive.
>> This was never said.
Quotes don't have to used solely to indicate somebody explicitly said something.
>> This was never said.
Quotes don't have to used solely to indicate somebody explicitly said something.
> You know, those hugging face guys kindaaaaa do know what they’re talking about.
> Like, really.
Trust me, bro?
> Like, really.
Trust me, bro?
> writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
Have you tried "sticking" with it for a while? Try forcing yoursel to use it[1]. I find that's the best way to learn it (and try reading more code from open source project in Rust you like).
I have an opposite experience writing Rust: it allows me to focus on the problem more easily, especially when the problem domain is complex and the codebase grows.
[1] a trick often used while learning human languages
Have you tried "sticking" with it for a while? Try forcing yoursel to use it[1]. I find that's the best way to learn it (and try reading more code from open source project in Rust you like).
I have an opposite experience writing Rust: it allows me to focus on the problem more easily, especially when the problem domain is complex and the codebase grows.
[1] a trick often used while learning human languages
Have you written a lot of things in Python? I remember coming "back" to Typescript and C# from Python and how much of a slog it felt. Last year I did quite a lot of work in Rust and I personally found it to be one of the worst languages I've ever worked with in terms of actually writing it.
I even like Rust, a lot. I really love things like how you actively need to make things mutable. I wish that was a standard feature in any language I work with. But it's just so verbose.
I even like Rust, a lot. I really love things like how you actively need to make things mutable. I wish that was a standard feature in any language I work with. But it's just so verbose.
> Have you written a lot of things in Python?
Not the person you responded to but yes. I have written a lot of production software in Python. I remember developing APIs with Python and the day we switched to using Typescript all these bugs we used to have just disappeared. Going from Typescript to Rust was not as big a change but well designed statically typed languages are a productivity booster not killer.
We can agree to disagree but I've personally experienced a large Python codebase at a Unicorn tech startup and I'd like to never experience it again.
Not the person you responded to but yes. I have written a lot of production software in Python. I remember developing APIs with Python and the day we switched to using Typescript all these bugs we used to have just disappeared. Going from Typescript to Rust was not as big a change but well designed statically typed languages are a productivity booster not killer.
We can agree to disagree but I've personally experienced a large Python codebase at a Unicorn tech startup and I'd like to never experience it again.
I don't think we necessarily disagree. I really enjoy using Typescript these days, but I do remember getting into it and being annoyed I had to write {}'s again. These days I'm annoyed when I can't export functions without putting them into a class though, so maybe people shouldn't listen too much to me.
> Have you written a lot of things in Python?
I've been exposed to Python through work, but I was never a fan. Backwhen I was faced with learning Ruby or Python, I picked Ruby; just felt more ergonomic. I even prefer more recent versions of TypeScript to Python too. I never really felt productive in Python, since the language is very finnicky (I would say more so than JS).
Rust might be verbose in parts of the language (but I don't find it as verbose as C or C++), but you get a lot of goodies the the price of a bit of verbosity. So, I don't mind it
I've been exposed to Python through work, but I was never a fan. Backwhen I was faced with learning Ruby or Python, I picked Ruby; just felt more ergonomic. I even prefer more recent versions of TypeScript to Python too. I never really felt productive in Python, since the language is very finnicky (I would say more so than JS).
Rust might be verbose in parts of the language (but I don't find it as verbose as C or C++), but you get a lot of goodies the the price of a bit of verbosity. So, I don't mind it
> Rust might be verbose in parts of the language (but I don't find it as verbose as C or C++), but you get a lot of goodies the the price of a bit of verbosity. So, I don't mind it
I think it's worse than C. That being said, I'm probably also biased in a way where I'm more forgiving to C++ than to Rust because C++ is old and Rust is new. What I found when I was working with Rust was that I wished you'd write it like you write Go. As for Go I quite quickly wished it worked like Rust. So I guess you can't win with me?
These days we mainly use Typescript because it's useful to mainly use a single language in small teams since it allows the one frontend guy to go on vacations without a laptop. We still do some C#, some C and some Rust + the Python which runs our ML/AI/BI parts since that's what those people work with in my area of the world.
I think it's worse than C. That being said, I'm probably also biased in a way where I'm more forgiving to C++ than to Rust because C++ is old and Rust is new. What I found when I was working with Rust was that I wished you'd write it like you write Go. As for Go I quite quickly wished it worked like Rust. So I guess you can't win with me?
These days we mainly use Typescript because it's useful to mainly use a single language in small teams since it allows the one frontend guy to go on vacations without a laptop. We still do some C#, some C and some Rust + the Python which runs our ML/AI/BI parts since that's what those people work with in my area of the world.
I agree that Rust makes it easier to program... But I'm not sure "you just have to force yourself to use it" is the best advert for Rust!
It's likely that they just haven't got to the point in their programming where they understand the need for the things Rust solves. Like if you only ever write 100 line Python scripts you probably haven't run into the performance issues or difficulties that come with working on large dynamically typed codebases.
It's likely that they just haven't got to the point in their programming where they understand the need for the things Rust solves. Like if you only ever write 100 line Python scripts you probably haven't run into the performance issues or difficulties that come with working on large dynamically typed codebases.
There are two kinds of complexity. Upfront complexity and hidden complexity. Rust with the borrow checker significantly shrinks the space of possible programs that can be written to solve a problem. Some people think this is bad because they can't write software in a way they find natural, I like it because of the possible programs that rust makes impossible to write the vast majority are incorrect and have bugs.
I also really like that when I use libraries, I know that all the library authors also subject to these same restrictions and I can trust their code much more than I could in other languages.
I also really like that when I use libraries, I know that all the library authors also subject to these same restrictions and I can trust their code much more than I could in other languages.
I was with you until you mentioned Python. The use cases for the two languages are entirely orthogonal. Rust is a fantastic C++ replacement (nothing will replace C).
My problem with Rust, now on month 3 of trying to love it, is exactly what you mentioned. It's a slog. Comparing it to other languages you have to know a lot more about systems. Initially, this isn't a problem, but as it turns out it becomes a problem with Rc/Arc/Box/etc. These are really just leaky abstractions in practice with strange behavior in many cases. Sure, you can write Rust without ever needing that. But if you ever need something more specialized than BTreeMap you're going to be chest deep in a kafkaesque nightmare of lifetimes, weird garbage collection, crazy types (Option<Box<Rc<Cell...) etc. It's at this point the helpful compiler becomes a mortal enemy. A sisyphean task made worse by the compiler suggesting the wrong things at the wrong time because you're so deep in lifetime/box/[A]Rc hell that not even God himself can save you.
As long as you can mostly use the standard library and mostly avoid ever needing to do something where resources are shared Rust isn't that bad. Once you start sharing resources your life becomes extremely painful extremely fast. In some cases this is good, requiring you to think, but in many cases it's bad due to the leaky abstractions over memory when you need to do something more. In this case, despite it's major flaws, C++ wins in my opinion. I have a problem with the abstractions over pointers in Rust because they require me to STILL know how things underlying those abstractions works.
Rust is going through its 15 minutes of fame. Much like Ruby, Elixir, etc you will have fans trying to shoehorn every possible case into it. It's a systems language. It is very good at what it is. It definitely does not belong in Data Science though it could be used to replace the underlying systems-level code found in Tensorflow/Torch/Pandas/etc a little safer. But at what cost?
I'm being hard on Rust because I think in 5 years time it'll begin to replace C++ for most new systems level code. I hope they continue to revise the API.
My problem with Rust, now on month 3 of trying to love it, is exactly what you mentioned. It's a slog. Comparing it to other languages you have to know a lot more about systems. Initially, this isn't a problem, but as it turns out it becomes a problem with Rc/Arc/Box/etc. These are really just leaky abstractions in practice with strange behavior in many cases. Sure, you can write Rust without ever needing that. But if you ever need something more specialized than BTreeMap you're going to be chest deep in a kafkaesque nightmare of lifetimes, weird garbage collection, crazy types (Option<Box<Rc<Cell...) etc. It's at this point the helpful compiler becomes a mortal enemy. A sisyphean task made worse by the compiler suggesting the wrong things at the wrong time because you're so deep in lifetime/box/[A]Rc hell that not even God himself can save you.
As long as you can mostly use the standard library and mostly avoid ever needing to do something where resources are shared Rust isn't that bad. Once you start sharing resources your life becomes extremely painful extremely fast. In some cases this is good, requiring you to think, but in many cases it's bad due to the leaky abstractions over memory when you need to do something more. In this case, despite it's major flaws, C++ wins in my opinion. I have a problem with the abstractions over pointers in Rust because they require me to STILL know how things underlying those abstractions works.
Rust is going through its 15 minutes of fame. Much like Ruby, Elixir, etc you will have fans trying to shoehorn every possible case into it. It's a systems language. It is very good at what it is. It definitely does not belong in Data Science though it could be used to replace the underlying systems-level code found in Tensorflow/Torch/Pandas/etc a little safer. But at what cost?
I'm being hard on Rust because I think in 5 years time it'll begin to replace C++ for most new systems level code. I hope they continue to revise the API.
Prototype in Python, but scale in Rust.
I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in parallel with Rust. Simple and Fast.
You can even make custom-made python modules with a Rust core when Numpy matrices are not enough to crack your graph-traversing problem efficiently ! (Thanks PyO3 and Maturin)
I do a lot of AI Engineering in Python, but production Data Engineering, and Feature Engineering workloads are done in parallel with Rust. Simple and Fast.
You can even make custom-made python modules with a Rust core when Numpy matrices are not enough to crack your graph-traversing problem efficiently ! (Thanks PyO3 and Maturin)
> Prototype in Python, but scale in Rust.
Not even. Some people start building big systems with Java or Go. It's entirely appropriate to start in Rust too once you're more proficient in it.
Not even. Some people start building big systems with Java or Go. It's entirely appropriate to start in Rust too once you're more proficient in it.
a bit difficult and controversial in Data Science, but okay. I’ve taught data scientists to use docker, I might be able to teach some Rust !
For anything that is not related to DS/ML/Opt, yes, start with Rust. I had tremendous success with distributed Rust microservices using ZeroMQ for backbone communication.
For anything that is not related to DS/ML/Opt, yes, start with Rust. I had tremendous success with distributed Rust microservices using ZeroMQ for backbone communication.
Since you mentioned Python / Rust interop I wouldn't be surprised to see this have a Python binding made for it and code will likely be easily translated over when it needs to be productionized.
I would be surprised if it does not happen. If it is not provided officially then very likey someone else will make it.
Python is the lingua franca of the ML domain, after all.
This has been my experience, too. Somewhere in Rust, there is a beautiful, functional(ish), pragmatic and simple language, hiding in between all the crappy things.
So it's just like C++ ;) except it has pattern matching and const by default.
So it's just like C++ ;) except it has pattern matching and const by default.
what being a kitchen sink language does to a mf
It'll never not have a trade-off! But there are some things that can make it easier in many cases:
- Copy structs- ownership issues vanish when things can be bitwise-copied
- .clone()- don't be afraid to clone outside of hot loops! If allocations are a problem, look at using Rc instead of Box where applicable
- Pure functions/methods- you don't need mutable references if you aren't mutating things
- Macros- reduce boilerplate
- Purpose-built traits- you can hang methods off of different structs, including std or builtin structs, that can make certain flows really ergonomic
In general: Rust asks you to care about all the little details at a language level, but it also empowers you in different ways to build high-level abstractions on top of those details so that you often don't have to care about them. I'm finding that I have the smoothest time with it when I use traits and macros to make my own little DSL for the problem I'm trying to solve, and then write my business logic in that.
And then- many libraries offer you their own abstractions that give you the same sort of benefit. Concurrency normally involves a bunch of ownership and locking headaches, but something like rayon makes it breezy by giving you an abstraction that handles those details for you. Async web route handlers with shared state from scratch would normally be hard, but axum and Rocket give you abstractions that handle the gross stuff for you. One of my favorite crates I recently discovered is just called "memoize", and it just gives you a macro you can slap on any old function to memoize its results! (global mutable state!) https://crates.io/crates/memoize. This would be a mess to implement yourself, but the abstraction makes it breezy
And then personally: the main reason I'd use a Rust library like the OP over a Python library is the tooling/dependency management. Running a Rust project with dependencies Just Works and that's extremely valuable to me. But, it'll always be weighed against the costs of using a lower-level language.
- Copy structs- ownership issues vanish when things can be bitwise-copied
- .clone()- don't be afraid to clone outside of hot loops! If allocations are a problem, look at using Rc instead of Box where applicable
- Pure functions/methods- you don't need mutable references if you aren't mutating things
- Macros- reduce boilerplate
- Purpose-built traits- you can hang methods off of different structs, including std or builtin structs, that can make certain flows really ergonomic
In general: Rust asks you to care about all the little details at a language level, but it also empowers you in different ways to build high-level abstractions on top of those details so that you often don't have to care about them. I'm finding that I have the smoothest time with it when I use traits and macros to make my own little DSL for the problem I'm trying to solve, and then write my business logic in that.
And then- many libraries offer you their own abstractions that give you the same sort of benefit. Concurrency normally involves a bunch of ownership and locking headaches, but something like rayon makes it breezy by giving you an abstraction that handles those details for you. Async web route handlers with shared state from scratch would normally be hard, but axum and Rocket give you abstractions that handle the gross stuff for you. One of my favorite crates I recently discovered is just called "memoize", and it just gives you a macro you can slap on any old function to memoize its results! (global mutable state!) https://crates.io/crates/memoize. This would be a mess to implement yourself, but the abstraction makes it breezy
And then personally: the main reason I'd use a Rust library like the OP over a Python library is the tooling/dependency management. Running a Rust project with dependencies Just Works and that's extremely valuable to me. But, it'll always be weighed against the costs of using a lower-level language.
Maybe I’m a bit ate for the party.
But I hope to get a suggestion for switching to the ext language in the real life. Like I’m quite proficient with swift language, I’m making money with it for years, so I consider my proficient comes not just from general computer science knowledge, but it comes from the getting o know the environment as well. Like I known the sdt lib, I know common caveats with the project, I have a pretty much experience it’s it after all.
And so I’ve tried rust. I mean it’s fantastic, it have awesome tooling, concise syntax and well crafted design, but I barely could write script with it. And I’m seeing clearly that this will lasts for a year or something, and even then I’ll suck at it towards an average senior dev focused on it a way beefier me.
So I really can’t crack it myself, like how do you folks switching to a new lang to get paid for it?
But I hope to get a suggestion for switching to the ext language in the real life. Like I’m quite proficient with swift language, I’m making money with it for years, so I consider my proficient comes not just from general computer science knowledge, but it comes from the getting o know the environment as well. Like I known the sdt lib, I know common caveats with the project, I have a pretty much experience it’s it after all.
And so I’ve tried rust. I mean it’s fantastic, it have awesome tooling, concise syntax and well crafted design, but I barely could write script with it. And I’m seeing clearly that this will lasts for a year or something, and even then I’ll suck at it towards an average senior dev focused on it a way beefier me.
So I really can’t crack it myself, like how do you folks switching to a new lang to get paid for it?
I've been using Rust for about five years, but I've never been paid for it and I'm not sure I ever will be. I just like it!
Personally I don't think it's worth trying to predict which technologies will be the most valuable and learning them based on that, unless you're struggling to make money right now. It sounds like Swift is working great as a money-maker for you right now, so I recommend just following your interests outside of that! Maybe they'll lead to another money-maker, maybe they won't and that's okay
I don't see Rust eating any domains outside of low-level systems development anytime soon, so I wouldn't worry that you're going to get left behind or something. It's not going to make Swift, or JavaScript, or Python, or Java, or Go, or many other things obsolete
Personally I don't think it's worth trying to predict which technologies will be the most valuable and learning them based on that, unless you're struggling to make money right now. It sounds like Swift is working great as a money-maker for you right now, so I recommend just following your interests outside of that! Maybe they'll lead to another money-maker, maybe they won't and that's okay
I don't see Rust eating any domains outside of low-level systems development anytime soon, so I wouldn't worry that you're going to get left behind or something. It's not going to make Swift, or JavaScript, or Python, or Java, or Go, or many other things obsolete
And the implementation behind those abstractions tend to be of really good quality. Thanks to the borrow checker and the scrutiny around the unsafe keyword.
In other words I am more willing to trust abstractions over tricky problems in Rust than I would in other languages.
Just for a second, try to imagine a generic memoize in C++.
In other words I am more willing to trust abstractions over tricky problems in Rust than I would in other languages.
Just for a second, try to imagine a generic memoize in C++.
Yep, exactly. The implementation - and more importantly, the contract - are much more dependable. For better and worse Rust is a language that leans all-in on abstraction, going to great lengths to make those as safe and scalable as possible, and if you aren't using them you're going to have a harder time than you need to
I'd invite you to keep up on improvements in tooling like analyzer and auto-fixing clippy warnings (tho understanding internals of linters, borrow and type checkers is a full time job and then some)
https://www.infoworld.com/article/3267624/whats-new-in-the-r...
https://www.infoworld.com/article/3267624/whats-new-in-the-r...
Rust needs a prototyping/scripting/application language on top of it, with 80% of the speed and 99% of the safety, with automatic memory management (reference counting or GC) and at least one idiomatic way of doing concurrency that disallows sharing mutable state. A language that is tailor-made for building applications quickly on top of the Rust ecosystem, with a build tool that gracefully handles mixed projects if you need to build part of your application in Rust, and support for writing small shebang scripts.
Also, I'd like a pony, a jetpack, and VIP passes to the premiere of Dune: Part 2.
Also, I'd like a pony, a jetpack, and VIP passes to the premiere of Dune: Part 2.
I've worked on a Haskell + Rust project that followed roughly the dynamic you're describing and was pretty happy with it. I wrote some simple tooling that let us share algebraic types between the two to manage our interfaces and it worked pretty well. In our case we needed to serialize/deserialize data between our components so the interfaces were at a bit of a distance, but, in principle, it would have been totally doable to extend the same kind of tooling to provide a legitimately easy-to-use FFI layer between the two languages.
Haskell is a bit of a tricky language organizationally for mostly social/political reasons, but if I end up starting my own company in the future, Haskell + Rust is definitely the tech stack I'd lean towards.
Haskell is a bit of a tricky language organizationally for mostly social/political reasons, but if I end up starting my own company in the future, Haskell + Rust is definitely the tech stack I'd lean towards.
I made the call between Haskell and Rust for my fintech startup. I chose Rust when I realized how few Haskell programmers there were available to hire - at 20 engineers, we'd be one of the largest shops on the West Coast of the US AFAIK.
That said, we were one of the largest Rust shops outside of the FAANGs, but Rust was straightforward to teach to C/C++ folks.
That said, we were one of the largest Rust shops outside of the FAANGs, but Rust was straightforward to teach to C/C++ folks.
We actually had the opposite experience in hiring for our Haskell team: it was substantially easier to find legitimately strong engineers for Haskell than for other teams using Python and Scala. I figure it comes down to two things:
1. A lot more people want to do Haskell than there are Haskell jobs
2. Haskell is an easy way to signal (show not tell) that our team is willing to do things differently—especially important because our team was at Target, which did not have much of a reputation as a tech employer!
We didn't try hiring Rust programmers because the existing Haskell folks were more than happy to pick it up.
So making hiring easier is one of the reasons I'd want to use Haskell, counterintuitively!
1. A lot more people want to do Haskell than there are Haskell jobs
2. Haskell is an easy way to signal (show not tell) that our team is willing to do things differently—especially important because our team was at Target, which did not have much of a reputation as a tech employer!
We didn't try hiring Rust programmers because the existing Haskell folks were more than happy to pick it up.
So making hiring easier is one of the reasons I'd want to use Haskell, counterintuitively!
! Amazing. I had seen so many strong Haskell teams that Rust seemed like the play, for the same reasoning.
I'm glad it worked out for both of us.
I'm glad it worked out for both of us.
I think you’re describing Swift
This would infect the language ecosystem. People that use Rust are usually using it because they don't want the performance sacrifice that gc or reference counting would bring. I am convinced that one of the reasons D did not catch on is because it had gc, an anti feature for the target demographic. You could disable it but it split the community and some of the standard libraries weren't compatible. Moreover, just about every mainstream language has automatic memory management. There are plenty of choices for those that want that, Rust is really the only mainstream choice for its niche so it's for the best to not try to turn it into something else.
A similar situation here for me. Rust might be a modern Ada for a particular use case(system programming), I was puzzled why stackoverflow ranked it as 'the most loved' language for years. For each language related post at HN, you will always see rust lover's posts there saying "I wish it's in rust", or "there is a rust version", or "rust is even better". The reality does not match up with the enthusiasm for me though, what am I missing?
After each try with Rust, I ended up going back to c++ myself, it's just more productive in practice.
After each try with Rust, I ended up going back to c++ myself, it's just more productive in practice.
Those who are enthusiastic are very much so. I would say it's less contentious compared to when Node.js started to become popular. It's still one of my favorite tools/languages, though I'd prefer to use Deno more these days.
As to falling back to any given language, that's not uncommon. Learning a new language is difficult, especially if the paradigms don't align well to what we already know, or have experience in. This may come down to experience, or it may come down to how you now think through problems. That said, I don't think that C++ is inherently more productive than Rust, I think saying your experience makes you more productive in C++ is a far more likely truth.
As to falling back to any given language, that's not uncommon. Learning a new language is difficult, especially if the paradigms don't align well to what we already know, or have experience in. This may come down to experience, or it may come down to how you now think through problems. That said, I don't think that C++ is inherently more productive than Rust, I think saying your experience makes you more productive in C++ is a far more likely truth.
Talking about my personal experience, getting downvoted heavily which was expected and not a surprise, in fact the insanity is another big reason I will not use rust, no other language has such patterns as far as I can tell.
> After each try with Rust, I ended up going back to c++ myself, it's just more productive in practice.
That has been my experience as well. Rust and the borrow checker get too much in my way. If I want a high level language I will use Python but if I want control all the way down to the hardware C++ is where I go. Plus, it /feels/ that ISO C++ has significantly picked up on momentum and is seeing more investments than Rust.
That has been my experience as well. Rust and the borrow checker get too much in my way. If I want a high level language I will use Python but if I want control all the way down to the hardware C++ is where I go. Plus, it /feels/ that ISO C++ has significantly picked up on momentum and is seeing more investments than Rust.
I write rust professionally and I really, really enjoy it. It’s lots of fun, and has found a good (and novel!) point in the space of safety vs. speed. That said, I think in the 80-90% case, a language with a garbage collector is probably more appropriate for the task at hand.
Python stands on the shoulders of giants writing C libraries. Those people think a lot of about problems you haven't had to deal with.
I can attest to this. I really think it depends on your background and what your project is. Rust has a lot of baggage from it's use in systems programming that many folks coming from C++ already know, and a lot of strange functional programming things that ML-family language folks already know, so there's just a lot of new things crammed together. Now add move-by-default semantics, and tight restrictions on pointers, and it can get really hard. They key is getting to the point where you just try to use value semantics as much as possible, and stop trying to write everything perfectly the first time (i.e. use lots of `.clone` and `.unwrap` so you can get your code written down to do the thing you're trying to do).
I am hopeful that languages like Mojo and Val are going to find cleaner ways of doing what Rust does best, and that Rust can learn from them.
I am hopeful that languages like Mojo and Val are going to find cleaner ways of doing what Rust does best, and that Rust can learn from them.
> I am hopeful that languages like Mojo and Val are going to find cleaner ways of doing what Rust does best, and that Rust can learn from them.
I suspect they'll be great for a subset of the things that makes Rust great. There's probably a bunch of essential complexity that can't be hand waved around via a "smarter compiler" at least at this point in time (I'm not saying Rust is as simple as it could be, but I suspect there's no order of magnitude improvement while fulfilling the niche that it occupies).
I suspect they'll be great for a subset of the things that makes Rust great. There's probably a bunch of essential complexity that can't be hand waved around via a "smarter compiler" at least at this point in time (I'm not saying Rust is as simple as it could be, but I suspect there's no order of magnitude improvement while fulfilling the niche that it occupies).
I don't do anything related to data science, but I feel like doing it in Rust would be nice.
You get operator overloading, so you can have ergonomic matrix operations that are typed also. Processing data on the CPU is fast, and crates like https://github.com/EmbarkStudios/rust-gpu make it very ergonomic to leverage the GPU.
I like this library for creating typed coordinate spaces for graphics programming (https://github.com/servo/euclid), I imagine something similar could be done to create refined types for matrices so you don't do matrix multiplication matrices of invalid sizes
You get operator overloading, so you can have ergonomic matrix operations that are typed also. Processing data on the CPU is fast, and crates like https://github.com/EmbarkStudios/rust-gpu make it very ergonomic to leverage the GPU.
I like this library for creating typed coordinate spaces for graphics programming (https://github.com/servo/euclid), I imagine something similar could be done to create refined types for matrices so you don't do matrix multiplication matrices of invalid sizes
Best I can see is that it helps incremental time that adds up over time
Like if your build server is backed up maybe the rust version of your language's compiler and interpreter will make your build server not be backed up because it does everything faster
But for everyone else its not that useful to be using the rust version of everything
Its just another one of those things where you see lucrative job offers requiring rust, so you make up reasons to have impactful-ish experiences in rust
Like if your build server is backed up maybe the rust version of your language's compiler and interpreter will make your build server not be backed up because it does everything faster
But for everyone else its not that useful to be using the rust version of everything
Its just another one of those things where you see lucrative job offers requiring rust, so you make up reasons to have impactful-ish experiences in rust
>So I'm not in a rush to rewrite anything written in Python just yet.
I think it's becoming clear that Python is to ML what JS is to the Web. We got stuck with it through a series of historical coincidences, and no one is super happy about it. But it mostly does the job and has evolved to fit the bill.
I'm not a huge fan but it's definitely here to stay. Hopefully Rust can take off but I don't see any other viable replacement.
I think it's becoming clear that Python is to ML what JS is to the Web. We got stuck with it through a series of historical coincidences, and no one is super happy about it. But it mostly does the job and has evolved to fit the bill.
I'm not a huge fan but it's definitely here to stay. Hopefully Rust can take off but I don't see any other viable replacement.
Yeah, Rust's sweat spot is systems programming and scenarios where no form of automatic memory management is tolerated, regardless of the reasons, for everything else there are enough AOT compiled languages with some form of automatic memory management, and features for low level coding if really needed.
I'm wondering as Mojo matures if it'll naturally take over these roles. That's what it's core mission is, and it's being developed by the very best in the biz.
I like Rust, but Mojo seems like it ought to be a much easier sell for projects that are very invested on the Python side.
I like Rust, but Mojo seems like it ought to be a much easier sell for projects that are very invested on the Python side.
[deleted]
[deleted]
I can understand why Rust feels like jumping thru hoops. In a way, working with Rust feels like working with Ruby/RoR. Is there similarities? I understand Rust is just as powerful and way safer and faster than Ruby.
I never understood why people say that rust isn’t appropriate for web dev. I think Axum is slick.
Typically the people I hear saying that are super Spring indoctrinated.
Typically the people I hear saying that are super Spring indoctrinated.
to me, rust has extra complexity related to memory management and raw performance, which are irrelevant to typical web dev.
I am not an expert in rust, but looking at it as potential choice for my next project.
I am not an expert in rust, but looking at it as potential choice for my next project.
I absolutely love the slog of writing Rust… but I would not inflict it on my coworkers for most business-type projects.
> writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
It's like this until it isn't. Eventually this falls completely away - I promise.
It's a lot like being a 100% backend engineer trying to develop in React for the first time. You just have to get used to it.
Rust is not difficult, it just takes some soaking in.
It's like this until it isn't. Eventually this falls completely away - I promise.
It's a lot like being a 100% backend engineer trying to develop in React for the first time. You just have to get used to it.
Rust is not difficult, it just takes some soaking in.
here comes the slew of rust people claiming you’re using it wrong and that anything another language does easier or more concisely is an antipattern
I've come to the personal conclusion that the issue with compiled languages and machine learning is that the API surface is so big and the operations themselves are very self-contained.
I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unrelated colab notebooks. My productivity goes up when I can quickly answer questions like "how does this transpose impact the shape of the output" or "how do I find the mean of this tensor along only these dimensions."
Compiled languages, and especially Rust, really require you to go in with a plan. I find most of the machine learning code I write is inspired by rapidly playing with ideas.
I find that I am not typically writing complex "language-level" logic that would be expressible in any type of language and would benefit from the performance of compilation. I am guessing APIs and quickly testing ideas in totally unrelated colab notebooks. My productivity goes up when I can quickly answer questions like "how does this transpose impact the shape of the output" or "how do I find the mean of this tensor along only these dimensions."
Compiled languages, and especially Rust, really require you to go in with a plan. I find most of the machine learning code I write is inspired by rapidly playing with ideas.
This seems to be explicitly targeted at deploying ML models rather than developing them
Which makes sense their serving framework has a big python component right now, but most of it is written in rust. Given that one of the lead devs for the serving framework is also working on this, I'd guess they're going to look to remove that python portion.
I'm a novice at ml, would the performance benefit here be large compared to wonnx or the onnx bindings?
I wonder about this. Is ML just plugging together prebuilt blocks because that covers the entire solution space, or are we limiting the solution space to that because that’s the only thing you can realistically do in Python?
It's worth noting that these pre-built blocks are both convenient and incredibly flexible. Before I started getting my hands dirty, it was hard to really see this. For example, here's a masked gather operation along the innermost dimension of an arbitrary rank tensor A:
A[..., idx[mask]]
Sure, but any language can provide that kind of pre-built block in a library. (Indeed, I suspect the library we're discussing has this operation, though I didn't check.) But if you want to implement a new operation that performs well, not every language can do that. In python, people tend to drop down to C or C++ or increasingly Rust to do that. But it seems (to me) like it would be nice to be able to use a single language for both things.
At least with the more common modern ML (LLMs etc) operations, no one drops down to Rust. It's really only good for IO/CPU work, but that's rarely a spot for new operations, just for data loading and high-latency network stuff.
People typically use Triton or CUDA (usually the C++ flavor) to implement operations. It'd be cool if Rust had a CUDA dialect.
People typically use Triton or CUDA (usually the C++ flavor) to implement operations. It'd be cool if Rust had a CUDA dialect.
The point is that someone dropped down to implement those operations, at some point.
It seems like you can target CUDA with Rust, because this project seems to be doing that?
It seems like you can target CUDA with Rust, because this project seems to be doing that?
https://github.com/huggingface/candle/blob/main/candle-kerne...
That’s the C++ variant of cuda, not Rust. They’re just binding it as one might in Python. At which point, what’s the value add?
That’s the C++ variant of cuda, not Rust. They’re just binding it as one might in Python. At which point, what’s the value add?
Aha! Thanks for digging that up!
You're limiting your solution space because it's the only thing exposed to you in Python.
You're not really doing "ML in Python", you're doing ML with those bunch of libraries that happens to expose limited bindings in Python.
You're not really doing "ML in Python", you're doing ML with those bunch of libraries that happens to expose limited bindings in Python.
With ANNs all you are doing is matmul, addition, log, exp, cosine, sine, and indexing.
I am saying “all” because all operations we are doing are reduced to a tree of just these operations.
It turns out that these are enough, and that’s good because floating point units are limited. You don’t want to waste transistors on operations that don’t provide a lot of value.
I am saying “all” because all operations we are doing are reduced to a tree of just these operations.
It turns out that these are enough, and that’s good because floating point units are limited. You don’t want to waste transistors on operations that don’t provide a lot of value.
My sense is it's more the latter, at least when exploring how to solve things, but it's hard to say. What is the only thing you can realistically do in Python?
I am so happy about them releasing this. A few years ago I wrote a multi-task syntax annotator in Rust using Laurent Mazare's excellent Torch binding (tch-rs, it looks like he is also working on Candle):
https://github.com/tensordot/syntaxdot
However, the deployment story was always quite difficult. The PyTorch C++ API is not stable, so a particular version of tch-rs will only work with a particular PyTorch version. So, anyone wanting to use SyntaxDot always had to get exactly the right version of libtorch (and set some environment variables) to build the project.
The idea of making an abstraction over Torch and Rust ndarray (similar to Burn) crossed my mind several times, but there is only so much that I could do as a solo developer. So Candle would be a god-given if I was still working on this project.
Seeing Candle wants to make me port curated-transformers to Candle for fun:
https://github.com/explosion/curated-transformers
https://github.com/tensordot/syntaxdot
However, the deployment story was always quite difficult. The PyTorch C++ API is not stable, so a particular version of tch-rs will only work with a particular PyTorch version. So, anyone wanting to use SyntaxDot always had to get exactly the right version of libtorch (and set some environment variables) to build the project.
The idea of making an abstraction over Torch and Rust ndarray (similar to Burn) crossed my mind several times, but there is only so much that I could do as a solo developer. So Candle would be a god-given if I was still working on this project.
Seeing Candle wants to make me port curated-transformers to Candle for fun:
https://github.com/explosion/curated-transformers
Oh man. I feel like this is a really big deal. I've been wondering what might move data scientists to Rust. Outside working through rustlings and trying new projects in Rust, I wonder if having this available will be it.
I agree. Two years ago, I tried writing data-intensive components for Python as rust extension modules. It was painful, tedious, and confusing.
I came back to it for a second try a month ago, and was blown away. PyO3 and maturin (especially maturin) are stunningly straightforward. I actually find maturin easier to use than “normal” Python packaging tools, to the extent such a thing even exists.
I really hope we will see a move towards rust, and away from extension modules written in C/C++/Fortran, just because the development experience is so much more approachable. Which is not something I thought I would say about Rust!
I came back to it for a second try a month ago, and was blown away. PyO3 and maturin (especially maturin) are stunningly straightforward. I actually find maturin easier to use than “normal” Python packaging tools, to the extent such a thing even exists.
I really hope we will see a move towards rust, and away from extension modules written in C/C++/Fortran, just because the development experience is so much more approachable. Which is not something I thought I would say about Rust!
You've convinced me. I've been waiting to see when Julia or Rust would be able to start encroaching on Python's production lockup. Time to deep dive!
[deleted]
How can I effectively approach extending Python with Rust? I encountered some challenges while attempting to translate Python objects to Rust. My intuition suggests that it's more straightforward to work with arrays or dictionaries of primitive types. Could you share your approach and perhaps point me to some helpful examples or resources for better understanding?
"I've been wondering what might move data scientists to Rust"
I mean, writ large, the answer is "The availability of the ecosystem they need to use daily in Rust, without having to rewrite it themselves." That is more than just a torch replacement, so this won't do it for data scientists, even if Python is ... painful in a lot of ways to productionize or even use.
Maybe it'll get some started on that ecosystem, though, so that a few years from now you see larger movement.
I mean, writ large, the answer is "The availability of the ecosystem they need to use daily in Rust, without having to rewrite it themselves." That is more than just a torch replacement, so this won't do it for data scientists, even if Python is ... painful in a lot of ways to productionize or even use.
Maybe it'll get some started on that ecosystem, though, so that a few years from now you see larger movement.
More than the ecosystem, it’s the speed and flexibility in prototyping python offers. I do not see rust becoming a replacement for that.
The uncomfortable fact is that you really don’t need thread safety or static typing to prototype and build models effectively.
The uncomfortable fact is that you really don’t need thread safety or static typing to prototype and build models effectively.
I guess the answer is when the ecosystem stops being composed of mostly experimentation, training and prototyping (data scientists) and turns into deployment and upkeep (ops) of said systems.
Prototyping and training doesn't pay the bills. (unless you're Hugging Face, lol)
Prototyping and training doesn't pay the bills. (unless you're Hugging Face, lol)
> I've been wondering what might move data scientists to Rust.
I think dynamic typing is the missing piece.
Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs. Ease of experimentation and flexibility are key for this use case.
I think dynamic typing is the missing piece.
Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs. Ease of experimentation and flexibility are key for this use case.
> Rust's safety guarantees are great and I like the language, but the type system is a barrier for learners, especially people who aren't full-time devs.
This surprises me. Hasn't everyone working with data (even at an "enthusiast" level) had to understand data types in the context of databases?
This surprises me. Hasn't everyone working with data (even at an "enthusiast" level) had to understand data types in the context of databases?
Yes, but Rust’s type system is waaaaay more sophisticated than your typical SQL database. Talk about “generic associated types” and you can see people start to shrink back in fear.
Plus, a surprisingly enormous fraction of the NoSQL hysteria was that “you don’t even need a schema!”
Plus, a surprisingly enormous fraction of the NoSQL hysteria was that “you don’t even need a schema!”
> Plus, a surprisingly enormous fraction of the NoSQL hysteria was that “you don’t even need a schema!”
The problem with this is that you have a schema regardless. In "schemaless" systems, you still have a schema. You just don't have it written down anywhere. And because its not written down, the database doesn't enforce your schema. So its very common for nosql databases to have records with missing fields or subtly corrupted data.
That said, there's a reason why nosql became popular. I think a large part of it is that SQL databases are too inflexible. You have to define the entire schema ahead of time, and schema evolution is way too difficult. When I'm writing software, I don't define all of my columns ahead of time. I want to add and change stuff as I go.
I wish databases were more flexible. Like, let me just add arbitrary extra fields to my records. Let the database recommend promoting a dynamic field to a static field. ("Hey, all your records have a name: string field. Wanna make that required?").
I love programming languages with similar features. I love typescript for prototyping because I can start with the type any (or wildcard fields in my interfaces - like [k: string]: any). And then I can formalize my types later when I understand my software better.
I don't think schemas are a mistake for beginners. I think the opposite - they're amazing guard rails.
But I agree with other commenters here. Rust's types include information about ownership, lifetime, containment (Box / Rc), traits and the stored data type all in one go. Its terrifying to beginners and exhausting for experts to use. I've been writing rust for years, and the Future trait still scares me.
The problem with this is that you have a schema regardless. In "schemaless" systems, you still have a schema. You just don't have it written down anywhere. And because its not written down, the database doesn't enforce your schema. So its very common for nosql databases to have records with missing fields or subtly corrupted data.
That said, there's a reason why nosql became popular. I think a large part of it is that SQL databases are too inflexible. You have to define the entire schema ahead of time, and schema evolution is way too difficult. When I'm writing software, I don't define all of my columns ahead of time. I want to add and change stuff as I go.
I wish databases were more flexible. Like, let me just add arbitrary extra fields to my records. Let the database recommend promoting a dynamic field to a static field. ("Hey, all your records have a name: string field. Wanna make that required?").
I love programming languages with similar features. I love typescript for prototyping because I can start with the type any (or wildcard fields in my interfaces - like [k: string]: any). And then I can formalize my types later when I understand my software better.
I don't think schemas are a mistake for beginners. I think the opposite - they're amazing guard rails.
But I agree with other commenters here. Rust's types include information about ownership, lifetime, containment (Box / Rc), traits and the stored data type all in one go. Its terrifying to beginners and exhausting for experts to use. I've been writing rust for years, and the Future trait still scares me.
A lot of science/engineering/ML/data analysis happens from hacking around with only a vague idea of what to expect. Appeasing a type checker simultaneously is an enormous blow to productivity. Obviously possible, but challenging to get into a flow state when you cannot quickly iterate on ideas.
I mean you can still hold data in hashmaps no?
Or even serde::Value(s)? Its not that everything has to be a type
Or even serde::Value(s)? Its not that everything has to be a type
If we're going to bypass the type system entirely then why are we using Rust at all? Might as well just use Python at this point
Who said entirely? I’ve used hashmaps prototyping my code, still used types in other areas, then converting dynamic maps to types too, Was just suggesting ways to handle types during prototyping
There's a bit of general dev snobbery to assume that data scientists can't code, or don't want to learn to code, or are able to understand so many other things but not a compiled language or a static type system.
It's not about ability to code, willingness to learn, or ability to understand. It's about friction. When you're iterating quickly and you're thinking about complicated mathematics and ideas, a complicated type system just adds a ton of friction. This is especially annoying when you know what you're doing is correct but the type system is unsatisfied and demands proof.
the place in DL/ML where typing could be the most useful is to keep you from trying to multiply differently sized tensors together. So the irony is that the one place where typing WOULD help DL/ML, is the place where rust is weak in its typing system (dependent types and const generics)
I keep checking the progress on dfdx for this reason. It does what I (and, I assume from context, you) want: Provides static checking of tensor shapes. Which is fantastic. Not quite as much inference as I'd like but I love getting compile-time errors that I forgot to transpose before a matmul.
It depends on the generic_const_exprs feature which is still, to quote, "highly experimental":
https://github.com/rust-lang/rust/issues/76560
Definitely not for production use, but it gives a flavor for where things can head in the medium term, and it's .. it's nice. You could imagine future type support allowing even more inference for some intermediate shapes, of course, but even what it has now is really nice. Like this cute little convnet example:
https://github.com/coreylowman/dfdx/blob/main/examples/night...
It depends on the generic_const_exprs feature which is still, to quote, "highly experimental":
https://github.com/rust-lang/rust/issues/76560
Definitely not for production use, but it gives a flavor for where things can head in the medium term, and it's .. it's nice. You could imagine future type support allowing even more inference for some intermediate shapes, of course, but even what it has now is really nice. Like this cute little convnet example:
https://github.com/coreylowman/dfdx/blob/main/examples/night...
You underestimate the gusto of data engineers/scientists to just assume types.
Not with an ownership model and lifetimes. Most of the scientists I know have not even dealt with parametric types.
It is not a barrier for learners. It is a barrier for people that just want to hack something together that works for one particular use-case.
Python is not popular because it is "easy" (far from it if you have to maintain it). But because it is easy to create executable code that does what you want, without having to have a background in coding.
Python is not popular because it is "easy" (far from it if you have to maintain it). But because it is easy to create executable code that does what you want, without having to have a background in coding.
The type system is certainly not a barrier, it’s actually rather straightforward unless I am missing something.
If you refer to lifetimes and borrow checker, those are different.
If you refer to lifetimes and borrow checker, those are different.
> I've been wondering what might move data scientists to Rust.
If they didn't migrate to Julia they certainly won't migrate to Rust.
If they didn't migrate to Julia they certainly won't migrate to Rust.
As someone who has made that choice in this context, Rust is low-level but an increasing part of the stack, and Julia always had weird edge cases for me.
I tried to do some data analysis in Rust but I came back to Python quickly. There are Numpy alternatives for most features but they're all scarcely documented, not entirely compatible, or are missing methods and implementations that Numpy readily provides.
With intensive Python processing being mostly C anyway, most data processing doesn't really benefit much from a Rust implementation. The actual native code implementation can probably be a lot better if that part switches over to Rust, with all of Rust's safeties and guarantees, but I don't think data science is going to switch to Rust for gluing together their actual calculations any time soon.
With intensive Python processing being mostly C anyway, most data processing doesn't really benefit much from a Rust implementation. The actual native code implementation can probably be a lot better if that part switches over to Rust, with all of Rust's safeties and guarantees, but I don't think data science is going to switch to Rust for gluing together their actual calculations any time soon.
I sense that most folks will end up using this via a Python wrapper, if at all, much like how HF has Transformers written in Rust but with a Python wrapper.
This library is targeting production, a much smaller subset of engineers. See the 'why candle' section: https://github.com/huggingface/candle#why-candle. So you could use PyTorch to build models and Candle to serve them.
This library is targeting production, a much smaller subset of engineers. See the 'why candle' section: https://github.com/huggingface/candle#why-candle. So you could use PyTorch to build models and Candle to serve them.
> I've been wondering what might move data scientists to Rust.
The two things that keep Python as the incumbent in this space is A.) It gets out of the way, sometimes to a fault, such that you can kinda write it without knowing what you're doing. This is a feature for most people, not a bug.
And B.) All the math/science libraries that already exist for it.
You can solve B, but until you solve A, Python stays put I think.
The two things that keep Python as the incumbent in this space is A.) It gets out of the way, sometimes to a fault, such that you can kinda write it without knowing what you're doing. This is a feature for most people, not a bug.
And B.) All the math/science libraries that already exist for it.
You can solve B, but until you solve A, Python stays put I think.
And C) it is taught at every University that teaches ML/DS.
I think unseating Python as the lingua franca will takes as long as it has taken to unseat MATLAB in engineering. 20+ years...
Of course other things will also exist, and all languages will have decent ML support (especially for inference).
I think unseating Python as the lingua franca will takes as long as it has taken to unseat MATLAB in engineering. 20+ years...
Of course other things will also exist, and all languages will have decent ML support (especially for inference).
[deleted]
id imagine repl, junpyter notebooks are pretty useful and not doable in rust atm
Rust in Jupyter Notebooks: https://users.rust-lang.org/t/getting-started-with-rust-jupy...
REPLit for Rust: https://replit.com/languages/rust
For general REPL and rust appears there are a few options.
I don't think getting rid of Python completely will happen. But current approaches _typically_ looks something like:
- Jupyter Notebook for custom parts of EDA
- Jupyter Notebook for testing new libraries or learning new methods
- Pipeline (replaceable with Rust) for standard EDA
- Pipeline (replaceable with Rust) for data quality
- Application/pipeline for product development
REPLit for Rust: https://replit.com/languages/rust
For general REPL and rust appears there are a few options.
I don't think getting rid of Python completely will happen. But current approaches _typically_ looks something like:
- Jupyter Notebook for custom parts of EDA
- Jupyter Notebook for testing new libraries or learning new methods
- Pipeline (replaceable with Rust) for standard EDA
- Pipeline (replaceable with Rust) for data quality
- Application/pipeline for product development
hmm ill look later, i thought they needed some compiler work done and miri to do it
Interesting. The repo was owned by Laurent Mazare[0] a few days ago, I'm wondering if HuggingFace just hired them to work on candle full time?
[0]: Also maintainer of https://github.com/LaurentMazare/tch-rs
[0]: Also maintainer of https://github.com/LaurentMazare/tch-rs
I think the first Rust ML framework to gain massive traction will be the one that supports quantisation out of the box.
At the moment there's no Rust ML library that does what llama.cpp or ggml can do.
At the moment there's no Rust ML library that does what llama.cpp or ggml can do.
Nowhere near as neat as candle or ggml, but just released a 4-bit rust llama2 implementation with simd. Runs pretty fast.
https://github.com/srush/llama2.rs/
https://github.com/srush/llama2.rs/
That is _really_ cool. Would be interested in some requirements being posted. Eg RAM required for the 70B model if CPU, VRAM required if GPU, etc.
edit: I know it's memory mapping, but that still loads data in RAM to execute so if you had 512MiB of RAM it would likely be slow as hell.
edit: I know it's memory mapping, but that still loads data in RAM to execute so if you had 512MiB of RAM it would likely be slow as hell.
Yeah, it's CPU only, and it is using about 38g for 70B and 7g 7B. Guessing that is mostly from the large caches that it keeps around for efficiency. If you wanted to pay some computational cost, you could likely get that down by quantizing activations. Llama2 has some of these tricks built in automatically, for instance grouped query attention.
That's really cool.
They already support quantization out of the box!
https://github.com/huggingface/candle/pull/314/files
https://github.com/huggingface/candle/pull/314/files
I would love to see a single binary application with bleeding edge neural networks in it without all the complexity and bloat added when using Python in distributed software programs.
I guess you would be able to achieve it even before but this library should help a lot.
I guess you would be able to achieve it even before but this library should help a lot.
What's the multithreading story? tch-rs's (and libtorch's) lack of real threading support was causing problems for my async code - at least, I haven't found a way to work around it yet, given my level of rust knowledge.
Tangentially: I keep thinking an ML framework could be done based on Rust macros, so the automatic differentiation is done entirely at compile time. `let grad = autodiff!(x + y);` (The `autodiff` crate does this at runtime IIUC). Then the compiler's optimizations can apply to the static computation graph.
I always thought that was the vision for TF in the early days, but clearly it moved in a more dynamic direction under PyTorch's influence.
Tangentially: I keep thinking an ML framework could be done based on Rust macros, so the automatic differentiation is done entirely at compile time. `let grad = autodiff!(x + y);` (The `autodiff` crate does this at runtime IIUC). Then the compiler's optimizations can apply to the static computation graph.
I always thought that was the vision for TF in the early days, but clearly it moved in a more dynamic direction under PyTorch's influence.
The tensors should be Send and Sync so can be manipulated from multiple threads, the underlying data is protected by a RWLock to guard against data races. The heavy operations such as matrix multiplication will be run on multiple cores even without using some explicit threading.
"Torch replacement"? Title should be "Candle: Minimalist ML framework for Rust" if anything.
Exciting tech, looking forward to see it running on wgpu! There is an issue about WebGPU although I don’t notice references to the native version of it.
https://github.com/huggingface/candle/issues/344
https://github.com/huggingface/candle/issues/344
Burn (deep learning framework in rust) has WGPU backend (WebGPU) already. Check it out https://github.com/burn-rs/burn. It was released recently.
ML on WebGPU is coming don't worry!
I have been occasionally checking https://www.arewelearningyet.com for where these problems are being solved. It doesn't look like Candle is listed there yet.
A logically connected name for a Torch-like project in Rust could have been Thermite Torch. You can make a thermite torch given that rust is iron oxide :)
Love the shoutut to dfdx. Haven't tried that crate yet but type checked matrix shapes sounds pretty good
This looks like a great project!
I have a question: Why is it necessary to specify the device for every Tensor? Wouldn't it be possible to set the device once and then all allocations are made to that device?
I have a question: Why is it necessary to specify the device for every Tensor? Wouldn't it be possible to set the device once and then all allocations are made to that device?
Anyone remember that Torch was first glued with Lua before PyTorch?
Hopefully the library will support in the future the shapes of the tensors in the type system, that would be great to prevent many errors.
After a few years, I am coming to an opposite opinion on this. Yeah, shape checking is nice and all and you should be able to run them before-hand (might be a linter pass). But inside type-system while most people have LSP running in the background and bark at you on any type mismatch? It is not going to be a pleasant development experience.
I cannot really see how it can be a worse experience than waiting for the model to run, fix the error found, wait a second time, fix another error; when you can just check instantly if the model architecture is sound.
Because that's not my workflow looks like?
* Fire up Jupyter notebook, create some PyTorch nn.Module, validate the output is expected.
* Built out the model, validate the output is expected.
* Check the data shape, do a eval() run with the data.
* Move out of Jupyter notebook (or not), to actually run the trainer.
Each of the "build out model" phase, shape inference at compile time would be kind of annoying especially if you have a LSP constantly running compilation.
* Fire up Jupyter notebook, create some PyTorch nn.Module, validate the output is expected.
* Built out the model, validate the output is expected.
* Check the data shape, do a eval() run with the data.
* Move out of Jupyter notebook (or not), to actually run the trainer.
Each of the "build out model" phase, shape inference at compile time would be kind of annoying especially if you have a LSP constantly running compilation.
That's also my workflow but the "Check the data shape, do a eval() run with the data." is the annoying process I was describing, the iterative "waiting for the model to run", "see that it doesn't work", "print random tensor shape trying the debug the code" etc... There is nothing annoying about having an oracle that can check it instantly unless you think that suffering should be part of the experience.
I am not arguing against having an oracle. Just saying in the static type-system (like the ones in Rust / C++ / Swift) would be too frequent to invoke to inhibit development. I think having a linter pass or similar is great. Much like TypeScript actually, where the type hint is the addon rather than intrinsic to the development process, making various things easier at REPL time.
Is this just an issue with the way the editor is presenting the LSP feedback to you?
Looks great! Rust is the wrong language for developing ML models, but it has a clear potential for being the best one for deploying them.
In the video game Rust, you spawn with a Torch. They also just did a torch replacement in game. Maybe it's time for a break...
I've been playing Rust lately. Take from that what you will, hahah.
Could this also run quantized models like llama.cpp?
Neat. This is the library I’ve been waiting to see. Rust has some nice potential but big parts of the ecosystem are missing. Now let’s see if it actually gets traction.
Would really wish something like that for Go. Rust is cool and all, but I just can’t be arsed to care about the language more than solving actual business problems.
this couldn’t happen soon enough.
Oh, another Candle and Torch. There are also programs called "Candle" and "Torch" inside the WIX toolset, a widely used generator of Microsoft Windows installers. (Not to be confused with Wix.com, the heavily promoted low-end web hosting service.)
I'll say it though I expect violent downmoderation: Rust is way too political for me to ever consider using it for anything real. I've been starting to worry the same about python too (and really all of software) but Rust and Mozilla have that vibe of wanting to go way beyond software development and into identity politics, and it's dangerous to have dependencies with those priorities. Especially in AI because there's an overlap with the "safety" nonsense. New frameworks are always interesting but I wouldn't use rust seriously for anything until the ecosystem broadens enough to incorporate all viewpoints and get away from politics.
Despite all the drama that happens every second month, I'm still "Rust curious". Working with Rust is fun, and I hope it will be my main language in my next job.
You are not the first that noticed the overly political nature of the Rust "community", not a coincidence that we see a blue haired individual in this skit https://youtu.be/TGfQu0bQTKc
I didn't downvote you, but you might be downvoted not because you are wrong, but because it derails the conversation. Turning every Rust thread into a "The Foundation is too woke" can be annoying.
You are not the first that noticed the overly political nature of the Rust "community", not a coincidence that we see a blue haired individual in this skit https://youtu.be/TGfQu0bQTKc
I didn't downvote you, but you might be downvoted not because you are wrong, but because it derails the conversation. Turning every Rust thread into a "The Foundation is too woke" can be annoying.
I'm sure it's annoying, but it's better than staying quiet. If people like the language on it's technical merits, which is believe is the majority of Rust enthusiasts, it's important for them not to stand by and let ideologues hijack their software project for political aims.
I couldn't agree more. This has turned me away from Rust as well.
Rust is in a weird place where people are trying to push it in a bunch of places where it doesn't belong naturally (web dev, data science) but then when you complain that it's difficult to use you get told that you're using the wrong language for your use case...
I really want to like Rust - I've read the Rust book, done the Rustling exercises, read Zero to Production in Rust and did the project there, and still it's not clicking - writing Rust is a slog where you have to jump through hoops and spend a lot more time thinking about programming language abstractions than thinking about the problem you're solving.
So I'm not in a rush to rewrite anything written in Python just yet.