Using Rust for 'Scripting'(chriskrycho.com)
chriskrycho.com
Using Rust for 'Scripting'
http://www.chriskrycho.com/2016/using-rust-for-scripting.html
125 comments
I've been using Rust to build a relatively large cross-platform command-line tool, and I've been genuinely impressed at how easy it is to get things to work on Windows. The Rust standard libraries Just Work on Windows, and they provide pathname handling, threads, environment variables, and a wealth of other tools. Many third party libraries will build for Windows without any problems as well.
I actually do think that Rust's type system helps a bit, because it forces me to consider weird portability issues (such as the way Windows permits slightly invalid Unicode, which Rust represents as OSString instead of String).
And of course, cross-compilation with MinGW is a real help. Sadly, OpenSSL still poses a fair number of portability issues, since it's notoriously difficult to link statically.
I actually do think that Rust's type system helps a bit, because it forces me to consider weird portability issues (such as the way Windows permits slightly invalid Unicode, which Rust represents as OSString instead of String).
And of course, cross-compilation with MinGW is a real help. Sadly, OpenSSL still poses a fair number of portability issues, since it's notoriously difficult to link statically.
Yup, as someone who's done x-platform C++(Windows/OSX/X360/PS3/etc) for a large part of my career the cross platform support is incredibly well done.
Not having to deal with differences between MSVC, LLVM, GCC and other obscure toolchains is a blessing.
Not having to deal with differences between MSVC, LLVM, GCC and other obscure toolchains is a blessing.
This is one reason why I enjoyed using Java so much in the early days, even without JIT.
The differences between MSVC, LLVM, GCC are nothing to worry about, compared with the early days of C89 or C++ARM in the process of being standardized, coupled with different levels of POSIX support.
The differences between MSVC, LLVM, GCC are nothing to worry about, compared with the early days of C89 or C++ARM in the process of being standardized, coupled with different levels of POSIX support.
> such as the way Windows permits slightly invalid Unicode
Windows allows invalid UTF-16 while Unixes allow invalid UTF-8.
Windows allows invalid UTF-16 while Unixes allow invalid UTF-8.
Totally fair. I summed up that way because this is the kind of thing I'd have done with Python historically, and it struck me as neat that it's so easy to do with Rust. As someone pointed out on Reddit: this isn't really "scripting", and that's correct. I put it in "scare quotes" for a reason. ;p
Is it fair to assume the actual task was more complicated than renaming .cha to .txt and you just made it more generic here for example's sake?
No, actually: it really was that simple, and I told my friend it was a super simple thing.
I took the fact that it was so simple as an excuse to do the more complicated things alongside a very simple piece of code. That way, it's easy to see what the code does and then think about the implications of being able to run something like it (including something more complicated) cross-platform.
I took the fact that it was so simple as an excuse to do the more complicated things alongside a very simple piece of code. That way, it's easy to see what the code does and then think about the implications of being able to run something like it (including something more complicated) cross-platform.
>Writing it in Rust means I can compile it and hand it to him, and he can run it. And that’s it. As wonderful as they are, the fact that languages like Python, Perl, Ruby, JavaScript, etc. require having the runtime bundled up with them makes just shipping a tool a lot harder—especially on systems which aren’t a Unix derivative and don’t have them installed by default.
I'm wondering what part of the twelve step compilation process was found to be easier than simply installing Python on the target machine?
Does the friend have any ability to audit the compiled binary before he runs it? Can he make a small change when he wants to re-use part of the solution?
I'm wondering what part of the twelve step compilation process was found to be easier than simply installing Python on the target machine?
Does the friend have any ability to audit the compiled binary before he runs it? Can he make a small change when he wants to re-use part of the solution?
> I'm wondering what part of the twelve step compilation process was found to be easier than simply installing Python on the target machine?
That part is something that you can control.
Giving instructions for someone non-technical to install Python and run a script with it, and then helping them figure out how to interpret those instructions, is a heck of a lot more complicated.
And yeah, you can diddle around with cx_freeze and then having to spend some time with InnoSetup or WiX to build an installer, and then do the same on OS X but putting everything into an app bundle inside of a disk image instead, and finally on Linux trying to figure out how to bundle something up to play nicely with all of the major distros and packaging systems, and get something to work, though it doesn't support cross compilation so you now need to set up and maintain three different build machines, which is definitely no simpler than the solution described.
All of these solutions have their ups and downs. But being able to relatively easily (modulo installing a linker and a few libraries) cross compile from Rust to a static executable for any of the major desktop platforms is pretty nice, after you've dealt with enough of these funky build and packaging systems or trying to walk people through figuring out how to add Python to their path on Windows.
That part is something that you can control.
Giving instructions for someone non-technical to install Python and run a script with it, and then helping them figure out how to interpret those instructions, is a heck of a lot more complicated.
And yeah, you can diddle around with cx_freeze and then having to spend some time with InnoSetup or WiX to build an installer, and then do the same on OS X but putting everything into an app bundle inside of a disk image instead, and finally on Linux trying to figure out how to bundle something up to play nicely with all of the major distros and packaging systems, and get something to work, though it doesn't support cross compilation so you now need to set up and maintain three different build machines, which is definitely no simpler than the solution described.
All of these solutions have their ups and downs. But being able to relatively easily (modulo installing a linker and a few libraries) cross compile from Rust to a static executable for any of the major desktop platforms is pretty nice, after you've dealt with enough of these funky build and packaging systems or trying to walk people through figuring out how to add Python to their path on Windows.
Not inconveniencing non-technical users cannot be underlined enough. I'd rather write this in C++ which I haven't touched in ten years than explaining my parents over the phone how to install Python or even just a screen sharing solution so that I can do it. I'm still traumatized from trying to get my dad to visit 192.168.1.1 in his browser.
Oh my dad wouldn't stand for that. If something happens, like malware, it would be my fault for making him put some "dodgy coding" in the internet and now every result in Google redirects to Adult Friend Finder and I am obliged to admit my wrongdoing and fix the problem - even though it's the address to his router portal...
There are multiple ways of making exes on Windows from Python. It's not too much work either.
> Giving instructions for someone non-technical to install Python and run a script with it, and then helping them figure out how to interpret those instructions, is a heck of a lot more complicated.
Training someone to accept binary file and run it without question is just dangerous.
Training someone to accept binary file and run it without question is just dangerous.
Have you been using windows in the past 5 years?
Accepting random binaries from the net and running them is the main software distribution model.
Accepting random binaries from the net and running them is the main software distribution model.
I am talking about scenario that "a friend" gives you binary to run. It is different from downloading software from a trusted source.
Big difference between "run a binary from me" and "run a binary from anyone"
Not really. It rather easy to assume someone's identity online and then make target person to run the binary without thinking twice, especially if such practice is already established.
> It rather easy to assume someone's identity online
So you just tell you friend "Only run a binary when I physically come over to your house with a USB drive" or "Only when I call you on the phone"
So you just tell you friend "Only run a binary when I physically come over to your house with a USB drive" or "Only when I call you on the phone"
My friend actually has the source code, so yes, he could do all of this. :)
Also, worth note that these "twelve steps" are:
- inclusive of literally everything including installing Rust and adding dependencies; this is like including "install Python" and "pip install" for a Python tutorial. The point was to be approachable for anyone. I think it succeeded.
- inclusive of the "copy the executable to hand to a friend" step in both cases, which is intentionally over the top in the true listicle style
- inclusive of a bunch of steps you only ever have to do once to get full MSVC-compatible cross compilation happening for Windows.
It's almost like I was intentionally using this tiny project (which isn't especially interesting in its own right; as many have noted it's a one-liner batch file) to provide a one-stop shop for the basics of getting a cross-compiled setup with Rust. ;)
Also, worth note that these "twelve steps" are:
- inclusive of literally everything including installing Rust and adding dependencies; this is like including "install Python" and "pip install" for a Python tutorial. The point was to be approachable for anyone. I think it succeeded.
- inclusive of the "copy the executable to hand to a friend" step in both cases, which is intentionally over the top in the true listicle style
- inclusive of a bunch of steps you only ever have to do once to get full MSVC-compatible cross compilation happening for Windows.
It's almost like I was intentionally using this tiny project (which isn't especially interesting in its own right; as many have noted it's a one-liner batch file) to provide a one-stop shop for the basics of getting a cross-compiled setup with Rust. ;)
> It's almost like I was intentionally using this tiny project (which isn't especially interesting in its own right; as many have noted it's a one-liner batch file) to provide a one-stop shop for the basics of getting a cross-compiled setup with Rust. ;)
That was my initial thought reading this. Even the most devout Rust enthusiast wouldn't go to this much trouble to avoid executing 'find' one would hope (yes, find does everything you need here for those wondering)
That was my initial thought reading this. Even the most devout Rust enthusiast wouldn't go to this much trouble to avoid executing 'find' one would hope (yes, find does everything you need here for those wondering)
Windows, remember? The OP is building a program on his Mac and sending it to his Windows-using friend. Windows has a "find" command, but it's a full-text search thing, unlike on *nix where it does find-by-name.
See: http://www.computerhope.com/findhlp.htm
See: http://www.computerhope.com/findhlp.htm
Cygwin / WSL. ;)
I think it's a great tutorial for setting up multi-platform cross-compilation. I'm just not sold on Rust as a 'scripting' language :)
Python scripts can also be compiled to an exe file, although I think that the following batch one-liner is the best for this job:
for /r startdir %%i in (*.cha) do ECHO ren "%%i" "%%~ni.txt"For the record, as the author of the linked post… this is exactly what one "should" do for convenience in this situation.
Compiling Python to exe from Mac to run on Windows is actually a lot of work, though it's doable (I've done it more than once in the past).
Compiling Python to exe from Mac to run on Windows is actually a lot of work, though it's doable (I've done it more than once in the past).
Thats pretty ugly
ls -Recurse -Include *.cha | % {mv $_.FullName $_. FullName.Replace(".cha", ".txt")}
ls -Recurse -Include *.cha | % {mv $_.FullName $_. FullName.Replace(".cha", ".txt")}
I agree.
Instead the "friend" is taught the bad behavior of running untrusted (and I didn't see any tests, so untested and undocumented too) code in binary format on their machine.
The glob-rename is exactly the sort of problem a script is supposed to solve easily. I feel the author justified the use of Rust and cross-compilation simply to avoid usage of a Windows OS (or VM) for such a trivial task.
If the goal was to avoid installing additional run time/deps on the user's machine it seems that (#1) using powershell would have been the best option followed by (#2) a classic .cmd shell file as second best solution.
Instead the "friend" is taught the bad behavior of running untrusted (and I didn't see any tests, so untested and undocumented too) code in binary format on their machine.
The glob-rename is exactly the sort of problem a script is supposed to solve easily. I feel the author justified the use of Rust and cross-compilation simply to avoid usage of a Windows OS (or VM) for such a trivial task.
If the goal was to avoid installing additional run time/deps on the user's machine it seems that (#1) using powershell would have been the best option followed by (#2) a classic .cmd shell file as second best solution.
And as the author said, they don't know PowerShell or batch files very well. They also didn't pretend they had a strong justification for doing it. Rust was what they wanted to use, and hey look, Rust let them do it _and_ cross-compile it. I don't think there was an intention to say that everyone should do this instead of using shell scripts.
It's not untrusted code, it's code that a friend wrote to help you out. That's not at all the same as installing random untrusted code off the internet. I write code to help out my friends with things they want to do - are you saying they shouldn't accept it from me just because they can't code themselves? I can give them the source code, but what good is that to them when they can't program and don't want to?
It's not untrusted code, it's code that a friend wrote to help you out. That's not at all the same as installing random untrusted code off the internet. I write code to help out my friends with things they want to do - are you saying they shouldn't accept it from me just because they can't code themselves? I can give them the source code, but what good is that to them when they can't program and don't want to?
What? You mean like every other program they've ever run in their lives if they're not a developer, except developed by a personal friend?
Compiling is easy. Configuring path variables and installing interpreters are arcane to non-developers.
Compiling is easy. Configuring path variables and installing interpreters are arcane to non-developers.
> Instead the "friend" is taught the bad behavior of running untrusted (and I didn't see any tests, so untested and undocumented too) code in binary format on their machine.
Being paranoid in the software you run on your machine is a double edge sword. How many people vet their software before using it, and how much vetting do you really need to do?
Being paranoid in the software you run on your machine is a double edge sword. How many people vet their software before using it, and how much vetting do you really need to do?
This topic is dear to me, I often deal with people without any software stacks on their machines and strict IT limits on installing new things. If I were on the same OS as the target user, I'd use pyinstaller. I've used it with very few issues over the past year on many projects. Most of them included Flask/Bottle and parts of numpy, it all worked like a charm. Cross compilation in Python is a bit of an issue, I for one have not resolved it in a satisfactory fashion (not saying it can't be done).
If one wants to be closer to the metal, Go's cross compilation works out of the box, without all the LLVM/MSVC work that Rust requires (for now, that is). As long as you use the standard library (and packages dependent on the standard library), it will work right away. I've cross compiled Go from my Mac for Windows users when I couldn't get my Python setup to do so, worked perfectly.
(Sure, you can script easy things as shown in other comments, this is a bit more general.)
If one wants to be closer to the metal, Go's cross compilation works out of the box, without all the LLVM/MSVC work that Rust requires (for now, that is). As long as you use the standard library (and packages dependent on the standard library), it will work right away. I've cross compiled Go from my Mac for Windows users when I couldn't get my Python setup to do so, worked perfectly.
(Sure, you can script easy things as shown in other comments, this is a bit more general.)
Re: Python, I've had success with running pyinstaller from within wine. I do most of my development from within linux, and so this covered my use case. No luck in cross-compiling into an OSX executable, though.
Re: Go, I've had the opposite experience. So long as you only use pure-Go libraries, cross-compiling works. If you use any C libraries (which last time I checked, include some of the standard library), you run into issues. For me, it was OpenGL bindings that made cross-compiling impossible.
Re: Go, I've had the opposite experience. So long as you only use pure-Go libraries, cross-compiling works. If you use any C libraries (which last time I checked, include some of the standard library), you run into issues. For me, it was OpenGL bindings that made cross-compiling impossible.
> If one wants to be closer to the metal, Go's cross compilation works out of the box, without all the LLVM/MSVC work that Rust requires (for now, that is).
That's not "closer to the metal"; it's reinventing parts of the native toolchain.
That's not "closer to the metal"; it's reinventing parts of the native toolchain.
Sorry, I meant closer to the metal relative to Python, not relative to Rust, poor wording.
I've had great luck with Go as well. The cross compilation just works and the binary is easy to distribute.
I love this idea. There's an implicit assumption that the reason scripting languages are suitable for scripts is that they play fast and loose with type systems, which allow enough shortcuts and escape hatches to be productive.
But that's only part of the story. Along with typing, scripting languages also had the major advantage of being fast to bootstrap and run (`ruby script.rb`) and were fully "batteries included" with good standard libraries.
These assumptions are certainly true, but they're much _more_ true when you're comparing them against "old world" languages like C, C++, or even Java. All of these require significant project scaffolding just to get running, and have varying levels of useful built-in utilities (C has almost nothing, C++ has the STL and maybe Boost if you're generous, and Java tried to have a useful core, but missed the mark in a number of places like HTTP). In the case of C and C++, complex build logic may also be required in the form of autoconf/automake/make.
Newer compiled languages like Rust just don't have these issues anymore (I'd also include Go, Crystal, and some others in the list). Project scaffolding is built right into the language (Cargo) and the standard library is about as complete as you can get (and probably better than the classical scripting languages — all of Ruby/Python/Perl missed a good API for some things like HTTP, which led to the rise of separate packages like Faraday/Requests/etc). Anything that's not in the standard library is easily retrievable through great package managers that are bundled in with the core language.
After you're over the initial learning curve of the language and standard library, it's possible to be nearly as productive in something like Rust as you can be in Python. As a bonus, you can also get reasonable reassurance that your program is correct before you run it — and even without writing an exhaustive test suite that exercises every line of code.
But that's only part of the story. Along with typing, scripting languages also had the major advantage of being fast to bootstrap and run (`ruby script.rb`) and were fully "batteries included" with good standard libraries.
These assumptions are certainly true, but they're much _more_ true when you're comparing them against "old world" languages like C, C++, or even Java. All of these require significant project scaffolding just to get running, and have varying levels of useful built-in utilities (C has almost nothing, C++ has the STL and maybe Boost if you're generous, and Java tried to have a useful core, but missed the mark in a number of places like HTTP). In the case of C and C++, complex build logic may also be required in the form of autoconf/automake/make.
Newer compiled languages like Rust just don't have these issues anymore (I'd also include Go, Crystal, and some others in the list). Project scaffolding is built right into the language (Cargo) and the standard library is about as complete as you can get (and probably better than the classical scripting languages — all of Ruby/Python/Perl missed a good API for some things like HTTP, which led to the rise of separate packages like Faraday/Requests/etc). Anything that's not in the standard library is easily retrievable through great package managers that are bundled in with the core language.
After you're over the initial learning curve of the language and standard library, it's possible to be nearly as productive in something like Rust as you can be in Python. As a bonus, you can also get reasonable reassurance that your program is correct before you run it — and even without writing an exhaustive test suite that exercises every line of code.
>[...] the standard library is about as complete as you can get (and probably better than the classical scripting languages — all of Ruby/Python/Perl missed a good API for some things like HTTP, which led to the rise of separate packages like Faraday/Requests/etc)
What? Rust basically has no standard library. At least Python doesn't require pulling in third-party libraries to do globbing, create a zip/tar, create a tempfile or send/serve http. Hell, it doesn't even do command-line arguments outside of manually parsing and iterating over the args vector.
What? Rust basically has no standard library. At least Python doesn't require pulling in third-party libraries to do globbing, create a zip/tar, create a tempfile or send/serve http. Hell, it doesn't even do command-line arguments outside of manually parsing and iterating over the args vector.
Admittedly they did ship a lot out of core outright, but a single line in `Cargo.toml` fetches any of these things pretty expediently, and they're still well standardized across the ecosystem.
zip = "0.2"
tempfile = "2.1.4"
getopts = "0.2"
hyper = "0.9.12"it sort of does, though. Trying to use urllib2 is like pulling teeth
Which is why you use httplib[1] for sending HTTP requests.
>>> import httplib, json
>>> conn = httplib.HTTPConnection("httpbin.org")
>>> conn.request("GET", "/get")
>>> res = conn.getresponse()
>>> print res.status, res.reason
200 OK
>>> data = json.load(res)
>>> print data
{u'origin': '0.0.0.0', ...}
>>> conn.close()
So, I'm going to stop you right there on the fast to bootstrap & run...
Certainly Java is kind of a hybrid worst if both worlds kind of thing, but C/C++? For simple scripting problems you don't need complex build logic. You can usually get by with "cc foo.c -o foo", which you can embed in the source file. Sure, if you have library dependencies it can be a bit more involved, but most modern IDE's (even less than modern) will manage that for you. It's particularly simple if you do static linking (which often makes sense with small scripts anyway). These days there are so many header only libraries that can remove even that boy if pain. On the deploy side, it's simple and likely faster to run than Rust.
Now, many C/C++ projects start with auto tools/cmake or similar heavy lifting, and long build times. But there's selection bias there: those projects aren't simple scripting jobs, but actually intend to exploit close system integration and support building with a broad variety of compilers and linkers. They do it because they want people to use the code and for a larger project, it isn't that much extra overhead.
Nah, the main pain with C++ in particular is the lack of a standard, simple library for network services like HTTP & databases. That problem seems to be finally getting resolved, but historically I've addressed by just standardizing on one of curl, cpp-http, etc. Once you do that with a modern C++17 compiler, and maybe their in Folly as well to make it truly easy, it starts to become a remarkably viable scripting language.
You will type a bit more, but the compiler will also catch a lot more stuff for you automatically, reducing the cognitive load. There are better alternatives, but if you are in a shop with C++ already, the advantages of just using the same tool as everywhere else are more than enough to justify it.
Certainly Java is kind of a hybrid worst if both worlds kind of thing, but C/C++? For simple scripting problems you don't need complex build logic. You can usually get by with "cc foo.c -o foo", which you can embed in the source file. Sure, if you have library dependencies it can be a bit more involved, but most modern IDE's (even less than modern) will manage that for you. It's particularly simple if you do static linking (which often makes sense with small scripts anyway). These days there are so many header only libraries that can remove even that boy if pain. On the deploy side, it's simple and likely faster to run than Rust.
Now, many C/C++ projects start with auto tools/cmake or similar heavy lifting, and long build times. But there's selection bias there: those projects aren't simple scripting jobs, but actually intend to exploit close system integration and support building with a broad variety of compilers and linkers. They do it because they want people to use the code and for a larger project, it isn't that much extra overhead.
Nah, the main pain with C++ in particular is the lack of a standard, simple library for network services like HTTP & databases. That problem seems to be finally getting resolved, but historically I've addressed by just standardizing on one of curl, cpp-http, etc. Once you do that with a modern C++17 compiler, and maybe their in Folly as well to make it truly easy, it starts to become a remarkably viable scripting language.
You will type a bit more, but the compiler will also catch a lot more stuff for you automatically, reducing the cognitive load. There are better alternatives, but if you are in a shop with C++ already, the advantages of just using the same tool as everywhere else are more than enough to justify it.
A bit tangential but here goes: I have some friends whose background is mostly algorithmic problems in competitive programming environments. I enjoy that kind of practice occasionally, but for them, it's the 90% of code they've written, and they are just too good at that stuff.
This means they know C++ best, as this is usually the language you use in those competitions. It's interesting to see the approach to scripting of these guys. While I tend to resort to shell pipelines and maybe throw in a short Lua program in-between, they write C++ code for everything, escaping to the shell for some stuff while using ordinary text files instead of piping, e.g. system("curl somesite >input.txt") and then fopen("input.txt", "r"). They write the glue between these calls, call the C compiler and run the binary.
They know their C++ and know how to write it fast; as a result, they can get a valid script running about as fast as I can do the same using a shell pipeline.
When I see this, I'm reminded how awesomely tools can be bent to do something they're not primarily designed for, when in the hands of someone skilled who knows them inside-out.
This means they know C++ best, as this is usually the language you use in those competitions. It's interesting to see the approach to scripting of these guys. While I tend to resort to shell pipelines and maybe throw in a short Lua program in-between, they write C++ code for everything, escaping to the shell for some stuff while using ordinary text files instead of piping, e.g. system("curl somesite >input.txt") and then fopen("input.txt", "r"). They write the glue between these calls, call the C compiler and run the binary.
They know their C++ and know how to write it fast; as a result, they can get a valid script running about as fast as I can do the same using a shell pipeline.
When I see this, I'm reminded how awesomely tools can be bent to do something they're not primarily designed for, when in the hands of someone skilled who knows them inside-out.
Throw in cling or cint and you are good to go!
This is over engineering if I've ever seen it. For one this is a single line bash script that would allow you to even handle drag and drop. Best user experience.
Second, if you're going to over engineering just look for someone who has already done that for you: http://www.den4b.com/products/renamer
It's no use to engineer something that has been solved (unless you're learning) if there are already better solutions. Even if it is an example a better one should be used.
The moral of the story should he use the best tool for the job, don't shoe horn a bad language for a into a solution, and also Google is usually the best tool for every job.
Second, if you're going to over engineering just look for someone who has already done that for you: http://www.den4b.com/products/renamer
It's no use to engineer something that has been solved (unless you're learning) if there are already better solutions. Even if it is an example a better one should be used.
The moral of the story should he use the best tool for the job, don't shoe horn a bad language for a into a solution, and also Google is usually the best tool for every job.
I don't think the author is really expecting people to go out and start uninstalling Python to do all their scripting in Rust. Seems to me like he saw this as an opportunity to show off the clap and glob crates, as well as write a little documentation for how to cross-compile Rust from Mac to Windows.
1. Of course it's over-engineered.
2. It wouldn't be a bash script; it would have to be batch or PowerShell, neither of which I know well, and neither of which I like.
3. It was fun.
2. It wouldn't be a bash script; it would have to be batch or PowerShell, neither of which I know well, and neither of which I like.
3. It was fun.
I'm pretty sure the author was just practicing Rust. And although in this case using Bash would be fine, in most cases you shouldn't distribute Bash scripts. They're virtually always filled with bugs and they don't work on Windows.
This is over engineering if I've ever seen it.
Overcomplicating is more like it, for engineering implies a formal specification document which includes requirements, as well as documentation to go along with the product, not to mention complete integration with the target operating system substrate.
This quickly hacked-together Rust program is light years away from that.
Overcomplicating is more like it, for engineering implies a formal specification document which includes requirements, as well as documentation to go along with the product, not to mention complete integration with the target operating system substrate.
This quickly hacked-together Rust program is light years away from that.
It's a shame the author didn't frame this article differently by strictly focusing on how to cross compile a simple rust app on macOS and Windows. The focus on "scripting" seems to have really brought out the negativity in people here. I definitely learned something, I had no idea I could compile a Windows binary from my Mac.
The author here: I agree with your assessment. Apparently the scare quotes didn't convey my intent: it's easy to do something (including cross-platform) where I normally would have used a traditional scripting language, and the portability is nice. ¯\_(ツ)_/¯
I agree that it was very useful, despite the issue with the topic of choice. Thanks for writing it!
Well thanks anyway, I enjoyed it!
As someone who only very recently caught the Rust bug; seeing a reasonably small, familiar example like this is really helpful. Thanks for sharing. Maybe I can learn something by seeing if I can introduce threads/channels to the renaming part of the script as an exercise.
For anyone else curious about Rust, my own learning spree was triggered after reading the "A very brief intro to Rust" Slide Deck [1] by Ashley Williams [2].
[1] https://ashleygwilliams.github.io/a-very-brief-intro-to-rust...
[2] https://twitter.com/ag_dubs
For anyone else curious about Rust, my own learning spree was triggered after reading the "A very brief intro to Rust" Slide Deck [1] by Ashley Williams [2].
[1] https://ashleygwilliams.github.io/a-very-brief-intro-to-rust...
[2] https://twitter.com/ag_dubs
One could also use Haskell [0] for scripting [1] using the turtle [2] library.
[1] http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...
[2] http://hackage.haskell.org/package/turtle
turtle is a reimplementation of the Unix command line
environment in Haskell so that you can use Haskell as both a
shell and a scripting language.
[1] http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...
[2] http://hackage.haskell.org/package/turtle
If we're talking about alternatives, xonsh is also an excellent alternative (Python + subprocess mode) http://xon.sh/
Big +1 to turtle, which is a really neat piece of software that manages both to be super useful on its own and to provide what looks like a really helpful way of learning Haskell.
For folks who want to do Bash-style shellouts and pipelines from Rust code, I'm working on a library to make it easier: https://github.com/oconnor663/duct.rs
I think the author wanted to write "Using Rust instead of 'Scripting'". Perfectly makes sense. Nicer features, static types, compiled code, no runtime on the host system, etc. I prefer OCaml over Python,Bash etc for CLI apps, for the same reasons.
I admit, as a longtime Rust user I don't often get around to using Rust for scripting, probably because Python is available on all my personal machines and I'm so accustomed to it. But after reading this article I'm immensely impressed at how far Rust has come in the past few years WRT suitability for scripting, I really need to take a look at the glob and clap crates.
Also, I'm very excited to hear that LLD is apparently usable for linking on Windows! It seems like we've been waiting for LLD since time immemorial. :)
Also, I'm very excited to hear that LLD is apparently usable for linking on Windows! It seems like we've been waiting for LLD since time immemorial. :)
I found myself doing the same just a few days ago when my local server suddenly had a new ip address. I made a simple script to scan IP addresses in the range: https://gist.github.com/johshoff/bed463ce8a122123e60bbc84bd1...
There are probably faster ways, but none that I could think of a the time.
There are probably faster ways, but none that I could think of a the time.
> There are probably faster ways, but none that I could think of a the time
For future reference, `nmap -p22 --open 192.168.1.*` does everything your rust script currently does.
Nmap is THE tool for the job of network scans. IMO, it should be in the toolbelt of every techie that deals with any sort of computer network.
For future reference, `nmap -p22 --open 192.168.1.*` does everything your rust script currently does.
Nmap is THE tool for the job of network scans. IMO, it should be in the toolbelt of every techie that deals with any sort of computer network.
Cool program.
Other quick ways besides the mentioned nmap: ping the broadcast address (if ICMP is allowed) or install avahi-daemon and you can just refer to "your-machines-hostname.local"
Other quick ways besides the mentioned nmap: ping the broadcast address (if ICMP is allowed) or install avahi-daemon and you can just refer to "your-machines-hostname.local"
I know this isn't the point.. but I can't help myself... PowerShell:
gci c:\somedir -r -filter "*.ext1" | %{Move-Item $_.FullName -Dest ($_.DirectoryName + "\" + $_.BaseName + ".ext2")}
gci c:\somedir -r -filter "*.ext1" | %{Move-Item $_.FullName -Dest ($_.DirectoryName + "\" + $_.BaseName + ".ext2")}
Nice to see how good Rust is for 'scripting'. I think Go would have been a better choice from a purely technical point of view - it's easier to write, and has even easier setup and cross-compilation than Rust. But I guess part of the point of this was to practice Rust so fair enough!
> As wonderful as they are, the fact that languages like Python, Perl, Ruby, JavaScript, etc. require having the runtime bundled up with them makes just shipping a tool a lot harder
Not true for js on Windows, though. Windows runs JScript natively (same language as JavaScript, only a different name).
Not true for js on Windows, though. Windows runs JScript natively (same language as JavaScript, only a different name).
There's also this handy cargo plugin that helps for using Rust for "scripting": https://github.com/DanielKeep/cargo-script
It's especially handy for a one-file solution.
It's especially handy for a one-file solution.
While this can be seen as an interesting exercise, I can't help but feel that rust isn't the best tool here.
First, there are a handful of existing tools, as other have pointed out (I would personally go for AntRenamer).
Then, there are a few scripting languages already installed (admittedly maybe not the best or most well-known). A quick Google search would probably have given a result right away, since this is a pretty common scenario.
Lastly, I am a bit curious, why go with msvc and not mingw/clang ?
First, there are a handful of existing tools, as other have pointed out (I would personally go for AntRenamer).
Then, there are a few scripting languages already installed (admittedly maybe not the best or most well-known). A quick Google search would probably have given a result right away, since this is a pretty common scenario.
Lastly, I am a bit curious, why go with msvc and not mingw/clang ?
For the "right tool for the problem", you're right; really, see up-thread where someone posted the one-liner.
For MSVC over MinGW/Clang: part of my point here was to first learn and then document for the community how you do cross-compilation for MSVC. There are lots of projects out there where being able to link against MSVC is important, and it's also harder than linking against MinGW (which would have just been `rustup target add x86_64-pc-windows-gnu` and `cargo build --release --target=x86_64-pc-windows-gnu`, I believe; no extra linker stuff needed), so there's more value in documenting it and putting it out there for MSVC.
For MSVC over MinGW/Clang: part of my point here was to first learn and then document for the community how you do cross-compilation for MSVC. There are lots of projects out there where being able to link against MSVC is important, and it's also harder than linking against MinGW (which would have just been `rustup target add x86_64-pc-windows-gnu` and `cargo build --release --target=x86_64-pc-windows-gnu`, I believe; no extra linker stuff needed), so there's more value in documenting it and putting it out there for MSVC.
Thanks, I didn't realise that it was meant to be this way. To me, it sounded a bit like MSVC was the only option, and you had to install if for cross compiling.
Since this is actually harder to do (from your own post), I worry a bit that a beginner might try to do it right away, to the risk of failing at it.
Since this is actually harder to do (from your own post), I worry a bit that a beginner might try to do it right away, to the risk of failing at it.
This guy has likely not prototyped in environment such as Jupyter notebooks or ptipython. It is not just language - there is whole ecosystem build around Python scripting: pandas, matplotlib, numpy, etc. And don't forget about debuggers and IDEs once your projects grow out of scripting environment. Did I mention hooking up to GUI or web app?
I've done a lot in those kinds of things, actually; they're awesome. I don't feel obliged to mention every possible alternative in every blog post; that would be absurdly tiring to do, and probably wouldn't add much value. (Edited so it didn't read as hostile as the original version of the comment did.)
Which is all completely irrelevant for the task he had.
Its all about using the right tools / language for the job. And most of the solutions mentioned here are not the right tool for the job. Install python?? install this, install that. All the guy needs is a simple .exe file for windows that doesn't come with a manual howto install it.
Was it too difficult to ask his friend to do right click -> create text document-> open it and write 'ren STAR.cha STAR.txt' -> save it as rename.bat and double click on it?
Seriously, I'm not sure if this whole thing is a joke...
edit: not sure why I cannot write the jolly character here
edit: not sure why I cannot write the jolly character here
It was an excuse to figure out the cross-compilation story with a tiny tool; he got his tool, I got my experience with a part of the Rust ecosystem I hadn't interacted with before. It was a win all around. (Note as well that I said explicitly that I asked if this was a "just give me a tool" or a "teach me something about scripting" kind of day!) :)
It looks like the rust script would rename files in the current directory and all subdirectories, whereas a batch rename would only work in the current directory.
This looks like an article on scripting, but it's in fact a subtle allegory about reinventing the wheel and overengineering.
It's so subtle that even the author might have missed it.
It's so subtle that even the author might have missed it.
Doesn't Windows have a built-in JavaScript runtime, the Windows Script Host? Sure, it may not be a very pretty language either, but it's still relatively sane compared to batch scripts. (I can't speak of PowerShell.)
I do like activex/js and wsh, but Wsh might require permissions to modify the filesystem, so installation/setup step can be harder than just copying and running a statically compiled executable.
Last time I checked, doing new ActiveXObject("Scripting.FileSystemObject") did not require any extra 'permissions'.
I remembered the problem, it was the execution of unsigned/remotely signed powershell scripts where you have to open powershell and give a permission[1] by typing something like:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
I personally like rust as a systems language. But the syntax really hurts my eyes as apposed to Python. A well crafted python code is soo easy to read. Rust can soo become very messy.
You could have done the same and given him a nice GUI in the process using lazarus.
To install a cross-compiler on mac:
To install a cross-compiler on mac:
$ fink install fpc-cross-i386-win32[deleted]
I considered rust for daily scripts for the performance and statically compiled binary. But had to give up when the complexities from borrower checker came up and not to mention the limitation of libraries.
http://www.pyinstaller.org/
Single file executable. Job done.
Of course, if you want to write something in Rust, it's not much use.
Single file executable. Job done.
Of course, if you want to write something in Rust, it's not much use.
Oh, nice! PyInstaller didn't have Python 3 support the last time I looked at using it (mid-2015; I think it was in-work at that point), so I'm glad it's landed.
> Add the dependencies...
Nope. The beauty of a decent systems scripting language is that it includes everything you need (and ideally is already installed everywhere).
Nope. The beauty of a decent systems scripting language is that it includes everything you need (and ideally is already installed everywhere).
The author explains that the intended target is Windows, which has neither Perl nor Python (nor even Bash) installed by default, so "already installed everywhere" is already off the table. At that point, since Rust can produce static binaries, it doesn't matter what deps you pull in.
Sure, but it's not that bad. Cargo is fantastic, once you have the dependencies in your manifest, `cargo build` downloads and compiles all your dependencies for you. It's pretty hassle free.
By the way, to solve the specific problem mentioned in the article, see "man 1 rename".
Is this available on Windows? The author made it pretty clear the target user was on a Windows machine
The Windows rename command doesn't support recursively processing subfolders, but it can be achieved using one line of powershell:
Get-ChildItem . -Recurse -Include *.cha | Rename-Item -NewName { $_.name -Replace '\.cha','.txt' }You're probably right that Rust code isn't the most minimal way of doing this specific task, but I think the author's intended point was not "This specific task is best done by cross-compiling Rust for Windows" but rather "Rust is a viable alternative to using Python (or the like) for general-purpose cross-platform scripting; here is an example of something that's surprisingly easy to do with Rust". In this case, using a simpler example is just a way of making the demonstration of the concept easier, which I think is fairly effective.
From what I remember Include will only match items from the base directory and powershell gets stupidly complicated very quickly:
http://flukus.github.io/2015/03/13/2015_03_13_Powershell-is-...
http://flukus.github.io/2015/03/13/2015_03_13_Powershell-is-...
That compares Powershell to Nant. Nant tasks (inspired by Ant tasks) are more powerful than regular commands. Not even shell commands match those. He's comparing apples with an apple pie there.
I started the article as a nant vs psake (PowerShell based build tool). Psake relies on PowerShell tools for things like copying files, which makes sense.
I'm working on a similar one using make instead, which uses shell commands and it's a lot better than both.
I'm working on a similar one using make instead, which uses shell commands and it's a lot better than both.
Does Windows have an equivalent? I assume one could get it exactly via cygwin or msys or something?
This...felt misleading.
[deleted]
Does Rust really support macOS, or did he mean to say OSX?
The amount of contortions the author has had to go through is ridiculous, and all because the target operating system is Windows, and doesn't yet ship a production ripe UNIX userland (hence no AWK, sed, or zsh/tcsh/ksh out of the box). As a colleague of mine always says: give Wintendo no chance!
Despite that, Windows is still the most popular desktop OS in the world by an enormous margin. You don't help a friend who needs to rename a load of files by saying "well, it'd be much easier if you first installed Linux". Or cygwin, or even the Windows Subsystem for Linux if they're running Windows 10 Anniversary Update.
You help a friend by giving him a one liner COMMAND.COM shell script, instead of turning it into a cross-compiling hackfest just because you're a Rust fanboy (in author's own words).
How is the one-liner shell script more helpful to the friend than the resulting binary? Yes it was convoluted but the point was the code author had to jump through hoops, not the end user.
Then the code author should have jumped through the hoops to learn Windows shell scripting, instead of purposely overcomplicating.
When I see things like these, it honestly makes me hate computers and IT.
When I see things like these, it honestly makes me hate computers and IT.
No real hoops to jump; I'm sure I could have googled the batch script invocation for him easily.
Again: the point was simply to use an interesting "hook" and then segue into providing resources on cross-compiling with a super-simple example.
Again: the point was simply to use an interesting "hook" and then segue into providing resources on cross-compiling with a super-simple example.
Or, you figure out the cross-compilation steps and document them for the whole Rust community, because there isn't yet a readily searchable resource on it. I actually knew exactly what I was doing with this, and it wasn't just solving that one specific problem. It was solving a bunch of different issues, including some for the whole Rust community, some for my own personal experience, and that one for my friend. Heck, I asked him if he minded waiting while I did all of this, and he gave the thumbs-up. ;)
There's lots of room for improvement here and I'm not sure Rust is the right tool for the job.
First of all, why doesn't this have a UI? It would be much easier for the friend to select the folder with the .cha files and they could see a nice progress bar while they are renamed.
However, if there are few files the renaming process would be too fast and the progress bar would flicker. An easy solution is to add sleep calls in the worker thread to ensure that the progress is visible.
Of course the user could have lots of .cha files, and the friend might get bored. A popular pattern is to display an overview of the program features in picture + text format while a long procedure is running. There could be one about .cha file renaming for instance.
It might make sense to also show ads, depending on the monetization strategy. Maybe the friend wants to buy something and they don't realise it yet.
Finally, when the whole thing is done, the friend should be offered the option to post it on twitter and/or facebook. Something along the lines "I have just renamed 100 .cha files using rename-it. Try it yourself at <address>". The address needs to be different from the blog, probably an SPA, but I'm not an expert at that. And one more tip: don't ask for the facebook login before the renaming, this makes users suspicious. It's much better to ask before posting and explain why you need it. This builds trust.
I said that Rust is not the right tool. I would maybe try Electron, it's also cross-platform, but the code can be reused on the back-end. For instance one can upload a zip of all their .cha files and they get a zip of txt files back.
Ultimately desktop apps such as these are a thing of the past. But I guess it's a nice exercise.
First of all, why doesn't this have a UI? It would be much easier for the friend to select the folder with the .cha files and they could see a nice progress bar while they are renamed.
However, if there are few files the renaming process would be too fast and the progress bar would flicker. An easy solution is to add sleep calls in the worker thread to ensure that the progress is visible.
Of course the user could have lots of .cha files, and the friend might get bored. A popular pattern is to display an overview of the program features in picture + text format while a long procedure is running. There could be one about .cha file renaming for instance.
It might make sense to also show ads, depending on the monetization strategy. Maybe the friend wants to buy something and they don't realise it yet.
Finally, when the whole thing is done, the friend should be offered the option to post it on twitter and/or facebook. Something along the lines "I have just renamed 100 .cha files using rename-it. Try it yourself at <address>". The address needs to be different from the blog, probably an SPA, but I'm not an expert at that. And one more tip: don't ask for the facebook login before the renaming, this makes users suspicious. It's much better to ask before posting and explain why you need it. This builds trust.
I said that Rust is not the right tool. I would maybe try Electron, it's also cross-platform, but the code can be reused on the back-end. For instance one can upload a zip of all their .cha files and they get a zip of txt files back.
Ultimately desktop apps such as these are a thing of the past. But I guess it's a nice exercise.
Not that this is a bad thing, but it feels like a bit of a far cry from my hacked together workflow of throw code in iPython and write out when I’m happy. I don’t see anything here that makes Rust any more attractive than whatever your multiplatform language of choice is for this use case scenario.
That being said, the code is concise and readable, and the article was a short, pleasant read.