Xonsh, a Python-ish, Bash-compatible shell language and command prompt(xonsh.org)

242 pointsby ngoldbaum11 tahun yang lalu61 comments

Xonsh, a Python-ish, Bash-compatible shell language and command prompt

http://xonsh.org/

62 comments

I'm a lead dev on the fish shell (http://fishshell.com)

xonsh looks very cool. Shell scripting languages are famously obtuse and underpowered, and we've thought about how to cleanly merge shell scripting with a real language. It's heartening to see an attempt at it.

Ambiguous syntax is a sticking point. For example, say the user runs this:

    [ 1 ]

is this constructing a Python list with the value 1, or is it invoking /bin/[ with two arguments? It's ambiguous! It looks like xonsh interprets it the first way. Any thoughts on this issue?

A related issue is the relationship between bash functions and Python functions. I'd be very interested to see how xonsh approaches this.

Also, here's some unsolicited advise, from one non-POSIX shell maintainer to another: The claim that xonsh is "bash-compatible" is delicate. Full bash compatibility requires your shell to be insane. E.g.

   echo foo | cd /
   cd / | echo foo

In bash, only the second line will change your working directory. Or consider this command:

this finds the first (for some meaning of first) file in your directory and invokes it with all other files as arguments. I tried it: xonsh doesn't handle either of these "correctly," which is to say that xonsh isn't yet insane. That's good news! On the other hand, this means it's not really bash compatible.

IME nobody really cares about full bash compatibility. What they want is for their existing bash stuff to just work. Bash shebang scripts will just run in real bash, but that leaves shell-integration scripts: virtualenv, rbenv, etc. Supporting that stuff is one of the most common requests that the fish shell gets, and one of the most commonly cited reasons for not using fish.

Here's a piece of rbenv's bash integration: https://github.com/sstephenson/rbenv/blob/master/libexec/rbe... . It's short but there's a lot of features being used: set, shopt, IFS, functions, etc. If you're aiming for bash compatibility, that's your target!

In the meantime, I suggest changing the language to express that xonsh is bash-like, not bash-compatible. Otherwise users are going to be confused and disappointed.

scopatz4211 tahun yang lalu

Thanks for posting this @ridiculous_fish! I am the main author of xonsh, and I really appreciate fish. Its awesome and I am certainly going to borrow ideas :). I agree that making shell languages into 'real' languages is strong goal that we should all have. And like with real languages, many can co-exist happily.

I'd like to point out though, that the goal is not to have bash-compatibility. As you point out, bash does some insane things and that is exactly what I want to avoid with xonsh. I should be able to give xonsh to a programming newbie and they should not experience any gotchas once they know the language.

I have been pretty careful to say BASHwards-compatibility in the docs and elsewhere, for exactly the reasons you mention. The goal isn't to be bash with some python-isms, but to be python with the useful parts of the shell syntax. Like any good technology, it should interface as well as it can (ie when not insane to do so) with the previous technologies. For example, read in the bashrc or take advantage of the bash completion functionality. That is what BASHwards was meant to mean. @ngoldbaum seems to have submitted the link here, under 'bash-compatibility.' Oh well. This probably is proof enough that I should change the 'BASHwards' language elsewhere. Thanks for the tip.

Keep doing great things with fish!

dragonwriter11 tahun yang lalu

"BASHwards" is a bad word to use for this, since the natural interpretation is that it is a combination of "backwards" and "BASH" meaning thatn xonsh is an "upgrade" that is backwards-compatible with BASH.

It seems to be more intended to be not backward-compatible, but to have some attention to minimizing the pain of transition, which is valuable, but needs different language to communicate.

geoelectric11 tahun yang lalu

FYI, I interpreted "BASH-wards compatible" as "BASH-compatible" as well. The subtlety seems a little lost on me.

eddieplan911 tahun yang lalu

> that leaves shell-integration scripts: virtualenv, rbenv, etc. Supporting that stuff is one of the most common requests that the fish shell gets, and one of the most commonly cited reasons for not using fish.

shameless plug, but I made a utility to solve the problem: https://github.com/edc/bass

So you can write `bass source /usr/local/bin/virtualenvwrapper.sh ';' mkvirtualenv env1`, which of course can be shortened with a function.

CADBOT11 tahun yang lalu

I'm testing this out later this week... if it works... you're my new hero!

lmm11 tahun yang lalu

> Shell scripting languages are famously obtuse and underpowered, and we've thought about how to cleanly merge shell scripting with a real language. It's heartening to see an attempt at it.

I used tclsh for a while. IMO a shell language needs the "function arg1 arg2" function-application syntax to fit with the way programs are called. Fortunately that style seems to be getting more popular - Haskell does it and to a certain extent so does Scala.

LukeShu11 tahun yang lalu

       echo foo | cd /
       cd / | echo foo
    
    In bash, only the second line will change your working
    directory.

Um... in Bash, neither of those change your working directory (unless `shopt -s lastpipe` is set, in which case: the first does).

ridiculous_fish11 tahun yang lalu

Whoops, you're right! Egg on my face.

danbmil9911 tahun yang lalu

you two should get a room, and fail to emerge until you are ready to bless the world with one shell to rule them all (and in the darkness bind them)

ngoldbaum11 tahun yang lalu

I'm pretty sure it said "BASHwards-compatible" when I submitted the story to HN, so I guess one of the admins changed the title.

pjmlp11 tahun yang lalu

> Shell scripting languages are famously obtuse and underpowered, and we've thought about how to cleanly merge shell scripting with a real language.

It was called REPL (Interlisp-D, Lisp Machine, Cedar, Oberon), Workspace (Smalltalk), but then UNIX won.

bkeroack11 tahun yang lalu

This is great. When dealing with bash stupid-isms I've always thought it would be awesome to have a shell with Python syntax. The tough part is figuring out how to combine Python with "normal" shell commands in a clean, intuitive manner but it looks like Xonsh has gone a long way towards solving that!

Just installed on my personal Macbook. Will see how it goes.

spystath11 tahun yang lalu

You could already do some minimal shell-like scripting with ipython[1]. Just put your ipython-flavored script in a file with .ipy extension an call it with ipython, or just use #!/usr/bin/ipython as a shebang.

  #!/usr/bin/ipython
  files = !ls
  for f in files:
    print(f)

That being said, it is a bit clunky to use since it's not a proper shell, so xonsh is definitely a very welcome improvement.

[1]: http://ipython.org/ipython-doc/stable/interactive/shell.html

meddlepal11 tahun yang lalu

I agree, this is awesome. I hate dealing with bash but it's pretty common when you're DevOps.

doomrobo11 tahun yang lalu

I genuinely wish there was an alternative that could gain enough traction to reach critical mass and hopefully start shipping with operating systems. I've never bothered to learn bash because it was always so damned confusing, and I just couldn't get over the idiosyncrasies when playing around with it. I wouldn't mind putting in time to learn something new and actually user-friendly.

bishc11 tahun yang lalu

This is quite apropos, as just last week I released a little language called 'bish' designed to overcome exactly this issue. It compiles to bash for portability, but has more modern and familiar syntax.

See https://github.com/tdenniston/bish for the repo and https://news.ycombinator.com/item?id=9166386 for some previous discussion.

The language is still brand new and under active development. It's not stable yet.

seanp2k211 tahun yang lalu

Well, if you do ever want to learn the "finer points" of bash, http://wiki.bash-hackers.org/doku.php and http://mywiki.wooledge.org/BashGuide have tons and tons of great info.

fiatjaf11 tahun yang lalu

Have you looked at fish? It is in the package repositories everywhere, I think.

doomrobo11 tahun yang lalu

I actually gave fish a try after you recommended it. I can't use it in my workflow :(. Virtualenv has a bash script to initiate the environment and fish's incompatibility with bash scripts makes it impossible for me to work with it.

fiatjaf11 tahun yang lalu

virtualenv deployments come with a fish specific script, it is at `venv/bin/activate.fish`.

You can source it with `. venv/bin/activate.fish`

doomrobo11 tahun yang lalu

Awesome! thanks

doomrobo11 tahun yang lalu

Even so, the single most attractive trait of bash is that it ships with almost everything. If I have to write a script on a machine with no internet, then bash is the only option. This its strongest source of staying power.

seanp2k211 tahun yang lalu

You would likely have a regular sh, be it bash in "sh-mode" or a real sh, plus likely csh and maybe ksh. You'd probably also have awk, and maybe even perl.

dijit11 tahun yang lalu

perl is everywhere, but like python, it's power is in it's modules.

and those are not everywhere.

fastviper11 tahun yang lalu

.. you say a shell command line should be user friendly? What kind of user would use command line? We are all bofhs here.

Also some of us are being paid to write bash scripts. I'm for example forced into bash because of AIX server. Learing bash is fun and profitable...

cstoner11 tahun yang lalu

> I'm for example forced into bash because of AIX server.

What? Consider yourself lucky. All of my AIX was ksh. If I wanted to be especially rebellious, I would use ksh93.

s_kilk11 tahun yang lalu

Having played with it for about an hour, I think Xonsh might actually be that alternative.

digi_owl11 tahun yang lalu

the demo gif could use some delays before each clear...

dmix11 tahun yang lalu

Yup, transitions way too fast to know whats going on.

blakehaven11 tahun yang lalu

I was able to read the gif as the information was displayed.

eximius11 tahun yang lalu

Read it, yes. Appreciate it, no.

scopatz4211 tahun yang lalu

I'd readily accept any new demo videos :) Just use a white background so it matches the website theme

strunz11 tahun yang lalu

Is there a reason xonsh doesn't honor the PS1 variable from bash to keep your terminal prefix? Or is there a way to customize xonsh's?

Example:

  10:54:01 foo@bar:~/scripts/xonsh$ xonsh
  foo@bar ~/scripts/xonsh $ # no time prefix

untothebreach11 tahun yang lalu

If you go to [http://xonsh.org/tutorial.html#environment-variables] and scroll down to the table, you can see how to change the prompt :)

strunz11 tahun yang lalu

Thanks, doesn't look like you can use anything that isn't in the supported keywords though (like date/time)

roryokane11 tahun yang lalu

It looks to me like you can.

> PROMPT: The prompt text, may be str or function which returns a str.

If you need non-built-in values in your prompt, just set PROMPT to a function, and call something like `date` within the function. Use `%` to format `date`’s output into a template string, and then pass that template string to `xonsh.environ.format_prompt` to get the final value to return from the function.

krick11 tahun yang lalu

Pretty cool, but I feel unsure what output something will produce. Look:

    a = $(ls)
    a  # '1\n2\n3\n4\n'
    b = `ls`
    b  # []  <--  What?!

Because of these little surprises I cannot really feel confident using it.

AstroChimpHam11 tahun yang lalu

Is there a way to switch explicitly between Python and Bash mode? The "ls -l" example makes me pretty worried that this could mess up basic bash scripts that already work just by defining a variable like "ls" and forgetting about it, or forgetting where it's defined. It's a great idea, but there's a lot of room for stuff breaking that worries me.

ptx11 tahun yang lalu

I don't think you would replace /bin/bash with this, but rather switch to it as your interactive shell, using "chsh". On FreeBSD, for example, the default shell is /bin/tcsh but scripts are usually written for /bin/sh (with a shebang line).

Even if this thing can execute unmodified bash scripts (can it?), it seems to me that would be missing the point, which is to avoid the horror of writing complex bash scripts. When I start writing something in bash, because of those few things it makes so easy (e.g. running subprocesses), I usually regret it as the script grows. If xonsh can make those things just as easy, every script can be written in the right language – which is of course Python :) – from the start.

sciurus11 tahun yang lalu

You can use an uncaptured subprocess to run a command explicitly in bash mode. I'm unsure how you think xonsh would break a bash script, though. When you run a script in xonsh it's still going to be interpreted by whatever is declared in the script's shebang. Just like you can run a perl script from bash, you can run a bash script from xonsh. E.G.

  $ ls='foo'
  $ cat test.sh 
  #!/bin/bash
  ls ~
  $ print(ls)
  foo
  $ ./test.sh 
  bin    C:\nppdf32Log\debuglog.txt  foo	       initrd.img.old  lost+found  opt	 run   sys  var
  boot   dev			   home        lib	       media	   proc  sbin  tmp  vmlinuz
  cdrom  etc			   initrd.img  lib64	       mnt	   root  srv   usr  vmlinuz.old

ridiculous_fish11 tahun yang lalu

The stuff that gets broken are bash scripts that are designed to be sourced directly: virtualenv, rbenv, various prompts, etc.

bkeroack11 tahun yang lalu

Presumably the shebang at the start of your script would cause it to be executed by bash directly.

eximius11 tahun yang lalu

So, for anyone like me who wanted to install it with pip, but has Python 2.7 as their default (and doesn't want to change for fear of breaking things), do the following (on a Debian based system):

curl -O https://raw.githubusercontent.com/pypa/pip/master/contrib/ge...

sudo python3.4 get-pip.py

sudo mv /usr/local/bin/pip /usr/local/bin/pip3 # optional, adjust line below if ommited

sudo pip3 install xonsh

------

then:

chsh -s /usr/local/bin/xonsh <yourusername>

EDIT: Just realized I forgot a step. You have to edit /etc/shells to add /usr/local/bin/xonsh to change your shell to it using chsh.

ivh11 tahun yang lalu

For Debian/jessie it worked nicely for me to just install python3-ply and python3-pip with apt-get and then just do "pip3 install xonsh". Python3 seems to coexist just fine beside the default 2.7.

eximius11 tahun yang lalu

Somehow, python3-pip escaped me. I feel silly.

strunz11 tahun yang lalu

On OSX I just installed it in a virtualenv with python3:

  brew install python3 # will install python3 but not make it default
  mkdir xonsh && cd xonsh
  virtualenv -p /usr/local/bin/python3 venv
  source venv/bin/activate
  pip install xonsh
  xonsh

bishc11 tahun yang lalu

I see a lot of discussion on the awkwardness and idiosyncrasies of shells like bash.... So this is a great time for a shameless plug! Last week I released a language called 'bish' (https://github.com/tdenniston/bish) which compiles to bash, but has none of the same syntactical weirdness. So the idea is: write and maintain your script in bish, but compile to bash and ship the bash version.

Here (https://news.ycombinator.com/item?id=9166386) is the previous discussion from last week.

fastviper11 tahun yang lalu

Wow.. this only needs python. Which means.. it works on windows?

Sadly no support for python 2.7 so I'm out of possibility to even test this.

giancarlostoro11 tahun yang lalu

Forgive me for my lack of knowledge, but where would one use this vs. just using Python?

eximius11 tahun yang lalu

You can use it instead of Bash. It basically combines the two... it looks glorious.

yeukhon11 tahun yang lalu

I think another question is when would one use Python over Bash and vice versa. Personally, I write bash (and shell, reminder bash script not always == shell script) if I need handy command lines. I could have run in subprocess, but would be simpler to just invoke commands directly in my bash script.

rufugee11 tahun yang lalu

Is && and || supposed to supported? When I try "command && someothercommand", which is a common bash idiom, xonsh seems to just freeze up until a ctrl+c. It also jumps up to around 100% CPU.

rufugee11 tahun yang lalu

Additionally, redirection of stderr with 2> doesn't seem to work. Can you describe (or point out in the docs) how to accomplish this? Thank you.

osn11 tahun yang lalu

Is there oh-my-xonsh ready or should we wait for it?

patrickshan11 tahun yang lalu

hehe, you mean zsh ?

osn11 tahun yang lalu

I mean xonsh. Oh'my exists for zsh, fish, git...

agumonkey11 tahun yang lalu

Kinda like it. Not too big a step up from Bash.

[deleted]11 tahun yang lalu

vram2211 tahun yang lalu

Interesting thread. For some alternative ways of doing "pipe-like" stuff in Python (with more than one meaning of the term), see these links:

Swapping pipe components at runtime with pipe_controller:

http://jugad2.blogspot.in/2012/10/swapping-pipe-components-a...

That above post was the last in a series that I wrote over a period. The series describes both a) a small experimental tool I wrote, with examples of a few ways of using it, and b) some other Python-shell tool alternatives that I came across. pipe_controller was inspired by reading about and trying out some of those other tools.

The post above links recursively to all the other posts in the series, but for convenience, I'm pasting selected other links below:

Using PipeController to run a pipe incrementally:

http://jugad2.blogspot.in/2012/09/using-pipecontroller-to-ru...

Some ways of doing UNIX-style pipes in Python:

http://jugad2.blogspot.in/2011/09/some-ways-of-doing-unix-st...

Plumbum, UNIX shell-like library and tool in Python:

http://jugad2.blogspot.in/2012/08/plumbum-unix-shell-like-li...

morekozhambu11 tahun yang lalu

Is there a ruby-ish shell out there?

vram2211 tahun yang lalu

There was one called rush, by Adam Wiggins, co-founder of Heroku, IIRC. I had tried it out a bit after it first came out. Just now did a Google search for "ruby shell rush" - without the quotes, and it found it - first result, though the others about it look interesting too. Not sure if the project is active though.