The Dune Shell
(adam-mcdaniel.github.io)240 points by thunderbong 10 months ago
240 points by thunderbong 10 months ago
The problem of powershell is not merely thet it's new and different, and not merely that you have to learn a new and different way to get a given task done.
Powershell makes some things possible or easier that weren't possible or easy, but only at the expense of making other things either impossible or at least vastly less practical, and all in all it's a bad trade off and a wrong set of priorities for the job it claims to fill, or at least for the job bash currently fills. It's not that bash is a gold standard, just that it's not fundamentally wrong.
It's also just absurdly verbose for a language intended mainly for manual input. I get annoyed having to type out `sudo systemctl` multiple times in a row, in Powershell every single command is at minimum that long. Which makes it way more difficult to memorize the commands too.
There are ton of aliases. It's "impolite" [0] to use aliases in scripts and documentation so looking at PowerShell examples is way more verbose than actually using it day to day in a REPL. Sure I could write `Set-Location` a million times a day, but I just use `cd` in the old ways. Same with `Get-ChildItem`, I tend to just use `ls` myself because of ancient habits. (I find it interesting how many have been moving to `gci` instead as the more PowerShell-native `ls` alias. I've not been convinced to do that myself, but I think it has to do with `gci` is way more powerful than most shells' `ls` and is used to navigate everything from folder structures to object structures including that its not a bad `jq` if you convert JSON files to PowerShell objects.)
[0] It's from a version of the Python ethos that code is read way more often than it is written, so when you are polishing PowerShell code to share with others you expand all the aliases so that it is easier to read.
Really? I only have to remember Item instead of touch, mkdir, RM, rmdir, cp, mv. All commands that have just different verbs before the Noun in Powershell. And that's for every single cmdlet out there.
It sounds to me that you're unfairly putting your existing knowledge about Linux shells into your criticism of PowerShell. If you want to be fair that criticism should come from the POV of someone who does not now both and now wants to memorise it. And whether or not commands are short or long does not matter. People memorise lyrics, or names of hundreds of people. Length is not a factor, objectively.
And alias do exist.
True. PS has platform-specific API hooks that are orthogonal to the capabilities of other shells, and more-less replaces JScript, VBScript, and .bat files.
Putting that stuff aside, it is/was one of the first command shells/REPLs outside of SQL DBMS ones in a long time to support manipulation of structured information. For a smoother transition, shells like nu that can straddle structured and file-/line-/word-oriented will be more successful than leaping to too radical of upheaval.
Curious as to what's "vastly less practical" in powershell?
> too radical having to reimagine/replace/relearn almost everything
I switched to zsh because at the time I thought it had better autocomplete, but honestly I use it as I would bash. And I don't think it was worth it to change a little bit.
To change shell I think the change _should_ be radical.
I was scripting on a windows machine which didn’t have WSL setup and decided to learn PowerShell. After a week or so of scripting with PowerShell and seeing how intuitive and robust the syntax is, something changed in my brain and I decided I never again wanted to deal with bash and its endless quirks and brittleness. It’s unnatural to have to wrap every variable access with “${}”, that unnatural key sequence always caused a break in my flow. Anyway I reach for Python sooner and really hunger for a clean, robust nix shell. I hope powershell has instilled this in many people and it drives adoption of something better
I have tried to do the same, but i have opposite experience. I just can't grok the syntax, it always seems like a magician pulling out another "Haha! I bet you haven't seen this one coming !"
Like, i can't even construct the abstract model of how it's supposed to work
Would you be able to provide an example? I'm not doubting your experience, just incredibly curious since I had the opposite experience.
I'm even a windows .NET developer and I also find Powershell extremely weird. I'm not sure what it is.
Interesting thought experiment. Reminds me of my cousin's project Xonsh https://xon.sh/
A day or so there was this: https://news.ycombinator.com/item?id=41558081
Imagine your shell's model not being a command line console but a spreadsheet!
I wonder if there's a good middle ground where unix commands could opt into a structured output, and until they exist, you could shim via `ls -la | shim "ls -la" |> ".modified_time"`. Then one day it could be `ls -la |> ".modified_time"` when structured data is supported natively by the program or command. I feel every project ends up dying since it needs to support every command all at once to be useful.
Shades of Q, APL and other array languages, but visual by default. Maybe an APL spreadsheet-shell would fit the bill? :-D
> One can dream of a future where perhaps it will be possible to write a function, library, or program in any language one chooses and call it from any other shell or language without swig or ceremony
I mean, there's a reason text was selected, it's the lowest common denominator. What you're asking for is universal FFI, which we already have that for the most part: C. The problem ultimately turns into runtime and resource management issues when crossing language boundaries, that is, unless you use OS primatives (pipes, sockets, shared memory, processes, etc.) and use message passing/pipelines exclusively. Then we're back to plaintext.
There's a ton of non-printable characters in ASCII that are useful as delimiters for non-keyboard interactive programs that are largely vestigial. One could at least consider reusing them for delimiters and special control sequences for message passing, but without some sort of standardization it's limited in practical use.
The problem with those ascii values is editors don't display them and there are no keyboard keys to type them.
The reason csv exists is because you can actually see and type all parts of it, on anything, any platform, any hardware, any software, any age, even a mechanical typewriter, even a pencil. That is not merely nice, it's essntially priceless, infinitely, incalculably valuable. The utility outweighs the problems as big as they absolutely are. Actually the same is true for json, yaml, xml...
> The problem with those ascii values is editors don't display them and there are no keyboard keys to type them.
Did you read my post? Like, every word of it? Being able to type a control sequence is not particularly useful for ephemeral data in a message-passing between programs context. It's absolutely important for persistent, editable data. It seems like more of a feature to me to use these special characters for special contexts and not have escaping typeable characters be load-bearing.
Further, my editor seems to print control characters just fine, though entering them would be a bit of a pain and the behavior is likely configurable.
Instead of inventing some new delimited formats, we also already have nearly ubiquitous interchange formats like JSON. If you wanted to reduce the in memory footprint versus plaintext encoding the JSON there are good options like BSON and CBOR.
JSON isn't streamable. Neither is CBOR for that matter, though it's not quite as bad.
Nitpick but I wouldn’t call C a universal FFI, it’s merely the de facto FFI. C can’t meaningfully represent any complex language semantics (not that it should or even could)
It's a near universal de facto FFI if we're being pedantic. My whole point was the reason that's the case is it has essentially no runtime expectations at all, which is a requirement for a truly universal FFI.
> One can dream of a future where perhaps it will be possible to write a function, library, or program in any language one chooses and call it from any other shell or language without swig or ceremony.
Was that not VMS?
Here is a nice list: https://github.com/oils-for-unix/oils/wiki/Alternative-Shell...
PowerShell seems to work fine calling a nix executable, unless I'm missing something:
> ls -t *.js | head -n 10 | foreach { echo $_ ; Get-ChildItem $_ | fl Name, Length } product_original.js
Name : product_original.js Length : 3353
gulpfile.js
Name : gulpfile.js Length : 382
> One can dream of a future where perhaps it will be possible to write a function, library, or program in any language one chooses and call it from any other shell or language without swig or ceremony.
That's a neat idea, and sounds like something that could be built on graalvm.
To me the silly hang-up for PowerShell is none of that, but the strange syntax. I'd taken (even!) JavaScript-ish syntax over that.
Maybe tabular data is a credible intermediate step. Many shell tools already export data in a tabular format.
"Think of it as an unholy combination of bash and Lisp" reminded me of Scheme Shell: https://scsh.net/
My Ph.D. advisor Olin Shivers created this. It's more pulling bash into Lisp than Lisp into bash.
It doesn't seem to be maintained anymore, but some of its ideas live on in Racket.
And now there’s Rash: The Reckless Racket Shell
I’ve always wanted to ask about the notorious introduction to scsh’s manual… [1] could Olin have been less than 100% satirical there?
More white rat than desert mouse, I was disappointed. The mouse is also featured in the source material - the books :)
Also the one in the film looked more like a jerboa.
Don’t know what a jerboa is but I quite liked the depiction in the film.
In particular the attention to detail, when they showed a bead of sweat running down its optimally shaped ear, whereupon it was pawed into its mouth.
I loved that little scene so much.
In the domain and history that shells inhabit, the design should not prioritize arithmetic operations over indirection. '>>' is a biiig footgun. Elevation of a symbol to non-destructive operation into a destructive one is obviously a wrong choice. I wonder if it was a deliberate decision.
edit: just leave '>>' as append and make '>>>' destructive instead.
I was thinking the same.
They should at least make it an option to prompt the user when interactively redirecting to an already existing file. ZSH has this, except for the prompt (you'll need to run the command again using `>!`)
I like the ideas behind Powershell, but the DX has always felt ugly and clunky compared to standard shell. Unix/Posix are full of great tools, but they are all extremely painful for newbies and take years to master. There's got to be a middle ground where we can have the power and approachability of SQL with the beautiful elegance of standard Unix pipes.
Dune seems to take a step in that direction which is great.
It kind of sucks that the default shell language is so clunky on Linux O/Ses. Bash marginally better than POSIX sh, and Zsh marginally better.
Even worse is that POSIX forces this to be used.
Using a non POSIX shell is much easier to write (e.g. fish xonsh - and possibly Powershell or rash but I have not used the last two)
sh and derivatives are difficult for me to follow the syntax is impenetrateable - and I was a professional APL programmer.
bash has its own issues but my main gripe is handling of arrays and dictionaries - the syntax is so awkward and hard to remember that I inevitably have to have a reference open in another window. I wish they added some syntactic sugar to make that more ergonomic while retaining backwards compatibility.
The author writes:
With my last shell, Atom, I had accomplished some of the coziness that bash was missing, but I also introduced a lot of really fatal flaws in the syntax and the type system.
I would be interested to read more about this -- what the author learned from that experiment.
Obligatory mention of Emacs Eshell. https://www.gnu.org/software/emacs/manual/html_mono/eshell.h...
The advertisement promises a combination of bash and lisp. So it makes me wonder -- why couldn't we just use lisp?
What are the affordances needed for a "scripting" language (or for interactive use) vis-a-vis more "production use? Is it just about having minimal boilerplate, and a large corpus of ready-to-use functions in the namespace?
"Is it just about having minimal boilerplate"
That's a lot of it, but I think people don't realize how every keystroke counts with shell.
There's only a handful of languages where the "apply function" operator is space. Shell is one of them. (Haskell & Forth are the other two I know off the top of my head, possibly Factor (concatenative in general tends this way).) Most new shells that are successful copy this. I don't think that's a coincidence. Lisp's abundance of parens is something that people will have trouble with, even Lisp programmers, because this is not the usual whining about a foreign language paradigm and not being used to reading parens... this is literally about the effort required to physically enter them with a keyboard.
Regarding minimal syntax: https://yosefk.com/blog/i-cant-believe-im-praising-tcl.html
Yeah, it really is like that.
I've tried a couple of times to switch to things like IPython as my shell. And what nukes it every time is that while it may be better at the complicated commands I run maybe 1% of the time if you actually look at an unmodified history,
cd("some dir")
cd som TAB ENTER
It doesn't look like it when you're doing it once, but when you're doing it dozens of times a minute it adds up fast.
In addition to "space is function application" another important shell affordance seems to be "words are just words" -- you don't have to quote simple strings like `README.md`. If you want a single string with more than one word (where there is a space in between), that is in conflict with "space is function application", so then you have to put it in quotes.
https://github.com/babashka/babashka
Granted, if you're saying why can't we have the shell also... well, now you're talking about Emacs.
I like the ideas of Dune and Nu, which I think were trailblazed by powershell to a degree. The problems of pure *NIX pipe philosophy with text as the lingua franca were persistent parsing problems and escaping. However, the problems of going to far in the way powershell went was it was too radical having to reimagine/replace/relearn almost everything, and generally more difficult to accomplish simple text manipulation. Nu at least maintains most of the *NIX shell UX. Maybe at some point in the future, we will reach parity to where there will fusion of the main DSLs, shells, and programming languages into a runtime interaction system that is somewhere between psql, Erlang shell, nu, and Squeak where there is programming language-OS parity obliterating the distinctions between programming languages, command shells, and DBMS shells. One can dream of a future where perhaps it will be possible to write a function, library, or program in any language one chooses and call it from any other shell or language without swig or ceremony.