Comment by snatchpiesinger 2 days ago
Cool! My personal preference is Knuth-style line-breaks on binary operators and pipes, which means breaking before the operator/pipe symbol.
foo -a -b \
| bar -c -d -e \
| baz -e -f
foo -a -b | \
bar -c -d -e | \
baz -e -f
\ linebreaks are not something I love,and a while ago I started using chained blocks..
These are usually a step between "overely complicated one liner" and structured script, and often get refactored to functions etc if the script evolves that far. But lots don't, and if I just want something readable, that also lends itself to comments etc, this works for me.
{ foo -a -b }|{ bar -c -d -e }|{ baz -e -f }
But I suspect it's not everyone's cup of tea!
Next level:
foo -a -b \
| bar -c -d -e \
| baz -e -f \
&& echo "DONE." # && /bin/true
A pattern I typically do
foo && \
bar && \
baz && \
:
You don't need the backslashes in that case. As with lines ending in pipes and a few other places, the line continuation is implicit after the &&:
https://unix.stackexchange.com/questions/253518/where-are-ba...
Huh, neat. So I picked that habit up from writing Dockerfiles, which does let you do
RUN foo && \
bar && \
:
RUN foo &&
bar &&
:
RUN <<EOF
foo
bar
EOF
I do it this way but I indent the rest of the pipeline (like one typically would in a functional language with a pipeline operator or thread macro, or in an OOP language with method chaining via `.`):
foo --long-something \
| bar --long-other
foo --long-flag \
| bar \
--long-other-flag \
--long-option a \
--long-option b \
| baz \
--long-another-flag \
--long-flag-again \
> /path/to/some/file
foo \
| bar \
| baz
and echo done
test -n "$SOMETHING"
or set -x SOMETHING some-default
command $SOMETHING
One of my favorite patterns for readability is to use indented, long-form pipelines like this to set variables. They work fine inside subshells, but for uniformity and clarity I prefer to do
shopt -s lastpipe
foo \
| bar \
| baz \
| read SOMEVAR
If you do this, you end up with a script is mostly 'flat' (having very little nested control flow, largely avoiding loops), has very few variable assignments, and predictably locates the variable assignments it does have at the ends of pipelines. Each pipeline is a singular train of thought requiring you to consider context and state only at the very beginning and very end, and you can typically likewise think of all the intermediate steps/commands in functional terms.
I tend to write all of my shell scripts this way, including the ones I write interactively at my prompt. One really cool thing about shell languages is that unlike in 'real' programming languages, loops are actually composable! So you can freely mix ad-hoc/imperative and pipeline-centric styles like this (example is Fish):
find -name whatever -exec basename '{}' \;
| while read -l data
set -l temp (some-series $data)
set -l another (some-command $temp)
blahdiblah --something $temp --other $another
end \
| bar \
| baz \
> some-list.txt
I tend to follow my guideline suggested earlier and do this only close to the very beginning or very end of a pipeline if that loop actually modifies the filesystem or non-local variables, but it's really nice to have that flexibility imo. (It's especially handy if you want to embed some testing or conditional logic into the pipeline to filter results in a complex way.)
Shell script authors like yourself make me very happy. The pipe-to-read is a fun idea, I’ll use it.
One stanza I have at the beginning of every script:
[[ -n “${TRACE:-}” ]] && set -o xtrace
This was typed from memory on mobile so if the above is bugged, my bad :)
> Shell script authors like yourself make me very happy.
Shell script is really good for some things, so we're gonna end up writing it. And if we write it, it might as well be legible, right? Shell scripts deserve the same care that other programs do.
> This lets you trace any script just by setting the environment variable. And it’s nounset-safe.
Nice! I think I'll start adding the same.
> This was typed from memory on mobile so if the above is bugged, my bad :)
I think it's fine aside from the smart quote characters your phone inserted instead of plain vertical double quotes around the parameter expansion you use to make the line nounset-friendly!
Same w/ the "pipe to read" (although it doesn't seem to work right with OSX's bash-3.2). I found this gizmo somewhere and it's worth sharing as well...
# MAGIC DEBUGGING LINE
#trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND? ")' DEBUG
I need to write up some thoughts on bash being effectively a lisp if you stare at it the right way.
> Same w/ the "pipe to read" (although it doesn't seem to work right with OSX's bash-3.2). I found this gizmo somewhere and it's worth sharing as well...
Yes, you need a recent Bash for this (and lots of other nice things like 'associative arrays' (maps)). To free myself up to write more concise and legible scripts, I lean into bashisms and make use of external programs without restriction, regardless of what's shipped by default on any system. To recover portability where I need it, I use Nix to fix my Bash interpreter versions as well as the versions of external tools.
When my scripts are 'packaged' by Nix, their bodies are written as multiline strings in Nix. In that case, Nix takes care of setting the shebang to a fixed version of Bash, and I interpolate external commands in like this:
"${pkgs.coreutils}/bin/realpath" . # these do
"${pkgs.coreutils}/bin/pwd" --physical # the same thing
In other contexts (e.g., cloud-init), I use bash a bit more restrictively depending on what versions I'm targeting. But I still use Nix to provide dependencies so that my scripts use the same versions of external tools regardless of distro or platform:
nix shell nixpkgs#toybox --command which which # these do
nix run nixpkgs#which -- which # the same thing
There are is a really cool tool[1] by the Nixer abathur for automagically converting naively-written scripts to ones that pin their deps via Nix, as in the first example. I'm not using it yet but I likely will if the scripts I use for our development environments at work get much bigger-- that way I can store them as normal scripts without any special escaping/templating and linters will know how to read them and all that.
Anyhow, it's totally safe to install a newer Bash on macOS, and I recommend doing it for personal interactive use and private scripts. Pkgsrc and MacPorts both have Bash 5.x, if you don't have or want Nix.
> I need to write up some thoughts on bash being effectively a lisp if you stare at it the right way.
You should! It's totally true, since external commands and shell builtins are in a prefix syntax just like Lisp function calls. Some shells accentuate this a bit, like Fish, where subshells are just parens with no dollar signs so nested subcommands look very Lisp-y, and Elvish, whose author (xiaq) has tried to lean into that syntactic convergence designing a new shell (drawing inspiration from Scheme in various places).
> I found this gizmo somewhere and it's worth sharing as well...
# MAGIC DEBUGGING LINE
#trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND? ")' DEBUG
--
If you end with a pipe, you don't need the backslash before the newline. It's implicit.
https://unix.stackexchange.com/questions/253518/where-are-ba...
When writing bash, if I have a command with many switches, I use an array to avoid backslashes and make the intent clearer:
I also insist on long-form options in shell scripts since they are much more self-documenting as compared to single-letter options.