Notes on shell programming

I occasionally have to write Bourne shell scripts. Often enough that I've a small bag on knowledge about it, rarely enough that I tend to rediscover it far too often for my liking. Here is a repository of some of them, mainly for my use, but they could be useful for others.


#! /bin/ksh

usage() {
    printf "Usage: prog [options] args...\n"

help() {
    printf "Description...\n"
    printf "\nArguments\n"
    printf "\nOptions\n"

while getopts hab: opt ; do
    case $opt in
            exit 0
            printf "Received -a\n"
            printf "Received -b %s\n" "$OPTARG" 
            exit 1
shift $(expr $OPTIND - 1)


Some recommendations

Parameter quoting

Parameter should be expanded between double quotes "${var}" if one doesn't want to get their value split. There are a few contexts where the quoting isn't needed.

Parameter expansion

var is not set var is null var is not null
${var:-txt} txt txt $var
${var-txt} txt $var
${var:=txt} txt txt $var
${var=txt} txt $var
${var:+txt} txt
${var+txt} txt txt

The form with = also set $var. This is often used in a context where the parameter expansion isn't needed, just to set a default value. For instance as argument of : (which is a variant of true which never use its argument, GNU true outputs something with --version).

: ${var:=default}

A common use is to set the default value of optional arguments when the default is set from other (potentially optional) arguments. For instance:

: ${prefix:=/usr/local}
: ${bindir:=${prefix}/bin}
: ${libdir:=${prefix}/lib}

will set prefix, bindir and libdir if they are not already defined by the preceding arguments parsing.

To test if a parameter is set, the idiom is [ -n "${var+set}" ] and to test if it is not set, use [ -z "${var+set}" ].

There is also some way to remove a pattern at the start or end of a parameter:

remove the shortest pattern from the end of V.
remove the longest pattern from the end of V.
remove the shortest pattern from the start of V.
remove the longest pattern from the start of V.


[ (or test) can be used to compare strings and numbers:

strings integers
not equal!=-ne
less or equal<=-le
greater or equal>=-ge

-z and -n test for (non) null strings.

Starting with ! negates the test.

Testing files

-t fdfd is a terminal
-e pathentry exists
-f pathnormal file
-d pathdirectory
-L pathsymbolic link
-r pathreadable
-w pathwritable
-x pathexecutable
-s pathfile not empty

See also the note on test X"$var" = X"value".

Arithmetic expansion

Expressions in $(( ... )) are replaced by their value. Parameters in these expressions do not need to be preceded by $. The operators and their precedence are

unary +, unary -, ~, !
*, /, %
+, -
<<, >>
<, <=, >=, >
==, !=
=, *=, /=, %=, +=, -=, <<=, >>=, &=, ^=, |=


Forwarding arguments

Forwarding arguments to another program should be done with "$@" so that they are not split before the passing. If one want to process some arguments and pass the others, something like this is in order to handle the argument parsing.

#! /bin/sh -
set -e

checkarg() {
    if [ $(( $2 + 1 )) = $3 ] ; then
       printf "%s: argument expected\n" "$1"
       exit 1

while [ $cnt -ne $argc ] ; do
    case $1 in
            printf "Got -mine\n"
            checkarg "$1" $cnt $argc
            printf "Got -with %s\n" "$2"
            cnt=$(( cnt + 1 ))
            cnt=$(( cnt + 1 ))
            set -- "$@" "$1"
    cnt=$(( cnt + 1 ))

while [ $cnt -ne $argc ] ; do
    set -- "$@" "$1"
    cnt=$(( cnt + 1 ))

exec prog "$@"

Choice of shell

Although I'm somewhat concerned about portability, I don't write scripts for unconditional one. For instance all the machines I care about have a /bin/ksh and thus I write my scripts for it (trying to keep to POSIX features) instead of writing them to the lowest common denominator of /bin/sh (that would be the one from Solaris) or doing epic effort to respawn a better shell.


Rationale for things one often see in scripts

Here are some rationales for usage often seen in scripts but that I don't follow.

test X"$var" = X"value"

The idiom test X"$var" = X"value" is used traditionally to handle correctly the case where $var begins in a way which could confuse test about the fact it is something to be compared. Personally, I avoid -a, -o and the parenthesis in test arguments (thus I use the shell && and || and grouping) as they are marked as obsolete by POSIX and this avoidance suppresses the need of the trick.

Test or [

Some scripts are using

if test cond ; then
while others are using
if [ cond ] ; then

The only reason I know to choose the former is that the use of [ doesn't work with some preprocessing tools (m4 for instance for autotools scripts).

Use of negation

Some scripts are using

if cond ; then

Instead of

if ! cond ; then

The only shell I know which doesn't support ! is /bin/sh on Solaris. Better use some other shell on Solaris, that one is stable but has been frozen for so long that supporting it is painful.

! in test is AFAIK supported everywhere.

Conditional expression with [[

Some shells, at least bash and zsh, allow conditional expressions with [[. Those behave mostly like test but with some additional capabilities such as using && and || to combine subexpressions.

Arithmetic expression with ((

Some shells, at least bash and zsh, allow arithmetic expressions with ((.