Notes on shell programming

The point of a standard isn't even whether it's ideal or particularly sensible, it's that a compliant program produces consistent results on compliant platforms. Diverging in the name of whatever benefit just means that one has to work harder to produce a portable program. -- Richard L. Hamilton, in comp.unix.questions

I occasionally have to write Bourne shell scripts. Often enough that I've a small bag on knowledge about it, rarely enough that I tend to rediscover it far too often for my liking. Here is a repository of some of them, mainly for my use, but they could be useful for others.

On the given quote, my view is that portability is not a goal, it is a mean. Thus one shouldn't jeopardize the goal neither in the name of portability nor in the one of convenience.

Template

#! /bin/sh -

set -e

usage() {
    printf "Usage: prog [options] args...\n"
}

help() {
    usage
    printf "Description...\n"
    printf "\nArguments\n"
    printf "\nOptions\n"
}

while getopts hab: opt ; do
    case $opt in
        h) 
            help
            exit 0
            ;;
        a) 
            printf "Received -a\n"
            ;;
        b)
            printf "Received -b %s\n" "$OPTARG" 
            ;;
        ?) 
            usage
            exit 1
            ;;
    esac
done
shift $((OPTIND - 1))

Index

Some recommendations
Rationale for things one often see in scripts

Some recommendations

Parameter quoting

Parameter should be expanded between double quotes "${var}" if one doesn't want to get their value split. There are a few contexts where the quoting isn't needed.

parameter assignment:
```
var=$other
```
case control:
```
case $var in
```
one knows there is nothing to split ($# and other parameters restricted to numeral values by the logic of the script)

Parameter expansion

	`var` is not set	`var` is null	`var` is not null
`${var:-txt}`	txt	txt	$var
`${var-txt}`	txt		$var
`${var:=txt}`	txt	txt	$var
`${var=txt}`	txt		$var
`${var:+txt}`			txt
`${var+txt}`		txt	txt

The form with = also set $var. This is often used in a context where the parameter expansion isn't needed, just to set a default value. For instance as argument of : (which is a variant of true which never use its argument, GNU true outputs something with --version).

: ${var:=default}

A common use is to set the default value of optional arguments when the default is set from other (potentially optional) arguments. For instance:

: ${prefix:=/usr/local}
: ${bindir:=${prefix}/bin}
: ${libdir:=${prefix}/lib}

will set prefix, bindir and libdir if they are not already defined by the preceding arguments parsing.

To test if a parameter is set, the idiom is [ -n "${var+set}" ] and to test if it is not set, use [ -z "${var+set}" ]. The variants with :+ instead of + are handling a set but null parameter in the same way as an unset one.

There is also some way to remove a pattern at the start or end of a parameter:

${V%pattern}: remove the shortest pattern from the end of V.
${V%%pattern}: remove the longest pattern from the end of V.
${V#pattern}: remove the shortest pattern from the start of V.
${V##pattern}: remove the longest pattern from the start of V.

Test

[ (or test) can be used to compare strings and numbers:

	strings	integers
equal	=	-eq
not equal	!=	-ne
less	<	-lt
less or equal	<=	-le
greater or equal	>=	-ge
greater	>	-gt

-z and -n test for (non) null strings.

Starting with ! negates the test.

Testing files

-t fd	fd is a terminal
-e path	entry exists
-f path	normal file
-d path	directory
-L path	symbolic link
-r path	readable
-w path	writable
-x path	executable
-s path	file not empty

See also the note on test X"$var" = X"value".

Arithmetic expansion

Expressions in $(( ... )) are replaced by their value. Parameters in these expressions do not need to be preceded by $. The operators and their precedence are

Operator
unary +, unary -, ~, !
*, /, %
+, -
<<, >>
<, <=, >=, >
==, !=
&
^
\|
&&
\|\|
=, *=, /=, %=, +=, -=, <<=, >>=, &=, ^=, \|=

expr

expr is a standalone program which allows to evaluate expressions. For arithmetic expressions, $((...)) is probably more convenient and avoid to launch another process, for comparisons, test avoid the issue of different interpretations determined if the values are decimal representations or not, so the main usage of expr is the regular expression matching.

Operator	Description
( expr )	Grouping
expr1 : expr2	regular expression matching, either the number of matched characters or the match of the first subexpression `$ ...$`.
expr1 * expr2	Multiplication of decimal integers.
expr1 / expr2	Integer division of decimal integers.
expr1 % expr2	Remainder of integer division of decimal integers.
expr1 + expr2	Addition of decimal integers.
expr1 - expr2	Subtraction of decimal integer.
expr1 = expr2	Equal.
expr1 > expr2	Greater than.
expr1 >= expr2	Greater than or equal.
expr1 < expr2	Less than.
expr1 <= expr2	Less than or equal.
expr1 != expr2	Not equal.
expr1 & expr2	Returns the evaluation of expr1 if neither expression evaluates to null or zero; otherwise, returns zero.
expr1 \| expr2	Returns the evaluation of expr1 if it is neither null nor zero; otherwise, returns the evaluation of expr2 if it is not null; otherwise, zero.

Comparisons return the result of a decimal integer comparison if both arguments are integers; otherwise, returns the result of a string comparison using the locale-specific collation sequence. The result of each comparison is 1 if the specified relationship is true, or 0 if the relationship is false.

Forwarding arguments

Forwarding arguments to another program should be done with "$@" so that they are not split before the passing. If one want to process some arguments and pass the others, something like this is in order to handle the argument parsing.

#! /bin/sh -

set -e

checkarg() {
    if [ -z "$2" ] ; then
       printf "%s: argument expected\n" "$1"
       exit 1
    fi
}

cnt=0
argc=$#
while [ $cnt -ne $argc ] ; do
    case $1 in
        -mine)
            printf "Got -mine\n"
            ;;
        -with)
            checkarg "$1" "${2+set}"
            printf "Got -with '%s'\n" "$2"
            shift
            cnt=$((cnt + 1))
            ;;
        --)
            shift
            cnt=$((cnt + 1))
            break
            ;;
        *)
            set -- "$@" "$1"
            ;;
    esac
    shift
    cnt=$((cnt + 1))
done

while [ $cnt -ne $argc ] ; do
    set -- "$@" "$1"
    shift
    cnt=$((cnt + 1))
done

showargs prog "$@"

Choice of shell

Although I'm somewhat concerned about portability, I don't write scripts for unconditional one. For instance all the machines I care about had a /bin/ksh and thus I used write my scripts for it (trying to keep to POSIX features) instead of writing them to the lowest common denominator of /bin/sh (that would be the one from Solaris) or doing epic effort to respawn a better shell. Nowadays I'm using /bin/sh and using POSIX features without caring about the old Solaris Bourne shell.

Miscellaneous

! applies to the pipe (! false | true is false) but not the whole expression (! false || true is true and ! false && false is false).

Rationale for things one often see in scripts

Here are some rationales for usage often seen in scripts but that I don't follow.

`test X"$var" = X"value"`

The idiom test X"$var" = X"value" is used traditionally to handle correctly the case where $var begins in a way which could confuse test about the fact it is something to be compared. Personally, I avoid -a, -o and the parenthesis in test arguments (thus I use the shell && and || and grouping) as they are marked as obsolete by POSIX and this avoidance suppresses the need of the trick.

Test or [

Some scripts are using

if test cond ; then

while others are using

if [ cond ] ; then

The only reason I know to choose the former is that the use of [ doesn't work with some preprocessing tools (m4 for instance for autotools scripts).

Use of negation

Some scripts are using

if cond ; then
   :
else
   ...
fi

Instead of

if ! cond ; then
   ...
fi

The only shell I know which doesn't support ! is /bin/sh on Solaris. Better use some other shell on Solaris, that one is stable but has been frozen for so long that supporting it is painful.

! in test is AFAIK supported everywhere.

Conditional expression with [[

Some shells, at least bash and zsh, allow conditional expressions with [[. Those behave mostly like test but with some additional capabilities such as using && and || to combine subexpressions.

Arithmetic expression with ((

Some shells, at least bash and zsh, allow arithmetic expressions with ((. Note that like expr it has an non-null exit status when the result is 0, so that makes it often less convenient than $((...)).

References

Sven Mascheck on Portable Shell Programming (where I got the starting quote).
Stéphane Chazelas on #!/bin/sh - (his questions and answers in unix.se are a gold mine of information).
Sven Mascheck on #!/bin/sh - (with a table of the behavior of various systems).

Navigation

Topics