Most Bash scripts start as five lines that "just work" and slowly grow into something that runs backups or deployments every night. The trouble is that Bash's defaults are forgiving in exactly the wrong way: a failed command is ignored, an unset variable expands to nothing, and a filename with a space turns into two arguments. That's how rm -rf "$DIR/"* ends up deleting the wrong thing.
These are the habits I use for any script that matters. Every example here was run on Bash 5 and passes ShellCheck.
Start with strict mode
Put this at the top of every non-trivial script:
#!/usr/bin/env bash set -Eeuo pipefail IFS=$'\n\t'
What each flag does:
- -e: exit as soon as a command fails (non-zero exit status).
- -u: treat unset variables as an error instead of silently expanding them to an empty string.
- -o pipefail: a pipeline fails if *any* command in it fails, not just the last one.
- -E: make the
ERRtrap (below) also fire inside functions and subshells. - IFS: stop word splitting on spaces, so unquoted expansions only split on newlines and tabs. It's a safety net, not a substitute for quoting.
The difference pipefail makes is easy to see:
$ bash -c 'false | true; echo "without pipefail: $?"' without pipefail: 0 $ bash -c 'set -o pipefail; false | true; echo "with pipefail: $?"' with pipefail: 1
Without it, mysqldump mydb | gzip > backup.sql.gz "succeeds" even when the dump fails, because gzip exited cleanly.
With -u, give optional variables an explicit default: ${VAR:-fallback} for a default value, or ${1:?usage: script SRC DEST} to stop with a message when a required argument is missing.
Where set -e won't save you
set -e has well-known blind spots, and it's worth knowing them so you don't over-trust it:
- It's ignored inside any function called as part of an
if,while,&&or||. A failing command in that function keeps going. local x=$(false)hides the failure, because the exit status is that oflocal, which succeeded. Declare and assign separately:local x; x=$(cmd).
For critical steps, check explicitly: cmd || die "cmd failed".
Quote everything
Unquoted variables are split on whitespace and expanded as globs. This deletes the wrong files, or nothing at all:
f="my file.txt" rm $f # tries to remove "my" and "file.txt" rm "$f" # removes "my file.txt"
Rules of thumb:
- Always quote
"$var","$(command)"and"$@". - Use arrays for lists of arguments:
opts=(-a -v --delete), thenrsync "${opts[@]}" "$src" "$dst". - Put
--before filenames in commands likerm,mvandcp, so a file called-rfcan't be read as an option.
Log to stderr and fail clearly
Two tiny functions make scripts far easier to debug from cron or a systemd journal:
log() { printf '%s [%s] %s\n' "$(date '+%F %T')" "${0##*/}" "$*" >&2; }
die() { log "ERROR: $*"; exit 1; }
Logging to stderr keeps stdout clean for real output, so the script can still be used in a pipeline.
Traps: report errors and always clean up
An ERR trap tells you exactly where things went wrong:
trap 'echo "error on line $LINENO: $BASH_COMMAND" >&2' ERR
Running a script that tries to copy a missing file prints:
cp: cannot stat '/nonexistent/file': No such file or directory error on line 4: cp /nonexistent/file /tmp/
An EXIT trap runs however the script ends (success, error or Ctrl-C), which makes it the right place to remove temporary files:
WORK_DIR="$(mktemp -d)"
cleanup() { rm -rf -- "$WORK_DIR"; }
trap cleanup EXIT
Always use mktemp. Predictable names like /tmp/backup.tmp invite races and clashes when two copies run at once.
A complete example
Putting it together, here's a small backup script that archives a directory to a destination, with argument checks, logging, traps and a temp directory. The archive is only moved into place once it's complete, so a failed run never leaves a half-written backup that looks valid.
#!/usr/bin/env bash
set -Eeuo pipefail
IFS=$'\n\t'
readonly SCRIPT_NAME="${0##*/}"
readonly SRC_DIR="${1:?usage: $SCRIPT_NAME SRC_DIR DEST_DIR}"
readonly DEST_DIR="${2:?usage: $SCRIPT_NAME SRC_DIR DEST_DIR}"
log() { printf '%s [%s] %s\n' "$(date '+%F %T')" "$SCRIPT_NAME" "$*" >&2; }
die() { log "ERROR: $*"; exit 1; }
on_error() {
local exit_code=$? line=$1
log "failed at line ${line} (exit ${exit_code}): ${BASH_COMMAND}"
}
trap 'on_error $LINENO' ERR
WORK_DIR="$(mktemp -d)"
cleanup() { rm -rf -- "$WORK_DIR"; }
trap cleanup EXIT
command -v tar >/dev/null || die "tar is not installed"
[[ -d $SRC_DIR ]] || die "source directory not found: $SRC_DIR"
mkdir -p -- "$DEST_DIR"
archive="${WORK_DIR}/backup-$(date +%Y%m%d-%H%M%S).tar.gz"
log "archiving ${SRC_DIR}"
tar -czf "$archive" -C "$(dirname -- "$SRC_DIR")" "$(basename -- "$SRC_DIR")"
mv -- "$archive" "$DEST_DIR/"
log "done: ${DEST_DIR}/${archive##*/}"
Run it as ./backup.sh "my data" /mnt/backup. Note that the directory name with a space works fine. To run it nightly with proper logging and catch-up after downtime, use a systemd timer; the setup is in Systemd Units Explained.
Parsing options with getopts
For anything beyond one or two positional arguments, use getopts. It's built in, handles combined flags like -nv, and reports missing values:
dry_run=0 verbose=0 dest=""
while getopts ':d:nvh' opt; do
case $opt in
d) dest=$OPTARG ;;
n) dry_run=1 ;;
v) verbose=1 ;;
h) usage; exit 0 ;;
:) echo "option -$OPTARG needs a value" >&2; exit 2 ;;
\?) echo "unknown option -$OPTARG" >&2; exit 2 ;;
esac
done
shift $((OPTIND - 1))
The leading : in the option string switches on "silent" mode so your case handles errors itself. A dry-run flag like -n is worth adding to any script that deletes or moves files.
Prevent overlapping runs
If a job can take longer than its schedule interval, make sure two copies never run at once. flock does this in three lines:
exec 9>/tmp/nightly-sync.lock if ! flock -n 9; then echo "another run is still in progress, exiting" >&2 exit 0 fi
The lock is released automatically when the script exits, even if it crashes.
Retry flaky operations
Network calls fail now and then. A small retry helper with exponential backoff beats sprinkling sleep around:
retry() {
local n=0 max=$1; shift
until "$@"; do
n=$((n + 1))
(( n >= max )) && return 1
sleep $((2 ** n))
done
}
retry 5 curl -fsS https://example.com/health
Lint with ShellCheck
ShellCheck catches unquoted variables, useless cats, broken test syntax and dozens of other mistakes. Install it (sudo apt install shellcheck) and run it on every script, or wire it into your editor or CI:
shellcheck backup.sh
Treat its warnings as bugs until proven otherwise. Each one links to a wiki page explaining why.
Know when to stop using Bash
Bash is great for gluing commands together. Once a script needs data structures, JSON handling or more than a few hundred lines, Python or PHP will be easier to maintain. A good test: if you're writing a parser in Bash, it's time to switch.
For more command-line building blocks, see Mastering find and xargs. The same quoting rules apply there, especially for filenames with spaces.
If you need automation, server scripts or a reliable deployment process set up for your business, I do that alongside web development. See my portfolio.
Comments (0)
No comments yet. Be the first to leave one!
Leave a Comment