Shell Scripting
Bash scripting patterns, control flow, text processing, and automation for Linux administration
You are an expert in Bash shell scripting for automating Linux system administration tasks. You write scripts that are strict by default (`set -euo pipefail`), handle errors explicitly, quote all variables, and use shellcheck for validation before deployment.
## Key Points
- Always use `set -euo pipefail` at the top of scripts. This catches the vast majority of silent failures.
- Quote all variable expansions (`"$var"`, `"${array[@]}"`) to prevent word splitting and globbing surprises.
- Use `[[ ]]` instead of `[ ]` — it handles empty strings safely and supports regex.
- Use `mktemp` for temporary files and always clean up with a `trap EXIT`.
- Prefer `printf` over `echo` for portable, predictable output (especially with variables that might start with `-`).
- Use `shellcheck` to lint every script before deployment. It catches subtle bugs that are easy to miss.
- Write functions for any logic used more than once; use `local` for all function variables.
- Log to stderr (`>&2`) so stdout remains clean for piping data.
- **Unquoted variables** — `rm -rf $DIR/` with an empty `DIR` becomes `rm -rf /`. Always quote.
- **Parsing `ls` output** — `ls` output is not reliably parseable. Use `find` with `-print0` or shell globs instead.
- **Using `cat` unnecessarily** — `cat file | grep pattern` should be `grep pattern file`. Useless use of cat wastes a process.
- **Not handling filenames with spaces** — Use `"$file"` not `$file`; use `-print0`/`-d ''` with find loops.
## Quick Example
```bash
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
```skilldb get linux-admin-skills/Shell ScriptingFull skill: 299 linesShell Scripting — Linux Administration
You are an expert in Bash shell scripting for automating Linux system administration tasks. You write scripts that are strict by default (set -euo pipefail), handle errors explicitly, quote all variables, and use shellcheck for validation before deployment.
Core Philosophy
Shell scripts are the glue of Linux administration — they connect tools, automate repetitive tasks, and encode operational procedures. But Bash is also one of the most error-prone languages in common use: unquoted variables cause word splitting, unset variables silently expand to empty strings, failed commands in pipelines are ignored, and subtle syntax differences between [ and [[ cause hard-to-debug failures. The antidote is strict mode and defensive coding. Every production script starts with set -euo pipefail and treats shellcheck warnings as errors. This eliminates the largest class of shell scripting bugs before they reach production.
Scripts should be written for the person who will maintain them at 3 AM during an outage — and that person may be you, six months from now, with no memory of why the script works the way it does. This means clear variable names, consistent formatting, usage functions for argument handling, explicit error messages that name the file and operation that failed, and trap-based cleanup so temporary files are removed even on unexpected exits. A script that "works" but is unreadable is a liability.
Know when Bash is the wrong tool. Bash excels at orchestrating commands, processing line-oriented text, and automating sequential operations. It struggles with complex data structures, concurrent operations, error handling across deeply nested function calls, and anything requiring JSON or YAML parsing beyond trivial cases. When a script exceeds 200 lines, needs associative arrays of arrays, or requires robust error handling with rollback, it is time to reach for Python or Go. The best Bash scripts are short, focused, and do one job well.
Anti-Patterns
- Omitting
set -euo pipefail— Without strict mode, failed commands are silently ignored, unset variables expand to empty strings (potentially causingrm -rf /with an empty variable), and pipeline failures are hidden. This is the root cause of most shell script bugs in production. - Unquoted variable expansions —
$varwithout quotes undergoes word splitting and pathname expansion. A filename with spaces becomes multiple arguments; a variable containing*expands to every file in the current directory. Always use"$var"and"${array[@]}". - Parsing ls output —
lsoutput is not machine-parseable: filenames with spaces, newlines, or special characters break any script that processeslsoutput line by line. Usefindwith-print0andread -d '', or shell globs (for f in *.log) instead. - No cleanup trap for temporary files — Scripts that create temporary files with
mktempbut do not set atrap EXITto remove them leave debris on every run. Over time, this fills/tmpand causes unrelated failures. Always pairmktempwith a trap-based cleanup function. - Using backticks instead of
$()— Backtick command substitution (`cmd`) cannot be nested, is hard to read, and has inconsistent quoting behavior across shells.$(cmd)is nestable, clearer, and POSIX-compliant. There is no reason to use backticks in modern scripts.
Overview
Bash is the default shell on most Linux distributions and the primary language for system automation scripts. Effective shell scripting combines built-in Bash features with standard Unix utilities to build reliable, maintainable automation. It is the glue language of Linux administration — connecting tools, processing text, orchestrating services, and handling operational tasks.
Core Concepts
Script Structure and Safety
Every production script should start with strict mode settings:
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
| Flag | Effect |
|---|---|
set -e | Exit immediately on non-zero return |
set -u | Treat unset variables as errors |
set -o pipefail | Pipe fails if any command in the chain fails |
Variables and Parameter Expansion
# Assignment (no spaces around =)
name="backup"
readonly CONFIG_DIR="/etc/myapp"
# Parameter expansion
echo "${name}" # Basic
echo "${name:-default}" # Default if unset/empty
echo "${name:=default}" # Assign default if unset/empty
echo "${name:+alternative}" # Alternative if set
echo "${name:?'not set'}" # Error if unset/empty
# String operations
echo "${filename%.tar.gz}" # Remove suffix
echo "${filepath##*/}" # Remove longest prefix (basename)
echo "${filepath%/*}" # Remove shortest suffix (dirname)
echo "${var^^}" # Uppercase
echo "${var,,}" # Lowercase
echo "${var/old/new}" # Replace first match
echo "${var//old/new}" # Replace all matches
echo "${#var}" # String length
Arrays
# Indexed arrays
files=("one.txt" "two.txt" "three.txt")
files+=("four.txt")
echo "${files[0]}" # First element
echo "${files[@]}" # All elements
echo "${#files[@]}" # Length
# Associative arrays (Bash 4+)
declare -A config
config[host]="db.example.com"
config[port]="5432"
for key in "${!config[@]}"; do
echo "$key = ${config[$key]}"
done
Implementation Patterns
Argument Parsing
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: $(basename "$0") [OPTIONS] <target>
Options:
-v, --verbose Enable verbose output
-d, --dry-run Show what would be done
-c, --config FILE Path to config file
-h, --help Show this help
EOF
exit "${1:-0}"
}
VERBOSE=false
DRY_RUN=false
CONFIG_FILE="/etc/myapp/default.conf"
while [[ $# -gt 0 ]]; do
case "$1" in
-v|--verbose) VERBOSE=true; shift ;;
-d|--dry-run) DRY_RUN=true; shift ;;
-c|--config) CONFIG_FILE="$2"; shift 2 ;;
-h|--help) usage 0 ;;
--) shift; break ;;
-*) echo "Unknown option: $1" >&2; usage 1 ;;
*) break ;;
esac
done
TARGET="${1:?'Error: target argument required'}"
Error Handling and Cleanup
#!/usr/bin/env bash
set -euo pipefail
TMPDIR=""
cleanup() {
local exit_code=$?
if [[ -n "$TMPDIR" && -d "$TMPDIR" ]]; then
rm -rf "$TMPDIR"
fi
exit "$exit_code"
}
trap cleanup EXIT
die() {
echo "ERROR: $*" >&2
exit 1
}
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"
}
TMPDIR=$(mktemp -d) || die "Failed to create temp directory"
log "Working in $TMPDIR"
Text Processing Pipelines
# Extract and summarize failed SSH logins
journalctl -u sshd --since "24 hours ago" --no-pager \
| grep "Failed password" \
| awk '{print $(NF-3)}' \
| sort | uniq -c | sort -rn \
| head -20
# Parse CSV and extract fields
while IFS=',' read -r name email role; do
[[ "$role" == "admin" ]] && echo "$name ($email)"
done < users.csv
# Find large files, format output
find /var/log -type f -size +100M -printf '%s %p\n' 2>/dev/null \
| sort -rn \
| numfmt --to=iec --field=1 \
| head -10
Conditional Patterns
# File tests
[[ -f "$file" ]] # Regular file exists
[[ -d "$dir" ]] # Directory exists
[[ -r "$file" ]] # Readable
[[ -w "$file" ]] # Writable
[[ -x "$file" ]] # Executable
[[ -s "$file" ]] # Non-empty
[[ "$f1" -nt "$f2" ]] # f1 newer than f2
# String tests
[[ -z "$var" ]] # Empty string
[[ -n "$var" ]] # Non-empty string
[[ "$a" == "$b" ]] # String equality
[[ "$a" =~ ^[0-9]+$ ]] # Regex match
# Numeric tests
(( count > 10 ))
(( count >= min && count <= max ))
Looping Patterns
# Process files safely (handle spaces, special chars)
while IFS= read -r -d '' file; do
echo "Processing: $file"
done < <(find /data -name "*.log" -print0)
# Retry loop with exponential backoff
max_retries=5
for (( i=1; i<=max_retries; i++ )); do
if curl -sf "http://service/health" > /dev/null; then
echo "Service is up"
break
fi
if (( i == max_retries )); then
die "Service failed to come up after $max_retries attempts"
fi
sleep $(( 2 ** i ))
done
# Parallel execution with controlled concurrency
max_jobs=4
for host in "${hosts[@]}"; do
while (( $(jobs -r | wc -l) >= max_jobs )); do
sleep 0.5
done
ssh "$host" 'uptime' &
done
wait
Functions
# Function with local variables and return values
get_disk_usage() {
local mount_point="${1:?'mount_point required'}"
local usage
usage=$(df --output=pcent "$mount_point" | tail -1 | tr -d ' %')
echo "$usage"
}
check_disk() {
local mount="$1"
local threshold="${2:-90}"
local usage
usage=$(get_disk_usage "$mount")
if (( usage > threshold )); then
log "WARNING: $mount at ${usage}% (threshold: ${threshold}%)"
return 1
fi
return 0
}
Here Documents and Process Substitution
# Here document for multi-line input
mysql -u admin -p"$DB_PASS" <<SQL
SELECT user, host FROM mysql.user WHERE account_locked = 'N';
SQL
# Process substitution to compare outputs
diff <(ssh host1 'rpm -qa | sort') <(ssh host2 'rpm -qa | sort')
# Here string
grep "error" <<< "$log_output"
Best Practices
- Always use
set -euo pipefailat the top of scripts. This catches the vast majority of silent failures. - Quote all variable expansions (
"$var","${array[@]}") to prevent word splitting and globbing surprises. - Use
[[ ]]instead of[ ]— it handles empty strings safely and supports regex. - Use
mktempfor temporary files and always clean up with atrap EXIT. - Prefer
printfoverechofor portable, predictable output (especially with variables that might start with-). - Use
shellcheckto lint every script before deployment. It catches subtle bugs that are easy to miss. - Write functions for any logic used more than once; use
localfor all function variables. - Log to stderr (
>&2) so stdout remains clean for piping data.
Common Pitfalls
- Unquoted variables —
rm -rf $DIR/with an emptyDIRbecomesrm -rf /. Always quote. - Parsing
lsoutput —lsoutput is not reliably parseable. Usefindwith-print0or shell globs instead. - Using
catunnecessarily —cat file | grep patternshould begrep pattern file. Useless use of cat wastes a process. - Not handling filenames with spaces — Use
"$file"not$file; use-print0/-d ''with find loops. - Assuming GNU tools — On minimal or Alpine containers, tools may be BusyBox. Stick to POSIX when portability matters.
- Ignoring exit codes in conditionals —
set -edoes not trigger insideifconditions or||/&&chains. Check critical commands explicitly. - Modifying IFS globally — Set IFS locally within
readcommands (IFS=',' read -r ...) rather than changing it for the whole script.
Install this skill directly: skilldb add linux-admin-skills
Related Skills
Disk Management
Disk partitioning, filesystems, LVM, RAID, mount management, and storage monitoring on Linux
File Permissions
Linux file permissions, ownership, special bits, ACLs, and file attribute management
Log Management
Log management with journalctl, rsyslog, logrotate, and centralized logging strategies on Linux
Networking Tools
Linux networking tools including ss, ip, iptables, nftables, and diagnostic utilities
Process Management
Process lifecycle, monitoring, signals, cgroups, and performance analysis on Linux systems
Systemd
Systemd service units, timers, targets, and dependency management for Linux init systems