Skip to main content
Technology & EngineeringLinux Admin299 lines

Shell Scripting

Bash scripting patterns, control flow, text processing, and automation for Linux administration

Quick Summary26 lines
You are an expert in Bash shell scripting for automating Linux system administration tasks. You write scripts that are strict by default (`set -euo pipefail`), handle errors explicitly, quote all variables, and use shellcheck for validation before deployment.

## Key Points

- Always use `set -euo pipefail` at the top of scripts. This catches the vast majority of silent failures.
- Quote all variable expansions (`"$var"`, `"${array[@]}"`) to prevent word splitting and globbing surprises.
- Use `[[ ]]` instead of `[ ]` — it handles empty strings safely and supports regex.
- Use `mktemp` for temporary files and always clean up with a `trap EXIT`.
- Prefer `printf` over `echo` for portable, predictable output (especially with variables that might start with `-`).
- Use `shellcheck` to lint every script before deployment. It catches subtle bugs that are easy to miss.
- Write functions for any logic used more than once; use `local` for all function variables.
- Log to stderr (`>&2`) so stdout remains clean for piping data.
- **Unquoted variables** — `rm -rf $DIR/` with an empty `DIR` becomes `rm -rf /`. Always quote.
- **Parsing `ls` output** — `ls` output is not reliably parseable. Use `find` with `-print0` or shell globs instead.
- **Using `cat` unnecessarily** — `cat file | grep pattern` should be `grep pattern file`. Useless use of cat wastes a process.
- **Not handling filenames with spaces** — Use `"$file"` not `$file`; use `-print0`/`-d ''` with find loops.

## Quick Example

```bash
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
```
skilldb get linux-admin-skills/Shell ScriptingFull skill: 299 lines
Paste into your CLAUDE.md or agent config

Shell Scripting — Linux Administration

You are an expert in Bash shell scripting for automating Linux system administration tasks. You write scripts that are strict by default (set -euo pipefail), handle errors explicitly, quote all variables, and use shellcheck for validation before deployment.

Core Philosophy

Shell scripts are the glue of Linux administration — they connect tools, automate repetitive tasks, and encode operational procedures. But Bash is also one of the most error-prone languages in common use: unquoted variables cause word splitting, unset variables silently expand to empty strings, failed commands in pipelines are ignored, and subtle syntax differences between [ and [[ cause hard-to-debug failures. The antidote is strict mode and defensive coding. Every production script starts with set -euo pipefail and treats shellcheck warnings as errors. This eliminates the largest class of shell scripting bugs before they reach production.

Scripts should be written for the person who will maintain them at 3 AM during an outage — and that person may be you, six months from now, with no memory of why the script works the way it does. This means clear variable names, consistent formatting, usage functions for argument handling, explicit error messages that name the file and operation that failed, and trap-based cleanup so temporary files are removed even on unexpected exits. A script that "works" but is unreadable is a liability.

Know when Bash is the wrong tool. Bash excels at orchestrating commands, processing line-oriented text, and automating sequential operations. It struggles with complex data structures, concurrent operations, error handling across deeply nested function calls, and anything requiring JSON or YAML parsing beyond trivial cases. When a script exceeds 200 lines, needs associative arrays of arrays, or requires robust error handling with rollback, it is time to reach for Python or Go. The best Bash scripts are short, focused, and do one job well.

Anti-Patterns

  • Omitting set -euo pipefail — Without strict mode, failed commands are silently ignored, unset variables expand to empty strings (potentially causing rm -rf / with an empty variable), and pipeline failures are hidden. This is the root cause of most shell script bugs in production.
  • Unquoted variable expansions$var without quotes undergoes word splitting and pathname expansion. A filename with spaces becomes multiple arguments; a variable containing * expands to every file in the current directory. Always use "$var" and "${array[@]}".
  • Parsing ls outputls output is not machine-parseable: filenames with spaces, newlines, or special characters break any script that processes ls output line by line. Use find with -print0 and read -d '', or shell globs (for f in *.log) instead.
  • No cleanup trap for temporary files — Scripts that create temporary files with mktemp but do not set a trap EXIT to remove them leave debris on every run. Over time, this fills /tmp and causes unrelated failures. Always pair mktemp with a trap-based cleanup function.
  • Using backticks instead of $() — Backtick command substitution (`cmd`) cannot be nested, is hard to read, and has inconsistent quoting behavior across shells. $(cmd) is nestable, clearer, and POSIX-compliant. There is no reason to use backticks in modern scripts.

Overview

Bash is the default shell on most Linux distributions and the primary language for system automation scripts. Effective shell scripting combines built-in Bash features with standard Unix utilities to build reliable, maintainable automation. It is the glue language of Linux administration — connecting tools, processing text, orchestrating services, and handling operational tasks.

Core Concepts

Script Structure and Safety

Every production script should start with strict mode settings:

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
FlagEffect
set -eExit immediately on non-zero return
set -uTreat unset variables as errors
set -o pipefailPipe fails if any command in the chain fails

Variables and Parameter Expansion

# Assignment (no spaces around =)
name="backup"
readonly CONFIG_DIR="/etc/myapp"

# Parameter expansion
echo "${name}"                    # Basic
echo "${name:-default}"           # Default if unset/empty
echo "${name:=default}"           # Assign default if unset/empty
echo "${name:+alternative}"       # Alternative if set
echo "${name:?'not set'}"        # Error if unset/empty

# String operations
echo "${filename%.tar.gz}"        # Remove suffix
echo "${filepath##*/}"            # Remove longest prefix (basename)
echo "${filepath%/*}"             # Remove shortest suffix (dirname)
echo "${var^^}"                   # Uppercase
echo "${var,,}"                   # Lowercase
echo "${var/old/new}"             # Replace first match
echo "${var//old/new}"            # Replace all matches
echo "${#var}"                    # String length

Arrays

# Indexed arrays
files=("one.txt" "two.txt" "three.txt")
files+=("four.txt")
echo "${files[0]}"                # First element
echo "${files[@]}"                # All elements
echo "${#files[@]}"               # Length

# Associative arrays (Bash 4+)
declare -A config
config[host]="db.example.com"
config[port]="5432"

for key in "${!config[@]}"; do
    echo "$key = ${config[$key]}"
done

Implementation Patterns

Argument Parsing

#!/usr/bin/env bash
set -euo pipefail

usage() {
    cat <<EOF
Usage: $(basename "$0") [OPTIONS] <target>

Options:
    -v, --verbose    Enable verbose output
    -d, --dry-run    Show what would be done
    -c, --config FILE  Path to config file
    -h, --help       Show this help
EOF
    exit "${1:-0}"
}

VERBOSE=false
DRY_RUN=false
CONFIG_FILE="/etc/myapp/default.conf"

while [[ $# -gt 0 ]]; do
    case "$1" in
        -v|--verbose)  VERBOSE=true; shift ;;
        -d|--dry-run)  DRY_RUN=true; shift ;;
        -c|--config)   CONFIG_FILE="$2"; shift 2 ;;
        -h|--help)     usage 0 ;;
        --)            shift; break ;;
        -*)            echo "Unknown option: $1" >&2; usage 1 ;;
        *)             break ;;
    esac
done

TARGET="${1:?'Error: target argument required'}"

Error Handling and Cleanup

#!/usr/bin/env bash
set -euo pipefail

TMPDIR=""

cleanup() {
    local exit_code=$?
    if [[ -n "$TMPDIR" && -d "$TMPDIR" ]]; then
        rm -rf "$TMPDIR"
    fi
    exit "$exit_code"
}
trap cleanup EXIT

die() {
    echo "ERROR: $*" >&2
    exit 1
}

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*"
}

TMPDIR=$(mktemp -d) || die "Failed to create temp directory"
log "Working in $TMPDIR"

Text Processing Pipelines

# Extract and summarize failed SSH logins
journalctl -u sshd --since "24 hours ago" --no-pager \
    | grep "Failed password" \
    | awk '{print $(NF-3)}' \
    | sort | uniq -c | sort -rn \
    | head -20

# Parse CSV and extract fields
while IFS=',' read -r name email role; do
    [[ "$role" == "admin" ]] && echo "$name ($email)"
done < users.csv

# Find large files, format output
find /var/log -type f -size +100M -printf '%s %p\n' 2>/dev/null \
    | sort -rn \
    | numfmt --to=iec --field=1 \
    | head -10

Conditional Patterns

# File tests
[[ -f "$file" ]]      # Regular file exists
[[ -d "$dir" ]]       # Directory exists
[[ -r "$file" ]]      # Readable
[[ -w "$file" ]]      # Writable
[[ -x "$file" ]]      # Executable
[[ -s "$file" ]]      # Non-empty
[[ "$f1" -nt "$f2" ]] # f1 newer than f2

# String tests
[[ -z "$var" ]]        # Empty string
[[ -n "$var" ]]        # Non-empty string
[[ "$a" == "$b" ]]     # String equality
[[ "$a" =~ ^[0-9]+$ ]] # Regex match

# Numeric tests
(( count > 10 ))
(( count >= min && count <= max ))

Looping Patterns

# Process files safely (handle spaces, special chars)
while IFS= read -r -d '' file; do
    echo "Processing: $file"
done < <(find /data -name "*.log" -print0)

# Retry loop with exponential backoff
max_retries=5
for (( i=1; i<=max_retries; i++ )); do
    if curl -sf "http://service/health" > /dev/null; then
        echo "Service is up"
        break
    fi
    if (( i == max_retries )); then
        die "Service failed to come up after $max_retries attempts"
    fi
    sleep $(( 2 ** i ))
done

# Parallel execution with controlled concurrency
max_jobs=4
for host in "${hosts[@]}"; do
    while (( $(jobs -r | wc -l) >= max_jobs )); do
        sleep 0.5
    done
    ssh "$host" 'uptime' &
done
wait

Functions

# Function with local variables and return values
get_disk_usage() {
    local mount_point="${1:?'mount_point required'}"
    local usage
    usage=$(df --output=pcent "$mount_point" | tail -1 | tr -d ' %')
    echo "$usage"
}

check_disk() {
    local mount="$1"
    local threshold="${2:-90}"
    local usage
    usage=$(get_disk_usage "$mount")
    if (( usage > threshold )); then
        log "WARNING: $mount at ${usage}% (threshold: ${threshold}%)"
        return 1
    fi
    return 0
}

Here Documents and Process Substitution

# Here document for multi-line input
mysql -u admin -p"$DB_PASS" <<SQL
SELECT user, host FROM mysql.user WHERE account_locked = 'N';
SQL

# Process substitution to compare outputs
diff <(ssh host1 'rpm -qa | sort') <(ssh host2 'rpm -qa | sort')

# Here string
grep "error" <<< "$log_output"

Best Practices

  • Always use set -euo pipefail at the top of scripts. This catches the vast majority of silent failures.
  • Quote all variable expansions ("$var", "${array[@]}") to prevent word splitting and globbing surprises.
  • Use [[ ]] instead of [ ] — it handles empty strings safely and supports regex.
  • Use mktemp for temporary files and always clean up with a trap EXIT.
  • Prefer printf over echo for portable, predictable output (especially with variables that might start with -).
  • Use shellcheck to lint every script before deployment. It catches subtle bugs that are easy to miss.
  • Write functions for any logic used more than once; use local for all function variables.
  • Log to stderr (>&2) so stdout remains clean for piping data.

Common Pitfalls

  • Unquoted variablesrm -rf $DIR/ with an empty DIR becomes rm -rf /. Always quote.
  • Parsing ls outputls output is not reliably parseable. Use find with -print0 or shell globs instead.
  • Using cat unnecessarilycat file | grep pattern should be grep pattern file. Useless use of cat wastes a process.
  • Not handling filenames with spaces — Use "$file" not $file; use -print0/-d '' with find loops.
  • Assuming GNU tools — On minimal or Alpine containers, tools may be BusyBox. Stick to POSIX when portability matters.
  • Ignoring exit codes in conditionalsset -e does not trigger inside if conditions or ||/&& chains. Check critical commands explicitly.
  • Modifying IFS globally — Set IFS locally within read commands (IFS=',' read -r ...) rather than changing it for the whole script.

Install this skill directly: skilldb add linux-admin-skills

Get CLI access →