Linux

Shell Scripting Basics for Automating Bioinformatics Tasks

Variables, loops, and functions in Bash - enough to stop repeating yourself across sample files.

6 September 20262 min read

linuxbashautomation

Most bioinformatics work involves running the same command across dozens or hundreds of sample files. A few lines of Bash turn that from an afternoon of copy-pasting into a script you run once.

Variables

SAMPLE="patient_042"
REF="/data/reference/GRCh38.fa"
 
echo "Processing $SAMPLE against $REF"

Looping over files

The most common pattern in a bioinformatics context - run the same tool across every sample in a directory:

for fastq in /data/samples/*.fastq.gz; do
    sample=$(basename "$fastq" .fastq.gz)
    echo "Aligning $sample..."
    bwa mem "$REF" "$fastq" > "results/${sample}.sam"
done

basename strips the directory path and extension, so /data/samples/patient_042.fastq.gz becomes patient_042 - useful for naming output files consistently.

Conditionals

if [ ! -f "$REF" ]; then
    echo "Reference file not found: $REF" >&2
    exit 1
fi

Checking that inputs exist before a long pipeline runs saves you from discovering a typo an hour into a job.

Functions

Once a block of logic gets reused, wrap it in a function:

run_qc() {
    local input=$1
    fastqc "$input" -o qc_reports/
}
 
for fastq in /data/samples/*.fastq.gz; do
    run_qc "$fastq"
done

A complete, minimal pipeline script

#!/usr/bin/env bash
set -euo pipefail   # exit on error, undefined variable, or failed pipe
 
REF="/data/reference/GRCh38.fa"
OUTDIR="results"
mkdir -p "$OUTDIR"
 
for fastq in /data/samples/*.fastq.gz; do
    sample=$(basename "$fastq" .fastq.gz)
    echo "==> $sample"
    bwa mem "$REF" "$fastq" | samtools sort -o "$OUTDIR/${sample}.bam"
    samtools index "$OUTDIR/${sample}.bam"
done
 
echo "Done: $(ls "$OUTDIR"/*.bam | wc -l) samples processed."

set -euo pipefail is worth memorising - it makes the script stop immediately on any error instead of silently continuing with bad output, which matters a lot more in a clinical pipeline than in a one-off script.