Introduction
# Cron & Scheduling
Schedule and manage recurring tasks. Covers cron syntax, crontab management, systemd timers, one-off scheduling, timezone handling, monitoring, and common failure patterns.
## When to Use
- Running scripts on a schedule (backups, reports, cleanup) - Setting up systemd timers (modern cron alternative) - Debugging why a scheduled job didn't run - Handling timezones in scheduled tasks - Monitoring and alerting on job failures - Running one-off delayed commands
## Cron Syntax
### The five fields
``` ┌───────── minute (0-59) │ ┌─────── hour (0-23) │ │ ┌───── day of month (1-31) │ │ │ ┌─── month (1-12 or JAN-DEC) │ │ │ │ ┌─ day of week (0-7, 0 and 7 = Sunday, or SUN-SAT) │ │ │ │ │ * * * * * command ```
### Common schedules
```bash # Every minute * * * * * /path/to/script.sh
# Every 5 minutes */5 * * * * /path/to/script.sh
# Every hour at :00 0 * * * * /path/to/script.sh
# Every day at 2:30 AM 30 2 * * * /path/to/script.sh
# Every Monday at 9:00 AM 0 9 * * 1 /path/to/script.sh
# Every weekday at 8:00 AM 0 8 * * 1-5 /path/to/script.sh
# First day of every month at midnight 0 0 1 * * /path/to/script.sh
# Every 15 minutes during business hours (Mon-Fri 9-17) */15 9-17 * * 1-5 /path/to/script.sh
# Twice a day (9 AM and 5 PM) 0 9,17 * * * /path/to/script.sh
# Every quarter (Jan, Apr, Jul, Oct) on the 1st at midnight 0 0 1 1,4,7,10 * /path/to/script.sh
# Every Sunday at 3 AM 0 3 * * 0 /path/to/script.sh ```
### Special strings (shorthand)
```bash @reboot /path/to/script.sh # Run once at startup @yearly /path/to/script.sh # 0 0 1 1 * @monthly /path/to/script.sh # 0 0 1 * * @weekly /path/to/script.sh # 0 0 * * 0 @daily /path/to/script.sh # 0 0 * * * @hourly /path/to/script.sh # 0 * * * * ```
## Crontab Management
```bash # Edit current user's crontab crontab -e
# List current crontab crontab -l
# Edit another user's crontab (root) sudo crontab -u www-data -e
# Remove all cron jobs (be careful!) crontab -r
# Install crontab from file crontab mycrontab.txt
# Backup crontab crontab -l > crontab-backup-$(date +%Y%m%d).txt ```
### Crontab best practices
```bash # Set PATH explicitly (cron has minimal PATH) PATH=/usr/local/bin:/usr/bin:/bin
# Set MAILTO for error notifications [email protected]
# Set shell explicitly SHELL=/bin/bash
# Full crontab example PATH=/usr/local/bin:/usr/bin:/bin [email protected] SHELL=/bin/bash
# Backups 0 2 * * * /opt/scripts/backup.sh >> /var/log/backup.log 2>&1
# Cleanup old logs 0 3 * * 0 find /var/log/myapp -name "*.log" -mtime +30 -delete
# Health check */5 * * * * /opt/scripts/healthcheck.sh || /opt/scripts/alert.sh "Health check failed" ```
## Systemd Timers
### Create a timer (modern cron replacement)
```ini # /etc/systemd/system/backup.service [Unit] Description=Daily backup
[Service] Type=oneshot ExecStart=/opt/scripts/backup.sh User=backup StandardOutput=journal StandardError=journal ```
```ini # /etc/systemd/system/backup.timer [Unit] Description=Run backup daily at 2 AM
[Timer] OnCalendar=*-*-* 02:00:00 Persistent=true RandomizedDelaySec=300
[Install] WantedBy=timers.target ```
```bash # Enable and start the timer sudo systemctl daemon-reload sudo systemctl enable --now backup.timer
# Check timer status systemctl list-timers systemctl list-timers --all
# Check last run systemctl status backup.service journalctl -u backup.service --since today
# Run manually (for testing) sudo systemctl start backup.service
# Disable timer sudo systemctl disable --now backup.timer ```
### OnCalendar syntax
```ini # Systemd calendar expressions
# Daily at midnight OnCalendar=daily # or: OnCalendar=*-*-* 00:00:00
# Every Monday at 9 AM OnCalendar=Mon *-*-* 09:00:00
# Every 15 minutes OnCalendar=*:0/15
# Weekdays at 8 AM OnCalendar=Mon..Fri *-*-* 08:00:00
# First of every month OnCalendar=*-*-01 00:00:00
# Every 6 hours OnCalendar=0/6:00:00
# Specific dates OnCalendar=2026-02-03 12:00:00
# Test calendar expressions systemd-analyze calendar "Mon *-*-* 09:00:00" systemd-analyze calendar "*:0/15" systemd-analyze calendar --iterations=5 "Mon..Fri *-*-* 08:00:00" ```
### Advantages over cron
``` Systemd timers vs cron: + Logs in journald (journalctl -u service-name) + Persistent: catches up on missed runs after reboot + RandomizedDelaySec: prevents thundering herd + Dependencies: can depend on network, mounts, etc. + Resource limits: CPUQuota, MemoryMax, etc. + No lost-email problem (MAILTO often misconfigured) - More files to create (service + timer) - More verbose configuration ```
## One-Off Scheduling
### at (run once at a specific time)
```bash # Schedule a command echo "/opt/scripts/deploy.sh" | at 2:00 AM tomorrow echo "reboot" | at now + 30 minutes echo "/opt/scripts/report.sh" | at 5:00 PM Friday
# Interactive (type commands, Ctrl+D to finish) at 10:00 AM > /opt/scripts/task.sh > echo "Done" | mail -s "Task complete" [email protected] > <Ctrl+D>
# List pending jobs atq
# View job details at -c <job-number>
# Remove a job atrm <job-number> ```
### sleep-based (simplest)
```bash # Run something after a delay (sleep 3600 && /opt/scripts/task.sh) &
# With nohup (survives logout) nohup bash -c "sleep 7200 && /opt/scripts/task.sh" & ```
## Timezone Handling
```bash # Cron runs in the system timezone by default # Check system timezone timedatectl date +%Z
# Set timezone for a specific cron job # Method 1: TZ variable in crontab TZ=America/New_York 0 9 * * * /opt/scripts/report.sh
# Method 2: In the script itself #!/bin/bash export TZ=UTC # All date operations now use UTC
# Method 3: Wrapper TZ=Europe/London date '+%Y-%m-%d %H:%M:%S'
# List available timezones timedatectl list-timezones timedatectl list-timezones | grep America ```
### DST pitfalls
``` Problem: A job scheduled for 2:30 AM may run twice or not at all during DST transitions.
"Spring forward": 2:30 AM doesn't exist (clock jumps 2:00 → 3:00) "Fall back": 2:30 AM happens twice
Mitigation: 1. Schedule critical jobs outside 1:00-3:00 AM 2. Use UTC for the schedule: TZ=UTC in crontab 3. Make jobs idempotent (safe to run twice) 4. Systemd timers handle DST correctly ```
## Monitoring and Debugging
### Why didn't my cron job run?
```bash # 1. Check cron daemon is running systemctl status cron # Debian/Ubuntu systemctl status crond # CentOS/RHEL
# 2. Check cron logs grep CRON /var/log/syslog # Debian/Ubuntu grep CRON /var/log/cron # CentOS/RHEL journalctl -u cron --since today # systemd
# 3. Check crontab actually exists crontab -l
# 4. Test the command manually (with cron's environment) env -i HOME=$HOME SHELL=/bin/sh PATH=/usr/bin:/bin /opt/scripts/backup.sh # If it fails here but works normally → PATH or env issue
# 5. Check permissions ls -la /opt/scripts/backup.sh # Must be executable ls -la /var/spool/cron/ # Crontab file permissions
# 6. Check for syntax errors in crontab # cron silently ignores lines with errors
# 7. Check if output is being discarded # By default, cron emails output. If no MTA, output is lost. # Always redirect: >> /var/log/myjob.log 2>&1 ```
### Job wrapper with logging and alerting
```bash #!/bin/bash # cron-wrapper.sh — Run a command with logging, timing, and error alerting # Usage: cron-wrapper.sh <job-name> <command> [args...]
set -euo pipefail
JOB_NAME="${1:?Usage: cron-wrapper.sh <job-name> <command> [args...]}" shift COMMAND=("$@")
LOG_DIR="/var/log/cron-jobs" mkdir -p "$LOG_DIR" LOG_FILE="$LOG_DIR/$JOB_NAME.log"
log() { echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] $*" >> "$LOG_FILE"; }
log "START: ${COMMAND[*]}" START_TIME=$(date +%s)
if "${COMMAND[@]}" >> "$LOG_FILE" 2>&1; then ELAPSED=$(( $(date +%s) - START_TIME )) log "SUCCESS (${ELAPSED}s)" else EXIT_CODE=$? ELAPSED=$(( $(date +%s) - START_TIME )) log "FAILED with exit code $EXIT_CODE (${ELAPSED}s)" # Alert (customize as needed) echo "Cron job '$JOB_NAME' failed with exit $EXIT_CODE" | \ mail -s "CRON FAIL: $JOB_NAME" [email protected] 2>/dev/null || true exit $EXIT_CODE fi ```
```bash # Use in crontab: 0 2 * * * /opt/scripts/cron-wrapper.sh daily-backup /opt/scripts/backup.sh */5 * * * * /opt/scripts/cron-wrapper.sh health-check /opt/scripts/healthcheck.sh ```
### Lock to prevent overlap
```bash # Prevent concurrent runs (job takes longer than interval) # Method 1: flock * * * * * flock -n /tmp/myjob.lock /opt/scripts/slow-job.sh
# Method 2: In the script LOCKFILE="/tmp/myjob.lock" exec 200>"$LOCKFILE" flock -n 200 || { echo "Already running"; exit 0; } # ... do work ... ```
## Idempotent Job Patterns
```bash # Idempotent backup (only creates if newer than last backup) #!/bin/bash BACKUP_DIR="/backups/$(date +%Y%m%d)" [[ -d "$BACKUP_DIR" ]] && { echo "Backup already exists"; exit 0; } mkdir -p "$BACKUP_DIR" pg_dump mydb > "$BACKUP_DIR/mydb.sql"
# Idempotent cleanup (safe to run multiple times) find /tmp/uploads -mtime +7 -type f -delete 2>/dev/null || true
# Idempotent sync (rsync only transfers changes) rsync -az /data/ backup-server:/backups/data/ ```
## Tips
- Always redirect output in cron jobs: `>> /var/log/job.log 2>&1`. Without this, output goes to mail (if configured) or is silently lost. - Test cron jobs by running them with `env -i` to simulate cron's minimal environment. Most failures are caused by missing `PATH` or environment variables. - Use `flock` to prevent overlapping runs when a job might take longer than its schedule interval. - Make all scheduled jobs idempotent. If a job runs twice (DST, manual trigger, crash recovery), it should produce the same result. - `systemd-analyze calendar` is invaluable for verifying timer schedules before deploying. - Never schedule critical jobs between 1:00 AM and 3:00 AM if DST applies. Use UTC schedules instead. - Log the start time, end time, and exit code of every cron job. Without this, debugging failures after the fact is guesswork. - Prefer systemd timers over cron for production services: you get journald logging, missed-run catchup (`Persistent=true`), and resource limits for free.