Automated Docker Compose deployment script with health checks, logging, and update detection for production environments

Docker Compose Deployment Checker

Automated deployment script for Docker Compose environments with intelligent update detection, comprehensive logging, health monitoring, and rollback capabilities. Perfect for production environments requiring reliable automated deployments.

Overview

This script provides enterprise-grade deployment automation for Docker Compose applications, including change detection, health monitoring, and comprehensive logging for production environments.

What is the Deployment Checker?

The Docker Compose Deployment Checker is a comprehensive automation script that:

Monitors for changes: Detects updates in registry images and compose file modifications
Automated deployments: Performs safe deployments with health checks
Comprehensive logging: Maintains detailed logs with automatic rotation
Health monitoring: Verifies service health after deployments
Rollback support: Maintains backup states for quick recovery
Cron integration: Runs automatically via scheduled tasks

Key Features

Change Detection

Health Monitoring

Logging & Monitoring

Safety Features

Prerequisites

System Requirements

Docker Engine 20.10 or later
Docker Compose v2.0 or later
Bash 4.0 or later
Basic utilities: md5sum, stat, jq (optional but recommended)
File system access for logging and state management

Environment Setup

Ensure the script has write permissions to its directory for logging and state management.

Required directory structure:

Script directory: Contains the main script and docker-compose.yml
Logs directory: Stores deployment and activity logs
State directory: Maintains deployment state and backups

Installation and Setup

Step 1: Download and Setup Script

Download and set up the deployment script:

# Download the script
wget -O docker-compose-deployer.sh https://github.com/j551n-ncloud/docker-compose-deployer.sh

# Make executable
chmod +x docker-compose-deployer.sh

# Ensure docker-compose.yml is in the same directory
ls -la docker-compose.yml docker-compose-deployer.sh

Directory structure:

/path/to/your/project/
├── docker-compose.yml
├── docker-compose-deployer.sh
├── logs/                    # Created automatically
└── .deployment_state/       # Created automatically

Manual setup with custom configuration:

#!/bin/bash
# Create project structure
mkdir -p /opt/docker-app/{logs,.deployment_state}
cd /opt/docker-app

# Copy your docker-compose.yml
cp /path/to/docker-compose.yml .

# Create the deployment script
cat > docker-compose-deployer.sh << 'EOF'
[Script content here]
EOF

# Set permissions
chmod +x docker-compose-deployer.sh
chown -R docker-user:docker-group /opt/docker-app

Set up automated cron execution:

# Edit crontab for automated deployments
crontab -e

# Add entry for daily deployment checks at 2 AM
0 2 * * * /path/to/docker-compose-deployer.sh >> /var/log/docker-deployment-cron.log 2>&1

# Or every 6 hours
0 */6 * * * /path/to/docker-compose-deployer.sh

# For testing, run every 15 minutes
*/15 * * * * /path/to/docker-compose-deployer.sh

Cron best practices:

Use absolute paths to scripts and docker-compose.yml
Redirect output to log files
Set appropriate environment variables
Test cron execution manually first

Cron Environment Setup

Ensure proper environment variables for cron execution:

# Edit crontab with environment setup
crontab -e

# Set required environment variables at top of crontab
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DOCKER_HOST=unix:///var/run/docker.sock
HOME=/home/docker-deploy
SHELL=/bin/bash

# Docker Compose specific variables
COMPOSE_PROJECT_NAME=myapp
COMPOSE_FILE=/opt/docker-app/docker-compose.yml
DOCKER_CONFIG=/home/docker-deploy/.docker

# Application-specific variables
APP_ENV=production
LOG_LEVEL=INFO

# Cron job with proper directory context
0 2 * * * cd /opt/docker-app && ./docker-compose-deployer.sh

Important environment considerations:

PATH: Must include docker and docker-compose binaries
HOME: Required for docker configuration and credentials
DOCKER_HOST: Ensure docker socket access
Working directory: Always cd to script directory

Configure comprehensive logging for cron execution:

# Cron entries with different logging strategies
crontab -e

# Basic logging to single file
0 2 * * * cd /opt/docker-app && ./docker-compose-deployer.sh >> /var/log/docker-deployment.log 2>&1

# Separate error logging
0 2 * * * cd /opt/docker-app && ./docker-compose-deployer.sh >> /var/log/docker-deployment.log 2>> /var/log/docker-deployment-errors.log

# Date-based log rotation
0 2 * * * cd /opt/docker-app && ./docker-compose-deployer.sh >> /var/log/docker-deployment-$(date +\%Y\%m\%d).log 2>&1

# Syslog integration
0 2 * * * cd /opt/docker-app && ./docker-compose-deployer.sh 2>&1 | logger -t "docker-deployment"

# Email on failure only
0 2 * * * cd /opt/docker-app && ./docker-compose-deployer.sh || echo "Deployment failed" | mail -s "Alert: Docker Deployment Failed" [email protected]

Logging best practices:

Always redirect both stdout and stderr (2>&1)
Use dated log files for easier management
Consider log rotation to prevent disk space issues
Set up alerts for failures

Secure cron execution setup:

# Create dedicated user for deployments
sudo useradd -r -s /bin/bash -m docker-deploy
sudo usermod -aG docker docker-deploy

# Set up user crontab (not root)
sudo -u docker-deploy crontab -e

# Secure file permissions
sudo chmod 700 /opt/docker-app/docker-compose-deployer.sh
sudo chown docker-deploy:docker-deploy /opt/docker-app/docker-compose-deployer.sh

# Restrict cron file access
sudo chmod 600 /var/spool/cron/crontabs/docker-deploy

# Set up log directory with proper permissions
sudo mkdir -p /var/log/docker-deployment
sudo chown docker-deploy:docker-deploy /var/log/docker-deployment
sudo chmod 750 /var/log/docker-deployment

Security considerations:

Never run as root unless absolutely necessary
Use dedicated user account with minimal privileges
Secure log files and script permissions
Audit cron access regularly

Script Usage

Basic Operations

Run the script in automatic mode (default behavior):

# Run deployment check
./docker-compose-deployer.sh

# Check if deployment is needed
# Script will automatically:
# 1. Check for compose file changes
# 2. Check for registry image updates
# 3. Deploy if changes detected
# 4. Perform health checks
# 5. Log all activities

Automatic mode behavior:

Detects changes in docker-compose.yml file
Pulls latest images and compares with current
Deploys only if updates are available
Performs comprehensive health checks
Logs all activities and results

Run interactive deployment with status monitoring:

# Interactive run with detailed output
./docker-compose-deployer.sh

# Monitor logs in real-time during deployment
tail -f logs/deployment.log

# Check deployment history
cat logs/deployment_history.log

# View current container status
docker compose ps

Force deployment regardless of detected changes:

# Force deployment even if no changes detected
./docker-compose-deployer.sh --force

# Force deployment with custom reason logging
# (Useful for manual maintenance deployments)

When to use force deployment:

Manual configuration changes not detected by script
Recovery from failed deployments
Testing deployment process
Applying external configuration updates

Monitoring and Logging

Monitor deployment activities and troubleshoot issues:

# View recent deployment logs
tail -f logs/deployment.log

# Search for errors in logs
grep -i error logs/deployment.log

# View deployment summary
grep -A 5 -B 5 "Deployment Summary" logs/deployment.log

# Check log file sizes and rotation
ls -lh logs/

# View specific deployment attempt
grep "2024-01-15 02:00" logs/deployment.log

Log format:

[TIMESTAMP] [LEVEL] MESSAGE
Color-coded output for interactive sessions
Automatic rotation at 10MB with 10-file retention
Separate deployment history tracking

Monitor service health and deployment success:

# Check current service health
docker compose ps

# View health check logs
grep "health check" logs/deployment.log

# Monitor specific service health
docker compose ps [SERVICE_NAME]

# Check service logs after deployment
docker compose logs [SERVICE_NAME]

# View unhealthy services from logs
grep "unhealthy" logs/deployment.log

Health check features:

Configurable timeout (default: 2 minutes)
Detection of unhealthy and starting services
Automatic retry with exponential backoff
Integration with Docker health checks

Track deployment history and analyze patterns:

# View deployment history
cat logs/deployment_history.log

# Recent deployments
tail -20 logs/deployment_history.log

# Count successful deployments
grep "SUCCESS" logs/deployment_history.log | wc -l

# Find failed deployments
grep "FAILED" logs/deployment_history.log

# Deployment frequency analysis
awk '{print $1, $2}' logs/deployment_history.log | sort | uniq -c

History tracking includes:

Deployment timestamp
Success/failure status
Deployment reason (file change, registry update, forced)
Quick statistics and analysis

Advanced Configuration

Change Detection Customization

Customize docker-compose.yml change detection:

# Modify check_compose_file_changes() function
check_compose_file_changes() {
    local compose_hash_file="$STATE_DIR/compose_hash"
    
    # Include additional files in change detection
    local additional_files=(
        "$SCRIPT_DIR/.env"
        "$SCRIPT_DIR/config.yml"
        "$SCRIPT_DIR/nginx.conf"
    )
    
    local combined_hash=""
    for file in "${additional_files[@]}"; do
        if [[ -f "$file" ]]; then
            combined_hash+=$(md5sum "$file" | cut -d' ' -f1)
        fi
    done
    
    local current_hash=$(echo "$combined_hash" | md5sum | cut -d' ' -f1)
    # [Rest of function logic]
}

Enhanced monitoring:

Monitor multiple configuration files
Include environment files and configs
Custom hash calculations
Selective file monitoring

Configure registry update detection behavior:

# Customize check_registry_updates() function
check_registry_updates() {
    local registries_to_check=(
        "docker.io"
        "your-private-registry.com"
        "ghcr.io"
    )
    
    # Add authentication for private registries
    docker login your-private-registry.com
    
    # Custom update detection logic
    local services=$(docker compose config --services)
    for service in $services; do
        # Add custom logic for specific services
        case $service in
            "web")
                check_web_service_updates "$service"
                ;;
            "database")
                check_database_updates "$service"
                ;;
            *)
                check_standard_updates "$service"
                ;;
        esac
    done
}

Implement custom deployment triggers:

# Add custom trigger function
check_custom_triggers() {
    local needs_deployment=false
    
    # Check for webhook file
    if [[ -f "$STATE_DIR/deploy_webhook" ]]; then
        log INFO "Webhook deployment trigger found"
        rm -f "$STATE_DIR/deploy_webhook"
        needs_deployment=true
    fi
    
    # Check for time-based deployments
    local last_deploy=$(stat -c %Y "$STATE_DIR/last_deploy" 2>/dev/null || echo 0)
    local current_time=$(date +%s)
    local time_diff=$((current_time - last_deploy))
    
    # Force deployment every 7 days
    if [[ $time_diff -gt 604800 ]]; then
        log INFO "Weekly deployment trigger activated"
        needs_deployment=true
    fi
    
    echo $needs_deployment
}

Health Check Configuration

Configure health checks for different service types:

# Enhanced health_check() function
health_check() {
    log INFO "Performing comprehensive health check..."
    
    local services=$(docker compose ps --services)
    
    for service in $services; do
        case $service in
            "web"|"api")
                check_http_service_health "$service"
                ;;
            "database"|"postgres"|"mysql")
                check_database_health "$service"
                ;;
            "redis"|"cache")
                check_cache_health "$service"
                ;;
            *)
                check_generic_health "$service"
                ;;
        esac
    done
}

# HTTP service health check
check_http_service_health() {
    local service=$1
    local port=$(docker compose port "$service" 80 2>/dev/null | cut -d: -f2)
    
    if [[ -n "$port" ]]; then
        if curl -f "http://localhost:$port/health" >/dev/null 2>&1; then
            log SUCCESS "HTTP health check passed for $service"
            return 0
        else
            log ERROR "HTTP health check failed for $service"
            return 1
        fi
    fi
}

Implement custom health check logic:

# Custom application-specific health checks
custom_application_health() {
    local app_name=$1
    
    case $app_name in
        "my-web-app")
            # Check database connectivity
            docker compose exec -T database pg_isready >/dev/null 2>&1 || return 1
            
            # Check API endpoint
            local api_response=$(curl -s "http://localhost:3000/api/health")
            echo "$api_response" | grep -q "healthy" || return 1
            
            # Check log for errors
            docker compose logs --tail=50 web | grep -i error && return 1
            ;;
        "monitoring-stack")
            # Check Prometheus
            curl -f "http://localhost:9090/-/healthy" >/dev/null 2>&1 || return 1
            
            # Check Grafana
            curl -f "http://localhost:3000/api/health" >/dev/null 2>&1 || return 1
            ;;
    esac
    
    return 0
}

Integration with external monitoring systems:

# External monitoring integration
send_deployment_metrics() {
    local status=$1
    local deployment_time=$2
    
    # Send metrics to Prometheus Pushgateway
    cat << EOF | curl --data-binary @- "http://pushgateway:9091/metrics/job/docker-deployment"
deployment_status{environment="production"} $([[ "$status" == "SUCCESS" ]] && echo 1 || echo 0)
deployment_duration_seconds{environment="production"} $deployment_time
deployment_timestamp{environment="production"} $(date +%s)
EOF
    
    # Send notification to Slack/Discord
    if [[ "$status" == "FAILED" ]]; then
        curl -X POST -H 'Content-type: application/json' \
            --data "{\"text\":\"🚨 Deployment failed in production environment\"}" \
            "$SLACK_WEBHOOK_URL"
    elif [[ "$status" == "SUCCESS" ]]; then
        curl -X POST -H 'Content-type: application/json' \
            --data "{\"text\":\"✅ Deployment successful in production environment\"}" \
            "$SLACK_WEBHOOK_URL"
    fi
}

Troubleshooting

Common Issues and Solutions

Docker Compose Pull Failures:

# Check Docker daemon status
systemctl status docker

# Verify Docker Compose version
docker compose version

# Check network connectivity
ping registry.docker.com

# Test manual pull
docker pull [IMAGE_NAME]

# Check authentication for private registries
docker login [REGISTRY_URL]

# Review deployment logs
grep -A 10 -B 10 "Failed to pull" logs/deployment.log

Container Start Failures:

# Check container logs
docker compose logs [SERVICE_NAME]

# Inspect container configuration
docker compose config

# Check resource availability
docker system df
df -h

# Verify port availability
netstat -tulpn | grep [PORT]

Services Not Passing Health Checks:

# Check service-specific logs
docker compose logs --tail=100 [SERVICE_NAME]

# Manual health check
curl -v http://localhost:[PORT]/health

# Check container resource usage
docker stats

# Inspect container processes
docker compose exec [SERVICE_NAME] ps aux

# Check network connectivity between services
docker compose exec [SERVICE_A] ping [SERVICE_B]

Health Check Timeouts:

# Increase health check timeout in script
HEALTH_CHECK_ATTEMPTS=24     # 4 minutes instead of 2
HEALTH_CHECK_INTERVAL=10     # Keep 10-second intervals

# Check for slow-starting services
docker compose logs [SLOW_SERVICE] | grep -i "ready\|started\|listening"

Log Rotation Issues:

# Check disk space
df -h /path/to/logs

# Manual log rotation
./docker-compose-deployer.sh --rotate-logs

# Check log file permissions
ls -la logs/

# Fix permissions if needed
chown -R [USER]:[GROUP] logs/
chmod 755 logs/
chmod 644 logs/*.log

Missing Log Entries:

# Check script execution
ps aux | grep docker-compose-deployer

# Verify cron execution
tail -f /var/log/cron

# Check for script errors
bash -x ./docker-compose-deployer.sh --force 2>&1 | tee debug.log

Performance Optimization

Optimize resource usage during deployments:

# Configure deployment resources
export COMPOSE_PARALLEL_LIMIT=4  # Limit parallel operations
export DOCKER_BUILDKIT=1         # Enable BuildKit for faster builds

# Add resource limits to script
deploy() {
    # Set deployment-time resource limits
    ulimit -n 65536  # Increase file descriptor limit
    
    # Use --parallel flag for faster deployments
    docker compose pull --parallel
    
    # Deploy with resource constraints
    docker compose up -d --remove-orphans --quiet-pull
}

Resource monitoring during deployment:

Monitor CPU and memory usage
Track disk I/O during image pulls
Limit concurrent operations to prevent resource exhaustion

Optimize network operations:

# Network optimization settings
configure_network_optimization() {
    # Configure Docker daemon for better performance
    cat > /etc/docker/daemon.json << EOF
{
    "max-concurrent-downloads": 3,
    "max-concurrent-uploads": 5,
    "registry-mirrors": ["http://your-local-mirror:5000"]
}
EOF
    
    # Restart Docker daemon
    systemctl restart docker
}

# Use local registry mirror
setup_registry_mirror() {
    # Pull through local mirror for faster downloads
    docker pull your-mirror.com/library/nginx:latest
}

Optimize storage usage and cleanup:

# Enhanced cleanup function
perform_cleanup() {
    log INFO "Starting cleanup process..."
    
    # Remove unused images
    docker image prune -af --filter "until=168h"  # 1 week old
    
    # Remove unused volumes
    docker volume prune -f
    
    # Remove unused networks
    docker network prune -f
    
    # Clean build cache
    docker builder prune -af
    
    # Log cleanup results
    log INFO "Cleanup completed. Disk space freed: $(df -h /var/lib/docker | tail -1 | awk '{print $4}')"
}

Security Considerations

Best Practices

Script Security

Registry Security

Access Control

Data Protection

Security Configuration

# Security hardening for deployment script
security_hardening() {
    # Set secure file permissions
    chmod 700 /path/to/docker-compose-deployer.sh
    chmod 700 logs/
    chmod 600 .deployment_state/*
    
    # Create dedicated deployment user
    useradd -r -s /bin/bash -m docker-deploy
    usermod -aG docker docker-deploy
    
    # Secure Docker socket access
    chown root:docker /var/run/docker.sock
    chmod 660 /var/run/docker.sock
    
    # Setup log monitoring
    echo "docker-compose-deployer.sh" >> /etc/audit/audit.rules
}

Integration Examples

CI/CD Pipeline Integration

# .gitlab-ci.yml
stages:
  - deploy

production-deploy:
  stage: deploy
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - apk add --no-cache curl bash
  script:
    - chmod +x docker-compose-deployer.sh
    - ./docker-compose-deployer.sh --force
  artifacts:
    reports:
      junit: logs/deployment-report.xml
    paths:
      - logs/
    expire_in: 1 week
  only:
    - main

# .github/workflows/deploy.yml
name: Production Deployment
on:
  push:
    branches: [main]
  schedule:
    - cron: '0 2 * * *'  # Daily at 2 AM

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Docker
        uses: docker/setup-buildx-action@v2
        
      - name: Run Deployment
        run: |
          chmod +x docker-compose-deployer.sh
          ./docker-compose-deployer.sh --force
          
      - name: Upload logs
        uses: actions/upload-artifact@v3
        if: always()
        with:
          name: deployment-logs
          path: logs/

// Jenkinsfile
pipeline {
    agent any
    
    triggers {
        cron('0 2 * * *')  // Daily at 2 AM
    }
    
    stages {
        stage('Deploy') {
            steps {
                script {
                    sh '''
                        chmod +x docker-compose-deployer.sh
                        ./docker-compose-deployer.sh --force
                    '''
                }
            }
        }
    }
    
    post {
        always {
            archiveArtifacts artifacts: 'logs/**', allowEmptyArchive: true
        }
        failure {
            emailext (
                subject: "Deployment Failed: ${env.JOB_NAME}",
                body: "Deployment failed. Check logs for details.",
                to: "${env.CHANGE_AUTHOR_EMAIL}"
            )
        }
    }
}

Conclusion

The Docker Compose Deployment Checker provides:

Automated deployment management with intelligent change detection
Comprehensive health monitoring ensuring service reliability
Enterprise-grade logging with rotation and history tracking
Production-ready safety features including rollback capabilities
Flexible configuration for various deployment scenarios
Integration support for CI/CD pipelines and monitoring systems

Regular monitoring of deployment logs and health checks ensures optimal performance and reliability of your containerized applications in production environments.

Docker Compose Deployment Checker

On this page