metadata-agregator/docs/research/lidarr-metadata-api/analysis/DEPLOYMENT.md

Lidarr Metadata API - Deployment

Deployment Overview

The Lidarr Metadata API is designed for containerized deployment using Docker and Docker Compose. The architecture consists of 8+ containers working together to provide a complete metadata aggregation service.

Container Architecture

Container Inventory

Container	Image	Purpose	CPU	Memory	Storage
musicbrainz	ghcr.io/lidarr/mb-postgres:1.0.10	MusicBrainz database	2+ cores	4GB+	100GB+
solr	ghcr.io/lidarr/mb-solr:3.3.1.9	Search index	2 cores	2GB	8GB
redis	redis:6-alpine	Cache + rate limiting	1 core	512MB	None
rabbitmq	rabbitmq:3-management	Message queue	1 core	1GB	1GB
indexer	ghcr.io/lidarr/mb-sir:latest	Search index rebuilder	1 core	512MB	None
api-v0.3	ghcr.io/lidarr/lidarrapi.metadata:v0.3	Stable API version	2 cores	1GB	None
api-testing	ghcr.io/lidarr/lidarrapi.metadata:testing	Development API version	2 cores	1GB	None
crawler	ghcr.io/lidarr/lidarrapi.metadata:v0.3	Background cache warmer	1 core	512MB	None

Total minimum resources: 12 CPU cores, 11.5GB RAM, 110GB storage

Docker Compose Files

The project uses multiple compose files for different environments:

1. docker-compose.yml (Base)

Purpose: Core infrastructure services

Services:

• musicbrainz: PostgreSQL with MusicBrainz schema
• solr: Search index
• redis: Cache
• rabbitmq: Message queue

File location: docker-compose.yml

Key configuration:

version: '3.8'

services:
  musicbrainz:
    image: ghcr.io/lidarr/mb-postgres:1.0.10
    container_name: musicbrainz
    environment:
      POSTGRES_USER: abc
      POSTGRES_PASSWORD: abc
      POSTGRES_DB: musicbrainz_db
    volumes:
      - musicbrainz-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: unless-stopped
    shm_size: 2gb
  
  solr:
    image: ghcr.io/lidarr/mb-solr:3.3.1.9
    container_name: solr
    environment:
      SOLR_HEAP: 2g
    volumes:
      - solr-data:/var/solr
    ports:
      - "8983:8983"
    restart: unless-stopped
  
  redis:
    image: redis:6-alpine
    container_name: redis
    command: redis-server --maxmemory 512mb --maxmemory-policy allkeys-lfu
    ports:
      - "6379:6379"
    restart: unless-stopped
  
  rabbitmq:
    image: rabbitmq:3-management
    container_name: rabbitmq
    environment:
      RABBITMQ_DEFAULT_USER: abc
      RABBITMQ_DEFAULT_PASS: abc
    ports:
      - "5672:5672"
      - "15672:15672"
    restart: unless-stopped

volumes:
  musicbrainz-data:
  solr-data:

2. docker-compose.dev.yml (Development)

Purpose: Development environment with exposed ports

Extends: docker-compose.yml

Additional configuration:

version: '3.8'

services:
  api:
    image: ghcr.io/lidarr/lidarrapi.metadata:testing
    container_name: lidarr-metadata-api
    environment:
      LIDARR_METADATA_CONFIG: lidarrmetadata.config.DevelopmentConfig
      DATABASE__HOST: musicbrainz
      CACHE__REDIS_URL: redis://redis:6379/0
      SOLR__URL: http://solr:8983/solr
      RABBITMQ__HOST: rabbitmq
    ports:
      - "5001:5001"
    depends_on:
      - musicbrainz
      - solr
      - redis
      - rabbitmq
    restart: unless-stopped
  
  indexer:
    image: ghcr.io/lidarr/mb-sir:latest
    container_name: search-indexer
    environment:
      DATABASE_HOST: musicbrainz
      SOLR_URL: http://solr:8983/solr
      RABBITMQ_HOST: rabbitmq
    depends_on:
      - musicbrainz
      - solr
      - rabbitmq
    restart: unless-stopped

Usage:

docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d

3. docker-compose.prod.yml (Production)

Purpose: Production deployment with dual API versions

Extends: docker-compose.yml

Additional configuration:

version: '3.8'

services:
  api-v0.3:
    image: ghcr.io/lidarr/lidarrapi.metadata:v0.3
    container_name: lidarr-metadata-api-v0.3
    environment:
      LIDARR_METADATA_CONFIG: lidarrmetadata.config.ProductionConfig
      DATABASE__HOST: musicbrainz
      CACHE__REDIS_URL: redis://redis:6379/0
      SOLR__URL: http://solr:8983/solr
      RABBITMQ__HOST: rabbitmq
      SENTRY_DSN: ${SENTRY_DSN}
      CLOUDFLARE_ZONE_ID: ${CLOUDFLARE_ZONE_ID}
      CLOUDFLARE_API_TOKEN: ${CLOUDFLARE_API_TOKEN}
      FANART_API_KEY: ${FANART_API_KEY}
      SPOTIFY_CLIENT_ID: ${SPOTIFY_CLIENT_ID}
      SPOTIFY_CLIENT_SECRET: ${SPOTIFY_CLIENT_SECRET}
      LASTFM_API_KEY: ${LASTFM_API_KEY}
    deploy:
      replicas: 2
      resources:
        limits:
          cpus: '2'
          memory: 1G
        reservations:
          cpus: '1'
          memory: 512M
    depends_on:
      - musicbrainz
      - solr
      - redis
      - rabbitmq
    restart: unless-stopped
  
  api-testing:
    image: ghcr.io/lidarr/lidarrapi.metadata:testing
    container_name: lidarr-metadata-api-testing
    environment:
      LIDARR_METADATA_CONFIG: lidarrmetadata.config.ProductionConfig
      DATABASE__HOST: musicbrainz
      CACHE__REDIS_URL: redis://redis:6379/0
      SOLR__URL: http://solr:8983/solr
      RABBITMQ__HOST: rabbitmq
      SENTRY_DSN: ${SENTRY_DSN}
      CLOUDFLARE_ZONE_ID: ${CLOUDFLARE_ZONE_ID}
      CLOUDFLARE_API_TOKEN: ${CLOUDFLARE_API_TOKEN}
      FANART_API_KEY: ${FANART_API_KEY}
      SPOTIFY_CLIENT_ID: ${SPOTIFY_CLIENT_ID}
      SPOTIFY_CLIENT_SECRET: ${SPOTIFY_CLIENT_SECRET}
      LASTFM_API_KEY: ${LASTFM_API_KEY}
    deploy:
      replicas: 1
      resources:
        limits:
          cpus: '2'
          memory: 1G
    depends_on:
      - musicbrainz
      - solr
      - redis
      - rabbitmq
    restart: unless-stopped
  
  indexer:
    image: ghcr.io/lidarr/mb-sir:latest
    container_name: search-indexer
    environment:
      DATABASE_HOST: musicbrainz
      SOLR_URL: http://solr:8983/solr
      RABBITMQ_HOST: rabbitmq
    depends_on:
      - musicbrainz
      - solr
      - rabbitmq
    restart: unless-stopped

Usage:

docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

4. docker-compose.crawler.yml (Crawler)

Purpose: Background cache warming

Extends: docker-compose.yml

Additional configuration:

version: '3.8'

services:
  crawler:
    image: ghcr.io/lidarr/lidarrapi.metadata:v0.3
    container_name: lidarr-metadata-crawler
    command: lidarr-metadata-crawler
    environment:
      LIDARR_METADATA_CONFIG: lidarrmetadata.config.ProductionConfig
      DATABASE__HOST: musicbrainz
      CACHE__REDIS_URL: redis://redis:6379/0
      SOLR__URL: http://solr:8983/solr
      FANART_API_KEY: ${FANART_API_KEY}
      SPOTIFY_CLIENT_ID: ${SPOTIFY_CLIENT_ID}
      SPOTIFY_CLIENT_SECRET: ${SPOTIFY_CLIENT_SECRET}
      CRAWLER_INTERVAL: 3600
      CRAWLER_BATCH_SIZE: 100
    depends_on:
      - musicbrainz
      - redis
    restart: unless-stopped

Usage:

docker-compose -f docker-compose.yml -f docker-compose.crawler.yml up -d

Dockerfile

Multi-Stage Build

File location: Dockerfile

Build stages:

Stage 1: Dependencies

FROM python:3.9-alpine AS builder

# Install build dependencies
RUN apk add --no-cache \
    gcc \
    musl-dev \
    postgresql-dev \
    libffi-dev \
    openssl-dev

# Install Poetry
RUN pip install poetry==1.1.13

# Copy dependency files
WORKDIR /app
COPY pyproject.toml poetry.lock ./

# Install dependencies
RUN poetry config virtualenvs.create false \
    && poetry install --no-dev --no-interaction --no-ansi

Stage 2: Runtime

FROM python:3.9-alpine

# Install runtime dependencies
RUN apk add --no-cache \
    postgresql-libs \
    libffi \
    openssl

# Create non-root user
RUN addgroup -g 1000 lidarr \
    && adduser -D -u 1000 -G lidarr lidarr

# Copy dependencies from builder
COPY --from=builder /usr/local/lib/python3.9/site-packages /usr/local/lib/python3.9/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin

# Copy application code
WORKDIR /app
COPY --chown=lidarr:lidarr . .

# Switch to non-root user
USER lidarr

# Expose port
EXPOSE 5001

# Default command
CMD ["gunicorn", "-w", "1", "-k", "uvicorn.workers.UvicornWorker", "--bind", "0.0.0.0:5001", "--access-logfile", "-", "lidarrmetadata.server:app"]

Build Arguments

ARG VERSION=unknown
ARG GIT_SHA=unknown
ARG BUILD_DATE=unknown

LABEL org.opencontainers.image.version="${VERSION}"
LABEL org.opencontainers.image.revision="${GIT_SHA}"
LABEL org.opencontainers.image.created="${BUILD_DATE}"
LABEL org.opencontainers.image.source="https://github.com/Lidarr/LidarrAPI.Metadata"

Build Command

docker build \
  --build-arg VERSION=10.0.0.0 \
  --build-arg GIT_SHA=$(git rev-parse HEAD) \
  --build-arg BUILD_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ") \
  -t ghcr.io/lidarr/lidarrapi.metadata:v0.3 \
  .

Initialization Process

10-Step Initialization

The deployment requires a specific initialization sequence:

Step 1: Start PostgreSQL

docker-compose up -d musicbrainz

Wait for: PostgreSQL ready to accept connections

Verification:

docker-compose exec musicbrainz pg_isready -U abc

Step 2: Fetch MusicBrainz Dump

Options:

Option A: Full dump (recommended for production):

docker-compose exec musicbrainz /scripts/fetch-dump.sh

Duration: 4-8 hours (depends on network speed)

Size: ~30GB compressed, ~100GB uncompressed

Option B: Sample dump (for development):

docker-compose exec musicbrainz /scripts/fetch-sample-dump.sh

Duration: 30-60 minutes

Size: ~1GB compressed, ~5GB uncompressed

Step 3: Setup AMQP Triggers

docker-compose exec musicbrainz /scripts/setup-amqp.sh

Purpose: Create database triggers that publish changes to RabbitMQ

Verification:

SELECT * FROM pg_trigger WHERE tgname LIKE 'amqp_%';

Step 4: Create Custom Indices

docker-compose exec musicbrainz psql -U abc -d musicbrainz_db -f /scripts/create-indices.sql

Indices created:

• idx_artist_last_updated
• idx_release_group_last_updated
• idx_release_last_updated
• idx_l_artist_url_last_updated
• idx_cover_art_date_updated

Duration: 30-60 minutes

Step 5: Start RabbitMQ

docker-compose up -d rabbitmq

Wait for: RabbitMQ management interface available

Verification:

curl http://localhost:15672/api/overview -u abc:abc

Step 6: Start Solr

docker-compose up -d solr

Wait for: Solr cores created

Verification:

curl http://localhost:8983/solr/admin/cores?action=STATUS

Step 7: Load Search Indices

docker-compose run indexer rebuild-artist
docker-compose run indexer rebuild-album

Duration: 4-8 hours for full dataset

Verification:

curl "http://localhost:8983/solr/artist/select?q=*:*&rows=0" | jq '.response.numFound'

Step 8: Enable Replication

docker-compose exec musicbrainz /scripts/enable-replication.sh

Purpose: Start hourly replication from MusicBrainz master

Verification:

SELECT * FROM replication_control;

Step 9: Start Search Indexer

docker-compose up -d indexer

Purpose: Real-time search index updates via RabbitMQ

Verification:

docker-compose logs -f indexer

Step 10: Start API and Crawler

docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d api-v0.3 api-testing
docker-compose -f docker-compose.yml -f docker-compose.crawler.yml up -d crawler

Verification:

curl http://localhost:5001/

Initialization Script

Complete initialization script:

#!/bin/bash
set -e

echo "Step 1: Starting PostgreSQL..."
docker-compose up -d musicbrainz
sleep 30

echo "Step 2: Fetching MusicBrainz dump..."
docker-compose exec musicbrainz /scripts/fetch-dump.sh

echo "Step 3: Setting up AMQP triggers..."
docker-compose exec musicbrainz /scripts/setup-amqp.sh

echo "Step 4: Creating custom indices..."
docker-compose exec musicbrainz psql -U abc -d musicbrainz_db -f /scripts/create-indices.sql

echo "Step 5: Starting RabbitMQ..."
docker-compose up -d rabbitmq
sleep 10

echo "Step 6: Starting Solr..."
docker-compose up -d solr
sleep 10

echo "Step 7: Loading search indices..."
docker-compose run indexer rebuild-artist
docker-compose run indexer rebuild-album

echo "Step 8: Enabling replication..."
docker-compose exec musicbrainz /scripts/enable-replication.sh

echo "Step 9: Starting search indexer..."
docker-compose up -d indexer

echo "Step 10: Starting API and crawler..."
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d api-v0.3 api-testing
docker-compose -f docker-compose.yml -f docker-compose.crawler.yml up -d crawler

echo "Initialization complete!"

CI/CD Pipeline

Azure Pipelines

File location: azure-pipelines.yml

Trigger: Commits to master or develop branches

Stages:

1. Build and Test

stages:
  - stage: Build
    jobs:
      - job: BuildAndTest
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: UsePythonVersion@0
            inputs:
              versionSpec: '3.9'
          
          - script: |
              pip install poetry
              poetry install
            displayName: 'Install dependencies'
          
          # Tests commented out
          # - script: |
          #     poetry run pytest tests/
          #   displayName: 'Run tests'
          
          - task: SonarCloudPrepare@1
            inputs:
              SonarCloud: 'SonarCloud'
              organization: 'lidarr'
              scannerMode: 'CLI'
              configMode: 'manual'
              cliProjectKey: 'Lidarr_LidarrAPI.Metadata'
              cliProjectName: 'LidarrAPI.Metadata'
          
          - task: SonarCloudAnalyze@1
          
          - task: SonarCloudPublish@1
            inputs:
              pollingTimeoutSec: '300'

2. Docker Build and Push

  - stage: Docker
    dependsOn: Build
    jobs:
      - job: BuildAndPush
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: Docker@2
            inputs:
              containerRegistry: 'GitHub Container Registry'
              repository: 'lidarr/lidarrapi.metadata'
              command: 'buildAndPush'
              Dockerfile: 'Dockerfile'
              tags: |
                $(Build.SourceBranchName)
                $(Build.SourceVersion)
                latest
              buildArgs: |
                VERSION=10.0.0.0
                GIT_SHA=$(Build.SourceVersion)
                BUILD_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

3. Sentry Release

  - stage: Sentry
    dependsOn: Docker
    jobs:
      - job: CreateRelease
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: |
              curl -sL https://sentry.io/get-cli/ | bash
              sentry-cli releases new "lidarr-metadata@$(Build.SourceVersion)"
              sentry-cli releases set-commits "lidarr-metadata@$(Build.SourceVersion)" --auto
              sentry-cli releases finalize "lidarr-metadata@$(Build.SourceVersion)"
            env:
              SENTRY_AUTH_TOKEN: $(SENTRY_AUTH_TOKEN)
              SENTRY_ORG: lidarr
              SENTRY_PROJECT: lidarr-metadata
            displayName: 'Create Sentry release'

GitHub Actions (Alternative)

File location: .github/workflows/build.yml

name: Build and Deploy

on:
  push:
    branches: [master, develop]
  pull_request:
    branches: [master]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
      
      - name: Install dependencies
        run: |
          pip install poetry
          poetry install
      
      - name: Run tests
        run: poetry run pytest tests/
      
      - name: SonarCloud Scan
        uses: SonarSource/sonarcloud-github-action@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
      
      - name: Build Docker image
        run: |
          docker build \
            --build-arg VERSION=10.0.0.0 \
            --build-arg GIT_SHA=${{ github.sha }} \
            --build-arg BUILD_DATE=$(date -u +"%Y-%m-%dT%H:%M:%SZ") \
            -t ghcr.io/lidarr/lidarrapi.metadata:${{ github.ref_name }} \
            .
      
      - name: Push to GitHub Container Registry
        run: |
          echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
          docker push ghcr.io/lidarr/lidarrapi.metadata:${{ github.ref_name }}
      
      - name: Create Sentry release
        run: |
          curl -sL https://sentry.io/get-cli/ | bash
          sentry-cli releases new "lidarr-metadata@${{ github.sha }}"
          sentry-cli releases set-commits "lidarr-metadata@${{ github.sha }}" --auto
          sentry-cli releases finalize "lidarr-metadata@${{ github.sha }}"
        env:
          SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
          SENTRY_ORG: lidarr
          SENTRY_PROJECT: lidarr-metadata

Infrastructure Requirements

Minimum Requirements

Component	CPU	Memory	Storage	Network
MusicBrainz DB	2 cores	4GB	100GB SSD	1Gbps
Solr	2 cores	2GB	8GB SSD	1Gbps
Redis	1 core	512MB	None	1Gbps
RabbitMQ	1 core	1GB	1GB SSD	1Gbps
API (x2)	4 cores	2GB	None	1Gbps
Indexer	1 core	512MB	None	1Gbps
Crawler	1 core	512MB	None	1Gbps
Total	12 cores	11.5GB	110GB	1Gbps

Recommended Production Requirements

Component	CPU	Memory	Storage	Network
MusicBrainz DB	4 cores	8GB	200GB NVMe	10Gbps
Solr	4 cores	4GB	16GB NVMe	10Gbps
Redis	2 cores	1GB	None	10Gbps
RabbitMQ	2 cores	2GB	2GB SSD	10Gbps
API (x4)	8 cores	4GB	None	10Gbps
Indexer	2 cores	1GB	None	10Gbps
Crawler	2 cores	1GB	None	10Gbps
Total	24 cores	21GB	220GB	10Gbps

Cloud Provider Recommendations

AWS

Instance types:

• Database: r6g.xlarge (4 vCPU, 32GB RAM)
• Solr: c6g.xlarge (4 vCPU, 8GB RAM)
• API: c6g.large (2 vCPU, 4GB RAM) x 2
• Other: t4g.small (2 vCPU, 2GB RAM) x 3

Storage:

• Database: gp3 200GB (3000 IOPS, 125 MB/s)
• Solr: gp3 20GB (3000 IOPS)

Estimated cost: $400-600/month

Google Cloud

Instance types:

• Database: n2-highmem-4 (4 vCPU, 32GB RAM)
• Solr: n2-standard-4 (4 vCPU, 16GB RAM)
• API: n2-standard-2 (2 vCPU, 8GB RAM) x 2
• Other: e2-small (2 vCPU, 2GB RAM) x 3

Storage:

• Database: pd-ssd 200GB
• Solr: pd-ssd 20GB

Estimated cost: $450-650/month

DigitalOcean

Droplets:

• Database: Memory-Optimized 8GB ($80/month)
• Solr: General Purpose 4GB ($48/month)
• API: General Purpose 2GB ($24/month) x 2
• Other: Basic 2GB ($18/month) x 3

Volumes:

• Database: 200GB SSD ($20/month)
• Solr: 20GB SSD ($2/month)

Estimated cost: $250-350/month

Monitoring and Logging

Health Checks

Docker Compose health checks:

services:
  api-v0.3:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:5001/"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
  
  musicbrainz:
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "abc"]
      interval: 30s
      timeout: 5s
      retries: 3
  
  solr:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8983/solr/admin/ping"]
      interval: 30s
      timeout: 5s
      retries: 3
  
  redis:
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 30s
      timeout: 3s
      retries: 3

Logging

Centralized logging with Docker:

services:
  api-v0.3:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Log aggregation with Loki (optional):

services:
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"
    volumes:
      - loki-data:/loki
  
  promtail:
    image: grafana/promtail:latest
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - ./promtail-config.yml:/etc/promtail/config.yml
    command: -config.file=/etc/promtail/config.yml

Metrics

Prometheus scraping:

services:
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
  
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    volumes:
      - grafana-data:/var/lib/grafana
    environment:
      GF_SECURITY_ADMIN_PASSWORD: admin

Backup and Recovery

Database Backup

PostgreSQL backup script:

#!/bin/bash
BACKUP_DIR=/backups
DATE=$(date +%Y%m%d_%H%M%S)

# Backup MusicBrainz database
docker-compose exec -T musicbrainz pg_dump -U abc musicbrainz_db | gzip > $BACKUP_DIR/musicbrainz_$DATE.sql.gz

# Backup cache database
docker-compose exec -T musicbrainz pg_dump -U abc lm_cache_db | gzip > $BACKUP_DIR/cache_$DATE.sql.gz

# Cleanup old backups (keep 7 days)
find $BACKUP_DIR -name "*.sql.gz" -mtime +7 -delete

Cron schedule:

0 2 * * * /scripts/backup.sh

Solr Backup

Solr snapshot:

curl "http://localhost:8983/solr/artist/replication?command=backup&location=/var/solr/backups&name=artist_$(date +%Y%m%d)"
curl "http://localhost:8983/solr/release-group/replication?command=backup&location=/var/solr/backups&name=album_$(date +%Y%m%d)"

Recovery

Database restore:

# Stop services
docker-compose down

# Restore database
gunzip -c /backups/musicbrainz_20250428.sql.gz | docker-compose exec -T musicbrainz psql -U abc musicbrainz_db

# Restart services
docker-compose up -d

Security Hardening

Production Security Checklist

• Change default credentials (abc/abc)
• Use strong passwords for all services
• Enable TLS for PostgreSQL connections
• Enable TLS for Redis connections
• Use secrets management (Docker Secrets, Vault)
• Restrict network access (firewall rules)
• Enable API authentication
• Use read-only database user for API
• Enable Sentry error tracking
• Configure rate limiting
• Use HTTPS reverse proxy (Nginx, Traefik)
• Enable CORS restrictions
• Implement API key rotation
• Regular security updates
• Monitor for vulnerabilities (Snyk, Dependabot)

Secrets Management

Docker Secrets (Swarm mode):

services:
  api-v0.3:
    secrets:
      - db_password
      - redis_password
      - fanart_api_key
      - spotify_client_secret
    environment:
      DATABASE__PASSWORD_FILE: /run/secrets/db_password
      CACHE__REDIS_PASSWORD_FILE: /run/secrets/redis_password
      FANART_API_KEY_FILE: /run/secrets/fanart_api_key
      SPOTIFY_CLIENT_SECRET_FILE: /run/secrets/spotify_client_secret

secrets:
  db_password:
    external: true
  redis_password:
    external: true
  fanart_api_key:
    external: true
  spotify_client_secret:
    external: true

Scaling Strategies

Horizontal Scaling

API instances:

services:
  api-v0.3:
    deploy:
      replicas: 4
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

Load balancing (Nginx):

upstream lidarr_metadata_api {
    least_conn;
    server api-v0.3-1:5001;
    server api-v0.3-2:5001;
    server api-v0.3-3:5001;
    server api-v0.3-4:5001;
}

server {
    listen 80;
    server_name api.lidarr.audio;
    
    location / {
        proxy_pass http://lidarr_metadata_api;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Vertical Scaling

Increase worker count:

services:
  api-v0.3:
    command: gunicorn -w 4 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:5001 lidarrmetadata.server:app

Increase database connections:

DATABASE = {
    'min_pool_size': 20,
    'max_pool_size': 100
}

Conclusion

The deployment architecture demonstrates production-ready containerization with:

• 10-step initialization for complex setup
• Multi-environment support (dev, prod, crawler)
• Dual-version deployment for safe rollouts
• CI/CD integration with Azure Pipelines
• Comprehensive monitoring with health checks and metrics
• Backup and recovery procedures
• Security hardening recommendations
• Horizontal and vertical scaling strategies

The Docker Compose approach makes deployment straightforward while maintaining flexibility for different environments.