metadata-agregator/docs/research/meelo/analysis/DEPLOYMENT.md

Meelo Deployment

Deployment Overview

Meelo deploys as a multi-container Docker application orchestrated by Docker Compose. Three deployment variants support different use cases: production (pre-built images), development (hot reload), and local build (custom images).

Docker Compose Variants

Production (docker-compose.yml)

Use Case: End users running stable releases
Images: Pre-built from Docker Hub
Startup Time: Fast (no build step)
Updates: Pull new images, restart containers

services:
  server:
    image: arthichaud/meelo-server:latest
    restart: always
    depends_on:
      db:
        condition: service_healthy
      meilisearch:
        condition: service_healthy
      mq:
        condition: service_healthy
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/meelo
      - MEILISEARCH_URL=http://meilisearch:7700
      - RABBITMQ_URL=amqp://guest:guest@mq:5672
    volumes:
      - ${CONFIG_DIR}:/config
      - ${DATA_DIR}:/data

Key Features:

• restart: always for automatic recovery
• Health check dependencies ensure startup order
• Environment variables from .env
• Volumes for config and data persistence

Development (docker-compose.dev.yml)

Use Case: Contributors developing features
Images: Built from source with hot reload
Startup Time: Slower (build + watch)
Updates: Automatic on file save

services:
  server:
    build:
      context: ./server
      dockerfile: Dockerfile.dev
    volumes:
      - ./server/src:/app/src
      - ./server/prisma:/app/prisma
    ports:
      - "4000:4000"
    environment:
      - NODE_ENV=development
    command: npm run start:dev

Key Features:

• Source directories mounted for hot reload
• Exposed ports for debugging
• Development commands (start:dev, test:watch)
• No restart policy (manual control)

Local Build (docker-compose.local.yml)

Use Case: Testing Dockerfile changes, custom builds
Images: Built from source
Startup Time: Slow (full build)
Updates: Rebuild images manually

services:
  server:
    build:
      context: ./server
      dockerfile: Dockerfile
    restart: unless-stopped

Key Features:

• Builds production images locally
• Tests Dockerfile changes before pushing
• unless-stopped restart policy

Service Configuration

Server (NestJS)

Image: arthichaud/meelo-server
Port: 4000
Dependencies: PostgreSQL, MeiliSearch, RabbitMQ

Environment Variables:

DATABASE_URL=postgresql://postgres:postgres@db:5432/meelo
MEILISEARCH_URL=http://meilisearch:7700
RABBITMQ_URL=amqp://guest:guest@mq:5672
JWT_SIGNATURE=your_secret_key
PORT=4000
PUBLIC_URL=https://meelo.example.com
CONFIG_DIR=/config
DATA_DIR=/data

Volumes:

• ${CONFIG_DIR}:/config - settings.json
• ${DATA_DIR}:/data - music files (read-only)

Health Check:

healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:4000/api/health"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s

Scanner (Go)

Image: arthichaud/meelo-scanner
Port: 8133
Dependencies: Server

Environment Variables:

SERVER_URL=http://server:4000
API_KEY=your_api_key

Volumes:

• ${DATA_DIR}:/data - music files (read-only)

Health Check:

healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8133/"]
  interval: 30s
  timeout: 10s
  retries: 3

Matcher (Python)

Image: arthichaud/meelo-matcher
Port: 6789
Dependencies: Server, RabbitMQ

Environment Variables:

SERVER_URL=http://server:4000
RABBITMQ_URL=amqp://guest:guest@mq:5672
GENIUS_ACCESS_TOKEN=your_genius_token
DISCOGS_ACCESS_TOKEN=your_discogs_token

Health Check:

healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:6789/health"]
  interval: 30s
  timeout: 10s
  retries: 3

Front (Next.js)

Image: arthichaud/meelo-front
Port: 3000
Dependencies: Server

Environment Variables:

NEXT_PUBLIC_API_URL=http://localhost/api

Health Check:

healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:3000/api/health"]
  interval: 30s
  timeout: 10s
  retries: 3

PostgreSQL

Image: postgres:alpine3.14
Port: 5432 (internal only)
Volume: meelo_db

Environment Variables:

POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=meelo

Health Check:

healthcheck:
  test: ["CMD", "pg_isready", "-U", "postgres"]
  interval: 10s
  timeout: 5s
  retries: 5

MeiliSearch

Image: getmeili/meilisearch:v1.5
Port: 7700 (internal only)
Volume: meelo_search

Environment Variables:

MEILI_ENV=production
MEILI_NO_ANALYTICS=true

Health Check:

healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:7700/health"]
  interval: 10s
  timeout: 5s
  retries: 5

RabbitMQ

Image: rabbitmq:4.2-alpine
Port: 5672 (AMQP), 15672 (management UI)
Volume: meelo_rabbitmq_data

Health Check:

healthcheck:
  test: ["CMD", "rabbitmq-diagnostics", "ping"]
  interval: 10s
  timeout: 5s
  retries: 5

Kyoo Transcoder

Image: zoriya/kyoo_transcoder:latest
Port: 7666 (internal only)
Volume: meelo_transcoder_cache

Environment Variables:

TRANSCODER_CACHE_ROOT=/cache

No health check (optional service).

Nginx

Image: nginx:1.29.7-alpine
Port: 80 (exposed to host)
Config: Mounted from nginx.conf

Configuration:

server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://front:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api/ {
        proxy_pass http://server:4000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /scanner/ {
        proxy_pass http://scanner:8133;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /matcher/ {
        proxy_pass http://matcher:6789;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api/events {
        proxy_pass http://server:4000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Health Check:

healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/"]
  interval: 30s
  timeout: 10s
  retries: 3

Volumes

Named Volumes

volumes:
  meelo_db:
    driver: local
  meelo_search:
    driver: local
  meelo_rabbitmq_data:
    driver: local
  meelo_transcoder_cache:
    driver: local

Persistence:

• meelo_db: PostgreSQL data (critical, backup regularly)
• meelo_search: MeiliSearch index (can rebuild from database)
• meelo_rabbitmq_data: Message queue state (can lose without data loss)
• meelo_transcoder_cache: Transcoded video segments (can delete to free space)

Bind Mounts

volumes:
  - ${CONFIG_DIR}:/config
  - ${DATA_DIR}:/data:ro

Paths:

• CONFIG_DIR: Directory containing settings.json (default: ./config)
• DATA_DIR: Music library directory (default: ./data)

Permissions:

• DATA_DIR mounted read-only (:ro) to prevent accidental modification
• Services run as non-root user (UID 1000)

Startup Order

Docker Compose orchestrates startup using health checks:

1. PostgreSQL starts
   └─ Health check: pg_isready
2. MeiliSearch starts
   └─ Health check: GET /health
3. RabbitMQ starts
   └─ Health check: rabbitmq-diagnostics ping
4. Server starts (depends on db, meilisearch, mq)
   └─ Runs Prisma migrations
   └─ Seeds initial data
   └─ Health check: GET /api/health
5. Scanner starts (depends on server)
   └─ Registers with Server
   └─ Health check: GET /
6. Matcher starts (depends on server, mq)
   └─ Connects to RabbitMQ
   └─ Health check: GET /health
7. Front starts (depends on server)
   └─ SSR requires Server API
   └─ Health check: GET /api/health
8. Transcoder starts (no dependencies)
9. Nginx starts (depends on all application services)
   └─ Health check: GET /

Start Period: Each service has a start period (30-40s) before health checks begin. This allows initialization without false failures.

Configuration Files

.env

Environment variables for deployment:

# Ports
PORT=4000
FRONT_PORT=3000
SCANNER_PORT=8133
MATCHER_PORT=6789

# URLs
PUBLIC_URL=https://meelo.example.com

# Directories
CONFIG_DIR=./config
DATA_DIR=/path/to/music

# Database
DATABASE_URL=postgresql://postgres:postgres@db:5432/meelo

# Search
MEILISEARCH_URL=http://meilisearch:7700

# Message Queue
RABBITMQ_URL=amqp://guest:guest@mq:5672

# Authentication
JWT_SIGNATURE=your_secret_key_here
ALLOW_ANONYMOUS=0

# External Providers
GENIUS_ACCESS_TOKEN=your_genius_token
DISCOGS_ACCESS_TOKEN=your_discogs_token

# Last.fm OAuth
LASTFM_API_KEY=your_lastfm_key
LASTFM_API_SECRET=your_lastfm_secret

# CORS
CORS_ORIGINS=https://meelo.example.com

settings.json

User preferences (stored in CONFIG_DIR):

{
  "trackRegex": "(?P<artist>[^/]+)/(?P<album>[^/]+)/(?P<disc>\\d+)-(?P<track>\\d+) (?P<title>.+)\\.(?P<ext>\\w+)",
  "metadata": {
    "source": "providers",
    "order": ["musicbrainz", "genius", "wikipedia", "lrclib"]
  },
  "providers": {
    "musicbrainz": { "enabled": true },
    "genius": { "enabled": true },
    "wikipedia": { "enabled": true },
    "wikidata": { "enabled": true },
    "discogs": { "enabled": false },
    "allmusic": { "enabled": false },
    "metacritic": { "enabled": false },
    "lrclib": { "enabled": true }
  },
  "compilations": {
    "detectByArtist": true,
    "detectByFolder": true,
    "keywords": ["Various Artists", "Compilation", "Soundtrack"]
  }
}

First-Time Setup

1. Clone Repository

git clone https://github.com/Arthi-chaud/Meelo.git
cd Meelo

2. Configure Environment

cp .env.example .env
nano .env

Fill in required values:

• DATA_DIR: Path to music library
• JWT_SIGNATURE: Random secret key
• GENIUS_ACCESS_TOKEN: Genius API token (optional)
• DISCOGS_ACCESS_TOKEN: Discogs API token (optional)
• LASTFM_API_KEY, LASTFM_API_SECRET: Last.fm OAuth credentials (optional)

3. Create Settings File

mkdir -p config
nano config/settings.json

Copy example settings from above, adjust trackRegex to match your file naming.

4. Start Services

docker-compose up -d

Wait for all services to become healthy:

docker-compose ps

5. Register Admin User

Navigate to http://localhost and register first user (becomes admin automatically).

6. Create Library

1. Go to Settings > Libraries
2. Click "Add Library"
3. Enter name and path (must match DATA_DIR mount)
4. Save

7. Trigger Initial Scan

curl -X POST http://localhost/scanner/scan

Monitor progress:

curl http://localhost/scanner/tasks

8. Wait for Enrichment

Matcher processes files asynchronously. Check progress in UI (Artists/Albums pages populate as metadata arrives).

Updates

Pull New Images

docker-compose pull

Restart Services

docker-compose up -d

Docker Compose recreates containers with new images. Volumes persist data.

Database Migrations

Prisma migrations run automatically on Server startup. No manual intervention needed.

Backup

Database Backup

docker exec meelo-db pg_dump -U postgres meelo > backup.sql

Restore Database

docker exec -i meelo-db psql -U postgres meelo < backup.sql

Volume Backup

docker run --rm -v meelo_db:/data -v $(pwd):/backup alpine tar czf /backup/db.tar.gz /data

Restore Volume

docker run --rm -v meelo_db:/data -v $(pwd):/backup alpine tar xzf /backup/db.tar.gz -C /

Config Backup

cp -r config config.backup

Monitoring

Service Status

docker-compose ps

Shows health status for all services.

Logs

All Services:

docker-compose logs -f

Specific Service:

docker-compose logs -f server

Last 100 Lines:

docker-compose logs --tail=100 server

Resource Usage

docker stats

Shows CPU, memory, network, and disk I/O per container.

Troubleshooting

Service Won't Start

Check logs:

docker-compose logs <service>

Common issues:

• Database connection failed: PostgreSQL not healthy yet, wait longer
• Port already in use: Change port in .env
• Volume mount failed: Check DATA_DIR path exists and has correct permissions

Health Check Failing

Increase start period in docker-compose.yml:

healthcheck:
  start_period: 60s  # Increase from 40s

Out of Memory

Increase Docker memory limit (Docker Desktop settings) or reduce concurrent services.

Slow Performance

Check resource usage:

docker stats

Bottlenecks:

• High CPU on Matcher: Too many providers enabled, disable optional ones
• High memory on MeiliSearch: Large library, increase Docker memory
• High I/O on Scanner: Slow disk, use SSD

Production Deployment

Reverse Proxy

Use Nginx or Caddy as external reverse proxy:

server {
    listen 443 ssl http2;
    server_name meelo.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    location / {
        proxy_pass http://localhost:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

HTTPS

Use Let's Encrypt with Certbot:

certbot --nginx -d meelo.example.com

Or use Caddy (automatic HTTPS):

meelo.example.com {
    reverse_proxy localhost:80
}

Firewall

Open only port 443 (HTTPS):

ufw allow 443/tcp
ufw enable

Security Hardening

• Set ALLOW_ANONYMOUS=0 in .env
• Use strong JWT_SIGNATURE (32+ random characters)
• Restrict CORS_ORIGINS to your domain
• Run Docker in rootless mode
• Enable Docker Content Trust

Monitoring

Use Prometheus + Grafana (future enhancement, not built-in).

Backups

Automate database backups with cron:

0 2 * * * docker exec meelo-db pg_dump -U postgres meelo > /backups/meelo-$(date +\%Y\%m\%d).sql

Rotate backups:

find /backups -name "meelo-*.sql" -mtime +30 -delete

CI/CD

GitHub Actions

Meelo uses GitHub Actions for CI/CD. Workflows per service:

server.yml:

name: Server CI/CD

on:
  push:
    branches: [main]
    paths:
      - 'server/**'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 20
      - run: npm ci
        working-directory: server
      - run: npm run lint
        working-directory: server
      - run: npm test
        working-directory: server
      - uses: SonarSource/sonarcloud-github-action@master
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: docker/setup-buildx-action@v2
      - uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}
      - uses: docker/build-push-action@v4
        with:
          context: ./server
          push: true
          tags: arthichaud/meelo-server:latest

Similar workflows for scanner, matcher, front.

Quality Gates

SonarCloud enforces:

• Code coverage > 80%
• No critical bugs
• No security vulnerabilities
• Maintainability rating A

Failing quality gates block merges.

Scaling

Horizontal Scaling

Run multiple instances of stateless services:

services:
  scanner:
    image: arthichaud/meelo-scanner
    deploy:
      replicas: 3

Load balance with Nginx upstream:

upstream scanner {
    server scanner_1:8133;
    server scanner_2:8133;
    server scanner_3:8133;
}

location /scanner/ {
    proxy_pass http://scanner;
}

Vertical Scaling

Increase container resources:

services:
  server:
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 4G
        reservations:
          cpus: '1'
          memory: 2G

Summary

Meelo's deployment uses Docker Compose to orchestrate 8 services with health checks ensuring correct startup order. Three variants (production, development, local build) support different use cases. Configuration via .env and settings.json separates deployment and user preferences. Volumes persist data, bind mounts provide access to music files. First-time setup involves configuring environment, creating settings, starting services, registering admin, creating library, and triggering scan. Updates are simple (pull images, restart). Backups cover database, volumes, and config. Production deployment adds reverse proxy, HTTPS, firewall, and security hardening. CI/CD via GitHub Actions ensures quality. Scaling options include horizontal (multiple instances) and vertical (more resources).