metadata-agregator/docs/research/meelo/analysis/DEPLOYMENT.md

# Meelo Deployment

## Deployment Overview

Meelo deploys as a multi-container Docker application orchestrated by Docker Compose. Three deployment variants support different use cases: production (pre-built images), development (hot reload), and local build (custom images).

## Docker Compose Variants

### Production (docker-compose.yml)

**Use Case**: End users running stable releases  
**Images**: Pre-built from Docker Hub  
**Startup Time**: Fast (no build step)  
**Updates**: Pull new images, restart containers

```yaml
services:
  server:
    image: arthichaud/meelo-server:latest
    restart: always
    depends_on:
      db:
        condition: service_healthy
      meilisearch:
        condition: service_healthy
      mq:
        condition: service_healthy
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/meelo
      - MEILISEARCH_URL=http://meilisearch:7700
      - RABBITMQ_URL=amqp://guest:guest@mq:5672
    volumes:
      - ${CONFIG_DIR}:/config
      - ${DATA_DIR}:/data
```

**Key Features**:
- `restart: always` for automatic recovery
- Health check dependencies ensure startup order
- Environment variables from .env
- Volumes for config and data persistence

### Development (docker-compose.dev.yml)

**Use Case**: Contributors developing features  
**Images**: Built from source with hot reload  
**Startup Time**: Slower (build + watch)  
**Updates**: Automatic on file save

```yaml
services:
  server:
    build:
      context: ./server
      dockerfile: Dockerfile.dev
    volumes:
      - ./server/src:/app/src
      - ./server/prisma:/app/prisma
    ports:
      - "4000:4000"
    environment:
      - NODE_ENV=development
    command: npm run start:dev
```

**Key Features**:
- Source directories mounted for hot reload
- Exposed ports for debugging
- Development commands (start:dev, test:watch)
- No restart policy (manual control)

### Local Build (docker-compose.local.yml)

**Use Case**: Testing Dockerfile changes, custom builds  
**Images**: Built from source  
**Startup Time**: Slow (full build)  
**Updates**: Rebuild images manually

```yaml
services:
  server:
    build:
      context: ./server
      dockerfile: Dockerfile
    restart: unless-stopped
```

**Key Features**:
- Builds production images locally
- Tests Dockerfile changes before pushing
- `unless-stopped` restart policy

## Service Configuration

### Server (NestJS)

**Image**: arthichaud/meelo-server  
**Port**: 4000  
**Dependencies**: PostgreSQL, MeiliSearch, RabbitMQ

**Environment Variables**:
```bash
DATABASE_URL=postgresql://postgres:postgres@db:5432/meelo
MEILISEARCH_URL=http://meilisearch:7700
RABBITMQ_URL=amqp://guest:guest@mq:5672
JWT_SIGNATURE=your_secret_key
PORT=4000
PUBLIC_URL=https://meelo.example.com
CONFIG_DIR=/config
DATA_DIR=/data
```

**Volumes**:
- `${CONFIG_DIR}:/config` - settings.json
- `${DATA_DIR}:/data` - music files (read-only)

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:4000/api/health"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s
```

### Scanner (Go)

**Image**: arthichaud/meelo-scanner  
**Port**: 8133  
**Dependencies**: Server

**Environment Variables**:
```bash
SERVER_URL=http://server:4000
API_KEY=your_api_key
```

**Volumes**:
- `${DATA_DIR}:/data` - music files (read-only)

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:8133/"]
  interval: 30s
  timeout: 10s
  retries: 3
```

### Matcher (Python)

**Image**: arthichaud/meelo-matcher  
**Port**: 6789  
**Dependencies**: Server, RabbitMQ

**Environment Variables**:
```bash
SERVER_URL=http://server:4000
RABBITMQ_URL=amqp://guest:guest@mq:5672
GENIUS_ACCESS_TOKEN=your_genius_token
DISCOGS_ACCESS_TOKEN=your_discogs_token
```

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:6789/health"]
  interval: 30s
  timeout: 10s
  retries: 3
```

### Front (Next.js)

**Image**: arthichaud/meelo-front  
**Port**: 3000  
**Dependencies**: Server

**Environment Variables**:
```bash
NEXT_PUBLIC_API_URL=http://localhost/api
```

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:3000/api/health"]
  interval: 30s
  timeout: 10s
  retries: 3
```

### PostgreSQL

**Image**: postgres:alpine3.14  
**Port**: 5432 (internal only)  
**Volume**: meelo_db

**Environment Variables**:
```bash
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=meelo
```

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "pg_isready", "-U", "postgres"]
  interval: 10s
  timeout: 5s
  retries: 5
```

### MeiliSearch

**Image**: getmeili/meilisearch:v1.5  
**Port**: 7700 (internal only)  
**Volume**: meelo_search

**Environment Variables**:
```bash
MEILI_ENV=production
MEILI_NO_ANALYTICS=true
```

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:7700/health"]
  interval: 10s
  timeout: 5s
  retries: 5
```

### RabbitMQ

**Image**: rabbitmq:4.2-alpine  
**Port**: 5672 (AMQP), 15672 (management UI)  
**Volume**: meelo_rabbitmq_data

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "rabbitmq-diagnostics", "ping"]
  interval: 10s
  timeout: 5s
  retries: 5
```

### Kyoo Transcoder

**Image**: zoriya/kyoo_transcoder:latest  
**Port**: 7666 (internal only)  
**Volume**: meelo_transcoder_cache

**Environment Variables**:
```bash
TRANSCODER_CACHE_ROOT=/cache
```

No health check (optional service).

### Nginx

**Image**: nginx:1.29.7-alpine  
**Port**: 80 (exposed to host)  
**Config**: Mounted from nginx.conf

**Configuration**:
```nginx
server {
    listen 80;
    server_name localhost;

    location / {
        proxy_pass http://front:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api/ {
        proxy_pass http://server:4000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /scanner/ {
        proxy_pass http://scanner:8133;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /matcher/ {
        proxy_pass http://matcher:6789;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    location /api/events {
        proxy_pass http://server:4000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
```

**Health Check**:
```yaml
healthcheck:
  test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost/"]
  interval: 30s
  timeout: 10s
  retries: 3
```

## Volumes

### Named Volumes

```yaml
volumes:
  meelo_db:
    driver: local
  meelo_search:
    driver: local
  meelo_rabbitmq_data:
    driver: local
  meelo_transcoder_cache:
    driver: local
```

**Persistence**:
- `meelo_db`: PostgreSQL data (critical, backup regularly)
- `meelo_search`: MeiliSearch index (can rebuild from database)
- `meelo_rabbitmq_data`: Message queue state (can lose without data loss)
- `meelo_transcoder_cache`: Transcoded video segments (can delete to free space)

### Bind Mounts

```yaml
volumes:
  - ${CONFIG_DIR}:/config
  - ${DATA_DIR}:/data:ro
```

**Paths**:
- `CONFIG_DIR`: Directory containing settings.json (default: ./config)
- `DATA_DIR`: Music library directory (default: ./data)

**Permissions**:
- `DATA_DIR` mounted read-only (`:ro`) to prevent accidental modification
- Services run as non-root user (UID 1000)

## Startup Order

Docker Compose orchestrates startup using health checks:

```
1. PostgreSQL starts
   └─ Health check: pg_isready
2. MeiliSearch starts
   └─ Health check: GET /health
3. RabbitMQ starts
   └─ Health check: rabbitmq-diagnostics ping
4. Server starts (depends on db, meilisearch, mq)
   └─ Runs Prisma migrations
   └─ Seeds initial data
   └─ Health check: GET /api/health
5. Scanner starts (depends on server)
   └─ Registers with Server
   └─ Health check: GET /
6. Matcher starts (depends on server, mq)
   └─ Connects to RabbitMQ
   └─ Health check: GET /health
7. Front starts (depends on server)
   └─ SSR requires Server API
   └─ Health check: GET /api/health
8. Transcoder starts (no dependencies)
9. Nginx starts (depends on all application services)
   └─ Health check: GET /
```

**Start Period**: Each service has a start period (30-40s) before health checks begin. This allows initialization without false failures.

## Configuration Files

### .env

Environment variables for deployment:

```bash
# Ports
PORT=4000
FRONT_PORT=3000
SCANNER_PORT=8133
MATCHER_PORT=6789

# URLs
PUBLIC_URL=https://meelo.example.com

# Directories
CONFIG_DIR=./config
DATA_DIR=/path/to/music

# Database
DATABASE_URL=postgresql://postgres:postgres@db:5432/meelo

# Search
MEILISEARCH_URL=http://meilisearch:7700

# Message Queue
RABBITMQ_URL=amqp://guest:guest@mq:5672

# Authentication
JWT_SIGNATURE=your_secret_key_here
ALLOW_ANONYMOUS=0

# External Providers
GENIUS_ACCESS_TOKEN=your_genius_token
DISCOGS_ACCESS_TOKEN=your_discogs_token

# Last.fm OAuth
LASTFM_API_KEY=your_lastfm_key
LASTFM_API_SECRET=your_lastfm_secret

# CORS
CORS_ORIGINS=https://meelo.example.com
```

### settings.json

User preferences (stored in CONFIG_DIR):

```json
{
  "trackRegex": "(?P<artist>[^/]+)/(?P<album>[^/]+)/(?P<disc>\\d+)-(?P<track>\\d+) (?P<title>.+)\\.(?P<ext>\\w+)",
  "metadata": {
    "source": "providers",
    "order": ["musicbrainz", "genius", "wikipedia", "lrclib"]
  },
  "providers": {
    "musicbrainz": { "enabled": true },
    "genius": { "enabled": true },
    "wikipedia": { "enabled": true },
    "wikidata": { "enabled": true },
    "discogs": { "enabled": false },
    "allmusic": { "enabled": false },
    "metacritic": { "enabled": false },
    "lrclib": { "enabled": true }
  },
  "compilations": {
    "detectByArtist": true,
    "detectByFolder": true,
    "keywords": ["Various Artists", "Compilation", "Soundtrack"]
  }
}
```

## First-Time Setup

### 1. Clone Repository

```bash
git clone https://github.com/Arthi-chaud/Meelo.git
cd Meelo
```

### 2. Configure Environment

```bash
cp .env.example .env
nano .env
```

Fill in required values:
- `DATA_DIR`: Path to music library
- `JWT_SIGNATURE`: Random secret key
- `GENIUS_ACCESS_TOKEN`: Genius API token (optional)
- `DISCOGS_ACCESS_TOKEN`: Discogs API token (optional)
- `LASTFM_API_KEY`, `LASTFM_API_SECRET`: Last.fm OAuth credentials (optional)

### 3. Create Settings File

```bash
mkdir -p config
nano config/settings.json
```

Copy example settings from above, adjust `trackRegex` to match your file naming.

### 4. Start Services

```bash
docker-compose up -d
```

Wait for all services to become healthy:
```bash
docker-compose ps
```

### 5. Register Admin User

Navigate to `http://localhost` and register first user (becomes admin automatically).

### 6. Create Library

1. Go to Settings > Libraries
2. Click "Add Library"
3. Enter name and path (must match DATA_DIR mount)
4. Save

### 7. Trigger Initial Scan

```bash
curl -X POST http://localhost/scanner/scan
```

Monitor progress:
```bash
curl http://localhost/scanner/tasks
```

### 8. Wait for Enrichment

Matcher processes files asynchronously. Check progress in UI (Artists/Albums pages populate as metadata arrives).

## Updates

### Pull New Images

```bash
docker-compose pull
```

### Restart Services

```bash
docker-compose up -d
```

Docker Compose recreates containers with new images. Volumes persist data.

### Database Migrations

Prisma migrations run automatically on Server startup. No manual intervention needed.

## Backup

### Database Backup

```bash
docker exec meelo-db pg_dump -U postgres meelo > backup.sql
```

### Restore Database

```bash
docker exec -i meelo-db psql -U postgres meelo < backup.sql
```

### Volume Backup

```bash
docker run --rm -v meelo_db:/data -v $(pwd):/backup alpine tar czf /backup/db.tar.gz /data
```

### Restore Volume

```bash
docker run --rm -v meelo_db:/data -v $(pwd):/backup alpine tar xzf /backup/db.tar.gz -C /
```

### Config Backup

```bash
cp -r config config.backup
```

## Monitoring

### Service Status

```bash
docker-compose ps
```

Shows health status for all services.

### Logs

**All Services**:
```bash
docker-compose logs -f
```

**Specific Service**:
```bash
docker-compose logs -f server
```

**Last 100 Lines**:
```bash
docker-compose logs --tail=100 server
```

### Resource Usage

```bash
docker stats
```

Shows CPU, memory, network, and disk I/O per container.

## Troubleshooting

### Service Won't Start

Check logs:
```bash
docker-compose logs <service>
```

Common issues:
- **Database connection failed**: PostgreSQL not healthy yet, wait longer
- **Port already in use**: Change port in .env
- **Volume mount failed**: Check DATA_DIR path exists and has correct permissions

### Health Check Failing

Increase start period in docker-compose.yml:
```yaml
healthcheck:
  start_period: 60s  # Increase from 40s
```

### Out of Memory

Increase Docker memory limit (Docker Desktop settings) or reduce concurrent services.

### Slow Performance

Check resource usage:
```bash
docker stats
```

Bottlenecks:
- **High CPU on Matcher**: Too many providers enabled, disable optional ones
- **High memory on MeiliSearch**: Large library, increase Docker memory
- **High I/O on Scanner**: Slow disk, use SSD

## Production Deployment

### Reverse Proxy

Use Nginx or Caddy as external reverse proxy:

```nginx
server {
    listen 443 ssl http2;
    server_name meelo.example.com;

    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;

    location / {
        proxy_pass http://localhost:80;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

### HTTPS

Use Let's Encrypt with Certbot:

```bash
certbot --nginx -d meelo.example.com
```

Or use Caddy (automatic HTTPS):

```
meelo.example.com {
    reverse_proxy localhost:80
}
```

### Firewall

Open only port 443 (HTTPS):
```bash
ufw allow 443/tcp
ufw enable
```

### Security Hardening

- Set `ALLOW_ANONYMOUS=0` in .env
- Use strong `JWT_SIGNATURE` (32+ random characters)
- Restrict `CORS_ORIGINS` to your domain
- Run Docker in rootless mode
- Enable Docker Content Trust

### Monitoring

Use Prometheus + Grafana (future enhancement, not built-in).

### Backups

Automate database backups with cron:

```bash
0 2 * * * docker exec meelo-db pg_dump -U postgres meelo > /backups/meelo-$(date +\%Y\%m\%d).sql
```

Rotate backups:
```bash
find /backups -name "meelo-*.sql" -mtime +30 -delete
```

## CI/CD

### GitHub Actions

Meelo uses GitHub Actions for CI/CD. Workflows per service:

**server.yml**:
```yaml
name: Server CI/CD

on:
  push:
    branches: [main]
    paths:
      - 'server/**'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 20
      - run: npm ci
        working-directory: server
      - run: npm run lint
        working-directory: server
      - run: npm test
        working-directory: server
      - uses: SonarSource/sonarcloud-github-action@master
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: docker/setup-buildx-action@v2
      - uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}
      - uses: docker/build-push-action@v4
        with:
          context: ./server
          push: true
          tags: arthichaud/meelo-server:latest
```

Similar workflows for scanner, matcher, front.

### Quality Gates

SonarCloud enforces:
- Code coverage > 80%
- No critical bugs
- No security vulnerabilities
- Maintainability rating A

Failing quality gates block merges.

## Scaling

### Horizontal Scaling

Run multiple instances of stateless services:

```yaml
services:
  scanner:
    image: arthichaud/meelo-scanner
    deploy:
      replicas: 3
```

Load balance with Nginx upstream:

```nginx
upstream scanner {
    server scanner_1:8133;
    server scanner_2:8133;
    server scanner_3:8133;
}

location /scanner/ {
    proxy_pass http://scanner;
}
```

### Vertical Scaling

Increase container resources:

```yaml
services:
  server:
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 4G
        reservations:
          cpus: '1'
          memory: 2G
```

## Summary

Meelo's deployment uses Docker Compose to orchestrate 8 services with health checks ensuring correct startup order. Three variants (production, development, local build) support different use cases. Configuration via .env and settings.json separates deployment and user preferences. Volumes persist data, bind mounts provide access to music files. First-time setup involves configuring environment, creating settings, starting services, registering admin, creating library, and triggering scan. Updates are simple (pull images, restart). Backups cover database, volumes, and config. Production deployment adds reverse proxy, HTTPS, firewall, and security hardening. CI/CD via GitHub Actions ensures quality. Scaling options include horizontal (multiple instances) and vertical (more resources).