a1f6701bac
- gRPC service with MusicBrainz provider - PostgreSQL schema with migrations - Service layer with database-first caching - Repository pattern for data access - YAML configuration support - Research documentation for 17 music metadata projects
923 lines
24 KiB
Markdown
923 lines
24 KiB
Markdown
# Melodee: Deployment Analysis
|
|
|
|
## Deployment Strategy Overview
|
|
|
|
Melodee provides Docker-based deployment with multi-stage builds, Docker Compose orchestration, and automatic database migrations. The deployment architecture prioritizes ease of setup for self-hosted environments while supporting advanced configurations for production deployments.
|
|
|
|
Key deployment features:
|
|
- **Docker multi-stage build**: Optimized image size and security
|
|
- **Docker Compose**: Single-command deployment with PostgreSQL
|
|
- **Automatic migrations**: Database schema updates on container startup
|
|
- **12 persistent volumes**: Data persistence across container restarts
|
|
- **Raspberry Pi support**: ARM64 compatibility for low-power hardware
|
|
- **Podman compatibility**: Rootless container runtime support
|
|
|
|
## Docker Architecture
|
|
|
|
### Multi-Stage Dockerfile
|
|
|
|
```dockerfile
|
|
# Build stage
|
|
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
|
|
WORKDIR /src
|
|
|
|
# Copy project files
|
|
COPY ["Melodee.Web/Melodee.Web.csproj", "Melodee.Web/"]
|
|
COPY ["Melodee.Data/Melodee.Data.csproj", "Melodee.Data/"]
|
|
COPY ["Melodee.Core/Melodee.Core.csproj", "Melodee.Core/"]
|
|
|
|
# Restore dependencies
|
|
RUN dotnet restore "Melodee.Web/Melodee.Web.csproj"
|
|
|
|
# Copy source code
|
|
COPY . .
|
|
|
|
# Build application
|
|
WORKDIR "/src/Melodee.Web"
|
|
RUN dotnet build "Melodee.Web.csproj" -c Release -o /app/build
|
|
|
|
# Publish application
|
|
RUN dotnet publish "Melodee.Web.csproj" -c Release -o /app/publish /p:UseAppHost=false
|
|
|
|
# Runtime stage
|
|
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS runtime
|
|
WORKDIR /app
|
|
|
|
# Install FFmpeg for transcoding
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends ffmpeg && \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
# Copy published application
|
|
COPY --from=build /app/publish .
|
|
|
|
# Copy entrypoint script
|
|
COPY entrypoint.sh .
|
|
RUN chmod +x entrypoint.sh
|
|
|
|
# Expose port
|
|
EXPOSE 5000
|
|
|
|
# Set entrypoint
|
|
ENTRYPOINT ["./entrypoint.sh"]
|
|
```
|
|
|
|
**Multi-Stage Benefits**:
|
|
1. **Smaller image size**: Runtime image excludes SDK (saves ~500 MB)
|
|
2. **Faster deployments**: Smaller images transfer and start faster
|
|
3. **Security**: No build tools in production image
|
|
4. **Layer caching**: Dependencies cached separately from source code
|
|
|
|
**Image Size Comparison**:
|
|
- Single-stage (with SDK): ~1.2 GB
|
|
- Multi-stage (runtime only): ~700 MB
|
|
- Savings: ~500 MB (42% reduction)
|
|
|
|
### Entrypoint Script
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
set -e
|
|
|
|
echo "Melodee v1.8.0 starting..."
|
|
|
|
# Wait for PostgreSQL to be ready
|
|
echo "Waiting for PostgreSQL..."
|
|
until PGPASSWORD=$POSTGRES_PASSWORD psql -h "$POSTGRES_HOST" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c '\q' 2>/dev/null; do
|
|
echo "PostgreSQL is unavailable - sleeping"
|
|
sleep 2
|
|
done
|
|
|
|
echo "PostgreSQL is ready"
|
|
|
|
# Run database migrations
|
|
echo "Applying database migrations..."
|
|
dotnet ef database update --project /app/Melodee.Data.dll --no-build
|
|
|
|
if [ $? -ne 0 ]; then
|
|
echo "Migration failed, exiting..."
|
|
exit 1
|
|
fi
|
|
|
|
echo "Migrations applied successfully"
|
|
|
|
# Start application
|
|
echo "Starting Melodee..."
|
|
exec dotnet Melodee.Web.dll
|
|
```
|
|
|
|
**Entrypoint Responsibilities**:
|
|
1. **Database readiness check**: Waits for PostgreSQL before starting
|
|
2. **Automatic migrations**: Applies schema changes on startup
|
|
3. **Error handling**: Exits if migrations fail
|
|
4. **Process replacement**: `exec` replaces shell with .NET process for proper signal handling
|
|
|
|
**Signal Handling**:
|
|
The `exec` command is critical for graceful shutdown. Without it:
|
|
- Docker sends SIGTERM to shell process
|
|
- Shell doesn't forward signal to .NET process
|
|
- .NET process killed with SIGKILL after timeout
|
|
- No graceful shutdown (connections dropped, jobs interrupted)
|
|
|
|
With `exec`:
|
|
- Docker sends SIGTERM directly to .NET process
|
|
- .NET process handles shutdown gracefully
|
|
- Connections closed cleanly
|
|
- Background jobs complete or checkpoint
|
|
|
|
### Docker Compose Configuration
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
melodee:
|
|
image: melodee:1.8.0
|
|
container_name: melodee
|
|
restart: unless-stopped
|
|
ports:
|
|
- "5000:5000"
|
|
environment:
|
|
- ASPNETCORE_ENVIRONMENT=Production
|
|
- ASPNETCORE_URLS=http://+:5000
|
|
- ConnectionStrings__DefaultConnection=Host=postgres;Database=melodee;Username=melodee;Password=${POSTGRES_PASSWORD}
|
|
- MusicBrainz__CachePath=/data/mb-cache.db
|
|
- Library__Path=/music
|
|
- Spotify__ClientId=${SPOTIFY_CLIENT_ID}
|
|
- Spotify__ClientSecret=${SPOTIFY_CLIENT_SECRET}
|
|
- LastFm__ApiKey=${LASTFM_API_KEY}
|
|
- LastFm__SharedSecret=${LASTFM_SHARED_SECRET}
|
|
- Google__ClientId=${GOOGLE_CLIENT_ID}
|
|
- Google__ClientSecret=${GOOGLE_CLIENT_SECRET}
|
|
- Brave__ApiKey=${BRAVE_API_KEY}
|
|
volumes:
|
|
- music:/music
|
|
- data:/data
|
|
- logs:/var/log/melodee
|
|
- config:/app/config
|
|
- cache:/app/cache
|
|
- album-art:/app/album-art
|
|
- transcoding:/app/transcoding
|
|
depends_on:
|
|
- postgres
|
|
networks:
|
|
- melodee-network
|
|
healthcheck:
|
|
test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 40s
|
|
|
|
postgres:
|
|
image: postgres:17
|
|
container_name: melodee-postgres
|
|
restart: unless-stopped
|
|
environment:
|
|
- POSTGRES_DB=melodee
|
|
- POSTGRES_USER=melodee
|
|
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
|
|
volumes:
|
|
- postgres-data:/var/lib/postgresql/data
|
|
- postgres-backups:/backups
|
|
networks:
|
|
- melodee-network
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U melodee"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
|
|
volumes:
|
|
music:
|
|
driver: local
|
|
driver_opts:
|
|
type: none
|
|
o: bind
|
|
device: /path/to/music/library
|
|
data:
|
|
driver: local
|
|
logs:
|
|
driver: local
|
|
config:
|
|
driver: local
|
|
cache:
|
|
driver: local
|
|
album-art:
|
|
driver: local
|
|
transcoding:
|
|
driver: local
|
|
postgres-data:
|
|
driver: local
|
|
postgres-backups:
|
|
driver: local
|
|
|
|
networks:
|
|
melodee-network:
|
|
driver: bridge
|
|
```
|
|
|
|
**Volume Breakdown**:
|
|
|
|
| Volume | Purpose | Size | Backup Priority |
|
|
|--------|---------|------|-----------------|
|
|
| `music` | User's music library | Varies (100GB-10TB) | Critical (user data) |
|
|
| `data` | MusicBrainz cache, app data | 2-5 GB | Medium (rebuildable) |
|
|
| `logs` | Application logs | 1-10 GB | Low (rotated) |
|
|
| `config` | User settings, API keys | <1 MB | Critical (secrets) |
|
|
| `cache` | Metadata cache | 100 MB-1 GB | Low (rebuildable) |
|
|
| `album-art` | Album cover images | 1-10 GB | Medium (re-downloadable) |
|
|
| `transcoding` | Temporary transcoded files | 1-5 GB | None (temporary) |
|
|
| `postgres-data` | PostgreSQL database | 1-10 GB | Critical (user data) |
|
|
| `postgres-backups` | Database backups | 5-50 GB | Critical (disaster recovery) |
|
|
|
|
**Environment Variables**:
|
|
|
|
| Variable | Purpose | Required | Default |
|
|
|----------|---------|----------|---------|
|
|
| `ASPNETCORE_ENVIRONMENT` | Runtime environment | No | Production |
|
|
| `ASPNETCORE_URLS` | Listening URLs | No | http://+:5000 |
|
|
| `ConnectionStrings__DefaultConnection` | PostgreSQL connection | Yes | - |
|
|
| `MusicBrainz__CachePath` | SQLite cache location | No | /data/mb-cache.db |
|
|
| `Library__Path` | Music library path | Yes | - |
|
|
| `Spotify__ClientId` | Spotify API credentials | No | - |
|
|
| `Spotify__ClientSecret` | Spotify API credentials | No | - |
|
|
| `LastFm__ApiKey` | Last.fm API credentials | No | - |
|
|
| `LastFm__SharedSecret` | Last.fm API credentials | No | - |
|
|
| `Google__ClientId` | Google OAuth credentials | No | - |
|
|
| `Google__ClientSecret` | Google OAuth credentials | No | - |
|
|
| `Brave__ApiKey` | Brave Search API key | No | - |
|
|
|
|
**Health Checks**:
|
|
- **Melodee**: HTTP GET to `/health` endpoint every 30 seconds
|
|
- **PostgreSQL**: `pg_isready` command every 10 seconds
|
|
|
|
Health checks enable:
|
|
- **Automatic restarts**: Container restarts if unhealthy
|
|
- **Load balancer integration**: Remove unhealthy instances from rotation
|
|
- **Monitoring alerts**: Trigger notifications on health check failures
|
|
|
|
### Environment File (.env)
|
|
|
|
```bash
|
|
# PostgreSQL
|
|
POSTGRES_PASSWORD=your-secure-password
|
|
|
|
# Spotify (optional)
|
|
SPOTIFY_CLIENT_ID=your-spotify-client-id
|
|
SPOTIFY_CLIENT_SECRET=your-spotify-client-secret
|
|
|
|
# Last.fm (optional)
|
|
LASTFM_API_KEY=your-lastfm-api-key
|
|
LASTFM_SHARED_SECRET=your-lastfm-shared-secret
|
|
|
|
# Google OAuth (optional)
|
|
GOOGLE_CLIENT_ID=your-google-client-id
|
|
GOOGLE_CLIENT_SECRET=your-google-client-secret
|
|
|
|
# Brave Search (optional)
|
|
BRAVE_API_KEY=your-brave-api-key
|
|
```
|
|
|
|
**Security Considerations**:
|
|
- `.env` file should be in `.gitignore`
|
|
- Use strong passwords (20+ characters, mixed case, numbers, symbols)
|
|
- Rotate API keys periodically
|
|
- Restrict file permissions: `chmod 600 .env`
|
|
|
|
## Deployment Scenarios
|
|
|
|
### Single-Server Deployment
|
|
|
|
**Hardware Requirements**:
|
|
- **CPU**: 2+ cores (4+ recommended)
|
|
- **RAM**: 4 GB minimum (8 GB recommended)
|
|
- **Storage**: 50 GB minimum (varies with library size)
|
|
- **Network**: 100 Mbps+ for streaming
|
|
|
|
**Deployment Steps**:
|
|
|
|
1. **Install Docker and Docker Compose**:
|
|
```bash
|
|
# Ubuntu/Debian
|
|
sudo apt-get update
|
|
sudo apt-get install -y docker.io docker-compose
|
|
|
|
# Enable Docker service
|
|
sudo systemctl enable docker
|
|
sudo systemctl start docker
|
|
```
|
|
|
|
2. **Clone repository or create docker-compose.yml**:
|
|
```bash
|
|
mkdir melodee
|
|
cd melodee
|
|
# Create docker-compose.yml and .env files
|
|
```
|
|
|
|
3. **Configure environment variables**:
|
|
```bash
|
|
nano .env
|
|
# Set POSTGRES_PASSWORD and optional API keys
|
|
```
|
|
|
|
4. **Update music library path**:
|
|
```bash
|
|
# Edit docker-compose.yml
|
|
# Change device: /path/to/music/library to actual path
|
|
```
|
|
|
|
5. **Start services**:
|
|
```bash
|
|
docker-compose up -d
|
|
```
|
|
|
|
6. **Verify deployment**:
|
|
```bash
|
|
docker-compose ps
|
|
docker-compose logs -f melodee
|
|
curl http://localhost:5000/health
|
|
```
|
|
|
|
7. **Access web interface**:
|
|
```
|
|
http://localhost:5000
|
|
```
|
|
|
|
### Raspberry Pi Deployment
|
|
|
|
**Hardware Requirements**:
|
|
- **Model**: Raspberry Pi 4 (4GB+ RAM recommended)
|
|
- **Storage**: 64 GB+ microSD or USB SSD
|
|
- **OS**: Raspberry Pi OS 64-bit or Ubuntu Server ARM64
|
|
|
|
**ARM64 Image Build**:
|
|
```dockerfile
|
|
# Use ARM64 base images
|
|
FROM mcr.microsoft.com/dotnet/sdk:10.0-arm64v8 AS build
|
|
# ... build stage ...
|
|
|
|
FROM mcr.microsoft.com/dotnet/aspnet:10.0-arm64v8 AS runtime
|
|
# ... runtime stage ...
|
|
```
|
|
|
|
**Performance Optimizations**:
|
|
1. **Use SSD instead of microSD**: 10x faster I/O
|
|
2. **Disable transcoding**: Use direct streaming when possible
|
|
3. **Limit concurrent jobs**: Reduce background job parallelism
|
|
4. **Increase swap**: Add 2-4 GB swap for memory-intensive operations
|
|
|
|
**Deployment Steps**:
|
|
```bash
|
|
# Install Docker
|
|
curl -fsSL https://get.docker.com -o get-docker.sh
|
|
sudo sh get-docker.sh
|
|
|
|
# Add user to docker group
|
|
sudo usermod -aG docker $USER
|
|
|
|
# Install Docker Compose
|
|
sudo apt-get install -y docker-compose
|
|
|
|
# Deploy Melodee
|
|
docker-compose up -d
|
|
```
|
|
|
|
**Resource Limits**:
|
|
```yaml
|
|
services:
|
|
melodee:
|
|
# ... other config ...
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: '3'
|
|
memory: 3G
|
|
reservations:
|
|
cpus: '1'
|
|
memory: 1G
|
|
```
|
|
|
|
### Reverse Proxy Deployment
|
|
|
|
**Nginx Configuration**:
|
|
```nginx
|
|
upstream melodee {
|
|
server localhost:5000;
|
|
}
|
|
|
|
server {
|
|
listen 80;
|
|
server_name music.example.com;
|
|
|
|
# Redirect HTTP to HTTPS
|
|
return 301 https://$server_name$request_uri;
|
|
}
|
|
|
|
server {
|
|
listen 443 ssl http2;
|
|
server_name music.example.com;
|
|
|
|
# SSL certificates
|
|
ssl_certificate /etc/letsencrypt/live/music.example.com/fullchain.pem;
|
|
ssl_certificate_key /etc/letsencrypt/live/music.example.com/privkey.pem;
|
|
|
|
# SSL configuration
|
|
ssl_protocols TLSv1.2 TLSv1.3;
|
|
ssl_ciphers HIGH:!aNULL:!MD5;
|
|
ssl_prefer_server_ciphers on;
|
|
|
|
# Proxy settings
|
|
location / {
|
|
proxy_pass http://melodee;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
|
|
# Timeouts for streaming
|
|
proxy_read_timeout 3600s;
|
|
proxy_send_timeout 3600s;
|
|
}
|
|
|
|
# Increase max upload size for album art
|
|
client_max_body_size 50M;
|
|
}
|
|
```
|
|
|
|
**Traefik Configuration** (Docker labels):
|
|
```yaml
|
|
services:
|
|
melodee:
|
|
# ... other config ...
|
|
labels:
|
|
- "traefik.enable=true"
|
|
- "traefik.http.routers.melodee.rule=Host(`music.example.com`)"
|
|
- "traefik.http.routers.melodee.entrypoints=websecure"
|
|
- "traefik.http.routers.melodee.tls.certresolver=letsencrypt"
|
|
- "traefik.http.services.melodee.loadbalancer.server.port=5000"
|
|
```
|
|
|
|
### High Availability Deployment
|
|
|
|
**Architecture**:
|
|
```
|
|
┌─────────────┐
|
|
│ Load Balancer│
|
|
│ (HAProxy) │
|
|
└──────┬───────┘
|
|
│
|
|
┌──────────────────┼──────────────────┐
|
|
│ │ │
|
|
┌────▼────┐ ┌────▼────┐ ┌────▼────┐
|
|
│Melodee 1│ │Melodee 2│ │Melodee 3│
|
|
└────┬────┘ └────┬────┘ └────┬────┘
|
|
│ │ │
|
|
└──────────────────┼──────────────────┘
|
|
│
|
|
┌──────▼───────┐
|
|
│ PostgreSQL │
|
|
│ Primary │
|
|
└──────┬───────┘
|
|
│
|
|
┌──────▼───────┐
|
|
│ PostgreSQL │
|
|
│ Replica │
|
|
└──────────────┘
|
|
```
|
|
|
|
**Challenges**:
|
|
1. **Blazor Server state**: SignalR connections tied to specific server
|
|
2. **Session affinity**: Load balancer must route user to same server
|
|
3. **Shared storage**: Music library and album art must be accessible to all instances
|
|
|
|
**Solutions**:
|
|
|
|
**1. Redis Backplane for SignalR**:
|
|
```csharp
|
|
services.AddSignalR()
|
|
.AddStackExchangeRedis(options =>
|
|
{
|
|
options.Configuration.EndPoints.Add("redis:6379");
|
|
});
|
|
```
|
|
|
|
**2. HAProxy Sticky Sessions**:
|
|
```
|
|
backend melodee
|
|
balance roundrobin
|
|
cookie SERVERID insert indirect nocache
|
|
server melodee1 melodee1:5000 check cookie melodee1
|
|
server melodee2 melodee2:5000 check cookie melodee2
|
|
server melodee3 melodee3:5000 check cookie melodee3
|
|
```
|
|
|
|
**3. NFS for Shared Storage**:
|
|
```yaml
|
|
volumes:
|
|
music:
|
|
driver: local
|
|
driver_opts:
|
|
type: nfs
|
|
o: addr=nfs-server,rw
|
|
device: ":/music"
|
|
album-art:
|
|
driver: local
|
|
driver_opts:
|
|
type: nfs
|
|
o: addr=nfs-server,rw
|
|
device: ":/album-art"
|
|
```
|
|
|
|
**4. PostgreSQL Replication**:
|
|
```yaml
|
|
services:
|
|
postgres-primary:
|
|
image: postgres:17
|
|
environment:
|
|
- POSTGRES_REPLICATION_MODE=master
|
|
- POSTGRES_REPLICATION_USER=replicator
|
|
- POSTGRES_REPLICATION_PASSWORD=replicator-password
|
|
volumes:
|
|
- postgres-primary-data:/var/lib/postgresql/data
|
|
|
|
postgres-replica:
|
|
image: postgres:17
|
|
environment:
|
|
- POSTGRES_REPLICATION_MODE=slave
|
|
- POSTGRES_MASTER_HOST=postgres-primary
|
|
- POSTGRES_REPLICATION_USER=replicator
|
|
- POSTGRES_REPLICATION_PASSWORD=replicator-password
|
|
volumes:
|
|
- postgres-replica-data:/var/lib/postgresql/data
|
|
```
|
|
|
|
## Podman Deployment
|
|
|
|
Podman is a daemonless, rootless container runtime compatible with Docker.
|
|
|
|
**Advantages**:
|
|
- **Rootless**: Runs without root privileges
|
|
- **Daemonless**: No background daemon process
|
|
- **Systemd integration**: Native systemd service generation
|
|
|
|
**Deployment Steps**:
|
|
|
|
1. **Install Podman**:
|
|
```bash
|
|
# Ubuntu/Debian
|
|
sudo apt-get install -y podman podman-compose
|
|
|
|
# Fedora
|
|
sudo dnf install -y podman podman-compose
|
|
```
|
|
|
|
2. **Convert Docker Compose to Podman**:
|
|
```bash
|
|
# Podman Compose uses same syntax
|
|
podman-compose up -d
|
|
```
|
|
|
|
3. **Generate systemd service**:
|
|
```bash
|
|
# Generate service file for melodee container
|
|
podman generate systemd --new --name melodee > ~/.config/systemd/user/melodee.service
|
|
|
|
# Enable service
|
|
systemctl --user enable melodee.service
|
|
systemctl --user start melodee.service
|
|
```
|
|
|
|
**Rootless Considerations**:
|
|
- **Port binding**: Ports <1024 require root or `sysctl net.ipv4.ip_unprivileged_port_start=80`
|
|
- **Volume permissions**: Ensure user has read/write access to volume paths
|
|
- **Resource limits**: Rootless containers have lower default limits
|
|
|
|
## Backup and Recovery
|
|
|
|
### Database Backup
|
|
|
|
**Automated Daily Backups**:
|
|
```bash
|
|
#!/bin/bash
|
|
BACKUP_DIR="/backups/postgres"
|
|
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
|
BACKUP_FILE="$BACKUP_DIR/melodee_$TIMESTAMP.sql.gz"
|
|
|
|
# Create backup
|
|
docker exec melodee-postgres pg_dump -U melodee melodee | gzip > $BACKUP_FILE
|
|
|
|
# Verify backup
|
|
if [ $? -eq 0 ]; then
|
|
echo "Backup successful: $BACKUP_FILE"
|
|
else
|
|
echo "Backup failed"
|
|
exit 1
|
|
fi
|
|
|
|
# Retain last 30 days
|
|
find $BACKUP_DIR -name "melodee_*.sql.gz" -mtime +30 -delete
|
|
|
|
# Upload to S3 (optional)
|
|
aws s3 cp $BACKUP_FILE s3://melodee-backups/postgres/
|
|
```
|
|
|
|
**Cron Schedule**:
|
|
```cron
|
|
0 2 * * * /usr/local/bin/backup-melodee.sh
|
|
```
|
|
|
|
**Restore from Backup**:
|
|
```bash
|
|
# Stop Melodee
|
|
docker-compose stop melodee
|
|
|
|
# Restore database
|
|
gunzip -c /backups/postgres/melodee_20250428_020000.sql.gz | \
|
|
docker exec -i melodee-postgres psql -U melodee melodee
|
|
|
|
# Start Melodee
|
|
docker-compose start melodee
|
|
```
|
|
|
|
### Volume Backup
|
|
|
|
**Backup Script**:
|
|
```bash
|
|
#!/bin/bash
|
|
BACKUP_DIR="/backups/volumes"
|
|
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
|
|
|
|
# Backup config volume (contains API keys)
|
|
docker run --rm \
|
|
-v melodee_config:/data \
|
|
-v $BACKUP_DIR:/backup \
|
|
alpine tar czf /backup/config_$TIMESTAMP.tar.gz -C /data .
|
|
|
|
# Backup data volume (MusicBrainz cache)
|
|
docker run --rm \
|
|
-v melodee_data:/data \
|
|
-v $BACKUP_DIR:/backup \
|
|
alpine tar czf /backup/data_$TIMESTAMP.tar.gz -C /data .
|
|
|
|
# Backup album-art volume
|
|
docker run --rm \
|
|
-v melodee_album-art:/data \
|
|
-v $BACKUP_DIR:/backup \
|
|
alpine tar czf /backup/album-art_$TIMESTAMP.tar.gz -C /data .
|
|
```
|
|
|
|
**Restore Volumes**:
|
|
```bash
|
|
# Restore config volume
|
|
docker run --rm \
|
|
-v melodee_config:/data \
|
|
-v $BACKUP_DIR:/backup \
|
|
alpine tar xzf /backup/config_20250428_020000.tar.gz -C /data
|
|
|
|
# Restore data volume
|
|
docker run --rm \
|
|
-v melodee_data:/data \
|
|
-v $BACKUP_DIR:/backup \
|
|
alpine tar xzf /backup/data_20250428_020000.tar.gz -C /data
|
|
```
|
|
|
|
### Disaster Recovery
|
|
|
|
**Full System Recovery**:
|
|
|
|
1. **Install Docker and Docker Compose** on new server
|
|
2. **Restore docker-compose.yml and .env** files
|
|
3. **Create volumes**:
|
|
```bash
|
|
docker volume create melodee_config
|
|
docker volume create melodee_data
|
|
docker volume create melodee_album-art
|
|
docker volume create melodee_postgres-data
|
|
```
|
|
|
|
4. **Restore volume data** from backups
|
|
5. **Restore PostgreSQL database** from backup
|
|
6. **Start services**:
|
|
```bash
|
|
docker-compose up -d
|
|
```
|
|
|
|
7. **Verify health**:
|
|
```bash
|
|
docker-compose ps
|
|
curl http://localhost:5000/health
|
|
```
|
|
|
|
**Recovery Time Objective (RTO)**: 1-2 hours
|
|
**Recovery Point Objective (RPO)**: 24 hours (daily backups)
|
|
|
|
## Monitoring and Logging
|
|
|
|
### Prometheus Metrics
|
|
|
|
**Metrics Endpoint**:
|
|
```csharp
|
|
app.UseEndpoints(endpoints =>
|
|
{
|
|
endpoints.MapMetrics("/metrics");
|
|
});
|
|
```
|
|
|
|
**Prometheus Configuration**:
|
|
```yaml
|
|
scrape_configs:
|
|
- job_name: 'melodee'
|
|
static_configs:
|
|
- targets: ['melodee:5000']
|
|
metrics_path: '/metrics'
|
|
scrape_interval: 15s
|
|
```
|
|
|
|
**Key Metrics**:
|
|
- `http_requests_total`: Total HTTP requests
|
|
- `http_request_duration_seconds`: Request latency
|
|
- `dotnet_gc_collections_total`: Garbage collection count
|
|
- `process_cpu_seconds_total`: CPU usage
|
|
- `process_resident_memory_bytes`: Memory usage
|
|
- `melodee_scrobbles_total`: Total scrobbles submitted
|
|
- `melodee_library_tracks_total`: Total tracks in library
|
|
|
|
### Grafana Dashboard
|
|
|
|
**Dashboard Panels**:
|
|
1. **Request Rate**: Requests per second
|
|
2. **Response Time**: P50, P95, P99 latencies
|
|
3. **Error Rate**: 4xx and 5xx responses
|
|
4. **CPU Usage**: Process CPU percentage
|
|
5. **Memory Usage**: Resident memory
|
|
6. **Database Connections**: Active connections
|
|
7. **Scrobble Rate**: Scrobbles per hour
|
|
8. **Library Size**: Total tracks, albums, artists
|
|
|
|
### Log Aggregation
|
|
|
|
**Serilog to Elasticsearch**:
|
|
```csharp
|
|
Log.Logger = new LoggerConfiguration()
|
|
.WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://elasticsearch:9200"))
|
|
{
|
|
AutoRegisterTemplate = true,
|
|
IndexFormat = "melodee-logs-{0:yyyy.MM.dd}"
|
|
})
|
|
.CreateLogger();
|
|
```
|
|
|
|
**Kibana Queries**:
|
|
```
|
|
# Errors in last hour
|
|
level:Error AND @timestamp:[now-1h TO now]
|
|
|
|
# Slow requests (>1s)
|
|
http.request.duration:>1000
|
|
|
|
# Failed scrobbles
|
|
message:"scrobble failed"
|
|
```
|
|
|
|
## Security Hardening
|
|
|
|
### HTTPS Configuration
|
|
|
|
**Let's Encrypt with Certbot**:
|
|
```bash
|
|
# Install Certbot
|
|
sudo apt-get install -y certbot
|
|
|
|
# Obtain certificate
|
|
sudo certbot certonly --standalone -d music.example.com
|
|
|
|
# Configure Nginx with certificate (see Reverse Proxy section)
|
|
```
|
|
|
|
**Certificate Renewal**:
|
|
```cron
|
|
0 0 1 * * certbot renew --quiet && systemctl reload nginx
|
|
```
|
|
|
|
### Firewall Configuration
|
|
|
|
**UFW (Ubuntu)**:
|
|
```bash
|
|
# Allow SSH
|
|
sudo ufw allow 22/tcp
|
|
|
|
# Allow HTTP/HTTPS (if using reverse proxy)
|
|
sudo ufw allow 80/tcp
|
|
sudo ufw allow 443/tcp
|
|
|
|
# Allow Melodee (if direct access)
|
|
sudo ufw allow 5000/tcp
|
|
|
|
# Enable firewall
|
|
sudo ufw enable
|
|
```
|
|
|
|
### Secret Management
|
|
|
|
**Docker Secrets** (Swarm mode):
|
|
```yaml
|
|
services:
|
|
melodee:
|
|
secrets:
|
|
- postgres_password
|
|
- spotify_client_secret
|
|
environment:
|
|
- ConnectionStrings__DefaultConnection=Host=postgres;Database=melodee;Username=melodee;Password_FILE=/run/secrets/postgres_password
|
|
|
|
secrets:
|
|
postgres_password:
|
|
file: ./secrets/postgres_password.txt
|
|
spotify_client_secret:
|
|
file: ./secrets/spotify_client_secret.txt
|
|
```
|
|
|
|
**Vault Integration**:
|
|
```csharp
|
|
var vaultClient = new VaultClient(new VaultClientSettings("http://vault:8200", "vault-token"));
|
|
var secret = await vaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync("melodee/postgres");
|
|
var password = secret.Data.Data["password"].ToString();
|
|
```
|
|
|
|
## Performance Tuning
|
|
|
|
### PostgreSQL Optimization
|
|
|
|
```sql
|
|
-- Increase shared buffers (25% of RAM)
|
|
ALTER SYSTEM SET shared_buffers = '2GB';
|
|
|
|
-- Increase work memory for complex queries
|
|
ALTER SYSTEM SET work_mem = '64MB';
|
|
|
|
-- Increase maintenance work memory for VACUUM
|
|
ALTER SYSTEM SET maintenance_work_mem = '512MB';
|
|
|
|
-- Optimize for SSD
|
|
ALTER SYSTEM SET random_page_cost = 1.1;
|
|
|
|
-- Enable query planning statistics
|
|
ALTER SYSTEM SET track_activity_query_size = 2048;
|
|
|
|
-- Reload configuration
|
|
SELECT pg_reload_conf();
|
|
```
|
|
|
|
### .NET Runtime Optimization
|
|
|
|
**Environment Variables**:
|
|
```yaml
|
|
environment:
|
|
- DOTNET_GCServer=1 # Server GC mode
|
|
- DOTNET_GCConcurrent=1 # Concurrent GC
|
|
- DOTNET_GCRetainVM=1 # Retain virtual memory
|
|
- DOTNET_ThreadPool_MinThreads=50 # Minimum thread pool size
|
|
- DOTNET_ThreadPool_MaxThreads=500 # Maximum thread pool size
|
|
```
|
|
|
|
### Caching Configuration
|
|
|
|
**Redis Cache**:
|
|
```yaml
|
|
services:
|
|
redis:
|
|
image: redis:7
|
|
command: redis-server --maxmemory 1gb --maxmemory-policy allkeys-lru
|
|
volumes:
|
|
- redis-data:/data
|
|
```
|
|
|
|
**Application Configuration**:
|
|
```csharp
|
|
services.AddStackExchangeRedisCache(options =>
|
|
{
|
|
options.Configuration = "redis:6379";
|
|
options.InstanceName = "melodee:";
|
|
});
|
|
```
|
|
|
|
## Conclusion
|
|
|
|
Melodee's deployment architecture demonstrates production-ready containerization with Docker multi-stage builds, automatic migrations, and comprehensive volume management. The 12 persistent volumes ensure data persistence, while health checks and logging enable robust monitoring.
|
|
|
|
Key strengths:
|
|
- **Easy deployment**: Single-command Docker Compose setup
|
|
- **Automatic migrations**: Database schema updates on startup
|
|
- **Raspberry Pi support**: ARM64 compatibility for low-power deployments
|
|
- **Podman compatibility**: Rootless container runtime support
|
|
|
|
Key challenges:
|
|
- **Horizontal scaling**: Blazor Server requires sticky sessions and Redis backplane
|
|
- **Backup complexity**: 12 volumes require coordinated backup strategy
|
|
- **Secret management**: API keys in environment variables (consider Vault)
|
|
|
|
The architecture positions Melodee for both simple self-hosted deployments and advanced production configurations with high availability and monitoring.
|