a1f6701bac
- gRPC service with MusicBrainz provider - PostgreSQL schema with migrations - Service layer with database-first caching - Repository pattern for data access - YAML configuration support - Research documentation for 17 music metadata projects
704 lines
16 KiB
Markdown
704 lines
16 KiB
Markdown
# minim: Deployment
|
|
|
|
## Deployment Model
|
|
|
|
minim is a **Python library**, not a deployable service. There is no server, no daemon, no container to deploy. Users install the library and import it into their own Python code.
|
|
|
|
**Installation Target:** Developer workstations, scripts, Jupyter notebooks, personal automation tools.
|
|
|
|
**Not Applicable:** Production web servers, cloud deployments, containerized services, serverless functions.
|
|
|
|
## Installation Methods
|
|
|
|
### From Source (Current)
|
|
|
|
**Clone Repository:**
|
|
```bash
|
|
git clone https://github.com/bbye98/minim.git
|
|
cd minim
|
|
```
|
|
|
|
**Install in Development Mode:**
|
|
```bash
|
|
python -m pip install -e .
|
|
```
|
|
|
|
**Install in Production Mode:**
|
|
```bash
|
|
python -m pip install .
|
|
```
|
|
|
|
**Editable Install (`-e`):**
|
|
- Changes to source code immediately reflected without reinstalling
|
|
- Useful for development and testing
|
|
- Creates symlink to source directory
|
|
|
|
**Production Install:**
|
|
- Copies files to site-packages
|
|
- Requires reinstall after code changes
|
|
- Cleaner for end users
|
|
|
|
### Via Conda
|
|
|
|
**Environment File:**
|
|
```yaml
|
|
# environment.yml
|
|
name: minim
|
|
channels:
|
|
- conda-forge
|
|
- defaults
|
|
dependencies:
|
|
- python>=3.9
|
|
- cryptography
|
|
- mutagen
|
|
- requests
|
|
- pip
|
|
- pip:
|
|
- -e .
|
|
```
|
|
|
|
**Create Environment:**
|
|
```bash
|
|
conda env create -f environment.yml
|
|
conda activate minim
|
|
```
|
|
|
|
**Update Environment:**
|
|
```bash
|
|
conda env update -f environment.yml
|
|
```
|
|
|
|
### Via PyPI (Planned for v2)
|
|
|
|
**Not Yet Available.** minim is not published to PyPI as of v1.1.0.
|
|
|
|
**Planned for v2:**
|
|
```bash
|
|
pip install minim
|
|
```
|
|
|
|
**Package Metadata (setup.py):**
|
|
```python
|
|
from setuptools import setup, find_packages
|
|
|
|
setup(
|
|
name="minim",
|
|
version="1.1.0",
|
|
author="Benjamin Ye",
|
|
author_email="bbye98@gmail.com",
|
|
description="Comprehensive music metadata library",
|
|
long_description=open("README.md").read(),
|
|
long_description_content_type="text/markdown",
|
|
url="https://github.com/bbye98/minim",
|
|
packages=find_packages(),
|
|
classifiers=[
|
|
"Development Status :: 4 - Beta",
|
|
"Intended Audience :: Developers",
|
|
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
|
|
"Programming Language :: Python :: 3",
|
|
"Programming Language :: Python :: 3.9",
|
|
"Programming Language :: Python :: 3.10",
|
|
"Programming Language :: Python :: 3.11",
|
|
],
|
|
python_requires=">=3.9",
|
|
install_requires=[
|
|
"cryptography",
|
|
"mutagen",
|
|
"requests",
|
|
],
|
|
extras_require={
|
|
"full": [
|
|
"ffmpeg-python",
|
|
"flask",
|
|
"levenshtein",
|
|
"numpy",
|
|
"pillow",
|
|
"playwright",
|
|
],
|
|
},
|
|
)
|
|
```
|
|
|
|
## Dependencies
|
|
|
|
### Required (Core)
|
|
|
|
**cryptography:**
|
|
- Purpose: TIDAL manifest decryption, secure token handling
|
|
- Version: Not pinned (latest compatible)
|
|
- Install: `pip install cryptography`
|
|
|
|
**mutagen:**
|
|
- Purpose: Audio file metadata reading/writing
|
|
- Version: Not pinned
|
|
- Install: `pip install mutagen`
|
|
|
|
**requests:**
|
|
- Purpose: HTTP client for all API calls
|
|
- Version: Not pinned
|
|
- Install: `pip install requests`
|
|
|
|
### Optional (Features)
|
|
|
|
**ffmpeg:**
|
|
- Purpose: Audio format conversion
|
|
- Type: System binary (not Python package)
|
|
- Install: `apt install ffmpeg` (Ubuntu), `brew install ffmpeg` (macOS), download from ffmpeg.org (Windows)
|
|
- Detection: `shutil.which("ffmpeg")`
|
|
|
|
**flask:**
|
|
- Purpose: OAuth callback server (alternative to http.server)
|
|
- Install: `pip install flask`
|
|
|
|
**levenshtein:**
|
|
- Purpose: Fuzzy string matching for search results
|
|
- Install: `pip install levenshtein`
|
|
|
|
**numpy:**
|
|
- Purpose: Audio analysis features
|
|
- Install: `pip install numpy`
|
|
|
|
**pillow:**
|
|
- Purpose: Image processing for album artwork
|
|
- Install: `pip install pillow`
|
|
|
|
**playwright:**
|
|
- Purpose: Browser automation for OAuth flows
|
|
- Install: `pip install playwright && playwright install chromium`
|
|
|
|
### Dependency Management
|
|
|
|
**No Lock File:** minim does not use `requirements.txt` or `Pipfile.lock` for version pinning.
|
|
|
|
**Version Constraints:** None specified. Uses latest compatible versions.
|
|
|
|
**Risk:** Dependency updates may introduce breaking changes.
|
|
|
|
**Recommendation for Production:**
|
|
```bash
|
|
# Generate lock file
|
|
pip freeze > requirements.txt
|
|
|
|
# Install from lock file
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## System Requirements
|
|
|
|
### Python Version
|
|
|
|
**Minimum:** Python 3.9
|
|
**Tested:** Python 3.9, 3.10, 3.11
|
|
**Recommended:** Python 3.11 (latest stable)
|
|
|
|
**Version-Specific Features:**
|
|
- Type hints (PEP 585): `list[str]` instead of `List[str]` (requires 3.9+)
|
|
- Union operator: `str | None` instead of `Optional[str]` (requires 3.10+, not used in v1)
|
|
|
|
### Operating Systems
|
|
|
|
**Supported:**
|
|
- Linux (Ubuntu 20.04+, Debian 11+, Fedora 35+, Arch)
|
|
- macOS (10.15 Catalina+)
|
|
- Windows (10, 11)
|
|
|
|
**Tested in CI:** Ubuntu 22.04 (GitHub Actions)
|
|
|
|
**Platform-Specific Considerations:**
|
|
|
|
**Linux:**
|
|
- FFmpeg available via package manager (`apt`, `dnf`, `pacman`)
|
|
- Config file at `~/.minim.cfg` or `/home/username/minim.cfg`
|
|
|
|
**macOS:**
|
|
- FFmpeg via Homebrew (`brew install ffmpeg`)
|
|
- Config file at `~/minim.cfg` or `/Users/username/minim.cfg`
|
|
|
|
**Windows:**
|
|
- FFmpeg requires manual download and PATH configuration
|
|
- Config file at `C:\Users\username\minim.cfg`
|
|
- Path handling uses `os.path.expanduser("~")` (cross-platform)
|
|
|
|
### External Dependencies
|
|
|
|
**FFmpeg (Optional):**
|
|
- Required for audio format conversion
|
|
- Not required for metadata reading/writing or API access
|
|
- Version: 4.0+ recommended
|
|
|
|
**Browser (Optional):**
|
|
- Required for OAuth flows using Playwright
|
|
- Chromium installed via `playwright install chromium`
|
|
- Not required for http.server or Flask callback methods
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
minim checks environment variables for credentials:
|
|
|
|
**Discogs:**
|
|
- `DISCOGS_CONSUMER_KEY`
|
|
- `DISCOGS_CONSUMER_SECRET`
|
|
- `DISCOGS_ACCESS_TOKEN`
|
|
- `DISCOGS_ACCESS_TOKEN_SECRET`
|
|
- `DISCOGS_PERSONAL_ACCESS_TOKEN`
|
|
|
|
**Qobuz:**
|
|
- `QOBUZ_APP_ID`
|
|
- `QOBUZ_APP_SECRET`
|
|
- `QOBUZ_EMAIL`
|
|
- `QOBUZ_PASSWORD`
|
|
|
|
**Spotify:**
|
|
- `SPOTIFY_CLIENT_ID`
|
|
- `SPOTIFY_CLIENT_SECRET`
|
|
- `SPOTIFY_REDIRECT_URI`
|
|
|
|
**TIDAL:**
|
|
- `TIDAL_CLIENT_ID`
|
|
- `TIDAL_CLIENT_SECRET`
|
|
- `TIDAL_REDIRECT_URI`
|
|
|
|
**Precedence:** Environment variables > config file > constructor parameters
|
|
|
|
**Use Case:** CI/CD pipelines, containerized environments, shared systems
|
|
|
|
**Example:**
|
|
```bash
|
|
export SPOTIFY_CLIENT_ID="abc123"
|
|
export SPOTIFY_CLIENT_SECRET="def456"
|
|
|
|
python script.py # Automatically uses environment variables
|
|
```
|
|
|
|
### Config File
|
|
|
|
**Location:** `~/minim.cfg`
|
|
|
|
**Format:** INI-style (ConfigParser)
|
|
|
|
**Auto-Creation:** Created automatically when tokens are saved via `set_access_token()`
|
|
|
|
**Manual Creation:**
|
|
```ini
|
|
[spotify]
|
|
client_id = abc123
|
|
client_secret = def456
|
|
access_token = BQC...
|
|
refresh_token = AQD...
|
|
expires_at = 1672531200
|
|
```
|
|
|
|
**Permissions:** Default (0644 on Unix). Recommendation: `chmod 600 ~/minim.cfg` for security.
|
|
|
|
## CI/CD
|
|
|
|
### GitHub Actions
|
|
|
|
**Workflow File:** `.github/workflows/ci.yml`
|
|
|
|
```yaml
|
|
name: CI
|
|
|
|
on:
|
|
push:
|
|
branches: [main, dev]
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- uses: actions/checkout@v3
|
|
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v4
|
|
with:
|
|
python-version: '3.9'
|
|
|
|
- name: Install FFmpeg
|
|
run: sudo apt-get update && sudo apt-get install -y ffmpeg
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
python -m pip install --upgrade pip
|
|
pip install -e .
|
|
pip install pytest ruff
|
|
|
|
- name: Lint with ruff
|
|
run: ruff check .
|
|
|
|
- name: Run tests
|
|
env:
|
|
SPOTIFY_CLIENT_ID: ${{ secrets.SPOTIFY_CLIENT_ID }}
|
|
SPOTIFY_CLIENT_SECRET: ${{ secrets.SPOTIFY_CLIENT_SECRET }}
|
|
TIDAL_CLIENT_ID: ${{ secrets.TIDAL_CLIENT_ID }}
|
|
TIDAL_CLIENT_SECRET: ${{ secrets.TIDAL_CLIENT_SECRET }}
|
|
run: pytest tests/
|
|
```
|
|
|
|
**Secrets Management:**
|
|
- API credentials stored in GitHub Secrets
|
|
- Accessed via `${{ secrets.SECRET_NAME }}`
|
|
- Not exposed in logs
|
|
|
|
**Test Execution:**
|
|
- Real API calls (not mocked)
|
|
- Requires valid credentials
|
|
- May fail if rate limits exceeded or services change APIs
|
|
|
|
**Linting:**
|
|
- `ruff`: Fast Python linter (replaces flake8, pylint)
|
|
- Configuration in `pyproject.toml` or `ruff.toml`
|
|
|
|
### Coverage
|
|
|
|
**Tool:** `coverage.py`
|
|
|
|
**Configuration:** `.coveragerc`
|
|
|
|
```ini
|
|
[run]
|
|
source = minim
|
|
omit =
|
|
*/tests/*
|
|
*/__init__.py
|
|
*/site-packages/*
|
|
|
|
[report]
|
|
exclude_lines =
|
|
pragma: no cover
|
|
def __repr__
|
|
raise AssertionError
|
|
raise NotImplementedError
|
|
if __name__ == .__main__.:
|
|
```
|
|
|
|
**Execution:**
|
|
```bash
|
|
coverage run -m pytest tests/
|
|
coverage report
|
|
coverage html # Generate HTML report
|
|
```
|
|
|
|
**Current Coverage:** Not documented in repository. Likely 60-80% based on test file count.
|
|
|
|
## Documentation
|
|
|
|
### ReadTheDocs
|
|
|
|
**URL:** https://minim.readthedocs.io
|
|
|
|
**Build System:** Sphinx
|
|
|
|
**Configuration:** `docs/conf.py`
|
|
|
|
```python
|
|
project = 'minim'
|
|
author = 'Benjamin Ye'
|
|
release = '1.1.0'
|
|
|
|
extensions = [
|
|
'sphinx.ext.autodoc',
|
|
'sphinx.ext.napoleon',
|
|
'sphinx.ext.viewcode',
|
|
]
|
|
|
|
html_theme = 'sphinx_rtd_theme'
|
|
```
|
|
|
|
**Auto-Deploy:**
|
|
- Triggered on push to `main` branch
|
|
- Builds from `docs/` directory
|
|
- Parses docstrings from source code
|
|
|
|
**Docstring Format:** Google-style
|
|
|
|
```python
|
|
def search(self, query: str, types: list[str] = ["track"]) -> dict:
|
|
"""
|
|
Search Spotify catalog.
|
|
|
|
Args:
|
|
query: Search query string
|
|
types: Result types (track, album, artist, playlist)
|
|
|
|
Returns:
|
|
Dict with type-specific results arrays
|
|
|
|
Raises:
|
|
RuntimeError: If API request fails
|
|
|
|
Example:
|
|
>>> api = WebAPI(client_id="...", client_secret="...")
|
|
>>> results = api.search("Radiohead", types=["artist"])
|
|
>>> print(results["artists"]["items"][0]["name"])
|
|
Radiohead
|
|
"""
|
|
```
|
|
|
|
### Local Documentation Build
|
|
|
|
```bash
|
|
cd docs
|
|
pip install sphinx sphinx_rtd_theme
|
|
make html
|
|
open _build/html/index.html
|
|
```
|
|
|
|
## Versioning
|
|
|
|
**Scheme:** Semantic Versioning (SemVer)
|
|
|
|
**Format:** `MAJOR.MINOR.PATCH`
|
|
|
|
**Current Version:** 1.1.0
|
|
|
|
**Version History:**
|
|
- 1.0.0: Initial release
|
|
- 1.1.0: Bug fixes, minor feature additions
|
|
|
|
**Version Location:** `minim/__init__.py`
|
|
|
|
```python
|
|
__version__ = "1.1.0"
|
|
```
|
|
|
|
**Git Tags:**
|
|
```bash
|
|
git tag v1.1.0
|
|
git push origin v1.1.0
|
|
```
|
|
|
|
## Release Process
|
|
|
|
**Current (Manual):**
|
|
|
|
1. Update version in `minim/__init__.py`
|
|
2. Update `CHANGELOG.md` (if exists)
|
|
3. Commit changes: `git commit -m "Bump version to 1.1.0"`
|
|
4. Create tag: `git tag v1.1.0`
|
|
5. Push: `git push origin main --tags`
|
|
6. GitHub automatically triggers ReadTheDocs build
|
|
|
|
**Planned for v2 (Automated):**
|
|
|
|
1. Create release branch: `git checkout -b release/1.2.0`
|
|
2. Update version and changelog
|
|
3. Open pull request
|
|
4. Merge to main
|
|
5. GitHub Actions workflow:
|
|
- Run tests
|
|
- Build package: `python -m build`
|
|
- Publish to PyPI: `twine upload dist/*`
|
|
- Create GitHub release with changelog
|
|
- Trigger ReadTheDocs build
|
|
|
|
## Distribution Channels
|
|
|
|
### Current
|
|
|
|
**GitHub Releases:**
|
|
- Source code archives (`.tar.gz`, `.zip`)
|
|
- No pre-built binaries
|
|
- Download: https://github.com/bbye98/minim/releases
|
|
|
|
**ReadTheDocs:**
|
|
- Documentation only
|
|
- No package distribution
|
|
|
|
### Planned (v2)
|
|
|
|
**PyPI:**
|
|
- `pip install minim`
|
|
- Versioned releases
|
|
- Automatic dependency resolution
|
|
|
|
**Conda-Forge:**
|
|
- `conda install -c conda-forge minim`
|
|
- Cross-platform binaries
|
|
- Dependency management via conda
|
|
|
|
## Containerization
|
|
|
|
**Not Applicable:** minim is a library, not a service.
|
|
|
|
**Hypothetical Use Case:** Containerized script using minim
|
|
|
|
**Dockerfile:**
|
|
```dockerfile
|
|
FROM python:3.11-slim
|
|
|
|
# Install FFmpeg
|
|
RUN apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/*
|
|
|
|
# Install minim
|
|
WORKDIR /app
|
|
COPY . .
|
|
RUN pip install --no-cache-dir -e .
|
|
|
|
# Run script
|
|
CMD ["python", "script.py"]
|
|
```
|
|
|
|
**Docker Compose:**
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
minim-script:
|
|
build: .
|
|
environment:
|
|
- SPOTIFY_CLIENT_ID=${SPOTIFY_CLIENT_ID}
|
|
- SPOTIFY_CLIENT_SECRET=${SPOTIFY_CLIENT_SECRET}
|
|
volumes:
|
|
- ./audio:/app/audio
|
|
- ./config:/root/.minim.cfg
|
|
```
|
|
|
|
## Monitoring and Logging
|
|
|
|
**Not Applicable:** minim is a library. Monitoring and logging are the responsibility of the calling application.
|
|
|
|
**Library Behavior:**
|
|
- No built-in logging (uses `warnings` module for non-critical issues)
|
|
- Errors raised as exceptions (caller handles logging)
|
|
- No metrics, no telemetry, no health checks
|
|
|
|
**Caller Responsibility:**
|
|
|
|
```python
|
|
import logging
|
|
from minim import spotify
|
|
|
|
# Configure logging
|
|
logging.basicConfig(level=logging.INFO)
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# Use minim with logging
|
|
try:
|
|
api = spotify.WebAPI(client_id="...", client_secret="...")
|
|
api.set_flow("client_credentials")
|
|
api.set_access_token()
|
|
|
|
results = api.search("Radiohead", types=["artist"])
|
|
logger.info(f"Found {len(results['artists']['items'])} artists")
|
|
|
|
except RuntimeError as e:
|
|
logger.error(f"API error: {e}")
|
|
except Exception as e:
|
|
logger.exception(f"Unexpected error: {e}")
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
### Credential Storage
|
|
|
|
**Risk:** Plain text tokens in `~/minim.cfg`
|
|
|
|
**Mitigation (Not Implemented):**
|
|
- Encrypt config file
|
|
- Use OS keychain (Keyring library)
|
|
- Use environment variables only
|
|
- Set restrictive file permissions (`chmod 600`)
|
|
|
|
### Dependency Vulnerabilities
|
|
|
|
**Risk:** Outdated dependencies with known CVEs
|
|
|
|
**Mitigation:**
|
|
```bash
|
|
# Scan for vulnerabilities
|
|
pip install safety
|
|
safety check
|
|
|
|
# Update dependencies
|
|
pip install --upgrade cryptography mutagen requests
|
|
```
|
|
|
|
### API Key Exposure
|
|
|
|
**Risk:** Hardcoded credentials in scripts
|
|
|
|
**Mitigation:**
|
|
- Use environment variables
|
|
- Use config file outside version control
|
|
- Add `minim.cfg` to `.gitignore`
|
|
|
|
### Private API Usage
|
|
|
|
**Risk:** Terms of service violations (Qobuz, TIDAL, Spotify lyrics)
|
|
|
|
**Mitigation:**
|
|
- Use only public APIs in production
|
|
- Document risks in README
|
|
- Obtain official API access if possible
|
|
|
|
## Scalability
|
|
|
|
**Not Applicable:** minim is a library for personal use, not a scalable service.
|
|
|
|
**Limitations:**
|
|
- Synchronous, blocking operations
|
|
- No connection pooling
|
|
- No rate limiting
|
|
- No caching
|
|
- Single-threaded
|
|
|
|
**For High-Volume Use:**
|
|
- Implement async version using `aiohttp`
|
|
- Add connection pooling
|
|
- Implement rate limiting and backoff
|
|
- Cache API responses (Redis, Memcached)
|
|
- Use task queue (Celery, RQ) for background processing
|
|
|
|
## Backup and Recovery
|
|
|
|
**Config File Backup:**
|
|
```bash
|
|
# Backup
|
|
cp ~/minim.cfg ~/minim.cfg.backup
|
|
|
|
# Restore
|
|
cp ~/minim.cfg.backup ~/minim.cfg
|
|
```
|
|
|
|
**Recommendation:** Exclude from cloud backup (contains sensitive tokens) or encrypt backups.
|
|
|
|
## Maintenance
|
|
|
|
**Current Status:** v1 in maintenance mode
|
|
|
|
**Maintenance Activities:**
|
|
- Bug fixes for critical issues
|
|
- Security updates for dependencies
|
|
- No new features
|
|
|
|
**Active Development:** v2 rewrite on `dev` branch
|
|
|
|
**Support Channels:**
|
|
- GitHub Issues: https://github.com/bbye98/minim/issues
|
|
- GitHub Discussions: https://github.com/bbye98/minim/discussions
|
|
|
|
## Summary
|
|
|
|
minim deployment is straightforward:
|
|
|
|
1. **Install from source:** `git clone` + `pip install -e .`
|
|
2. **Configure credentials:** Environment variables or `~/minim.cfg`
|
|
3. **Import and use:** `from minim import spotify`
|
|
|
|
No server deployment, no containers, no orchestration. The library runs in the caller's Python process.
|
|
|
|
For production use cases requiring scalability, security, and robustness, consider:
|
|
- Wrapping minim in a web service (Flask, FastAPI)
|
|
- Implementing async operations
|
|
- Adding rate limiting and caching
|
|
- Using secure credential storage
|
|
- Monitoring and logging
|
|
|
|
The v2 rewrite addresses many deployment concerns (PyPI publication, async support, secure storage) while maintaining the simple library architecture.
|