a1f6701bac
- gRPC service with MusicBrainz provider - PostgreSQL schema with migrations - Service layer with database-first caching - Repository pattern for data access - YAML configuration support - Research documentation for 17 music metadata projects
1144 lines
35 KiB
Markdown
1144 lines
35 KiB
Markdown
# Melodee: Integrations Analysis
|
|
|
|
## Integration Strategy Overview
|
|
|
|
Melodee integrates with six external metadata providers, two scrobbling services, and supports Google OAuth for authentication. This multi-provider approach maximizes metadata quality by aggregating data from diverse sources, each with different strengths and coverage.
|
|
|
|
The integration architecture prioritizes:
|
|
- **Resilience**: Provider failures don't block library operations
|
|
- **Performance**: Local caching reduces API calls and latency
|
|
- **Flexibility**: Provider priority allows customization
|
|
- **Cost efficiency**: Free tiers and local caching minimize API costs
|
|
|
|
## Metadata Provider Integrations
|
|
|
|
### MusicBrainz
|
|
|
|
**Type**: Primary metadata source
|
|
**Priority**: 100 (highest)
|
|
**Authentication**: None (public API)
|
|
**Rate Limit**: 1 request per second (enforced by MusicBrainz)
|
|
**Cache Strategy**: Local SQLite database, monthly updates
|
|
|
|
#### Overview
|
|
|
|
MusicBrainz is an open music encyclopedia that collects music metadata and makes it available to the public. With over 2 million artists and 3 million releases, MusicBrainz provides authoritative data for mainstream and obscure music.
|
|
|
|
Melodee uses MusicBrainz as the primary metadata source due to:
|
|
- **Comprehensive coverage**: Largest open music database
|
|
- **Community curation**: Human-verified data, not algorithmic
|
|
- **Stable identifiers**: MBIDs (MusicBrainz IDs) provide permanent references
|
|
- **Free access**: No API keys or usage fees
|
|
|
|
#### Local Cache Architecture
|
|
|
|
Rather than querying the MusicBrainz API for every lookup, Melodee maintains a local SQLite database mirroring MusicBrainz data.
|
|
|
|
**Cache Database Schema**:
|
|
```sql
|
|
CREATE TABLE mb_release (
|
|
id TEXT PRIMARY KEY, -- MusicBrainz Release ID (MBID)
|
|
title TEXT NOT NULL,
|
|
artist_credit TEXT NOT NULL,
|
|
release_date TEXT,
|
|
country TEXT,
|
|
barcode TEXT,
|
|
release_group_id TEXT,
|
|
updated_at INTEGER NOT NULL
|
|
);
|
|
|
|
CREATE TABLE mb_recording (
|
|
id TEXT PRIMARY KEY, -- MusicBrainz Recording ID
|
|
title TEXT NOT NULL,
|
|
artist_credit TEXT NOT NULL,
|
|
length INTEGER, -- Duration in milliseconds
|
|
updated_at INTEGER NOT NULL
|
|
);
|
|
|
|
CREATE TABLE mb_artist (
|
|
id TEXT PRIMARY KEY, -- MusicBrainz Artist ID
|
|
name TEXT NOT NULL,
|
|
sort_name TEXT,
|
|
type TEXT, -- Person, Group, Orchestra, etc.
|
|
country TEXT,
|
|
begin_date TEXT,
|
|
end_date TEXT,
|
|
updated_at INTEGER NOT NULL
|
|
);
|
|
|
|
CREATE TABLE mb_release_group (
|
|
id TEXT PRIMARY KEY,
|
|
title TEXT NOT NULL,
|
|
artist_credit TEXT NOT NULL,
|
|
type TEXT, -- Album, EP, Single, Compilation
|
|
first_release_date TEXT,
|
|
updated_at INTEGER NOT NULL
|
|
);
|
|
```
|
|
|
|
**Cache Update Process**:
|
|
1. Download MusicBrainz database dump (monthly, first day of month)
|
|
2. Parse dump files (XML or JSON format)
|
|
3. Update SQLite tables with new/changed entries
|
|
4. Mark stale entries for deletion
|
|
5. Rebuild indexes for query performance
|
|
|
|
**Dump File Sources**:
|
|
- **Full dump**: Complete database snapshot (20+ GB compressed)
|
|
- **Incremental dump**: Changes since last full dump (smaller, faster)
|
|
|
|
Melodee likely uses incremental dumps after initial full dump to reduce download time and storage.
|
|
|
|
#### Query Strategy
|
|
|
|
**Lookup by MBID** (fastest):
|
|
```csharp
|
|
public async Task<MbRelease> GetReleaseByIdAsync(string mbid)
|
|
{
|
|
using var command = _connection.CreateCommand();
|
|
command.CommandText = "SELECT * FROM mb_release WHERE id = @id";
|
|
command.Parameters.AddWithValue("@id", mbid);
|
|
|
|
using var reader = await command.ExecuteReaderAsync();
|
|
if (await reader.ReadAsync())
|
|
{
|
|
return new MbRelease
|
|
{
|
|
Id = reader.GetString(0),
|
|
Title = reader.GetString(1),
|
|
ArtistCredit = reader.GetString(2),
|
|
ReleaseDate = reader.GetString(3)
|
|
};
|
|
}
|
|
|
|
return null;
|
|
}
|
|
```
|
|
|
|
**Search by Artist and Album** (slower, requires fuzzy matching):
|
|
```csharp
|
|
public async Task<List<MbRelease>> SearchReleasesAsync(string artist, string album)
|
|
{
|
|
using var command = _connection.CreateCommand();
|
|
command.CommandText = @"
|
|
SELECT * FROM mb_release
|
|
WHERE artist_credit LIKE @artist
|
|
AND title LIKE @album
|
|
ORDER BY updated_at DESC
|
|
LIMIT 10
|
|
";
|
|
command.Parameters.AddWithValue("@artist", $"%{artist}%");
|
|
command.Parameters.AddWithValue("@album", $"%{album}%");
|
|
|
|
var results = new List<MbRelease>();
|
|
using var reader = await command.ExecuteReaderAsync();
|
|
while (await reader.ReadAsync())
|
|
{
|
|
results.Add(new MbRelease { /* ... */ });
|
|
}
|
|
|
|
return results;
|
|
}
|
|
```
|
|
|
|
**Fuzzy Matching**:
|
|
For better search results, Melodee likely uses Levenshtein distance or similar algorithms to handle typos and variations:
|
|
|
|
```csharp
|
|
public int LevenshteinDistance(string a, string b)
|
|
{
|
|
var matrix = new int[a.Length + 1, b.Length + 1];
|
|
|
|
for (int i = 0; i <= a.Length; i++)
|
|
matrix[i, 0] = i;
|
|
for (int j = 0; j <= b.Length; j++)
|
|
matrix[0, j] = j;
|
|
|
|
for (int i = 1; i <= a.Length; i++)
|
|
{
|
|
for (int j = 1; j <= b.Length; j++)
|
|
{
|
|
int cost = (a[i - 1] == b[j - 1]) ? 0 : 1;
|
|
matrix[i, j] = Math.Min(
|
|
Math.Min(matrix[i - 1, j] + 1, matrix[i, j - 1] + 1),
|
|
matrix[i - 1, j - 1] + cost
|
|
);
|
|
}
|
|
}
|
|
|
|
return matrix[a.Length, b.Length];
|
|
}
|
|
```
|
|
|
|
#### Fallback to API
|
|
|
|
If the local cache doesn't contain a release, Melodee falls back to the MusicBrainz API:
|
|
|
|
```csharp
|
|
public async Task<MbRelease> GetReleaseAsync(string artist, string album)
|
|
{
|
|
// Try local cache first
|
|
var cached = await _cache.SearchReleasesAsync(artist, album);
|
|
if (cached.Any())
|
|
return cached.First();
|
|
|
|
// Fallback to API
|
|
await Task.Delay(1000); // Rate limit: 1 request per second
|
|
|
|
var response = await _httpClient.GetAsync(
|
|
$"https://musicbrainz.org/ws/2/release?query=artist:{artist} AND release:{album}&fmt=json"
|
|
);
|
|
|
|
var data = await response.Content.ReadFromJsonAsync<MbSearchResponse>();
|
|
|
|
if (data.Releases.Any())
|
|
{
|
|
var release = data.Releases.First();
|
|
|
|
// Cache for future lookups
|
|
await _cache.InsertReleaseAsync(release);
|
|
|
|
return release;
|
|
}
|
|
|
|
return null;
|
|
}
|
|
```
|
|
|
|
#### Data Mapping
|
|
|
|
MusicBrainz data maps to Melodee entities:
|
|
|
|
```csharp
|
|
public Album MapToAlbum(MbRelease mbRelease)
|
|
{
|
|
return new Album
|
|
{
|
|
Title = mbRelease.Title,
|
|
ReleaseDate = ParseDate(mbRelease.ReleaseDate),
|
|
Country = mbRelease.Country,
|
|
Barcode = mbRelease.Barcode,
|
|
ExternalIds = new Dictionary<string, string>
|
|
{
|
|
["musicbrainz"] = mbRelease.Id,
|
|
["musicbrainz_release_group"] = mbRelease.ReleaseGroupId
|
|
}
|
|
};
|
|
}
|
|
```
|
|
|
|
### Spotify
|
|
|
|
**Type**: Commercial metadata and album art
|
|
**Priority**: 80
|
|
**Authentication**: Client Credentials Flow (OAuth 2.0)
|
|
**Rate Limit**: Varies by endpoint (typically 180 requests per minute)
|
|
**Cache Strategy**: In-memory cache with 1-hour expiration
|
|
|
|
#### Overview
|
|
|
|
Spotify provides high-quality album art, popularity metrics, and commercial metadata. The Spotify Web API offers comprehensive data for mainstream releases.
|
|
|
|
Melodee uses Spotify for:
|
|
- **Album art**: High-resolution cover images (640x640, 300x300, 64x64)
|
|
- **Popularity scores**: Algorithmic ranking of tracks and albums
|
|
- **Preview URLs**: 30-second audio previews
|
|
- **Related artists**: Recommendation data
|
|
|
|
#### Authentication
|
|
|
|
Spotify uses OAuth 2.0 Client Credentials flow for server-to-server authentication:
|
|
|
|
```csharp
|
|
public class SpotifyAuthService
|
|
{
|
|
private readonly HttpClient _httpClient;
|
|
private readonly string _clientId;
|
|
private readonly string _clientSecret;
|
|
private string _accessToken;
|
|
private DateTime _tokenExpiry;
|
|
|
|
public async Task<string> GetAccessTokenAsync()
|
|
{
|
|
if (_accessToken != null && DateTime.UtcNow < _tokenExpiry)
|
|
return _accessToken;
|
|
|
|
var credentials = Convert.ToBase64String(
|
|
Encoding.UTF8.GetBytes($"{_clientId}:{_clientSecret}")
|
|
);
|
|
|
|
var request = new HttpRequestMessage(HttpMethod.Post, "https://accounts.spotify.com/api/token");
|
|
request.Headers.Authorization = new AuthenticationHeaderValue("Basic", credentials);
|
|
request.Content = new FormUrlEncodedContent(new Dictionary<string, string>
|
|
{
|
|
["grant_type"] = "client_credentials"
|
|
});
|
|
|
|
var response = await _httpClient.SendAsync(request);
|
|
var data = await response.Content.ReadFromJsonAsync<SpotifyTokenResponse>();
|
|
|
|
_accessToken = data.AccessToken;
|
|
_tokenExpiry = DateTime.UtcNow.AddSeconds(data.ExpiresIn - 60); // 60s buffer
|
|
|
|
return _accessToken;
|
|
}
|
|
}
|
|
```
|
|
|
|
#### API Queries
|
|
|
|
**Search for Album**:
|
|
```csharp
|
|
public async Task<SpotifyAlbum> SearchAlbumAsync(string artist, string album)
|
|
{
|
|
var token = await _authService.GetAccessTokenAsync();
|
|
|
|
var query = $"artist:{artist} album:{album}";
|
|
var encodedQuery = Uri.EscapeDataString(query);
|
|
|
|
var request = new HttpRequestMessage(
|
|
HttpMethod.Get,
|
|
$"https://api.spotify.com/v1/search?q={encodedQuery}&type=album&limit=1"
|
|
);
|
|
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
|
|
|
|
var response = await _httpClient.SendAsync(request);
|
|
var data = await response.Content.ReadFromJsonAsync<SpotifySearchResponse>();
|
|
|
|
return data.Albums.Items.FirstOrDefault();
|
|
}
|
|
```
|
|
|
|
**Get Album Details**:
|
|
```csharp
|
|
public async Task<SpotifyAlbum> GetAlbumAsync(string spotifyId)
|
|
{
|
|
var token = await _authService.GetAccessTokenAsync();
|
|
|
|
var request = new HttpRequestMessage(
|
|
HttpMethod.Get,
|
|
$"https://api.spotify.com/v1/albums/{spotifyId}"
|
|
);
|
|
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
|
|
|
|
var response = await _httpClient.SendAsync(request);
|
|
return await response.Content.ReadFromJsonAsync<SpotifyAlbum>();
|
|
}
|
|
```
|
|
|
|
#### Data Extraction
|
|
|
|
```csharp
|
|
public AlbumMetadata ExtractMetadata(SpotifyAlbum spotifyAlbum)
|
|
{
|
|
return new AlbumMetadata
|
|
{
|
|
Title = spotifyAlbum.Name,
|
|
Artist = spotifyAlbum.Artists.First().Name,
|
|
ReleaseDate = DateTime.Parse(spotifyAlbum.ReleaseDate),
|
|
Genres = spotifyAlbum.Genres,
|
|
CoverArtUrl = spotifyAlbum.Images.FirstOrDefault()?.Url,
|
|
ExternalIds = new Dictionary<string, string>
|
|
{
|
|
["spotify"] = spotifyAlbum.Id,
|
|
["spotify_uri"] = spotifyAlbum.Uri
|
|
},
|
|
Popularity = spotifyAlbum.Popularity,
|
|
Tracks = spotifyAlbum.Tracks.Items.Select(t => new TrackMetadata
|
|
{
|
|
Title = t.Name,
|
|
Position = t.TrackNumber,
|
|
Duration = t.DurationMs / 1000,
|
|
PreviewUrl = t.PreviewUrl
|
|
}).ToList()
|
|
};
|
|
}
|
|
```
|
|
|
|
#### Rate Limiting
|
|
|
|
Spotify enforces rate limits per application (not per user):
|
|
|
|
```csharp
|
|
public class SpotifyRateLimiter
|
|
{
|
|
private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(3, 3); // 3 concurrent requests
|
|
private readonly Queue<DateTime> _requestTimes = new Queue<DateTime>();
|
|
private readonly int _maxRequestsPerMinute = 180;
|
|
|
|
public async Task WaitForSlotAsync()
|
|
{
|
|
await _semaphore.WaitAsync();
|
|
|
|
try
|
|
{
|
|
var now = DateTime.UtcNow;
|
|
|
|
// Remove requests older than 1 minute
|
|
while (_requestTimes.Count > 0 && _requestTimes.Peek() < now.AddMinutes(-1))
|
|
_requestTimes.Dequeue();
|
|
|
|
// Wait if at rate limit
|
|
if (_requestTimes.Count >= _maxRequestsPerMinute)
|
|
{
|
|
var oldestRequest = _requestTimes.Peek();
|
|
var waitTime = oldestRequest.AddMinutes(1) - now;
|
|
if (waitTime > TimeSpan.Zero)
|
|
await Task.Delay(waitTime);
|
|
}
|
|
|
|
_requestTimes.Enqueue(now);
|
|
}
|
|
finally
|
|
{
|
|
_semaphore.Release();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Last.fm
|
|
|
|
**Type**: Social metadata and scrobbling
|
|
**Priority**: 70
|
|
**Authentication**: API key + shared secret
|
|
**Rate Limit**: Varies by method (typically unlimited for reads, limited for writes)
|
|
**Cache Strategy**: In-memory cache with 24-hour expiration
|
|
|
|
#### Overview
|
|
|
|
Last.fm provides social metadata (tags, similar artists, listener counts) and scrobbling services. With 20+ years of listening data, Last.fm offers unique insights into music popularity and relationships.
|
|
|
|
Melodee uses Last.fm for:
|
|
- **Tags**: User-generated genre tags
|
|
- **Similar artists**: Collaborative filtering recommendations
|
|
- **Play counts**: Global and user-specific listening statistics
|
|
- **Scrobbling**: Submitting listening history
|
|
|
|
#### Authentication
|
|
|
|
Last.fm uses API key authentication with MD5 signature for write operations:
|
|
|
|
```csharp
|
|
public class LastFmAuthService
|
|
{
|
|
private readonly string _apiKey;
|
|
private readonly string _sharedSecret;
|
|
|
|
public string GenerateSignature(Dictionary<string, string> parameters)
|
|
{
|
|
var sorted = parameters.OrderBy(p => p.Key);
|
|
var signatureString = string.Join("", sorted.Select(p => $"{p.Key}{p.Value}"));
|
|
signatureString += _sharedSecret;
|
|
|
|
using var md5 = MD5.Create();
|
|
var hash = md5.ComputeHash(Encoding.UTF8.GetBytes(signatureString));
|
|
return BitConverter.ToString(hash).Replace("-", "").ToLower();
|
|
}
|
|
}
|
|
```
|
|
|
|
#### API Queries
|
|
|
|
**Get Album Info**:
|
|
```csharp
|
|
public async Task<LastFmAlbum> GetAlbumInfoAsync(string artist, string album)
|
|
{
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["method"] = "album.getInfo",
|
|
["artist"] = artist,
|
|
["album"] = album,
|
|
["api_key"] = _apiKey,
|
|
["format"] = "json"
|
|
};
|
|
|
|
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
|
|
var response = await _httpClient.GetAsync($"https://ws.audioscrobbler.com/2.0/?{queryString}");
|
|
|
|
var data = await response.Content.ReadFromJsonAsync<LastFmAlbumResponse>();
|
|
return data.Album;
|
|
}
|
|
```
|
|
|
|
**Get Similar Artists**:
|
|
```csharp
|
|
public async Task<List<LastFmArtist>> GetSimilarArtistsAsync(string artist)
|
|
{
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["method"] = "artist.getSimilar",
|
|
["artist"] = artist,
|
|
["api_key"] = _apiKey,
|
|
["format"] = "json",
|
|
["limit"] = "20"
|
|
};
|
|
|
|
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
|
|
var response = await _httpClient.GetAsync($"https://ws.audioscrobbler.com/2.0/?{queryString}");
|
|
|
|
var data = await response.Content.ReadFromJsonAsync<LastFmSimilarArtistsResponse>();
|
|
return data.SimilarArtists.Artist;
|
|
}
|
|
```
|
|
|
|
#### Data Extraction
|
|
|
|
```csharp
|
|
public AlbumMetadata ExtractMetadata(LastFmAlbum lastFmAlbum)
|
|
{
|
|
return new AlbumMetadata
|
|
{
|
|
Title = lastFmAlbum.Name,
|
|
Artist = lastFmAlbum.Artist,
|
|
Tags = lastFmAlbum.Tags.Tag.Select(t => t.Name).ToList(),
|
|
PlayCount = lastFmAlbum.PlayCount,
|
|
Listeners = lastFmAlbum.Listeners,
|
|
Wiki = lastFmAlbum.Wiki?.Summary,
|
|
ExternalIds = new Dictionary<string, string>
|
|
{
|
|
["lastfm"] = lastFmAlbum.Url
|
|
}
|
|
};
|
|
}
|
|
```
|
|
|
|
### iTunes
|
|
|
|
**Type**: Commercial metadata
|
|
**Priority**: 60
|
|
**Authentication**: None (public API)
|
|
**Rate Limit**: 20 requests per minute (unofficial, enforced by Apple)
|
|
**Cache Strategy**: In-memory cache with 1-hour expiration
|
|
|
|
#### Overview
|
|
|
|
iTunes Search API provides metadata for music available in the iTunes Store. Coverage is strong for mainstream releases but limited for independent or regional music.
|
|
|
|
Melodee uses iTunes for:
|
|
- **Album art**: High-quality cover images
|
|
- **Preview URLs**: 30-second audio previews
|
|
- **Pricing data**: Commercial availability
|
|
- **Release dates**: Official release information
|
|
|
|
#### API Queries
|
|
|
|
**Search for Album**:
|
|
```csharp
|
|
public async Task<ItunesAlbum> SearchAlbumAsync(string artist, string album)
|
|
{
|
|
var query = $"{artist} {album}";
|
|
var encodedQuery = Uri.EscapeDataString(query);
|
|
|
|
var response = await _httpClient.GetAsync(
|
|
$"https://itunes.apple.com/search?term={encodedQuery}&entity=album&limit=1"
|
|
);
|
|
|
|
var data = await response.Content.ReadFromJsonAsync<ItunesSearchResponse>();
|
|
return data.Results.FirstOrDefault();
|
|
}
|
|
```
|
|
|
|
#### Data Extraction
|
|
|
|
```csharp
|
|
public AlbumMetadata ExtractMetadata(ItunesAlbum itunesAlbum)
|
|
{
|
|
return new AlbumMetadata
|
|
{
|
|
Title = itunesAlbum.CollectionName,
|
|
Artist = itunesAlbum.ArtistName,
|
|
ReleaseDate = DateTime.Parse(itunesAlbum.ReleaseDate),
|
|
Genre = itunesAlbum.PrimaryGenreName,
|
|
CoverArtUrl = itunesAlbum.ArtworkUrl100.Replace("100x100", "600x600"),
|
|
TrackCount = itunesAlbum.TrackCount,
|
|
ExternalIds = new Dictionary<string, string>
|
|
{
|
|
["itunes"] = itunesAlbum.CollectionId.ToString(),
|
|
["itunes_artist"] = itunesAlbum.ArtistId.ToString()
|
|
}
|
|
};
|
|
}
|
|
```
|
|
|
|
### Deezer
|
|
|
|
**Type**: European market metadata
|
|
**Priority**: 50
|
|
**Authentication**: None (public API)
|
|
**Rate Limit**: 50 requests per 5 seconds
|
|
**Cache Strategy**: In-memory cache with 1-hour expiration
|
|
|
|
#### Overview
|
|
|
|
Deezer provides strong coverage for European music markets, particularly French, German, and Italian releases. The API is free and doesn't require authentication for basic queries.
|
|
|
|
Melodee uses Deezer for:
|
|
- **European releases**: Better coverage than US-centric services
|
|
- **Album art**: High-quality cover images
|
|
- **Track listings**: Accurate track metadata
|
|
- **Genre data**: European genre classifications
|
|
|
|
#### API Queries
|
|
|
|
**Search for Album**:
|
|
```csharp
|
|
public async Task<DeezerAlbum> SearchAlbumAsync(string artist, string album)
|
|
{
|
|
var query = $"{artist} {album}";
|
|
var encodedQuery = Uri.EscapeDataString(query);
|
|
|
|
var response = await _httpClient.GetAsync(
|
|
$"https://api.deezer.com/search/album?q={encodedQuery}&limit=1"
|
|
);
|
|
|
|
var data = await response.Content.ReadFromJsonAsync<DeezerSearchResponse>();
|
|
return data.Data.FirstOrDefault();
|
|
}
|
|
```
|
|
|
|
**Get Album Details**:
|
|
```csharp
|
|
public async Task<DeezerAlbum> GetAlbumAsync(int deezerId)
|
|
{
|
|
var response = await _httpClient.GetAsync($"https://api.deezer.com/album/{deezerId}");
|
|
return await response.Content.ReadFromJsonAsync<DeezerAlbum>();
|
|
}
|
|
```
|
|
|
|
#### Data Extraction
|
|
|
|
```csharp
|
|
public AlbumMetadata ExtractMetadata(DeezerAlbum deezerAlbum)
|
|
{
|
|
return new AlbumMetadata
|
|
{
|
|
Title = deezerAlbum.Title,
|
|
Artist = deezerAlbum.Artist.Name,
|
|
ReleaseDate = DateTime.Parse(deezerAlbum.ReleaseDate),
|
|
Genre = deezerAlbum.Genres.Data.FirstOrDefault()?.Name,
|
|
CoverArtUrl = deezerAlbum.CoverXl,
|
|
TrackCount = deezerAlbum.NbTracks,
|
|
Duration = deezerAlbum.Duration,
|
|
ExternalIds = new Dictionary<string, string>
|
|
{
|
|
["deezer"] = deezerAlbum.Id.ToString(),
|
|
["deezer_artist"] = deezerAlbum.Artist.Id.ToString()
|
|
}
|
|
};
|
|
}
|
|
```
|
|
|
|
### Brave Search
|
|
|
|
**Type**: Fallback web search
|
|
**Priority**: 10 (lowest)
|
|
**Authentication**: API key
|
|
**Rate Limit**: Varies by plan (free tier: 2000 queries per month)
|
|
**Cache Strategy**: In-memory cache with 7-day expiration
|
|
|
|
#### Overview
|
|
|
|
Brave Search serves as a fallback when other providers don't have metadata for obscure releases. The search API can find album information from web sources.
|
|
|
|
Melodee uses Brave Search for:
|
|
- **Obscure releases**: Independent labels, regional music
|
|
- **Missing metadata**: When all other providers fail
|
|
- **Web scraping**: Extracting data from music websites
|
|
|
|
#### API Queries
|
|
|
|
**Search for Album**:
|
|
```csharp
|
|
public async Task<BraveSearchResult> SearchAlbumAsync(string artist, string album)
|
|
{
|
|
var query = $"{artist} {album} album";
|
|
var encodedQuery = Uri.EscapeDataString(query);
|
|
|
|
var request = new HttpRequestMessage(
|
|
HttpMethod.Get,
|
|
$"https://api.search.brave.com/res/v1/web/search?q={encodedQuery}"
|
|
);
|
|
request.Headers.Add("X-Subscription-Token", _apiKey);
|
|
|
|
var response = await _httpClient.SendAsync(request);
|
|
return await response.Content.ReadFromJsonAsync<BraveSearchResult>();
|
|
}
|
|
```
|
|
|
|
#### Data Extraction
|
|
|
|
Brave Search returns web results, not structured metadata. Melodee must parse HTML or extract data from snippets:
|
|
|
|
```csharp
|
|
public AlbumMetadata ExtractMetadata(BraveSearchResult searchResult)
|
|
{
|
|
var firstResult = searchResult.Web.Results.FirstOrDefault();
|
|
if (firstResult == null)
|
|
return null;
|
|
|
|
// Extract year from snippet using regex
|
|
var yearMatch = Regex.Match(firstResult.Description, @"\b(19|20)\d{2}\b");
|
|
var year = yearMatch.Success ? int.Parse(yearMatch.Value) : (int?)null;
|
|
|
|
return new AlbumMetadata
|
|
{
|
|
Title = ExtractAlbumTitle(firstResult.Title),
|
|
Artist = ExtractArtistName(firstResult.Title),
|
|
ReleaseDate = year.HasValue ? new DateTime(year.Value, 1, 1) : null,
|
|
ExternalIds = new Dictionary<string, string>
|
|
{
|
|
["web_source"] = firstResult.Url
|
|
}
|
|
};
|
|
}
|
|
```
|
|
|
|
This approach is fragile and depends on consistent web page structures. Brave Search is truly a last resort.
|
|
|
|
## Metadata Aggregation Strategy
|
|
|
|
### Provider Priority
|
|
|
|
Providers are queried in priority order. Higher-priority providers override lower-priority providers for conflicting data:
|
|
|
|
```csharp
|
|
public class MetadataAggregator
|
|
{
|
|
private readonly List<IMetadataProvider> _providers;
|
|
|
|
public MetadataAggregator(IEnumerable<IMetadataProvider> providers)
|
|
{
|
|
_providers = providers.OrderByDescending(p => p.Priority).ToList();
|
|
}
|
|
|
|
public async Task<AlbumMetadata> AggregateAlbumMetadataAsync(string artist, string album)
|
|
{
|
|
var results = new List<AlbumMetadata>();
|
|
|
|
foreach (var provider in _providers)
|
|
{
|
|
try
|
|
{
|
|
var metadata = await provider.GetAlbumMetadataAsync(artist, album);
|
|
if (metadata != null)
|
|
{
|
|
metadata.Source = provider.Name;
|
|
results.Add(metadata);
|
|
}
|
|
}
|
|
catch (Exception ex)
|
|
{
|
|
_logger.LogWarning(ex, "Provider {Provider} failed for {Artist} - {Album}",
|
|
provider.Name, artist, album);
|
|
}
|
|
}
|
|
|
|
return MergeMetadata(results);
|
|
}
|
|
}
|
|
```
|
|
|
|
### Merge Strategy
|
|
|
|
```csharp
|
|
private AlbumMetadata MergeMetadata(List<AlbumMetadata> results)
|
|
{
|
|
if (!results.Any())
|
|
return null;
|
|
|
|
var merged = new AlbumMetadata();
|
|
|
|
// Title: prefer highest-priority provider
|
|
merged.Title = results.FirstOrDefault(r => !string.IsNullOrEmpty(r.Title))?.Title;
|
|
|
|
// Artist: prefer highest-priority provider
|
|
merged.Artist = results.FirstOrDefault(r => !string.IsNullOrEmpty(r.Artist))?.Artist;
|
|
|
|
// Release date: prefer most specific date
|
|
merged.ReleaseDate = results
|
|
.Where(r => r.ReleaseDate.HasValue)
|
|
.OrderByDescending(r => r.ReleaseDate.Value)
|
|
.FirstOrDefault()?.ReleaseDate;
|
|
|
|
// Genres: combine from all providers, deduplicate
|
|
merged.Genres = results
|
|
.SelectMany(r => r.Genres ?? new List<string>())
|
|
.Distinct()
|
|
.ToList();
|
|
|
|
// Cover art: prefer highest resolution
|
|
merged.CoverArtUrl = results
|
|
.Where(r => !string.IsNullOrEmpty(r.CoverArtUrl))
|
|
.OrderByDescending(r => EstimateImageResolution(r.CoverArtUrl))
|
|
.FirstOrDefault()?.CoverArtUrl;
|
|
|
|
// External IDs: combine from all providers
|
|
merged.ExternalIds = results
|
|
.SelectMany(r => r.ExternalIds ?? new Dictionary<string, string>())
|
|
.GroupBy(kvp => kvp.Key)
|
|
.ToDictionary(g => g.Key, g => g.First().Value);
|
|
|
|
return merged;
|
|
}
|
|
```
|
|
|
|
## Scrobbling Integrations
|
|
|
|
### Last.fm Scrobbling
|
|
|
|
**Authentication**: Session key (obtained via OAuth)
|
|
**Rate Limit**: 50 scrobbles per request, unlimited requests
|
|
|
|
#### Session Key Acquisition
|
|
|
|
Users must authorize Melodee to scrobble on their behalf:
|
|
|
|
```csharp
|
|
public class LastFmScrobblingService
|
|
{
|
|
public string GetAuthorizationUrl()
|
|
{
|
|
return $"https://www.last.fm/api/auth/?api_key={_apiKey}&cb={_callbackUrl}";
|
|
}
|
|
|
|
public async Task<string> GetSessionKeyAsync(string token)
|
|
{
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["method"] = "auth.getSession",
|
|
["api_key"] = _apiKey,
|
|
["token"] = token
|
|
};
|
|
|
|
var signature = _authService.GenerateSignature(parameters);
|
|
parameters["api_sig"] = signature;
|
|
parameters["format"] = "json";
|
|
|
|
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
|
|
var response = await _httpClient.GetAsync($"https://ws.audioscrobbler.com/2.0/?{queryString}");
|
|
|
|
var data = await response.Content.ReadFromJsonAsync<LastFmSessionResponse>();
|
|
return data.Session.Key;
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Scrobble Submission
|
|
|
|
```csharp
|
|
public async Task ScrobbleAsync(int userId, int trackId, DateTime playedAt)
|
|
{
|
|
var user = await _userRepository.GetByIdAsync(userId);
|
|
if (string.IsNullOrEmpty(user.Settings.LastFmSessionKey))
|
|
return;
|
|
|
|
var track = await _trackRepository.GetByIdAsync(trackId);
|
|
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["method"] = "track.scrobble",
|
|
["artist"] = track.Artist.Name,
|
|
["track"] = track.Title,
|
|
["album"] = track.Album.Title,
|
|
["timestamp"] = ((DateTimeOffset)playedAt).ToUnixTimeSeconds().ToString(),
|
|
["api_key"] = _apiKey,
|
|
["sk"] = user.Settings.LastFmSessionKey
|
|
};
|
|
|
|
var signature = _authService.GenerateSignature(parameters);
|
|
parameters["api_sig"] = signature;
|
|
parameters["format"] = "json";
|
|
|
|
var content = new FormUrlEncodedContent(parameters);
|
|
var response = await _httpClient.PostAsync("https://ws.audioscrobbler.com/2.0/", content);
|
|
|
|
if (!response.IsSuccessStatusCode)
|
|
{
|
|
_logger.LogError("Last.fm scrobble failed: {Status}", response.StatusCode);
|
|
throw new ScrobbleException("Last.fm scrobble failed");
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Batch Scrobbling
|
|
|
|
Last.fm supports batch scrobbling (up to 50 tracks per request):
|
|
|
|
```csharp
|
|
public async Task BatchScrobbleAsync(int userId, List<Scrobble> scrobbles)
|
|
{
|
|
var user = await _userRepository.GetByIdAsync(userId);
|
|
if (string.IsNullOrEmpty(user.Settings.LastFmSessionKey))
|
|
return;
|
|
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["method"] = "track.scrobble",
|
|
["api_key"] = _apiKey,
|
|
["sk"] = user.Settings.LastFmSessionKey
|
|
};
|
|
|
|
for (int i = 0; i < scrobbles.Count; i++)
|
|
{
|
|
var scrobble = scrobbles[i];
|
|
var track = await _trackRepository.GetByIdAsync(scrobble.TrackId);
|
|
|
|
parameters[$"artist[{i}]"] = track.Artist.Name;
|
|
parameters[$"track[{i}]"] = track.Title;
|
|
parameters[$"album[{i}]"] = track.Album.Title;
|
|
parameters[$"timestamp[{i}]"] = ((DateTimeOffset)scrobble.PlayedAt).ToUnixTimeSeconds().ToString();
|
|
}
|
|
|
|
var signature = _authService.GenerateSignature(parameters);
|
|
parameters["api_sig"] = signature;
|
|
parameters["format"] = "json";
|
|
|
|
var content = new FormUrlEncodedContent(parameters);
|
|
var response = await _httpClient.PostAsync("https://ws.audioscrobbler.com/2.0/", content);
|
|
|
|
if (response.IsSuccessStatusCode)
|
|
{
|
|
foreach (var scrobble in scrobbles)
|
|
{
|
|
scrobble.SubmittedToLastFm = true;
|
|
scrobble.LastFmSubmittedAt = DateTime.UtcNow;
|
|
}
|
|
|
|
await _scrobbleRepository.UpdateRangeAsync(scrobbles);
|
|
}
|
|
}
|
|
```
|
|
|
|
### Internal Melodee Scrobbler
|
|
|
|
Melodee maintains its own scrobble database for analytics and charts:
|
|
|
|
```csharp
|
|
public class MelodeeScrobbler : IScrobbler
|
|
{
|
|
private readonly IScrobbleRepository _scrobbleRepository;
|
|
|
|
public async Task ScrobbleAsync(int userId, int trackId, DateTime playedAt)
|
|
{
|
|
var scrobble = new Scrobble
|
|
{
|
|
UserId = userId,
|
|
TrackId = trackId,
|
|
PlayedAt = playedAt,
|
|
CreatedAt = DateTime.UtcNow
|
|
};
|
|
|
|
await _scrobbleRepository.CreateAsync(scrobble);
|
|
}
|
|
}
|
|
```
|
|
|
|
### Composite Scrobbler
|
|
|
|
Both scribblers run in parallel:
|
|
|
|
```csharp
|
|
public class CompositeScrobbler : IScrobbler
|
|
{
|
|
private readonly List<IScrobbler> _scribblers;
|
|
|
|
public async Task ScrobbleAsync(int userId, int trackId, DateTime playedAt)
|
|
{
|
|
var tasks = _scribblers.Select(s => s.ScrobbleAsync(userId, trackId, playedAt));
|
|
await Task.WhenAll(tasks);
|
|
}
|
|
}
|
|
```
|
|
|
|
## Authentication Integrations
|
|
|
|
### Google OAuth
|
|
|
|
**Flow**: Authorization Code Flow
|
|
**Scopes**: `openid`, `email`, `profile`
|
|
**Token Storage**: Access tokens not stored (only used during authentication)
|
|
|
|
#### Authorization Flow
|
|
|
|
```csharp
|
|
public class GoogleOAuthService
|
|
{
|
|
public string GetAuthorizationUrl(string state)
|
|
{
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["client_id"] = _clientId,
|
|
["redirect_uri"] = _redirectUri,
|
|
["response_type"] = "code",
|
|
["scope"] = "openid email profile",
|
|
["state"] = state
|
|
};
|
|
|
|
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
|
|
return $"https://accounts.google.com/o/oauth2/v2/auth?{queryString}";
|
|
}
|
|
|
|
public async Task<GoogleTokenResponse> ExchangeCodeForTokenAsync(string code)
|
|
{
|
|
var parameters = new Dictionary<string, string>
|
|
{
|
|
["code"] = code,
|
|
["client_id"] = _clientId,
|
|
["client_secret"] = _clientSecret,
|
|
["redirect_uri"] = _redirectUri,
|
|
["grant_type"] = "authorization_code"
|
|
};
|
|
|
|
var content = new FormUrlEncodedContent(parameters);
|
|
var response = await _httpClient.PostAsync("https://oauth2.googleapis.com/token", content);
|
|
|
|
return await response.Content.ReadFromJsonAsync<GoogleTokenResponse>();
|
|
}
|
|
|
|
public async Task<GoogleUserInfo> GetUserInfoAsync(string accessToken)
|
|
{
|
|
var request = new HttpRequestMessage(
|
|
HttpMethod.Get,
|
|
"https://www.googleapis.com/oauth2/v1/userinfo"
|
|
);
|
|
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
|
|
|
|
var response = await _httpClient.SendAsync(request);
|
|
return await response.Content.ReadFromJsonAsync<GoogleUserInfo>();
|
|
}
|
|
}
|
|
```
|
|
|
|
#### User Account Creation
|
|
|
|
```csharp
|
|
public async Task<User> AuthenticateWithGoogleAsync(string code)
|
|
{
|
|
var tokenResponse = await _googleOAuthService.ExchangeCodeForTokenAsync(code);
|
|
var userInfo = await _googleOAuthService.GetUserInfoAsync(tokenResponse.AccessToken);
|
|
|
|
var user = await _userRepository.GetByGoogleIdAsync(userInfo.Id);
|
|
|
|
if (user == null)
|
|
{
|
|
user = new User
|
|
{
|
|
Email = userInfo.Email,
|
|
GoogleId = userInfo.Id,
|
|
ProfilePictureUrl = userInfo.Picture,
|
|
Role = "user",
|
|
IsActive = true,
|
|
CreatedAt = DateTime.UtcNow
|
|
};
|
|
|
|
await _userRepository.CreateAsync(user);
|
|
}
|
|
else
|
|
{
|
|
user.ProfilePictureUrl = userInfo.Picture;
|
|
user.LastLoginAt = DateTime.UtcNow;
|
|
await _userRepository.UpdateAsync(user);
|
|
}
|
|
|
|
return user;
|
|
}
|
|
```
|
|
|
|
## Integration Resilience
|
|
|
|
### Circuit Breaker Pattern
|
|
|
|
Prevents cascading failures when providers are down:
|
|
|
|
```csharp
|
|
public class CircuitBreakerMetadataProvider : IMetadataProvider
|
|
{
|
|
private readonly IMetadataProvider _inner;
|
|
private int _failureCount;
|
|
private DateTime? _circuitOpenedAt;
|
|
private readonly int _failureThreshold = 5;
|
|
private readonly TimeSpan _resetTimeout = TimeSpan.FromMinutes(5);
|
|
|
|
public async Task<AlbumMetadata> GetAlbumMetadataAsync(string artist, string album)
|
|
{
|
|
if (_circuitOpenedAt.HasValue)
|
|
{
|
|
if (DateTime.UtcNow - _circuitOpenedAt.Value < _resetTimeout)
|
|
throw new CircuitBreakerOpenException($"Circuit breaker open for {Name}");
|
|
|
|
// Try to close circuit
|
|
_circuitOpenedAt = null;
|
|
_failureCount = 0;
|
|
}
|
|
|
|
try
|
|
{
|
|
var result = await _inner.GetAlbumMetadataAsync(artist, album);
|
|
_failureCount = 0;
|
|
return result;
|
|
}
|
|
catch (Exception ex)
|
|
{
|
|
_failureCount++;
|
|
|
|
if (_failureCount >= _failureThreshold)
|
|
{
|
|
_circuitOpenedAt = DateTime.UtcNow;
|
|
_logger.LogError("Circuit breaker opened for {Provider}", Name);
|
|
}
|
|
|
|
throw;
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Retry with Exponential Backoff
|
|
|
|
```csharp
|
|
public class RetryMetadataProvider : IMetadataProvider
|
|
{
|
|
private readonly IMetadataProvider _inner;
|
|
private readonly int _maxRetries = 3;
|
|
|
|
public async Task<AlbumMetadata> GetAlbumMetadataAsync(string artist, string album)
|
|
{
|
|
for (int attempt = 0; attempt < _maxRetries; attempt++)
|
|
{
|
|
try
|
|
{
|
|
return await _inner.GetAlbumMetadataAsync(artist, album);
|
|
}
|
|
catch (HttpRequestException ex) when (attempt < _maxRetries - 1)
|
|
{
|
|
var delay = TimeSpan.FromSeconds(Math.Pow(2, attempt));
|
|
_logger.LogWarning("Retry {Attempt} for {Provider} after {Delay}s",
|
|
attempt + 1, Name, delay.TotalSeconds);
|
|
await Task.Delay(delay);
|
|
}
|
|
}
|
|
|
|
throw new MetadataProviderException($"Failed after {_maxRetries} retries");
|
|
}
|
|
}
|
|
```
|
|
|
|
## Conclusion
|
|
|
|
Melodee's integration strategy demonstrates sophisticated metadata aggregation from six diverse providers. The priority-based merging, local MusicBrainz cache, and resilience patterns (circuit breaker, retry) create a robust system that maximizes metadata quality while minimizing external dependencies.
|
|
|
|
Key strengths:
|
|
- **Multi-provider redundancy**: Provider failures don't block library operations
|
|
- **Local caching**: MusicBrainz SQLite cache eliminates most API calls
|
|
- **Intelligent merging**: Priority-based conflict resolution
|
|
- **Scrobbling**: Dual submission to Last.fm and internal database
|
|
|
|
Key challenges:
|
|
- **API key management**: Six providers require configuration
|
|
- **Rate limiting**: Complex coordination across providers
|
|
- **Data consistency**: Merging conflicting metadata from multiple sources
|
|
- **Maintenance**: Provider API changes require ongoing updates
|
|
|
|
The architecture positions Melodee as a metadata-first music server, prioritizing data quality over simplicity.
|