Files
metadata-agregator/docs/research/melodee/analysis/INTEGRATIONS.md
T
Alexander a1f6701bac feat: initial implementation of metadata aggregator
- gRPC service with MusicBrainz provider
- PostgreSQL schema with migrations
- Service layer with database-first caching
- Repository pattern for data access
- YAML configuration support
- Research documentation for 17 music metadata projects
2026-04-28 16:28:53 +02:00

1144 lines
35 KiB
Markdown

# Melodee: Integrations Analysis
## Integration Strategy Overview
Melodee integrates with six external metadata providers, two scrobbling services, and supports Google OAuth for authentication. This multi-provider approach maximizes metadata quality by aggregating data from diverse sources, each with different strengths and coverage.
The integration architecture prioritizes:
- **Resilience**: Provider failures don't block library operations
- **Performance**: Local caching reduces API calls and latency
- **Flexibility**: Provider priority allows customization
- **Cost efficiency**: Free tiers and local caching minimize API costs
## Metadata Provider Integrations
### MusicBrainz
**Type**: Primary metadata source
**Priority**: 100 (highest)
**Authentication**: None (public API)
**Rate Limit**: 1 request per second (enforced by MusicBrainz)
**Cache Strategy**: Local SQLite database, monthly updates
#### Overview
MusicBrainz is an open music encyclopedia that collects music metadata and makes it available to the public. With over 2 million artists and 3 million releases, MusicBrainz provides authoritative data for mainstream and obscure music.
Melodee uses MusicBrainz as the primary metadata source due to:
- **Comprehensive coverage**: Largest open music database
- **Community curation**: Human-verified data, not algorithmic
- **Stable identifiers**: MBIDs (MusicBrainz IDs) provide permanent references
- **Free access**: No API keys or usage fees
#### Local Cache Architecture
Rather than querying the MusicBrainz API for every lookup, Melodee maintains a local SQLite database mirroring MusicBrainz data.
**Cache Database Schema**:
```sql
CREATE TABLE mb_release (
id TEXT PRIMARY KEY, -- MusicBrainz Release ID (MBID)
title TEXT NOT NULL,
artist_credit TEXT NOT NULL,
release_date TEXT,
country TEXT,
barcode TEXT,
release_group_id TEXT,
updated_at INTEGER NOT NULL
);
CREATE TABLE mb_recording (
id TEXT PRIMARY KEY, -- MusicBrainz Recording ID
title TEXT NOT NULL,
artist_credit TEXT NOT NULL,
length INTEGER, -- Duration in milliseconds
updated_at INTEGER NOT NULL
);
CREATE TABLE mb_artist (
id TEXT PRIMARY KEY, -- MusicBrainz Artist ID
name TEXT NOT NULL,
sort_name TEXT,
type TEXT, -- Person, Group, Orchestra, etc.
country TEXT,
begin_date TEXT,
end_date TEXT,
updated_at INTEGER NOT NULL
);
CREATE TABLE mb_release_group (
id TEXT PRIMARY KEY,
title TEXT NOT NULL,
artist_credit TEXT NOT NULL,
type TEXT, -- Album, EP, Single, Compilation
first_release_date TEXT,
updated_at INTEGER NOT NULL
);
```
**Cache Update Process**:
1. Download MusicBrainz database dump (monthly, first day of month)
2. Parse dump files (XML or JSON format)
3. Update SQLite tables with new/changed entries
4. Mark stale entries for deletion
5. Rebuild indexes for query performance
**Dump File Sources**:
- **Full dump**: Complete database snapshot (20+ GB compressed)
- **Incremental dump**: Changes since last full dump (smaller, faster)
Melodee likely uses incremental dumps after initial full dump to reduce download time and storage.
#### Query Strategy
**Lookup by MBID** (fastest):
```csharp
public async Task<MbRelease> GetReleaseByIdAsync(string mbid)
{
using var command = _connection.CreateCommand();
command.CommandText = "SELECT * FROM mb_release WHERE id = @id";
command.Parameters.AddWithValue("@id", mbid);
using var reader = await command.ExecuteReaderAsync();
if (await reader.ReadAsync())
{
return new MbRelease
{
Id = reader.GetString(0),
Title = reader.GetString(1),
ArtistCredit = reader.GetString(2),
ReleaseDate = reader.GetString(3)
};
}
return null;
}
```
**Search by Artist and Album** (slower, requires fuzzy matching):
```csharp
public async Task<List<MbRelease>> SearchReleasesAsync(string artist, string album)
{
using var command = _connection.CreateCommand();
command.CommandText = @"
SELECT * FROM mb_release
WHERE artist_credit LIKE @artist
AND title LIKE @album
ORDER BY updated_at DESC
LIMIT 10
";
command.Parameters.AddWithValue("@artist", $"%{artist}%");
command.Parameters.AddWithValue("@album", $"%{album}%");
var results = new List<MbRelease>();
using var reader = await command.ExecuteReaderAsync();
while (await reader.ReadAsync())
{
results.Add(new MbRelease { /* ... */ });
}
return results;
}
```
**Fuzzy Matching**:
For better search results, Melodee likely uses Levenshtein distance or similar algorithms to handle typos and variations:
```csharp
public int LevenshteinDistance(string a, string b)
{
var matrix = new int[a.Length + 1, b.Length + 1];
for (int i = 0; i <= a.Length; i++)
matrix[i, 0] = i;
for (int j = 0; j <= b.Length; j++)
matrix[0, j] = j;
for (int i = 1; i <= a.Length; i++)
{
for (int j = 1; j <= b.Length; j++)
{
int cost = (a[i - 1] == b[j - 1]) ? 0 : 1;
matrix[i, j] = Math.Min(
Math.Min(matrix[i - 1, j] + 1, matrix[i, j - 1] + 1),
matrix[i - 1, j - 1] + cost
);
}
}
return matrix[a.Length, b.Length];
}
```
#### Fallback to API
If the local cache doesn't contain a release, Melodee falls back to the MusicBrainz API:
```csharp
public async Task<MbRelease> GetReleaseAsync(string artist, string album)
{
// Try local cache first
var cached = await _cache.SearchReleasesAsync(artist, album);
if (cached.Any())
return cached.First();
// Fallback to API
await Task.Delay(1000); // Rate limit: 1 request per second
var response = await _httpClient.GetAsync(
$"https://musicbrainz.org/ws/2/release?query=artist:{artist} AND release:{album}&fmt=json"
);
var data = await response.Content.ReadFromJsonAsync<MbSearchResponse>();
if (data.Releases.Any())
{
var release = data.Releases.First();
// Cache for future lookups
await _cache.InsertReleaseAsync(release);
return release;
}
return null;
}
```
#### Data Mapping
MusicBrainz data maps to Melodee entities:
```csharp
public Album MapToAlbum(MbRelease mbRelease)
{
return new Album
{
Title = mbRelease.Title,
ReleaseDate = ParseDate(mbRelease.ReleaseDate),
Country = mbRelease.Country,
Barcode = mbRelease.Barcode,
ExternalIds = new Dictionary<string, string>
{
["musicbrainz"] = mbRelease.Id,
["musicbrainz_release_group"] = mbRelease.ReleaseGroupId
}
};
}
```
### Spotify
**Type**: Commercial metadata and album art
**Priority**: 80
**Authentication**: Client Credentials Flow (OAuth 2.0)
**Rate Limit**: Varies by endpoint (typically 180 requests per minute)
**Cache Strategy**: In-memory cache with 1-hour expiration
#### Overview
Spotify provides high-quality album art, popularity metrics, and commercial metadata. The Spotify Web API offers comprehensive data for mainstream releases.
Melodee uses Spotify for:
- **Album art**: High-resolution cover images (640x640, 300x300, 64x64)
- **Popularity scores**: Algorithmic ranking of tracks and albums
- **Preview URLs**: 30-second audio previews
- **Related artists**: Recommendation data
#### Authentication
Spotify uses OAuth 2.0 Client Credentials flow for server-to-server authentication:
```csharp
public class SpotifyAuthService
{
private readonly HttpClient _httpClient;
private readonly string _clientId;
private readonly string _clientSecret;
private string _accessToken;
private DateTime _tokenExpiry;
public async Task<string> GetAccessTokenAsync()
{
if (_accessToken != null && DateTime.UtcNow < _tokenExpiry)
return _accessToken;
var credentials = Convert.ToBase64String(
Encoding.UTF8.GetBytes($"{_clientId}:{_clientSecret}")
);
var request = new HttpRequestMessage(HttpMethod.Post, "https://accounts.spotify.com/api/token");
request.Headers.Authorization = new AuthenticationHeaderValue("Basic", credentials);
request.Content = new FormUrlEncodedContent(new Dictionary<string, string>
{
["grant_type"] = "client_credentials"
});
var response = await _httpClient.SendAsync(request);
var data = await response.Content.ReadFromJsonAsync<SpotifyTokenResponse>();
_accessToken = data.AccessToken;
_tokenExpiry = DateTime.UtcNow.AddSeconds(data.ExpiresIn - 60); // 60s buffer
return _accessToken;
}
}
```
#### API Queries
**Search for Album**:
```csharp
public async Task<SpotifyAlbum> SearchAlbumAsync(string artist, string album)
{
var token = await _authService.GetAccessTokenAsync();
var query = $"artist:{artist} album:{album}";
var encodedQuery = Uri.EscapeDataString(query);
var request = new HttpRequestMessage(
HttpMethod.Get,
$"https://api.spotify.com/v1/search?q={encodedQuery}&type=album&limit=1"
);
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
var response = await _httpClient.SendAsync(request);
var data = await response.Content.ReadFromJsonAsync<SpotifySearchResponse>();
return data.Albums.Items.FirstOrDefault();
}
```
**Get Album Details**:
```csharp
public async Task<SpotifyAlbum> GetAlbumAsync(string spotifyId)
{
var token = await _authService.GetAccessTokenAsync();
var request = new HttpRequestMessage(
HttpMethod.Get,
$"https://api.spotify.com/v1/albums/{spotifyId}"
);
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
var response = await _httpClient.SendAsync(request);
return await response.Content.ReadFromJsonAsync<SpotifyAlbum>();
}
```
#### Data Extraction
```csharp
public AlbumMetadata ExtractMetadata(SpotifyAlbum spotifyAlbum)
{
return new AlbumMetadata
{
Title = spotifyAlbum.Name,
Artist = spotifyAlbum.Artists.First().Name,
ReleaseDate = DateTime.Parse(spotifyAlbum.ReleaseDate),
Genres = spotifyAlbum.Genres,
CoverArtUrl = spotifyAlbum.Images.FirstOrDefault()?.Url,
ExternalIds = new Dictionary<string, string>
{
["spotify"] = spotifyAlbum.Id,
["spotify_uri"] = spotifyAlbum.Uri
},
Popularity = spotifyAlbum.Popularity,
Tracks = spotifyAlbum.Tracks.Items.Select(t => new TrackMetadata
{
Title = t.Name,
Position = t.TrackNumber,
Duration = t.DurationMs / 1000,
PreviewUrl = t.PreviewUrl
}).ToList()
};
}
```
#### Rate Limiting
Spotify enforces rate limits per application (not per user):
```csharp
public class SpotifyRateLimiter
{
private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(3, 3); // 3 concurrent requests
private readonly Queue<DateTime> _requestTimes = new Queue<DateTime>();
private readonly int _maxRequestsPerMinute = 180;
public async Task WaitForSlotAsync()
{
await _semaphore.WaitAsync();
try
{
var now = DateTime.UtcNow;
// Remove requests older than 1 minute
while (_requestTimes.Count > 0 && _requestTimes.Peek() < now.AddMinutes(-1))
_requestTimes.Dequeue();
// Wait if at rate limit
if (_requestTimes.Count >= _maxRequestsPerMinute)
{
var oldestRequest = _requestTimes.Peek();
var waitTime = oldestRequest.AddMinutes(1) - now;
if (waitTime > TimeSpan.Zero)
await Task.Delay(waitTime);
}
_requestTimes.Enqueue(now);
}
finally
{
_semaphore.Release();
}
}
}
```
### Last.fm
**Type**: Social metadata and scrobbling
**Priority**: 70
**Authentication**: API key + shared secret
**Rate Limit**: Varies by method (typically unlimited for reads, limited for writes)
**Cache Strategy**: In-memory cache with 24-hour expiration
#### Overview
Last.fm provides social metadata (tags, similar artists, listener counts) and scrobbling services. With 20+ years of listening data, Last.fm offers unique insights into music popularity and relationships.
Melodee uses Last.fm for:
- **Tags**: User-generated genre tags
- **Similar artists**: Collaborative filtering recommendations
- **Play counts**: Global and user-specific listening statistics
- **Scrobbling**: Submitting listening history
#### Authentication
Last.fm uses API key authentication with MD5 signature for write operations:
```csharp
public class LastFmAuthService
{
private readonly string _apiKey;
private readonly string _sharedSecret;
public string GenerateSignature(Dictionary<string, string> parameters)
{
var sorted = parameters.OrderBy(p => p.Key);
var signatureString = string.Join("", sorted.Select(p => $"{p.Key}{p.Value}"));
signatureString += _sharedSecret;
using var md5 = MD5.Create();
var hash = md5.ComputeHash(Encoding.UTF8.GetBytes(signatureString));
return BitConverter.ToString(hash).Replace("-", "").ToLower();
}
}
```
#### API Queries
**Get Album Info**:
```csharp
public async Task<LastFmAlbum> GetAlbumInfoAsync(string artist, string album)
{
var parameters = new Dictionary<string, string>
{
["method"] = "album.getInfo",
["artist"] = artist,
["album"] = album,
["api_key"] = _apiKey,
["format"] = "json"
};
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
var response = await _httpClient.GetAsync($"https://ws.audioscrobbler.com/2.0/?{queryString}");
var data = await response.Content.ReadFromJsonAsync<LastFmAlbumResponse>();
return data.Album;
}
```
**Get Similar Artists**:
```csharp
public async Task<List<LastFmArtist>> GetSimilarArtistsAsync(string artist)
{
var parameters = new Dictionary<string, string>
{
["method"] = "artist.getSimilar",
["artist"] = artist,
["api_key"] = _apiKey,
["format"] = "json",
["limit"] = "20"
};
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
var response = await _httpClient.GetAsync($"https://ws.audioscrobbler.com/2.0/?{queryString}");
var data = await response.Content.ReadFromJsonAsync<LastFmSimilarArtistsResponse>();
return data.SimilarArtists.Artist;
}
```
#### Data Extraction
```csharp
public AlbumMetadata ExtractMetadata(LastFmAlbum lastFmAlbum)
{
return new AlbumMetadata
{
Title = lastFmAlbum.Name,
Artist = lastFmAlbum.Artist,
Tags = lastFmAlbum.Tags.Tag.Select(t => t.Name).ToList(),
PlayCount = lastFmAlbum.PlayCount,
Listeners = lastFmAlbum.Listeners,
Wiki = lastFmAlbum.Wiki?.Summary,
ExternalIds = new Dictionary<string, string>
{
["lastfm"] = lastFmAlbum.Url
}
};
}
```
### iTunes
**Type**: Commercial metadata
**Priority**: 60
**Authentication**: None (public API)
**Rate Limit**: 20 requests per minute (unofficial, enforced by Apple)
**Cache Strategy**: In-memory cache with 1-hour expiration
#### Overview
iTunes Search API provides metadata for music available in the iTunes Store. Coverage is strong for mainstream releases but limited for independent or regional music.
Melodee uses iTunes for:
- **Album art**: High-quality cover images
- **Preview URLs**: 30-second audio previews
- **Pricing data**: Commercial availability
- **Release dates**: Official release information
#### API Queries
**Search for Album**:
```csharp
public async Task<ItunesAlbum> SearchAlbumAsync(string artist, string album)
{
var query = $"{artist} {album}";
var encodedQuery = Uri.EscapeDataString(query);
var response = await _httpClient.GetAsync(
$"https://itunes.apple.com/search?term={encodedQuery}&entity=album&limit=1"
);
var data = await response.Content.ReadFromJsonAsync<ItunesSearchResponse>();
return data.Results.FirstOrDefault();
}
```
#### Data Extraction
```csharp
public AlbumMetadata ExtractMetadata(ItunesAlbum itunesAlbum)
{
return new AlbumMetadata
{
Title = itunesAlbum.CollectionName,
Artist = itunesAlbum.ArtistName,
ReleaseDate = DateTime.Parse(itunesAlbum.ReleaseDate),
Genre = itunesAlbum.PrimaryGenreName,
CoverArtUrl = itunesAlbum.ArtworkUrl100.Replace("100x100", "600x600"),
TrackCount = itunesAlbum.TrackCount,
ExternalIds = new Dictionary<string, string>
{
["itunes"] = itunesAlbum.CollectionId.ToString(),
["itunes_artist"] = itunesAlbum.ArtistId.ToString()
}
};
}
```
### Deezer
**Type**: European market metadata
**Priority**: 50
**Authentication**: None (public API)
**Rate Limit**: 50 requests per 5 seconds
**Cache Strategy**: In-memory cache with 1-hour expiration
#### Overview
Deezer provides strong coverage for European music markets, particularly French, German, and Italian releases. The API is free and doesn't require authentication for basic queries.
Melodee uses Deezer for:
- **European releases**: Better coverage than US-centric services
- **Album art**: High-quality cover images
- **Track listings**: Accurate track metadata
- **Genre data**: European genre classifications
#### API Queries
**Search for Album**:
```csharp
public async Task<DeezerAlbum> SearchAlbumAsync(string artist, string album)
{
var query = $"{artist} {album}";
var encodedQuery = Uri.EscapeDataString(query);
var response = await _httpClient.GetAsync(
$"https://api.deezer.com/search/album?q={encodedQuery}&limit=1"
);
var data = await response.Content.ReadFromJsonAsync<DeezerSearchResponse>();
return data.Data.FirstOrDefault();
}
```
**Get Album Details**:
```csharp
public async Task<DeezerAlbum> GetAlbumAsync(int deezerId)
{
var response = await _httpClient.GetAsync($"https://api.deezer.com/album/{deezerId}");
return await response.Content.ReadFromJsonAsync<DeezerAlbum>();
}
```
#### Data Extraction
```csharp
public AlbumMetadata ExtractMetadata(DeezerAlbum deezerAlbum)
{
return new AlbumMetadata
{
Title = deezerAlbum.Title,
Artist = deezerAlbum.Artist.Name,
ReleaseDate = DateTime.Parse(deezerAlbum.ReleaseDate),
Genre = deezerAlbum.Genres.Data.FirstOrDefault()?.Name,
CoverArtUrl = deezerAlbum.CoverXl,
TrackCount = deezerAlbum.NbTracks,
Duration = deezerAlbum.Duration,
ExternalIds = new Dictionary<string, string>
{
["deezer"] = deezerAlbum.Id.ToString(),
["deezer_artist"] = deezerAlbum.Artist.Id.ToString()
}
};
}
```
### Brave Search
**Type**: Fallback web search
**Priority**: 10 (lowest)
**Authentication**: API key
**Rate Limit**: Varies by plan (free tier: 2000 queries per month)
**Cache Strategy**: In-memory cache with 7-day expiration
#### Overview
Brave Search serves as a fallback when other providers don't have metadata for obscure releases. The search API can find album information from web sources.
Melodee uses Brave Search for:
- **Obscure releases**: Independent labels, regional music
- **Missing metadata**: When all other providers fail
- **Web scraping**: Extracting data from music websites
#### API Queries
**Search for Album**:
```csharp
public async Task<BraveSearchResult> SearchAlbumAsync(string artist, string album)
{
var query = $"{artist} {album} album";
var encodedQuery = Uri.EscapeDataString(query);
var request = new HttpRequestMessage(
HttpMethod.Get,
$"https://api.search.brave.com/res/v1/web/search?q={encodedQuery}"
);
request.Headers.Add("X-Subscription-Token", _apiKey);
var response = await _httpClient.SendAsync(request);
return await response.Content.ReadFromJsonAsync<BraveSearchResult>();
}
```
#### Data Extraction
Brave Search returns web results, not structured metadata. Melodee must parse HTML or extract data from snippets:
```csharp
public AlbumMetadata ExtractMetadata(BraveSearchResult searchResult)
{
var firstResult = searchResult.Web.Results.FirstOrDefault();
if (firstResult == null)
return null;
// Extract year from snippet using regex
var yearMatch = Regex.Match(firstResult.Description, @"\b(19|20)\d{2}\b");
var year = yearMatch.Success ? int.Parse(yearMatch.Value) : (int?)null;
return new AlbumMetadata
{
Title = ExtractAlbumTitle(firstResult.Title),
Artist = ExtractArtistName(firstResult.Title),
ReleaseDate = year.HasValue ? new DateTime(year.Value, 1, 1) : null,
ExternalIds = new Dictionary<string, string>
{
["web_source"] = firstResult.Url
}
};
}
```
This approach is fragile and depends on consistent web page structures. Brave Search is truly a last resort.
## Metadata Aggregation Strategy
### Provider Priority
Providers are queried in priority order. Higher-priority providers override lower-priority providers for conflicting data:
```csharp
public class MetadataAggregator
{
private readonly List<IMetadataProvider> _providers;
public MetadataAggregator(IEnumerable<IMetadataProvider> providers)
{
_providers = providers.OrderByDescending(p => p.Priority).ToList();
}
public async Task<AlbumMetadata> AggregateAlbumMetadataAsync(string artist, string album)
{
var results = new List<AlbumMetadata>();
foreach (var provider in _providers)
{
try
{
var metadata = await provider.GetAlbumMetadataAsync(artist, album);
if (metadata != null)
{
metadata.Source = provider.Name;
results.Add(metadata);
}
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Provider {Provider} failed for {Artist} - {Album}",
provider.Name, artist, album);
}
}
return MergeMetadata(results);
}
}
```
### Merge Strategy
```csharp
private AlbumMetadata MergeMetadata(List<AlbumMetadata> results)
{
if (!results.Any())
return null;
var merged = new AlbumMetadata();
// Title: prefer highest-priority provider
merged.Title = results.FirstOrDefault(r => !string.IsNullOrEmpty(r.Title))?.Title;
// Artist: prefer highest-priority provider
merged.Artist = results.FirstOrDefault(r => !string.IsNullOrEmpty(r.Artist))?.Artist;
// Release date: prefer most specific date
merged.ReleaseDate = results
.Where(r => r.ReleaseDate.HasValue)
.OrderByDescending(r => r.ReleaseDate.Value)
.FirstOrDefault()?.ReleaseDate;
// Genres: combine from all providers, deduplicate
merged.Genres = results
.SelectMany(r => r.Genres ?? new List<string>())
.Distinct()
.ToList();
// Cover art: prefer highest resolution
merged.CoverArtUrl = results
.Where(r => !string.IsNullOrEmpty(r.CoverArtUrl))
.OrderByDescending(r => EstimateImageResolution(r.CoverArtUrl))
.FirstOrDefault()?.CoverArtUrl;
// External IDs: combine from all providers
merged.ExternalIds = results
.SelectMany(r => r.ExternalIds ?? new Dictionary<string, string>())
.GroupBy(kvp => kvp.Key)
.ToDictionary(g => g.Key, g => g.First().Value);
return merged;
}
```
## Scrobbling Integrations
### Last.fm Scrobbling
**Authentication**: Session key (obtained via OAuth)
**Rate Limit**: 50 scrobbles per request, unlimited requests
#### Session Key Acquisition
Users must authorize Melodee to scrobble on their behalf:
```csharp
public class LastFmScrobblingService
{
public string GetAuthorizationUrl()
{
return $"https://www.last.fm/api/auth/?api_key={_apiKey}&cb={_callbackUrl}";
}
public async Task<string> GetSessionKeyAsync(string token)
{
var parameters = new Dictionary<string, string>
{
["method"] = "auth.getSession",
["api_key"] = _apiKey,
["token"] = token
};
var signature = _authService.GenerateSignature(parameters);
parameters["api_sig"] = signature;
parameters["format"] = "json";
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
var response = await _httpClient.GetAsync($"https://ws.audioscrobbler.com/2.0/?{queryString}");
var data = await response.Content.ReadFromJsonAsync<LastFmSessionResponse>();
return data.Session.Key;
}
}
```
#### Scrobble Submission
```csharp
public async Task ScrobbleAsync(int userId, int trackId, DateTime playedAt)
{
var user = await _userRepository.GetByIdAsync(userId);
if (string.IsNullOrEmpty(user.Settings.LastFmSessionKey))
return;
var track = await _trackRepository.GetByIdAsync(trackId);
var parameters = new Dictionary<string, string>
{
["method"] = "track.scrobble",
["artist"] = track.Artist.Name,
["track"] = track.Title,
["album"] = track.Album.Title,
["timestamp"] = ((DateTimeOffset)playedAt).ToUnixTimeSeconds().ToString(),
["api_key"] = _apiKey,
["sk"] = user.Settings.LastFmSessionKey
};
var signature = _authService.GenerateSignature(parameters);
parameters["api_sig"] = signature;
parameters["format"] = "json";
var content = new FormUrlEncodedContent(parameters);
var response = await _httpClient.PostAsync("https://ws.audioscrobbler.com/2.0/", content);
if (!response.IsSuccessStatusCode)
{
_logger.LogError("Last.fm scrobble failed: {Status}", response.StatusCode);
throw new ScrobbleException("Last.fm scrobble failed");
}
}
```
#### Batch Scrobbling
Last.fm supports batch scrobbling (up to 50 tracks per request):
```csharp
public async Task BatchScrobbleAsync(int userId, List<Scrobble> scrobbles)
{
var user = await _userRepository.GetByIdAsync(userId);
if (string.IsNullOrEmpty(user.Settings.LastFmSessionKey))
return;
var parameters = new Dictionary<string, string>
{
["method"] = "track.scrobble",
["api_key"] = _apiKey,
["sk"] = user.Settings.LastFmSessionKey
};
for (int i = 0; i < scrobbles.Count; i++)
{
var scrobble = scrobbles[i];
var track = await _trackRepository.GetByIdAsync(scrobble.TrackId);
parameters[$"artist[{i}]"] = track.Artist.Name;
parameters[$"track[{i}]"] = track.Title;
parameters[$"album[{i}]"] = track.Album.Title;
parameters[$"timestamp[{i}]"] = ((DateTimeOffset)scrobble.PlayedAt).ToUnixTimeSeconds().ToString();
}
var signature = _authService.GenerateSignature(parameters);
parameters["api_sig"] = signature;
parameters["format"] = "json";
var content = new FormUrlEncodedContent(parameters);
var response = await _httpClient.PostAsync("https://ws.audioscrobbler.com/2.0/", content);
if (response.IsSuccessStatusCode)
{
foreach (var scrobble in scrobbles)
{
scrobble.SubmittedToLastFm = true;
scrobble.LastFmSubmittedAt = DateTime.UtcNow;
}
await _scrobbleRepository.UpdateRangeAsync(scrobbles);
}
}
```
### Internal Melodee Scrobbler
Melodee maintains its own scrobble database for analytics and charts:
```csharp
public class MelodeeScrobbler : IScrobbler
{
private readonly IScrobbleRepository _scrobbleRepository;
public async Task ScrobbleAsync(int userId, int trackId, DateTime playedAt)
{
var scrobble = new Scrobble
{
UserId = userId,
TrackId = trackId,
PlayedAt = playedAt,
CreatedAt = DateTime.UtcNow
};
await _scrobbleRepository.CreateAsync(scrobble);
}
}
```
### Composite Scrobbler
Both scribblers run in parallel:
```csharp
public class CompositeScrobbler : IScrobbler
{
private readonly List<IScrobbler> _scribblers;
public async Task ScrobbleAsync(int userId, int trackId, DateTime playedAt)
{
var tasks = _scribblers.Select(s => s.ScrobbleAsync(userId, trackId, playedAt));
await Task.WhenAll(tasks);
}
}
```
## Authentication Integrations
### Google OAuth
**Flow**: Authorization Code Flow
**Scopes**: `openid`, `email`, `profile`
**Token Storage**: Access tokens not stored (only used during authentication)
#### Authorization Flow
```csharp
public class GoogleOAuthService
{
public string GetAuthorizationUrl(string state)
{
var parameters = new Dictionary<string, string>
{
["client_id"] = _clientId,
["redirect_uri"] = _redirectUri,
["response_type"] = "code",
["scope"] = "openid email profile",
["state"] = state
};
var queryString = string.Join("&", parameters.Select(p => $"{p.Key}={Uri.EscapeDataString(p.Value)}"));
return $"https://accounts.google.com/o/oauth2/v2/auth?{queryString}";
}
public async Task<GoogleTokenResponse> ExchangeCodeForTokenAsync(string code)
{
var parameters = new Dictionary<string, string>
{
["code"] = code,
["client_id"] = _clientId,
["client_secret"] = _clientSecret,
["redirect_uri"] = _redirectUri,
["grant_type"] = "authorization_code"
};
var content = new FormUrlEncodedContent(parameters);
var response = await _httpClient.PostAsync("https://oauth2.googleapis.com/token", content);
return await response.Content.ReadFromJsonAsync<GoogleTokenResponse>();
}
public async Task<GoogleUserInfo> GetUserInfoAsync(string accessToken)
{
var request = new HttpRequestMessage(
HttpMethod.Get,
"https://www.googleapis.com/oauth2/v1/userinfo"
);
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
var response = await _httpClient.SendAsync(request);
return await response.Content.ReadFromJsonAsync<GoogleUserInfo>();
}
}
```
#### User Account Creation
```csharp
public async Task<User> AuthenticateWithGoogleAsync(string code)
{
var tokenResponse = await _googleOAuthService.ExchangeCodeForTokenAsync(code);
var userInfo = await _googleOAuthService.GetUserInfoAsync(tokenResponse.AccessToken);
var user = await _userRepository.GetByGoogleIdAsync(userInfo.Id);
if (user == null)
{
user = new User
{
Email = userInfo.Email,
GoogleId = userInfo.Id,
ProfilePictureUrl = userInfo.Picture,
Role = "user",
IsActive = true,
CreatedAt = DateTime.UtcNow
};
await _userRepository.CreateAsync(user);
}
else
{
user.ProfilePictureUrl = userInfo.Picture;
user.LastLoginAt = DateTime.UtcNow;
await _userRepository.UpdateAsync(user);
}
return user;
}
```
## Integration Resilience
### Circuit Breaker Pattern
Prevents cascading failures when providers are down:
```csharp
public class CircuitBreakerMetadataProvider : IMetadataProvider
{
private readonly IMetadataProvider _inner;
private int _failureCount;
private DateTime? _circuitOpenedAt;
private readonly int _failureThreshold = 5;
private readonly TimeSpan _resetTimeout = TimeSpan.FromMinutes(5);
public async Task<AlbumMetadata> GetAlbumMetadataAsync(string artist, string album)
{
if (_circuitOpenedAt.HasValue)
{
if (DateTime.UtcNow - _circuitOpenedAt.Value < _resetTimeout)
throw new CircuitBreakerOpenException($"Circuit breaker open for {Name}");
// Try to close circuit
_circuitOpenedAt = null;
_failureCount = 0;
}
try
{
var result = await _inner.GetAlbumMetadataAsync(artist, album);
_failureCount = 0;
return result;
}
catch (Exception ex)
{
_failureCount++;
if (_failureCount >= _failureThreshold)
{
_circuitOpenedAt = DateTime.UtcNow;
_logger.LogError("Circuit breaker opened for {Provider}", Name);
}
throw;
}
}
}
```
### Retry with Exponential Backoff
```csharp
public class RetryMetadataProvider : IMetadataProvider
{
private readonly IMetadataProvider _inner;
private readonly int _maxRetries = 3;
public async Task<AlbumMetadata> GetAlbumMetadataAsync(string artist, string album)
{
for (int attempt = 0; attempt < _maxRetries; attempt++)
{
try
{
return await _inner.GetAlbumMetadataAsync(artist, album);
}
catch (HttpRequestException ex) when (attempt < _maxRetries - 1)
{
var delay = TimeSpan.FromSeconds(Math.Pow(2, attempt));
_logger.LogWarning("Retry {Attempt} for {Provider} after {Delay}s",
attempt + 1, Name, delay.TotalSeconds);
await Task.Delay(delay);
}
}
throw new MetadataProviderException($"Failed after {_maxRetries} retries");
}
}
```
## Conclusion
Melodee's integration strategy demonstrates sophisticated metadata aggregation from six diverse providers. The priority-based merging, local MusicBrainz cache, and resilience patterns (circuit breaker, retry) create a robust system that maximizes metadata quality while minimizing external dependencies.
Key strengths:
- **Multi-provider redundancy**: Provider failures don't block library operations
- **Local caching**: MusicBrainz SQLite cache eliminates most API calls
- **Intelligent merging**: Priority-based conflict resolution
- **Scrobbling**: Dual submission to Last.fm and internal database
Key challenges:
- **API key management**: Six providers require configuration
- **Rate limiting**: Complex coordination across providers
- **Data consistency**: Merging conflicting metadata from multiple sources
- **Maintenance**: Provider API changes require ongoing updates
The architecture positions Melodee as a metadata-first music server, prioritizing data quality over simplicity.