a1f6701bac
- gRPC service with MusicBrainz provider - PostgreSQL schema with migrations - Service layer with database-first caching - Repository pattern for data access - YAML configuration support - Research documentation for 17 music metadata projects
1283 lines
34 KiB
Markdown
1283 lines
34 KiB
Markdown
# Bedrock-API Architecture
|
|
|
|
## System Layers
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Transport Layer │
|
|
│ gRPC Server (:50052) │ HTTP Proxy (:8080) │
|
|
│ 23 RPC Methods │ /stream, /cover │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Service Layer │
|
|
│ main.go (1329 lines) │ resolver.go │ auth.go │
|
|
│ proxy.go │ lrclib.go │ genius.go │
|
|
│ - Request routing │
|
|
│ - Fan-out orchestration │
|
|
│ - Response aggregation │
|
|
│ - JWT validation (interceptor) │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Provider Adapter Layer │
|
|
│ spotify.go │ soundcloud.go │ deezer.go │
|
|
│ youtube.go │ yandex.go (stub) │ vk.go (stub) │
|
|
│ - Platform-specific API calls │
|
|
│ - Response normalization │
|
|
│ - Error handling │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Data Layer │
|
|
│ store/user.go │ db/migrations/ │
|
|
│ - PostgreSQL (pgx/v5) │
|
|
│ - User CRUD operations │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Transport Layer
|
|
|
|
### gRPC Server
|
|
|
|
**File**: `bedrock_server/main.go` (server initialization)
|
|
**Port**: `:50052`
|
|
**Protocol**: gRPC over HTTP/2
|
|
**Security**: No TLS (insecure credentials)
|
|
|
|
**Server Configuration**:
|
|
```go
|
|
grpcServer := grpc.NewServer(
|
|
grpc.UnaryInterceptor(authInterceptor),
|
|
grpc.StreamInterceptor(streamAuthInterceptor),
|
|
)
|
|
pb.RegisterBedrockServiceServer(grpcServer, &server{})
|
|
```
|
|
|
|
**Interceptors**:
|
|
- `authInterceptor`: Validates JWT on unary RPCs
|
|
- `streamAuthInterceptor`: Validates JWT on streaming RPCs
|
|
|
|
**Public Methods** (bypass auth):
|
|
- `Register`
|
|
- `Login`
|
|
- `RefreshToken`
|
|
- `GetServiceStatus`
|
|
|
|
### HTTP Proxy Server
|
|
|
|
**File**: `bedrock_server/proxy.go`
|
|
**Port**: `:8080`
|
|
**Routes**:
|
|
|
|
| Route | Method | Purpose | Range Support |
|
|
|-------|--------|---------|---------------|
|
|
| `/stream/{service}/{id}` | GET | Audio stream proxy | Yes |
|
|
| `/cover/{service}/{id}` | GET | Album art proxy | Yes |
|
|
|
|
**Range Request Handling**:
|
|
```go
|
|
rangeHeader := r.Header.Get("Range")
|
|
if rangeHeader != "" {
|
|
req.Header.Set("Range", rangeHeader)
|
|
// Forward range request to upstream
|
|
}
|
|
```
|
|
|
|
**Response Headers**:
|
|
- `Content-Type`: Forwarded from upstream
|
|
- `Content-Length`: Forwarded from upstream
|
|
- `Accept-Ranges`: `bytes`
|
|
- `Content-Range`: Forwarded from upstream (206 responses)
|
|
|
|
**Error Responses**:
|
|
- 400: Invalid service or ID
|
|
- 404: Stream not found
|
|
- 500: Upstream fetch failure
|
|
|
|
## Service Layer
|
|
|
|
### Main Service Implementation
|
|
|
|
**File**: `bedrock_server/main.go`
|
|
**Lines**: 1329
|
|
**Type**: Monolithic service struct implementing all 23 RPC methods
|
|
|
|
**Core Structure**:
|
|
```go
|
|
type server struct {
|
|
pb.UnimplementedBedrockServiceServer
|
|
db *pgxpool.Pool
|
|
jwtSecret []byte
|
|
}
|
|
|
|
func (s *server) SearchTracks(ctx context.Context, req *pb.SearchRequest) (*pb.SearchTracksResponse, error) {
|
|
// Fan-out to all providers
|
|
// Aggregate results
|
|
// Return with status
|
|
}
|
|
```
|
|
|
|
### Fan-Out Concurrency Pattern
|
|
|
|
**Implementation**: Every search and retrieval method uses parallel goroutines
|
|
|
|
**Example** (SearchTracks):
|
|
```go
|
|
var (
|
|
mu sync.Mutex
|
|
wg sync.WaitGroup
|
|
allTracks []*pb.Track
|
|
errors []*pb.ProviderError
|
|
)
|
|
|
|
providers := []trackProvider{spotifyProvider, soundcloudProvider, deezerProvider, youtubeProvider}
|
|
|
|
for _, provider := range providers {
|
|
wg.Add(1)
|
|
go func(p trackProvider) {
|
|
defer wg.Done()
|
|
|
|
tracks, err := p.SearchTracks(ctx, req.Query, req.Limit)
|
|
|
|
mu.Lock()
|
|
defer mu.Unlock()
|
|
|
|
if err != nil {
|
|
errors = append(errors, &pb.ProviderError{
|
|
Provider: p.Name(),
|
|
Message: err.Error(),
|
|
})
|
|
} else {
|
|
allTracks = append(allTracks, tracks...)
|
|
}
|
|
}(provider)
|
|
}
|
|
|
|
wg.Wait()
|
|
|
|
status := pb.ResponseStatus_OK
|
|
if len(errors) > 0 {
|
|
if len(allTracks) == 0 {
|
|
status = pb.ResponseStatus_ERROR
|
|
} else {
|
|
status = pb.ResponseStatus_PARTIAL
|
|
}
|
|
}
|
|
|
|
return &pb.SearchTracksResponse{
|
|
Tracks: allTracks,
|
|
Status: status,
|
|
Errors: errors,
|
|
}, nil
|
|
```
|
|
|
|
**Characteristics**:
|
|
- No timeout enforcement (relies on context cancellation)
|
|
- Mutex-protected result aggregation
|
|
- Partial success handling
|
|
- Error collection per provider
|
|
|
|
### Stream Resolution Bridge
|
|
|
|
**File**: `bedrock_server/resolver.go`
|
|
**Purpose**: Resolve streaming URLs for platforms that don't provide them
|
|
|
|
**Algorithm**:
|
|
|
|
```
|
|
Input: Platform ID (e.g., "spotify:track:abc123")
|
|
|
|
1. Parse platform and native ID from namespaced ID
|
|
2. If platform is SoundCloud or YouTube Music:
|
|
- Call platform's GetStreamURL directly
|
|
- Return URL
|
|
3. If platform is Spotify or Deezer:
|
|
- Get track metadata (artist, title)
|
|
- Search SoundCloud for "{artist} - {title}"
|
|
- If SoundCloud returns results:
|
|
- Get stream URL from first result
|
|
- Return URL
|
|
- If SoundCloud fails:
|
|
- Search YouTube Music for "{artist} - {title}"
|
|
- Get stream URL from first result
|
|
- Return URL
|
|
4. If all attempts fail:
|
|
- Return error "no stream available"
|
|
```
|
|
|
|
**Fallback Chain**:
|
|
```
|
|
Non-streaming platform (Spotify/Deezer)
|
|
↓
|
|
SoundCloud search + GetStreamURL
|
|
↓ (on failure)
|
|
YouTube Music search + GetStreamURL
|
|
↓ (on failure)
|
|
Error response
|
|
```
|
|
|
|
**Code Structure**:
|
|
```go
|
|
func (s *server) resolveStreamURL(ctx context.Context, platformID string) (string, error) {
|
|
platform, nativeID := parseNamespacedID(platformID)
|
|
|
|
switch platform {
|
|
case "soundcloud", "youtube":
|
|
return s.getDirectStreamURL(ctx, platform, nativeID)
|
|
case "spotify", "deezer":
|
|
track, err := s.getTrackMetadata(ctx, platformID)
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
|
|
query := fmt.Sprintf("%s - %s", track.Artist, track.Title)
|
|
|
|
// Try SoundCloud first
|
|
scURL, err := s.searchAndStream(ctx, "soundcloud", query)
|
|
if err == nil {
|
|
return scURL, nil
|
|
}
|
|
|
|
// Fallback to YouTube Music
|
|
ytURL, err := s.searchAndStream(ctx, "youtube", query)
|
|
if err == nil {
|
|
return ytURL, nil
|
|
}
|
|
|
|
return "", errors.New("no stream available")
|
|
default:
|
|
return "", errors.New("unsupported platform")
|
|
}
|
|
}
|
|
```
|
|
|
|
### Authentication Service
|
|
|
|
**File**: `bedrock_server/auth.go`
|
|
|
|
**Components**:
|
|
|
|
1. **Password Hashing**:
|
|
```go
|
|
func hashPassword(password string) (string, error) {
|
|
bytes, err := bcrypt.GenerateFromPassword([]byte(password), 10)
|
|
return string(bytes), err
|
|
}
|
|
|
|
func checkPasswordHash(password, hash string) bool {
|
|
err := bcrypt.CompareHashAndPassword([]byte(hash), []byte(password))
|
|
return err == nil
|
|
}
|
|
```
|
|
|
|
2. **JWT Generation**:
|
|
```go
|
|
func (s *server) generateTokens(userID, email string) (access, refresh string, err error) {
|
|
accessClaims := jwt.MapClaims{
|
|
"user_id": userID,
|
|
"email": email,
|
|
"exp": time.Now().Add(15 * time.Minute).Unix(),
|
|
}
|
|
accessToken := jwt.NewWithClaims(jwt.SigningMethodHS256, accessClaims)
|
|
access, err = accessToken.SignedString(s.jwtSecret)
|
|
|
|
refreshClaims := jwt.MapClaims{
|
|
"user_id": userID,
|
|
"email": email,
|
|
"exp": time.Now().Add(7 * 24 * time.Hour).Unix(),
|
|
}
|
|
refreshToken := jwt.NewWithClaims(jwt.SigningMethodHS256, refreshClaims)
|
|
refresh, err = refreshToken.SignedString(s.jwtSecret)
|
|
|
|
return
|
|
}
|
|
```
|
|
|
|
3. **gRPC Interceptors**:
|
|
```go
|
|
func (s *server) authInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
|
|
publicMethods := map[string]bool{
|
|
"/bedrock.BedrockService/Register": true,
|
|
"/bedrock.BedrockService/Login": true,
|
|
"/bedrock.BedrockService/RefreshToken": true,
|
|
"/bedrock.BedrockService/GetServiceStatus": true,
|
|
}
|
|
|
|
if publicMethods[info.FullMethod] {
|
|
return handler(ctx, req)
|
|
}
|
|
|
|
md, ok := metadata.FromIncomingContext(ctx)
|
|
if !ok {
|
|
return nil, status.Error(codes.Unauthenticated, "missing metadata")
|
|
}
|
|
|
|
tokens := md.Get("authorization")
|
|
if len(tokens) == 0 {
|
|
return nil, status.Error(codes.Unauthenticated, "missing token")
|
|
}
|
|
|
|
token := strings.TrimPrefix(tokens[0], "Bearer ")
|
|
|
|
claims := jwt.MapClaims{}
|
|
_, err := jwt.ParseWithClaims(token, claims, func(t *jwt.Token) (interface{}, error) {
|
|
return s.jwtSecret, nil
|
|
})
|
|
|
|
if err != nil {
|
|
return nil, status.Error(codes.Unauthenticated, "invalid token")
|
|
}
|
|
|
|
return handler(ctx, req)
|
|
}
|
|
```
|
|
|
|
### Lyrics Services
|
|
|
|
**Files**: `bedrock_server/lrclib.go`, `bedrock_server/genius.go`
|
|
|
|
**LrcLib Integration** (Synced Lyrics):
|
|
```go
|
|
func (s *server) GetSyncedLyrics(ctx context.Context, req *pb.LyricsRequest) (*pb.SyncedLyricsResponse, error) {
|
|
client := &http.Client{Timeout: 5 * time.Second}
|
|
|
|
url := fmt.Sprintf("https://lrclib.net/api/get?artist_name=%s&track_name=%s&album_name=%s&duration=%d",
|
|
url.QueryEscape(req.Artist),
|
|
url.QueryEscape(req.Title),
|
|
url.QueryEscape(req.Album),
|
|
req.Duration,
|
|
)
|
|
|
|
resp, err := client.Get(url)
|
|
// Parse LRC format
|
|
// Return timestamped lines
|
|
}
|
|
```
|
|
|
|
**Genius Integration** (Plain Lyrics):
|
|
```go
|
|
func (s *server) GetLyrics(ctx context.Context, req *pb.LyricsRequest) (*pb.LyricsResponse, error) {
|
|
geniusClient := genius.NewClient(os.Getenv("GENIUS_ACCESS_TOKEN"))
|
|
|
|
song, err := geniusClient.Search(fmt.Sprintf("%s %s", req.Artist, req.Title))
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
lyrics, err := geniusClient.GetLyrics(song.ID)
|
|
// Return plain text + annotations
|
|
}
|
|
```
|
|
|
|
**Parallel Lyrics Fetch**:
|
|
```go
|
|
func (s *server) GetAllLyrics(ctx context.Context, req *pb.LyricsRequest) (*pb.AllLyricsResponse, error) {
|
|
var (
|
|
wg sync.WaitGroup
|
|
syncedLyrics *pb.SyncedLyricsResponse
|
|
plainLyrics *pb.LyricsResponse
|
|
)
|
|
|
|
wg.Add(2)
|
|
|
|
go func() {
|
|
defer wg.Done()
|
|
syncedLyrics, _ = s.GetSyncedLyrics(ctx, req)
|
|
}()
|
|
|
|
go func() {
|
|
defer wg.Done()
|
|
plainLyrics, _ = s.GetLyrics(ctx, req)
|
|
}()
|
|
|
|
wg.Wait()
|
|
|
|
return &pb.AllLyricsResponse{
|
|
Synced: syncedLyrics,
|
|
Plain: plainLyrics,
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
## Provider Adapter Layer
|
|
|
|
### Provider Interface
|
|
|
|
**Definition** (implicit, not formally declared):
|
|
```go
|
|
type trackProvider interface {
|
|
Name() string
|
|
SearchTracks(ctx context.Context, query string, limit int32) ([]*pb.Track, error)
|
|
SearchAlbums(ctx context.Context, query string, limit int32) ([]*pb.Album, error)
|
|
SearchArtists(ctx context.Context, query string, limit int32) ([]*pb.Artist, error)
|
|
SearchPlaylists(ctx context.Context, query string, limit int32) ([]*pb.Playlist, error)
|
|
GetTrack(ctx context.Context, id string) (*pb.Track, error)
|
|
GetAlbum(ctx context.Context, id string) (*pb.Album, error)
|
|
GetArtist(ctx context.Context, id string) (*pb.Artist, error)
|
|
GetPlaylist(ctx context.Context, id string) (*pb.Playlist, error)
|
|
GetStreamURL(ctx context.Context, id string) (string, error)
|
|
GetSimilarTracks(ctx context.Context, id string, limit int32) ([]*pb.Track, error)
|
|
}
|
|
```
|
|
|
|
**Implementations**:
|
|
- `providers/spotify.go`: SpotifyProvider
|
|
- `providers/soundcloud.go`: SoundCloudProvider
|
|
- `providers/deezer.go`: DeezerProvider
|
|
- `providers/youtube.go`: YouTubeProvider
|
|
- `providers/yandex.go`: YandexProvider (stub)
|
|
- `providers/vk.go`: VKProvider (stub)
|
|
|
|
### Spotify Provider
|
|
|
|
**File**: `providers/spotify.go`
|
|
**Dependency**: `spotapi-go` submodule (wrapper around `zmb3/spotify/v2`)
|
|
|
|
**Authentication**:
|
|
```go
|
|
func NewSpotifyProvider() *SpotifyProvider {
|
|
clientID := os.Getenv("SPOTIFY_CLIENT_ID")
|
|
clientSecret := os.Getenv("SPOTIFY_CLIENT_SECRET")
|
|
|
|
auth := spotifyauth.New(
|
|
spotifyauth.WithClientID(clientID),
|
|
spotifyauth.WithClientSecret(clientSecret),
|
|
)
|
|
|
|
token, _ := auth.Token(context.Background())
|
|
client := spotify.New(auth.Client(context.Background(), token))
|
|
|
|
return &SpotifyProvider{client: client}
|
|
}
|
|
```
|
|
|
|
**ID Namespacing**:
|
|
```go
|
|
func (p *SpotifyProvider) SearchTracks(ctx context.Context, query string, limit int32) ([]*pb.Track, error) {
|
|
results, err := p.client.Search(ctx, query, spotify.SearchTypeTrack)
|
|
|
|
tracks := make([]*pb.Track, 0, len(results.Tracks.Tracks))
|
|
for _, t := range results.Tracks.Tracks {
|
|
tracks = append(tracks, &pb.Track{
|
|
Id: fmt.Sprintf("spotify:track:%s", t.ID),
|
|
Title: t.Name,
|
|
Artist: t.Artists[0].Name,
|
|
Album: t.Album.Name,
|
|
Duration: int32(t.Duration / 1000), // ms to seconds
|
|
// ...
|
|
})
|
|
}
|
|
|
|
return tracks, nil
|
|
}
|
|
```
|
|
|
|
**No Streaming**:
|
|
```go
|
|
func (p *SpotifyProvider) GetStreamURL(ctx context.Context, id string) (string, error) {
|
|
return "", errors.New("spotify does not provide streaming URLs")
|
|
}
|
|
```
|
|
|
|
### SoundCloud Provider
|
|
|
|
**File**: `providers/soundcloud.go`
|
|
**API**: SoundCloud api-v2 (public, no official SDK)
|
|
|
|
**Client ID Rotation**:
|
|
```go
|
|
type SoundCloudProvider struct {
|
|
clientIDs []string
|
|
currentID int
|
|
mu sync.Mutex
|
|
}
|
|
|
|
func (p *SoundCloudProvider) getClientID() string {
|
|
p.mu.Lock()
|
|
defer p.mu.Unlock()
|
|
|
|
id := p.clientIDs[p.currentID]
|
|
p.currentID = (p.currentID + 1) % len(p.clientIDs)
|
|
|
|
return id
|
|
}
|
|
```
|
|
|
|
**Batch Hydration**:
|
|
```go
|
|
func (p *SoundCloudProvider) hydrateTracks(ctx context.Context, ids []string) ([]*pb.Track, error) {
|
|
// SoundCloud allows up to 30 IDs per request
|
|
chunks := chunkSlice(ids, 30)
|
|
|
|
var allTracks []*pb.Track
|
|
for _, chunk := range chunks {
|
|
url := fmt.Sprintf("https://api-v2.soundcloud.com/tracks?ids=%s&client_id=%s",
|
|
strings.Join(chunk, ","),
|
|
p.getClientID(),
|
|
)
|
|
|
|
resp, err := http.Get(url)
|
|
// Parse and append tracks
|
|
}
|
|
|
|
return allTracks, nil
|
|
}
|
|
```
|
|
|
|
**Stream URL Resolution**:
|
|
```go
|
|
func (p *SoundCloudProvider) GetStreamURL(ctx context.Context, id string) (string, error) {
|
|
// Get track info
|
|
trackURL := fmt.Sprintf("https://api-v2.soundcloud.com/tracks/%s?client_id=%s", id, p.getClientID())
|
|
|
|
var track struct {
|
|
Media struct {
|
|
Transcodings []struct {
|
|
URL string `json:"url"`
|
|
Format struct {
|
|
Protocol string `json:"protocol"`
|
|
MimeType string `json:"mime_type"`
|
|
} `json:"format"`
|
|
} `json:"transcodings"`
|
|
} `json:"media"`
|
|
}
|
|
|
|
// Fetch track data
|
|
// Select progressive MP3 transcoding
|
|
for _, t := range track.Media.Transcodings {
|
|
if t.Format.Protocol == "progressive" && strings.Contains(t.Format.MimeType, "mp3") {
|
|
// Fetch actual stream URL from transcoding URL
|
|
streamURL := fmt.Sprintf("%s?client_id=%s", t.URL, p.getClientID())
|
|
return streamURL, nil
|
|
}
|
|
}
|
|
|
|
return "", errors.New("no progressive stream found")
|
|
}
|
|
```
|
|
|
|
**URL Resolution**:
|
|
```go
|
|
func (p *SoundCloudProvider) ResolveURL(ctx context.Context, url string) (string, error) {
|
|
resolveURL := fmt.Sprintf("https://api-v2.soundcloud.com/resolve?url=%s&client_id=%s",
|
|
url.QueryEscape(url),
|
|
p.getClientID(),
|
|
)
|
|
|
|
// Returns track ID that can be used with other methods
|
|
}
|
|
```
|
|
|
|
### Deezer Provider
|
|
|
|
**File**: `providers/deezer.go`
|
|
**API**: Deezer public API (no authentication required)
|
|
|
|
**Concurrent Artist Data Fetching**:
|
|
```go
|
|
func (p *DeezerProvider) GetArtist(ctx context.Context, id string) (*pb.Artist, error) {
|
|
var (
|
|
wg sync.WaitGroup
|
|
artist *pb.Artist
|
|
albums []*pb.Album
|
|
topTracks []*pb.Track
|
|
)
|
|
|
|
wg.Add(3)
|
|
|
|
go func() {
|
|
defer wg.Done()
|
|
// Fetch artist info
|
|
url := fmt.Sprintf("https://api.deezer.com/artist/%s", id)
|
|
// Parse into artist
|
|
}()
|
|
|
|
go func() {
|
|
defer wg.Done()
|
|
// Fetch artist albums
|
|
url := fmt.Sprintf("https://api.deezer.com/artist/%s/albums", id)
|
|
// Parse into albums
|
|
}()
|
|
|
|
go func() {
|
|
defer wg.Done()
|
|
// Fetch artist top tracks
|
|
url := fmt.Sprintf("https://api.deezer.com/artist/%s/top", id)
|
|
// Parse into topTracks
|
|
}()
|
|
|
|
wg.Wait()
|
|
|
|
artist.Albums = albums
|
|
artist.TopTracks = topTracks
|
|
|
|
return artist, nil
|
|
}
|
|
```
|
|
|
|
**Duration Handling**:
|
|
```go
|
|
// Deezer returns duration in seconds, not milliseconds
|
|
track := &pb.Track{
|
|
Duration: int32(deezerTrack.Duration), // Already in seconds
|
|
}
|
|
```
|
|
|
|
**No Streaming**:
|
|
```go
|
|
func (p *DeezerProvider) GetStreamURL(ctx context.Context, id string) (string, error) {
|
|
return "", errors.New("deezer public API does not provide streaming URLs")
|
|
}
|
|
```
|
|
|
|
### YouTube Music Provider
|
|
|
|
**File**: `providers/youtube.go`
|
|
**Dependency**: `github.com/kkdai/youtube/v2`
|
|
|
|
**7-Client Fallback Pool**:
|
|
```go
|
|
var youtubeClients = []struct {
|
|
name string
|
|
client youtube.Client
|
|
}{
|
|
{"TVHTML5_SIMPLY_EMBEDDED", youtube.Client{/* config */}},
|
|
{"TVHTML5", youtube.Client{/* config */}},
|
|
{"ANDROID_VR_1", youtube.Client{/* config */}},
|
|
{"ANDROID_VR_2", youtube.Client{/* config */}},
|
|
{"ANDROID", youtube.Client{/* config */}},
|
|
{"IOS", youtube.Client{/* config */}},
|
|
{"WEB", youtube.Client{/* config */}},
|
|
}
|
|
|
|
func (p *YouTubeProvider) GetStreamURL(ctx context.Context, id string) (string, error) {
|
|
for _, clientConfig := range youtubeClients {
|
|
client := clientConfig.client
|
|
|
|
video, err := client.GetVideoContext(ctx, id)
|
|
if err != nil {
|
|
log.Printf("[youtube] Client %s failed: %v", clientConfig.name, err)
|
|
continue
|
|
}
|
|
|
|
// Check for cipher (encrypted stream)
|
|
if video.Formats[0].Cipher != "" {
|
|
log.Printf("[youtube] Client %s returned ciphered stream, skipping", clientConfig.name)
|
|
continue
|
|
}
|
|
|
|
// Select best format by itag priority
|
|
streamURL := p.selectBestFormat(video.Formats)
|
|
if streamURL != "" {
|
|
return streamURL, nil
|
|
}
|
|
}
|
|
|
|
// All clients failed, fallback to SoundCloud
|
|
return p.fallbackToSoundCloud(ctx, id)
|
|
}
|
|
```
|
|
|
|
**Itag Priority** (audio quality):
|
|
```go
|
|
func (p *YouTubeProvider) selectBestFormat(formats youtube.FormatList) string {
|
|
// Priority: 251 (opus) > 140 (aac)
|
|
itagPriority := []int{251, 140}
|
|
|
|
for _, itag := range itagPriority {
|
|
for _, format := range formats {
|
|
if format.ItagNo == itag {
|
|
return format.URL
|
|
}
|
|
}
|
|
}
|
|
|
|
// Fallback to first available audio format
|
|
for _, format := range formats {
|
|
if strings.Contains(format.MimeType, "audio") {
|
|
return format.URL
|
|
}
|
|
}
|
|
|
|
return ""
|
|
}
|
|
```
|
|
|
|
**Metadata Client** (WEB_REMIX):
|
|
```go
|
|
func (p *YouTubeProvider) SearchTracks(ctx context.Context, query string, limit int32) ([]*pb.Track, error) {
|
|
// Use WEB_REMIX client (client 67) for YouTube Music metadata
|
|
client := youtube.Client{
|
|
ClientName: "WEB_REMIX",
|
|
ClientVersion: "1.20231122.01.00",
|
|
}
|
|
|
|
// Search YouTube Music, not regular YouTube
|
|
searchURL := fmt.Sprintf("https://music.youtube.com/youtubei/v1/search?key=%s", apiKey)
|
|
// Parse music-specific results
|
|
}
|
|
```
|
|
|
|
**Cookie Support** (age-restricted content):
|
|
```go
|
|
func NewYouTubeProvider() *YouTubeProvider {
|
|
cookies := os.Getenv("YOUTUBE_COOKIES")
|
|
|
|
client := youtube.Client{}
|
|
if cookies != "" {
|
|
client.HTTPClient = &http.Client{
|
|
Transport: &cookieTransport{cookies: cookies},
|
|
}
|
|
}
|
|
|
|
return &YouTubeProvider{client: client}
|
|
}
|
|
```
|
|
|
|
### Stub Providers
|
|
|
|
**Files**: `providers/yandex.go`, `providers/vk.go`
|
|
|
|
**Implementation**:
|
|
```go
|
|
type YandexProvider struct{}
|
|
|
|
func (p *YandexProvider) Name() string {
|
|
return "yandex"
|
|
}
|
|
|
|
func (p *YandexProvider) SearchTracks(ctx context.Context, query string, limit int32) ([]*pb.Track, error) {
|
|
return nil, errors.New("yandex provider not implemented")
|
|
}
|
|
|
|
// All other methods return errors
|
|
```
|
|
|
|
## Data Layer
|
|
|
|
### Database Connection
|
|
|
|
**File**: `bedrock_server/main.go` (initialization)
|
|
**Driver**: `github.com/jackc/pgx/v5/pgxpool`
|
|
|
|
**Connection Pool**:
|
|
```go
|
|
func initDB() (*pgxpool.Pool, error) {
|
|
dbURL := os.Getenv("DATABASE_URL")
|
|
|
|
config, err := pgxpool.ParseConfig(dbURL)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
config.MaxConns = 10
|
|
config.MinConns = 2
|
|
config.MaxConnLifetime = time.Hour
|
|
config.MaxConnIdleTime = 30 * time.Minute
|
|
|
|
pool, err := pgxpool.NewWithConfig(context.Background(), config)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
return pool, nil
|
|
}
|
|
```
|
|
|
|
### User Store
|
|
|
|
**File**: `store/user.go`
|
|
|
|
**Schema**:
|
|
```sql
|
|
CREATE TABLE users (
|
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
|
email VARCHAR(255) UNIQUE NOT NULL,
|
|
password_hash VARCHAR(255) NOT NULL,
|
|
role VARCHAR(50) DEFAULT 'user',
|
|
is_verified BOOLEAN DEFAULT false,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
);
|
|
```
|
|
|
|
**Operations**:
|
|
```go
|
|
type UserStore struct {
|
|
db *pgxpool.Pool
|
|
}
|
|
|
|
func (s *UserStore) Save(ctx context.Context, email, passwordHash string) (string, error) {
|
|
var userID string
|
|
err := s.db.QueryRow(ctx,
|
|
"INSERT INTO users (email, password_hash) VALUES ($1, $2) RETURNING id",
|
|
email, passwordHash,
|
|
).Scan(&userID)
|
|
|
|
return userID, err
|
|
}
|
|
|
|
func (s *UserStore) Find(ctx context.Context, email string) (*User, error) {
|
|
var user User
|
|
err := s.db.QueryRow(ctx,
|
|
"SELECT id, email, password_hash, role, is_verified, created_at FROM users WHERE email = $1",
|
|
email,
|
|
).Scan(&user.ID, &user.Email, &user.PasswordHash, &user.Role, &user.IsVerified, &user.CreatedAt)
|
|
|
|
return &user, err
|
|
}
|
|
```
|
|
|
|
### Migrations
|
|
|
|
**Directory**: `db/migrations/`
|
|
|
|
**Format**: Paired up/down SQL files
|
|
```
|
|
001_create_users_table.up.sql
|
|
001_create_users_table.down.sql
|
|
002_add_user_roles.up.sql
|
|
002_add_user_roles.down.sql
|
|
```
|
|
|
|
**No Migration Runner**: Manual execution required (no golang-migrate or similar tool integrated).
|
|
|
|
## ID Namespacing System
|
|
|
|
### Format
|
|
|
|
```
|
|
{platform}:{entity_type}:{native_id}
|
|
```
|
|
|
|
**Examples**:
|
|
```
|
|
spotify:track:3n3Ppam7vgaVa1iaRUc9Lp
|
|
soundcloud:track:1234567890
|
|
deezer:album:302127
|
|
youtube:video:dQw4w9WgXcQ
|
|
spotify:artist:0TnOYISbd1XYRBk9myaseg
|
|
```
|
|
|
|
### Parsing
|
|
|
|
```go
|
|
func parseNamespacedID(id string) (platform, nativeID string) {
|
|
parts := strings.Split(id, ":")
|
|
if len(parts) < 3 {
|
|
return "", ""
|
|
}
|
|
|
|
platform = parts[0]
|
|
nativeID = strings.Join(parts[2:], ":") // Handle IDs with colons
|
|
|
|
return
|
|
}
|
|
```
|
|
|
|
### Benefits
|
|
|
|
1. **Collision Prevention**: Different platforms can have overlapping numeric IDs
|
|
2. **Explicit Routing**: Service layer knows which provider to call without lookup
|
|
3. **Debugging**: IDs are self-documenting in logs
|
|
4. **Client Clarity**: API consumers know the source platform
|
|
|
|
### Drawbacks
|
|
|
|
1. **ID Length**: Longer than native IDs (storage overhead)
|
|
2. **Client Parsing**: Clients must handle namespaced format
|
|
3. **Migration Complexity**: Changing namespace format requires data migration
|
|
|
|
## Error Handling Patterns
|
|
|
|
### Partial Response Model
|
|
|
|
**Status Enum**:
|
|
```protobuf
|
|
enum ResponseStatus {
|
|
OK = 0; // All providers succeeded
|
|
PARTIAL = 1; // Some providers failed, some succeeded
|
|
ERROR = 2; // All providers failed
|
|
}
|
|
```
|
|
|
|
**Error Aggregation**:
|
|
```protobuf
|
|
message ProviderError {
|
|
string provider = 1; // Provider name (spotify, soundcloud, etc.)
|
|
string message = 2; // Error message
|
|
}
|
|
|
|
message SearchTracksResponse {
|
|
repeated Track tracks = 1;
|
|
ResponseStatus status = 2;
|
|
repeated ProviderError errors = 3;
|
|
}
|
|
```
|
|
|
|
**Client Handling**:
|
|
```go
|
|
resp, err := client.SearchTracks(ctx, &pb.SearchRequest{Query: "test"})
|
|
if err != nil {
|
|
// gRPC-level error (network, auth, etc.)
|
|
return err
|
|
}
|
|
|
|
switch resp.Status {
|
|
case pb.ResponseStatus_OK:
|
|
// All providers succeeded, use resp.Tracks
|
|
case pb.ResponseStatus_PARTIAL:
|
|
// Some providers failed, use resp.Tracks (partial results)
|
|
// Check resp.Errors for failure details
|
|
case pb.ResponseStatus_ERROR:
|
|
// All providers failed, resp.Tracks is empty
|
|
// Check resp.Errors for all failure reasons
|
|
}
|
|
```
|
|
|
|
### Provider-Level Error Handling
|
|
|
|
**Pattern**: Log and continue
|
|
```go
|
|
tracks, err := provider.SearchTracks(ctx, query, limit)
|
|
if err != nil {
|
|
log.Printf("[%s] Search failed: %v", provider.Name(), err)
|
|
// Don't return, continue to next provider
|
|
}
|
|
```
|
|
|
|
**No Circuit Breakers**: Failed providers are retried on every request (no temporary disabling).
|
|
|
|
## Concurrency Patterns
|
|
|
|
### WaitGroup Coordination
|
|
|
|
**Standard Pattern**:
|
|
```go
|
|
var wg sync.WaitGroup
|
|
|
|
for _, item := range items {
|
|
wg.Add(1)
|
|
go func(i Item) {
|
|
defer wg.Done()
|
|
// Process item
|
|
}(item)
|
|
}
|
|
|
|
wg.Wait()
|
|
```
|
|
|
|
### Mutex-Protected Aggregation
|
|
|
|
**Pattern**:
|
|
```go
|
|
var (
|
|
mu sync.Mutex
|
|
results []Result
|
|
)
|
|
|
|
for _, provider := range providers {
|
|
go func(p Provider) {
|
|
result := p.Fetch()
|
|
|
|
mu.Lock()
|
|
results = append(results, result)
|
|
mu.Unlock()
|
|
}(provider)
|
|
}
|
|
```
|
|
|
|
### No Worker Pools
|
|
|
|
All goroutines are spawned per-request (no bounded concurrency). For 4 providers, each search spawns 4 goroutines.
|
|
|
|
**Potential Issue**: High request volume could spawn thousands of goroutines.
|
|
|
|
## Configuration Architecture
|
|
|
|
### Environment Variable Loading
|
|
|
|
**Search Order**:
|
|
1. `.env` in current working directory
|
|
2. `.env` in `bedrock_server/` directory
|
|
3. `.env` in parent directory
|
|
|
|
**Loader**:
|
|
```go
|
|
func loadEnv() {
|
|
locations := []string{
|
|
".env",
|
|
"bedrock_server/.env",
|
|
"../.env",
|
|
}
|
|
|
|
for _, loc := range locations {
|
|
if err := godotenv.Load(loc); err == nil {
|
|
log.Printf("Loaded environment from %s", loc)
|
|
return
|
|
}
|
|
}
|
|
|
|
log.Println("No .env file found, using system environment")
|
|
}
|
|
```
|
|
|
|
### CLI Flag Overrides
|
|
|
|
**Flags**:
|
|
```go
|
|
var (
|
|
grpcPort = flag.Int("port", 50052, "gRPC server port")
|
|
proxyAddr = flag.String("proxy-addr", ":8080", "HTTP proxy address")
|
|
proxyHost = flag.String("proxy-host", "", "HTTP proxy host for URL generation")
|
|
)
|
|
|
|
func main() {
|
|
flag.Parse()
|
|
loadEnv()
|
|
|
|
// Flags take precedence over environment variables
|
|
}
|
|
```
|
|
|
|
### Provider Initialization
|
|
|
|
**Conditional Initialization**:
|
|
```go
|
|
func initProviders() []trackProvider {
|
|
var providers []trackProvider
|
|
|
|
if os.Getenv("SPOTIFY_CLIENT_ID") != "" {
|
|
providers = append(providers, NewSpotifyProvider())
|
|
}
|
|
|
|
if os.Getenv("SOUNDCLOUD_CLIENT_IDS") != "" {
|
|
providers = append(providers, NewSoundCloudProvider())
|
|
}
|
|
|
|
// Deezer has no required credentials
|
|
providers = append(providers, NewDeezerProvider())
|
|
|
|
if os.Getenv("YOUTUBE_COOKIES") != "" {
|
|
providers = append(providers, NewYouTubeProvider())
|
|
}
|
|
|
|
return providers
|
|
}
|
|
```
|
|
|
|
**Graceful Degradation**: Missing credentials disable specific providers, service continues with available providers.
|
|
|
|
## Deployment Architecture
|
|
|
|
### Docker Multi-Stage Build
|
|
|
|
**Dockerfile**:
|
|
```dockerfile
|
|
# Builder stage
|
|
FROM golang:1.23-alpine AS builder
|
|
|
|
WORKDIR /app
|
|
|
|
# Copy go mod files
|
|
COPY go.mod go.sum ./
|
|
RUN go mod download
|
|
|
|
# Copy source
|
|
COPY . .
|
|
|
|
# Build binary
|
|
RUN CGO_ENABLED=0 GOOS=linux go build -o bedrock-server ./bedrock_server
|
|
|
|
# Runtime stage
|
|
FROM alpine:latest
|
|
|
|
RUN apk --no-cache add ca-certificates
|
|
|
|
WORKDIR /root/
|
|
|
|
COPY --from=builder /app/bedrock-server .
|
|
|
|
EXPOSE 50052 8080
|
|
|
|
CMD ["./bedrock-server"]
|
|
```
|
|
|
|
**Version Mismatch**: Dockerfile uses Go 1.23, but `go.mod` specifies 1.25.
|
|
|
|
### Docker Compose
|
|
|
|
**docker-compose.yml**:
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
postgres:
|
|
image: postgres:15-alpine
|
|
environment:
|
|
POSTGRES_USER: bedrock
|
|
POSTGRES_PASSWORD: bedrock
|
|
POSTGRES_DB: bedrock
|
|
ports:
|
|
- "5432:5432"
|
|
volumes:
|
|
- postgres_data:/var/lib/postgresql/data
|
|
|
|
volumes:
|
|
postgres_data:
|
|
```
|
|
|
|
**Missing Services**:
|
|
- No Redis (planned for caching)
|
|
- No reverse proxy (TLS must be added externally)
|
|
- No application service (must be run separately or added to compose)
|
|
|
|
## Observability Architecture
|
|
|
|
### Logging
|
|
|
|
**Implementation**: Go stdlib `log` package
|
|
|
|
**Format**:
|
|
```go
|
|
log.Printf("[spotify] Searching for: %s", query)
|
|
log.Printf("[soundcloud] Client ID rotation: %d -> %d", old, new)
|
|
log.Printf("[youtube] Client %s failed: %v", clientName, err)
|
|
```
|
|
|
|
**Limitations**:
|
|
- No structured logging (JSON)
|
|
- No log levels (info/warn/error mixed)
|
|
- No log aggregation
|
|
- No correlation IDs for request tracing
|
|
|
|
### Health Checks
|
|
|
|
**Stub Implementation**:
|
|
```go
|
|
func (s *server) GetServiceStatus(ctx context.Context, req *pb.Empty) (*pb.ServiceStatusResponse, error) {
|
|
return &pb.ServiceStatusResponse{
|
|
Status: pb.ServiceStatus_HEALTHY,
|
|
Dependencies: []*pb.DependencyStatus{
|
|
{Name: "spotify", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "soundcloud", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "deezer", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "youtube", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
},
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
**Missing**:
|
|
- Actual provider health checks
|
|
- Latency measurement
|
|
- Database connection check
|
|
- Dependency version reporting
|
|
|
|
### Metrics
|
|
|
|
**Current**: None
|
|
|
|
**Missing**:
|
|
- Prometheus metrics
|
|
- Request counters
|
|
- Latency histograms
|
|
- Error rates
|
|
- Provider success/failure rates
|
|
- Active goroutine count
|
|
|
|
## Security Architecture
|
|
|
|
### Transport Security
|
|
|
|
**gRPC**: No TLS (insecure credentials)
|
|
```go
|
|
grpcServer := grpc.NewServer() // No TLS config
|
|
```
|
|
|
|
**HTTP Proxy**: No HTTPS
|
|
```go
|
|
http.ListenAndServe(":8080", handler) // No TLS
|
|
```
|
|
|
|
**Recommendation**: Deploy behind reverse proxy (nginx, Caddy) with TLS termination.
|
|
|
|
### Authentication Flow
|
|
|
|
```
|
|
Client Registration:
|
|
1. Client sends email + password to Register RPC
|
|
2. Server hashes password with bcrypt (cost 10)
|
|
3. Server stores user in PostgreSQL
|
|
4. Server returns access token (15min) + refresh token (7 days)
|
|
|
|
Client Login:
|
|
1. Client sends email + password to Login RPC
|
|
2. Server fetches user from PostgreSQL
|
|
3. Server verifies password with bcrypt
|
|
4. Server returns access token + refresh token
|
|
|
|
Authenticated Requests:
|
|
1. Client includes "Authorization: Bearer <access_token>" in gRPC metadata
|
|
2. Server intercepts request with authInterceptor
|
|
3. Server validates JWT signature and expiration
|
|
4. Server allows request to proceed
|
|
|
|
Token Refresh:
|
|
1. Client sends refresh token to RefreshToken RPC
|
|
2. Server validates refresh token
|
|
3. Server issues new access token + refresh token
|
|
```
|
|
|
|
### Security Gaps
|
|
|
|
1. **No Rate Limiting**: Brute force attacks on login are possible
|
|
2. **No Account Lockout**: Unlimited failed login attempts
|
|
3. **No Token Revocation**: Compromised tokens valid until expiration
|
|
4. **No Email Verification**: `is_verified` field exists but unused
|
|
5. **No Password Requirements**: No minimum length, complexity rules
|
|
6. **No HTTPS**: Credentials transmitted in plaintext without reverse proxy
|
|
7. **JWT Secret in Environment**: No key rotation, single secret for all tokens
|
|
|
|
## Performance Characteristics
|
|
|
|
### Concurrency Model
|
|
|
|
**Per-Request Goroutines**: 4 goroutines per search (one per active provider)
|
|
|
|
**Example Load**:
|
|
- 100 concurrent search requests
|
|
- 4 providers per request
|
|
- 400 goroutines spawned
|
|
|
|
**No Limits**: Unbounded goroutine creation (potential memory exhaustion under high load).
|
|
|
|
### Response Time Factors
|
|
|
|
**Parallel Provider Queries**: Response time = slowest provider
|
|
|
|
**Example**:
|
|
- Spotify: 200ms
|
|
- SoundCloud: 150ms
|
|
- Deezer: 100ms
|
|
- YouTube Music: 500ms
|
|
- **Total**: 500ms (not 950ms)
|
|
|
|
**Timeout Handling**: No explicit timeouts (relies on HTTP client defaults, typically 30s).
|
|
|
|
### Caching Strategy
|
|
|
|
**Current**: No caching
|
|
|
|
**Impact**:
|
|
- Every request hits provider APIs
|
|
- High latency for repeated queries
|
|
- Risk of rate limiting from providers
|
|
- Unnecessary API quota consumption
|
|
|
|
**Planned** (Redis):
|
|
- Stream URL cache (1hr TTL)
|
|
- Metadata cache (5min TTL)
|
|
- Service status cache (5min TTL)
|
|
- Play deduplication (30s window)
|