a1f6701bac
- gRPC service with MusicBrainz provider - PostgreSQL schema with migrations - Service layer with database-first caching - Repository pattern for data access - YAML configuration support - Research documentation for 17 music metadata projects
1301 lines
32 KiB
Markdown
1301 lines
32 KiB
Markdown
# Bedrock-API Codebase Analysis
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
bedrock-api/
|
|
├── bedrock_server/ # Main application
|
|
│ ├── main.go # Service implementation (1329 lines)
|
|
│ ├── resolver.go # Stream resolution logic
|
|
│ ├── proxy.go # HTTP streaming proxy
|
|
│ ├── auth.go # JWT authentication
|
|
│ ├── lrclib.go # Synced lyrics (LrcLib)
|
|
│ └── genius.go # Plain lyrics (Genius)
|
|
├── providers/ # Platform adapters
|
|
│ ├── spotify.go # Spotify integration
|
|
│ ├── soundcloud.go # SoundCloud integration
|
|
│ ├── deezer.go # Deezer integration
|
|
│ ├── youtube.go # YouTube Music integration
|
|
│ ├── yandex.go # Yandex stub
|
|
│ └── vk.go # VK stub
|
|
├── store/ # Data access layer
|
|
│ └── user.go # User CRUD operations
|
|
├── db/ # Database
|
|
│ └── migrations/ # SQL migration files
|
|
├── proto/ # Protocol buffers
|
|
│ └── bedrock_service.proto # gRPC service definition (622 lines)
|
|
├── tests/ # Integration tests
|
|
│ ├── auth_test.go
|
|
│ ├── spotify_test.go
|
|
│ ├── soundcloud_test.go
|
|
│ ├── youtube_test.go
|
|
│ ├── deezer_test.go
|
|
│ └── lyrics_test.go
|
|
├── spotapi-go/ # Git submodule (Spotify wrapper)
|
|
├── .github/
|
|
│ └── workflows/
|
|
│ ├── test.yml # Integration tests
|
|
│ └── lint.yml # Code linting
|
|
├── Dockerfile # Multi-stage build
|
|
├── docker-compose.yml # PostgreSQL only
|
|
├── go.mod # Go 1.25
|
|
├── go.sum
|
|
├── .env.example # Environment template
|
|
└── README.md
|
|
```
|
|
|
|
**Total Lines of Code**: ~5000+ (excluding tests, proto, submodules)
|
|
|
|
## Configuration Management
|
|
|
|
### Environment Variables
|
|
|
|
**Loading Strategy**: Three-location search
|
|
|
|
**File**: `bedrock_server/main.go`
|
|
|
|
```go
|
|
func loadEnv() {
|
|
locations := []string{
|
|
".env", // Current directory
|
|
"bedrock_server/.env", // Server directory
|
|
"../.env", // Parent directory
|
|
}
|
|
|
|
for _, loc := range locations {
|
|
if err := godotenv.Load(loc); err == nil {
|
|
log.Printf("Loaded environment from %s", loc)
|
|
return
|
|
}
|
|
}
|
|
|
|
log.Println("No .env file found, using system environment variables")
|
|
}
|
|
```
|
|
|
|
**Precedence**: First found file wins (no merging)
|
|
|
|
### Required Variables
|
|
|
|
```
|
|
DATABASE_URL=postgresql://user:pass@host:port/database
|
|
JWT_SECRET=your-secret-key
|
|
```
|
|
|
|
### Optional Variables (Provider Credentials)
|
|
|
|
```
|
|
SPOTIFY_CLIENT_ID=your_id
|
|
SPOTIFY_CLIENT_SECRET=your_secret
|
|
SOUNDCLOUD_CLIENT_IDS=id1,id2,id3
|
|
DEEZER_APP_ID=your_id
|
|
YOUTUBE_COOKIES=cookie-string
|
|
GENIUS_ACCESS_TOKEN=your_token
|
|
```
|
|
|
|
### CLI Flags
|
|
|
|
**File**: `bedrock_server/main.go`
|
|
|
|
```go
|
|
var (
|
|
grpcPort = flag.Int("port", 50052, "gRPC server port")
|
|
proxyAddr = flag.String("proxy-addr", ":8080", "HTTP proxy address")
|
|
proxyHost = flag.String("proxy-host", "", "HTTP proxy host for URL generation")
|
|
)
|
|
|
|
func main() {
|
|
flag.Parse()
|
|
loadEnv()
|
|
|
|
// Flags override environment variables
|
|
if *grpcPort != 50052 {
|
|
log.Printf("Using custom gRPC port: %d", *grpcPort)
|
|
}
|
|
}
|
|
```
|
|
|
|
**Usage**:
|
|
```bash
|
|
./bedrock-server -port 9090 -proxy-addr :8888 -proxy-host https://api.example.com
|
|
```
|
|
|
|
### Configuration Validation
|
|
|
|
**No Validation**: Application crashes if required variables are missing
|
|
|
|
**Example Crash**:
|
|
```go
|
|
dbURL := os.Getenv("DATABASE_URL")
|
|
pool, err := pgxpool.New(context.Background(), dbURL) // Panics if dbURL is empty
|
|
```
|
|
|
|
**Recommendation**: Add startup validation
|
|
|
|
```go
|
|
func validateConfig() error {
|
|
required := []string{"DATABASE_URL", "JWT_SECRET"}
|
|
|
|
for _, key := range required {
|
|
if os.Getenv(key) == "" {
|
|
return fmt.Errorf("required environment variable %s not set", key)
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
```
|
|
|
|
## Logging
|
|
|
|
### Implementation
|
|
|
|
**Library**: Go stdlib `log` package
|
|
**Format**: Plain text with provider prefixes
|
|
|
|
**Examples**:
|
|
```go
|
|
log.Printf("[spotify] Searching for: %s", query)
|
|
log.Printf("[soundcloud] Client ID rotation: %d -> %d", old, new)
|
|
log.Printf("[youtube] Client %s failed: %v", clientName, err)
|
|
log.Printf("[auth] User registered: %s", email)
|
|
```
|
|
|
|
### Log Levels
|
|
|
|
**No Levels**: All logs are info-level (no debug/warn/error distinction)
|
|
|
|
**Example** (no level):
|
|
```go
|
|
log.Printf("[spotify] Search failed: %v", err) // Is this error or warning?
|
|
```
|
|
|
|
**Recommendation**: Use structured logging with levels
|
|
|
|
```go
|
|
import "go.uber.org/zap"
|
|
|
|
logger.Info("search request", zap.String("provider", "spotify"), zap.String("query", query))
|
|
logger.Error("search failed", zap.String("provider", "spotify"), zap.Error(err))
|
|
```
|
|
|
|
### Log Output
|
|
|
|
**Destination**: stdout (default)
|
|
**Rotation**: No (relies on systemd or Docker log rotation)
|
|
**Aggregation**: No (manual collection required)
|
|
|
|
**Systemd Logging**:
|
|
```bash
|
|
journalctl -u bedrock-api -f
|
|
```
|
|
|
|
**Docker Logging**:
|
|
```bash
|
|
docker logs -f bedrock-api
|
|
```
|
|
|
|
### Correlation IDs
|
|
|
|
**Not Implemented**: No request tracing across logs
|
|
|
|
**Recommendation**: Add correlation IDs
|
|
|
|
```go
|
|
func (s *server) SearchTracks(ctx context.Context, req *pb.SearchRequest) (*pb.SearchTracksResponse, error) {
|
|
correlationID := uuid.New().String()
|
|
ctx = context.WithValue(ctx, "correlation_id", correlationID)
|
|
|
|
log.Printf("[%s] Search request: %s", correlationID, req.Query)
|
|
// Pass ctx to providers
|
|
}
|
|
```
|
|
|
|
## Authentication Implementation
|
|
|
|
### JWT Token Generation
|
|
|
|
**File**: `bedrock_server/auth.go`
|
|
|
|
```go
|
|
func (s *server) generateTokens(userID, email string) (accessToken, refreshToken string, err error) {
|
|
// Access token (15 minutes)
|
|
accessClaims := jwt.MapClaims{
|
|
"user_id": userID,
|
|
"email": email,
|
|
"exp": time.Now().Add(15 * time.Minute).Unix(),
|
|
"iat": time.Now().Unix(),
|
|
}
|
|
|
|
accessTokenObj := jwt.NewWithClaims(jwt.SigningMethodHS256, accessClaims)
|
|
accessToken, err = accessTokenObj.SignedString(s.jwtSecret)
|
|
if err != nil {
|
|
return "", "", fmt.Errorf("sign access token: %w", err)
|
|
}
|
|
|
|
// Refresh token (7 days)
|
|
refreshClaims := jwt.MapClaims{
|
|
"user_id": userID,
|
|
"email": email,
|
|
"exp": time.Now().Add(7 * 24 * time.Hour).Unix(),
|
|
"iat": time.Now().Unix(),
|
|
}
|
|
|
|
refreshTokenObj := jwt.NewWithClaims(jwt.SigningMethodHS256, refreshClaims)
|
|
refreshToken, err = refreshTokenObj.SignedString(s.jwtSecret)
|
|
if err != nil {
|
|
return "", "", fmt.Errorf("sign refresh token: %w", err)
|
|
}
|
|
|
|
return accessToken, refreshToken, nil
|
|
}
|
|
```
|
|
|
|
**Algorithm**: HS256 (HMAC with SHA-256)
|
|
**Secret**: Single shared secret from `JWT_SECRET` environment variable
|
|
|
|
**Security Considerations**:
|
|
- HS256 is symmetric (same key for signing and verification)
|
|
- No key rotation (single secret for all tokens)
|
|
- No token revocation (valid until expiration)
|
|
|
|
**Recommendation**: Use RS256 (asymmetric) for better security
|
|
|
|
```go
|
|
// Generate RSA key pair
|
|
privateKey, _ := rsa.GenerateKey(rand.Reader, 2048)
|
|
publicKey := &privateKey.PublicKey
|
|
|
|
// Sign with private key
|
|
token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
|
|
tokenString, _ := token.SignedString(privateKey)
|
|
|
|
// Verify with public key (can be distributed to other services)
|
|
token, _ := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
|
|
return publicKey, nil
|
|
})
|
|
```
|
|
|
|
### Password Hashing
|
|
|
|
**File**: `bedrock_server/auth.go`
|
|
|
|
```go
|
|
func hashPassword(password string) (string, error) {
|
|
bytes, err := bcrypt.GenerateFromPassword([]byte(password), 10)
|
|
return string(bytes), err
|
|
}
|
|
|
|
func checkPasswordHash(password, hash string) bool {
|
|
err := bcrypt.CompareHashAndPassword([]byte(hash), []byte(password))
|
|
return err == nil
|
|
}
|
|
```
|
|
|
|
**Algorithm**: bcrypt
|
|
**Cost Factor**: 10 (2^10 = 1024 iterations)
|
|
**Time**: ~100ms per hash (intentionally slow)
|
|
|
|
**Security**: Strong (salted, slow, resistant to brute force)
|
|
|
|
### gRPC Interceptors
|
|
|
|
**Unary Interceptor** (single request/response):
|
|
|
|
```go
|
|
func (s *server) authInterceptor(
|
|
ctx context.Context,
|
|
req interface{},
|
|
info *grpc.UnaryServerInfo,
|
|
handler grpc.UnaryHandler,
|
|
) (interface{}, error) {
|
|
// Public methods bypass auth
|
|
publicMethods := map[string]bool{
|
|
"/bedrock.BedrockService/Register": true,
|
|
"/bedrock.BedrockService/Login": true,
|
|
"/bedrock.BedrockService/RefreshToken": true,
|
|
"/bedrock.BedrockService/GetServiceStatus": true,
|
|
}
|
|
|
|
if publicMethods[info.FullMethod] {
|
|
return handler(ctx, req)
|
|
}
|
|
|
|
// Extract token from metadata
|
|
md, ok := metadata.FromIncomingContext(ctx)
|
|
if !ok {
|
|
return nil, status.Error(codes.Unauthenticated, "missing metadata")
|
|
}
|
|
|
|
tokens := md.Get("authorization")
|
|
if len(tokens) == 0 {
|
|
return nil, status.Error(codes.Unauthenticated, "missing authorization header")
|
|
}
|
|
|
|
// Remove "Bearer " prefix
|
|
tokenString := strings.TrimPrefix(tokens[0], "Bearer ")
|
|
|
|
// Validate token
|
|
claims := jwt.MapClaims{}
|
|
token, err := jwt.ParseWithClaims(tokenString, claims, func(t *jwt.Token) (interface{}, error) {
|
|
// Verify signing method
|
|
if _, ok := t.Method.(*jwt.SigningMethodHMAC); !ok {
|
|
return nil, fmt.Errorf("unexpected signing method: %v", t.Header["alg"])
|
|
}
|
|
return s.jwtSecret, nil
|
|
})
|
|
|
|
if err != nil || !token.Valid {
|
|
return nil, status.Error(codes.Unauthenticated, "invalid token")
|
|
}
|
|
|
|
// Add user info to context
|
|
ctx = context.WithValue(ctx, "user_id", claims["user_id"])
|
|
ctx = context.WithValue(ctx, "email", claims["email"])
|
|
|
|
return handler(ctx, req)
|
|
}
|
|
```
|
|
|
|
**Stream Interceptor** (streaming requests):
|
|
|
|
```go
|
|
func (s *server) streamAuthInterceptor(
|
|
srv interface{},
|
|
ss grpc.ServerStream,
|
|
info *grpc.StreamServerInfo,
|
|
handler grpc.StreamHandler,
|
|
) error {
|
|
// Similar logic to unary interceptor
|
|
// Validates token once at stream start
|
|
// No per-message validation
|
|
}
|
|
```
|
|
|
|
**Registration**:
|
|
```go
|
|
grpcServer := grpc.NewServer(
|
|
grpc.UnaryInterceptor(s.authInterceptor),
|
|
grpc.StreamInterceptor(s.streamAuthInterceptor),
|
|
)
|
|
```
|
|
|
|
### Registration Flow
|
|
|
|
**File**: `bedrock_server/main.go`
|
|
|
|
```go
|
|
func (s *server) Register(ctx context.Context, req *pb.AuthRequest) (*pb.AuthResponse, error) {
|
|
// Validate email format
|
|
if !isValidEmail(req.Email) {
|
|
return nil, status.Error(codes.InvalidArgument, "invalid email format")
|
|
}
|
|
|
|
// Hash password
|
|
passwordHash, err := hashPassword(req.Password)
|
|
if err != nil {
|
|
return nil, status.Error(codes.Internal, "failed to hash password")
|
|
}
|
|
|
|
// Save user
|
|
userStore := store.NewUserStore(s.db)
|
|
userID, err := userStore.Save(ctx, req.Email, passwordHash)
|
|
if err != nil {
|
|
if strings.Contains(err.Error(), "duplicate key") {
|
|
return nil, status.Error(codes.AlreadyExists, "email already registered")
|
|
}
|
|
return nil, status.Error(codes.Internal, "failed to create user")
|
|
}
|
|
|
|
// Generate tokens
|
|
accessToken, refreshToken, err := s.generateTokens(userID, req.Email)
|
|
if err != nil {
|
|
return nil, status.Error(codes.Internal, "failed to generate tokens")
|
|
}
|
|
|
|
return &pb.AuthResponse{
|
|
AccessToken: accessToken,
|
|
RefreshToken: refreshToken,
|
|
User: &pb.User{
|
|
Id: userID,
|
|
Email: req.Email,
|
|
Role: "user",
|
|
IsVerified: false,
|
|
},
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
**No Password Requirements**: Any password is accepted (no minimum length, complexity rules)
|
|
|
|
**Recommendation**: Add password validation
|
|
|
|
```go
|
|
func validatePassword(password string) error {
|
|
if len(password) < 8 {
|
|
return errors.New("password must be at least 8 characters")
|
|
}
|
|
|
|
hasUpper := regexp.MustCompile(`[A-Z]`).MatchString(password)
|
|
hasLower := regexp.MustCompile(`[a-z]`).MatchString(password)
|
|
hasDigit := regexp.MustCompile(`[0-9]`).MatchString(password)
|
|
|
|
if !hasUpper || !hasLower || !hasDigit {
|
|
return errors.New("password must contain uppercase, lowercase, and digit")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
```
|
|
|
|
### Login Flow
|
|
|
|
```go
|
|
func (s *server) Login(ctx context.Context, req *pb.AuthRequest) (*pb.AuthResponse, error) {
|
|
// Find user by email
|
|
userStore := store.NewUserStore(s.db)
|
|
user, err := userStore.Find(ctx, req.Email)
|
|
if err != nil {
|
|
return nil, status.Error(codes.NotFound, "user not found")
|
|
}
|
|
|
|
// Verify password
|
|
if !checkPasswordHash(req.Password, user.PasswordHash) {
|
|
return nil, status.Error(codes.Unauthenticated, "invalid credentials")
|
|
}
|
|
|
|
// Generate tokens
|
|
accessToken, refreshToken, err := s.generateTokens(user.ID, user.Email)
|
|
if err != nil {
|
|
return nil, status.Error(codes.Internal, "failed to generate tokens")
|
|
}
|
|
|
|
return &pb.AuthResponse{
|
|
AccessToken: accessToken,
|
|
RefreshToken: refreshToken,
|
|
User: &pb.User{
|
|
Id: user.ID,
|
|
Email: user.Email,
|
|
Role: user.Role,
|
|
IsVerified: user.IsVerified,
|
|
},
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
**No Rate Limiting**: Unlimited login attempts (brute force possible)
|
|
|
|
**Recommendation**: Add rate limiting
|
|
|
|
```go
|
|
import "golang.org/x/time/rate"
|
|
|
|
var loginLimiters = make(map[string]*rate.Limiter)
|
|
var mu sync.Mutex
|
|
|
|
func getLoginLimiter(email string) *rate.Limiter {
|
|
mu.Lock()
|
|
defer mu.Unlock()
|
|
|
|
limiter, exists := loginLimiters[email]
|
|
if !exists {
|
|
limiter = rate.NewLimiter(rate.Every(time.Minute), 5) // 5 attempts per minute
|
|
loginLimiters[email] = limiter
|
|
}
|
|
|
|
return limiter
|
|
}
|
|
|
|
func (s *server) Login(ctx context.Context, req *pb.AuthRequest) (*pb.AuthResponse, error) {
|
|
limiter := getLoginLimiter(req.Email)
|
|
if !limiter.Allow() {
|
|
return nil, status.Error(codes.ResourceExhausted, "too many login attempts")
|
|
}
|
|
|
|
// Continue with login logic
|
|
}
|
|
```
|
|
|
|
## Testing
|
|
|
|
### Test Structure
|
|
|
|
**Directory**: `tests/`
|
|
|
|
**Files**:
|
|
- `auth_test.go` - Authentication tests (register, login, refresh)
|
|
- `spotify_test.go` - Spotify provider tests
|
|
- `soundcloud_test.go` - SoundCloud provider tests
|
|
- `youtube_test.go` - YouTube Music provider tests
|
|
- `deezer_test.go` - Deezer provider tests
|
|
- `lyrics_test.go` - Lyrics integration tests (LrcLib, Genius)
|
|
|
|
### Integration Tests
|
|
|
|
**Example**: `tests/spotify_test.go`
|
|
|
|
```go
|
|
func TestSpotifySearch(t *testing.T) {
|
|
// Connect to test server
|
|
addr := os.Getenv("BEDROCK_TEST_ADDR")
|
|
if addr == "" {
|
|
addr = "localhost:50052"
|
|
}
|
|
|
|
conn, err := grpc.Dial(addr, grpc.WithInsecure())
|
|
if err != nil {
|
|
t.Fatalf("dial: %v", err)
|
|
}
|
|
defer conn.Close()
|
|
|
|
client := pb.NewBedrockServiceClient(conn)
|
|
|
|
// Register test user
|
|
authResp, err := client.Register(context.Background(), &pb.AuthRequest{
|
|
Email: "test@example.com",
|
|
Password: "password123",
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("register: %v", err)
|
|
}
|
|
|
|
// Authenticated context
|
|
ctx := metadata.AppendToOutgoingContext(
|
|
context.Background(),
|
|
"authorization", "Bearer "+authResp.AccessToken,
|
|
)
|
|
|
|
// Search tracks
|
|
resp, err := client.SearchTracks(ctx, &pb.SearchRequest{
|
|
Query: "Bohemian Rhapsody",
|
|
Limit: 10,
|
|
})
|
|
if err != nil {
|
|
t.Fatalf("search: %v", err)
|
|
}
|
|
|
|
// Verify results
|
|
if len(resp.Tracks) == 0 {
|
|
t.Fatal("no tracks returned")
|
|
}
|
|
|
|
// Verify Spotify results present
|
|
hasSpotify := false
|
|
for _, track := range resp.Tracks {
|
|
if track.Platform == pb.Platform_SPOTIFY {
|
|
hasSpotify = true
|
|
break
|
|
}
|
|
}
|
|
|
|
if !hasSpotify {
|
|
t.Error("no Spotify results found")
|
|
}
|
|
}
|
|
```
|
|
|
|
**Test Requirements**:
|
|
- Running server (BEDROCK_TEST_ADDR)
|
|
- PostgreSQL database
|
|
- Provider credentials (environment variables)
|
|
|
|
**Test Timeout**: 120 seconds (configured in GitHub Actions)
|
|
|
|
### No Unit Tests
|
|
|
|
**Missing**:
|
|
- Provider adapter unit tests (mocked HTTP responses)
|
|
- Database store unit tests (mocked database)
|
|
- Authentication unit tests (mocked JWT)
|
|
- Stream resolution unit tests
|
|
|
|
**Recommendation**: Add unit tests with mocks
|
|
|
|
```go
|
|
func TestSpotifyProvider_SearchTracks(t *testing.T) {
|
|
// Mock HTTP server
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Content-Type", "application/json")
|
|
w.Write([]byte(`{
|
|
"tracks": {
|
|
"items": [
|
|
{
|
|
"id": "abc123",
|
|
"name": "Test Track",
|
|
"artists": [{"id": "artist1", "name": "Test Artist"}],
|
|
"album": {"id": "album1", "name": "Test Album"}
|
|
}
|
|
]
|
|
}
|
|
}`))
|
|
}))
|
|
defer server.Close()
|
|
|
|
// Create provider with mock server URL
|
|
provider := &SpotifyProvider{
|
|
client: spotify.New(/* mock client */),
|
|
}
|
|
|
|
// Test search
|
|
tracks, err := provider.SearchTracks(context.Background(), "test", 10)
|
|
if err != nil {
|
|
t.Fatalf("search failed: %v", err)
|
|
}
|
|
|
|
if len(tracks) != 1 {
|
|
t.Errorf("expected 1 track, got %d", len(tracks))
|
|
}
|
|
}
|
|
```
|
|
|
|
### Test Coverage
|
|
|
|
**No Coverage Reports**: Coverage not measured
|
|
|
|
**Recommendation**: Add coverage reporting
|
|
|
|
```bash
|
|
go test -cover ./...
|
|
go test -coverprofile=coverage.out ./...
|
|
go tool cover -html=coverage.out -o coverage.html
|
|
```
|
|
|
|
**GitHub Actions Integration**:
|
|
```yaml
|
|
- name: Run tests with coverage
|
|
run: go test -v -coverprofile=coverage.out ./...
|
|
|
|
- name: Upload coverage to Codecov
|
|
uses: codecov/codecov-action@v3
|
|
with:
|
|
files: ./coverage.out
|
|
```
|
|
|
|
## Health Checks
|
|
|
|
### Service Status
|
|
|
|
**File**: `bedrock_server/main.go`
|
|
|
|
```go
|
|
func (s *server) GetServiceStatus(ctx context.Context, req *pb.Empty) (*pb.ServiceStatusResponse, error) {
|
|
// Stub implementation (always returns healthy)
|
|
return &pb.ServiceStatusResponse{
|
|
Status: pb.ServiceStatus_HEALTHY,
|
|
Dependencies: []*pb.DependencyStatus{
|
|
{Name: "spotify", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "soundcloud", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "deezer", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "youtube", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
{Name: "postgres", Health: pb.HealthStatus_HEALTHY, Latency: 0},
|
|
},
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
**Issues**:
|
|
- No actual health checks (stub only)
|
|
- No latency measurement
|
|
- No database connection check
|
|
- No provider API checks
|
|
|
|
**Recommendation**: Implement real health checks
|
|
|
|
```go
|
|
func (s *server) GetServiceStatus(ctx context.Context, req *pb.Empty) (*pb.ServiceStatusResponse, error) {
|
|
var dependencies []*pb.DependencyStatus
|
|
|
|
// Check database
|
|
dbStart := time.Now()
|
|
if err := s.db.Ping(ctx); err != nil {
|
|
dependencies = append(dependencies, &pb.DependencyStatus{
|
|
Name: "postgres",
|
|
Health: pb.HealthStatus_UNHEALTHY,
|
|
Latency: 0,
|
|
})
|
|
} else {
|
|
dependencies = append(dependencies, &pb.DependencyStatus{
|
|
Name: "postgres",
|
|
Health: pb.HealthStatus_HEALTHY,
|
|
Latency: int32(time.Since(dbStart).Milliseconds()),
|
|
})
|
|
}
|
|
|
|
// Check each provider
|
|
for _, provider := range s.providers {
|
|
providerStart := time.Now()
|
|
_, err := provider.SearchTracks(ctx, "test", 1)
|
|
|
|
health := pb.HealthStatus_HEALTHY
|
|
if err != nil {
|
|
health = pb.HealthStatus_UNHEALTHY
|
|
}
|
|
|
|
dependencies = append(dependencies, &pb.DependencyStatus{
|
|
Name: provider.Name(),
|
|
Health: health,
|
|
Latency: int32(time.Since(providerStart).Milliseconds()),
|
|
})
|
|
}
|
|
|
|
// Determine overall status
|
|
status := pb.ServiceStatus_HEALTHY
|
|
for _, dep := range dependencies {
|
|
if dep.Health == pb.HealthStatus_UNHEALTHY {
|
|
status = pb.ServiceStatus_DEGRADED
|
|
break
|
|
}
|
|
}
|
|
|
|
return &pb.ServiceStatusResponse{
|
|
Status: status,
|
|
Dependencies: dependencies,
|
|
}, nil
|
|
}
|
|
```
|
|
|
|
### Readiness vs Liveness
|
|
|
|
**Not Implemented**: No distinction between readiness and liveness
|
|
|
|
**Kubernetes Probes** (recommended):
|
|
|
|
```yaml
|
|
livenessProbe:
|
|
exec:
|
|
command:
|
|
- grpc_health_probe
|
|
- -addr=:50052
|
|
initialDelaySeconds: 10
|
|
periodSeconds: 10
|
|
|
|
readinessProbe:
|
|
exec:
|
|
command:
|
|
- grpc_health_probe
|
|
- -addr=:50052
|
|
- -service=bedrock.BedrockService
|
|
initialDelaySeconds: 5
|
|
periodSeconds: 5
|
|
```
|
|
|
|
**gRPC Health Checking Protocol**:
|
|
```go
|
|
import "google.golang.org/grpc/health/grpc_health_v1"
|
|
|
|
type healthServer struct {
|
|
grpc_health_v1.UnimplementedHealthServer
|
|
}
|
|
|
|
func (h *healthServer) Check(ctx context.Context, req *grpc_health_v1.HealthCheckRequest) (*grpc_health_v1.HealthCheckResponse, error) {
|
|
// Check if service is ready
|
|
return &grpc_health_v1.HealthCheckResponse{
|
|
Status: grpc_health_v1.HealthCheckResponse_SERVING,
|
|
}, nil
|
|
}
|
|
|
|
// Register health server
|
|
grpc_health_v1.RegisterHealthServer(grpcServer, &healthServer{})
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
### Error Patterns
|
|
|
|
**Provider Errors**: Log and continue
|
|
|
|
```go
|
|
tracks, err := provider.SearchTracks(ctx, query, limit)
|
|
if err != nil {
|
|
log.Printf("[%s] Search failed: %v", provider.Name(), err)
|
|
errors = append(errors, &pb.ProviderError{
|
|
Provider: provider.Name(),
|
|
Message: err.Error(),
|
|
})
|
|
// Don't return, continue to next provider
|
|
}
|
|
```
|
|
|
|
**Database Errors**: Return immediately
|
|
|
|
```go
|
|
user, err := userStore.Find(ctx, email)
|
|
if err != nil {
|
|
return nil, status.Error(codes.NotFound, "user not found")
|
|
}
|
|
```
|
|
|
|
**gRPC Status Codes**:
|
|
|
|
| Code | Usage |
|
|
|------|-------|
|
|
| `OK` | Successful operation |
|
|
| `InvalidArgument` | Invalid request parameters |
|
|
| `NotFound` | Entity not found |
|
|
| `AlreadyExists` | Duplicate entity (email) |
|
|
| `Unauthenticated` | Missing or invalid JWT |
|
|
| `Internal` | Server error |
|
|
| `ResourceExhausted` | Rate limit (not implemented) |
|
|
|
|
### Error Wrapping
|
|
|
|
**No Error Wrapping**: Errors are not wrapped with context
|
|
|
|
**Example** (no wrapping):
|
|
```go
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
```
|
|
|
|
**Recommendation**: Wrap errors with context
|
|
|
|
```go
|
|
if err != nil {
|
|
return nil, fmt.Errorf("search spotify: %w", err)
|
|
}
|
|
```
|
|
|
|
**Benefits**:
|
|
- Error chain for debugging
|
|
- Context preservation
|
|
- Stack trace (with errors package)
|
|
|
|
## Code Style
|
|
|
|
### Comment Linting
|
|
|
|
**Custom Linter**: `.github/workflows/lint.yml`
|
|
|
|
**Rules**:
|
|
1. No decorative comments (`// ========`, `// --------`, etc.)
|
|
2. No uppercase-leading comments (except `TODO`, `FIXME`, `NOTE`)
|
|
|
|
**Examples**:
|
|
|
|
**Forbidden**:
|
|
```go
|
|
// ========================================
|
|
// Spotify Provider
|
|
// ========================================
|
|
|
|
// This function searches for tracks
|
|
func SearchTracks() {}
|
|
```
|
|
|
|
**Allowed**:
|
|
```go
|
|
// searchTracks queries Spotify API for tracks matching the query
|
|
func searchTracks() {}
|
|
|
|
// TODO: Add caching
|
|
func searchTracks() {}
|
|
```
|
|
|
|
**Enforcement**: GitHub Actions fails on violations
|
|
|
|
### Naming Conventions
|
|
|
|
**Exported Functions**: PascalCase
|
|
```go
|
|
func SearchTracks() {}
|
|
func GetStreamURL() {}
|
|
```
|
|
|
|
**Unexported Functions**: camelCase
|
|
```go
|
|
func parseNamespacedID() {}
|
|
func selectBestFormat() {}
|
|
```
|
|
|
|
**Constants**: PascalCase or SCREAMING_SNAKE_CASE
|
|
```go
|
|
const DefaultLimit = 20
|
|
const MAX_RETRIES = 3
|
|
```
|
|
|
|
**Interfaces**: Noun or adjective ending in "er"
|
|
```go
|
|
type trackProvider interface {}
|
|
type streamResolver interface {}
|
|
```
|
|
|
|
### Code Organization
|
|
|
|
**Single File Service**: `main.go` (1329 lines)
|
|
|
|
**Issues**:
|
|
- All RPC methods in one file
|
|
- Hard to navigate
|
|
- Merge conflicts likely
|
|
|
|
**Recommendation**: Split by domain
|
|
|
|
```
|
|
bedrock_server/
|
|
├── main.go # Server setup, initialization
|
|
├── search.go # Search methods
|
|
├── retrieval.go # Get methods
|
|
├── streaming.go # Stream methods
|
|
├── recommendations.go # Similar tracks
|
|
├── statistics.go # Top tracks/albums/artists
|
|
├── import.go # Playlist import
|
|
├── auth.go # Authentication
|
|
└── lyrics.go # Lyrics methods
|
|
```
|
|
|
|
## Dependency Management
|
|
|
|
### Go Modules
|
|
|
|
**File**: `go.mod`
|
|
|
|
```go
|
|
module github.com/feralbureau/bedrock-api
|
|
|
|
go 1.25
|
|
|
|
require (
|
|
github.com/golang-jwt/jwt/v5 v5.2.1
|
|
github.com/jackc/pgx/v5 v5.7.2
|
|
github.com/joho/godotenv v1.5.1
|
|
github.com/kkdai/youtube/v2 v2.10.3
|
|
github.com/rhnvrm/lyric-api-go v0.1.4
|
|
golang.org/x/crypto v0.31.0
|
|
google.golang.org/grpc v1.79.1
|
|
google.golang.org/protobuf v1.36.4
|
|
)
|
|
```
|
|
|
|
**Direct Dependencies**: 8
|
|
**Indirect Dependencies**: ~50 (transitive)
|
|
|
|
### Submodule Dependency
|
|
|
|
**Submodule**: `spotapi-go` (custom Spotify wrapper)
|
|
|
|
**Issues**:
|
|
- Custom fork (not official library)
|
|
- Maintenance burden
|
|
- Submodule initialization required
|
|
|
|
**Recommendation**: Use official library directly
|
|
|
|
```go
|
|
import "github.com/zmb3/spotify/v2"
|
|
|
|
// Remove spotapi-go submodule
|
|
// Use spotify/v2 directly
|
|
```
|
|
|
|
### Dependency Updates
|
|
|
|
**No Automated Updates**: Dependabot not configured
|
|
|
|
**Recommendation**: Add Dependabot
|
|
|
|
**File**: `.github/dependabot.yml`
|
|
|
|
```yaml
|
|
version: 2
|
|
updates:
|
|
- package-ecosystem: "gomod"
|
|
directory: "/"
|
|
schedule:
|
|
interval: "weekly"
|
|
open-pull-requests-limit: 10
|
|
```
|
|
|
|
## Performance Considerations
|
|
|
|
### Goroutine Management
|
|
|
|
**Unbounded Goroutines**: No limit on concurrent goroutines
|
|
|
|
**Example**:
|
|
```go
|
|
for _, provider := range providers {
|
|
wg.Add(1)
|
|
go func(p trackProvider) {
|
|
defer wg.Done()
|
|
// Query provider
|
|
}(provider)
|
|
}
|
|
```
|
|
|
|
**Risk**: High request volume spawns thousands of goroutines
|
|
|
|
**Recommendation**: Use worker pool
|
|
|
|
```go
|
|
type workerPool struct {
|
|
workers int
|
|
tasks chan func()
|
|
}
|
|
|
|
func newWorkerPool(workers int) *workerPool {
|
|
p := &workerPool{
|
|
workers: workers,
|
|
tasks: make(chan func(), workers*2),
|
|
}
|
|
|
|
for i := 0; i < workers; i++ {
|
|
go p.worker()
|
|
}
|
|
|
|
return p
|
|
}
|
|
|
|
func (p *workerPool) worker() {
|
|
for task := range p.tasks {
|
|
task()
|
|
}
|
|
}
|
|
|
|
func (p *workerPool) submit(task func()) {
|
|
p.tasks <- task
|
|
}
|
|
```
|
|
|
|
### Connection Pooling
|
|
|
|
**HTTP Clients**: Reused per provider (good)
|
|
|
|
```go
|
|
type SoundCloudProvider struct {
|
|
httpClient *http.Client
|
|
}
|
|
|
|
func NewSoundCloudProvider() *SoundCloudProvider {
|
|
return &SoundCloudProvider{
|
|
httpClient: &http.Client{
|
|
Timeout: 10 * time.Second,
|
|
Transport: &http.Transport{
|
|
MaxIdleConns: 100,
|
|
MaxIdleConnsPerHost: 10,
|
|
IdleConnTimeout: 90 * time.Second,
|
|
},
|
|
},
|
|
}
|
|
}
|
|
```
|
|
|
|
**Database**: Connection pooling configured (good)
|
|
|
|
```go
|
|
config.MaxConns = 10
|
|
config.MinConns = 2
|
|
```
|
|
|
|
### Memory Allocation
|
|
|
|
**No Object Pooling**: Objects allocated per request
|
|
|
|
**Recommendation**: Use sync.Pool for frequently allocated objects
|
|
|
|
```go
|
|
var trackPool = sync.Pool{
|
|
New: func() interface{} {
|
|
return &pb.Track{}
|
|
},
|
|
}
|
|
|
|
func getTrack() *pb.Track {
|
|
return trackPool.Get().(*pb.Track)
|
|
}
|
|
|
|
func putTrack(t *pb.Track) {
|
|
// Reset fields
|
|
t.Id = ""
|
|
t.Title = ""
|
|
// ...
|
|
trackPool.Put(t)
|
|
}
|
|
```
|
|
|
|
## Security Best Practices
|
|
|
|
### Input Validation
|
|
|
|
**Minimal Validation**: Only email format checked
|
|
|
|
**Missing**:
|
|
- Query length limits (SQL injection via search)
|
|
- ID format validation
|
|
- Limit parameter bounds
|
|
|
|
**Recommendation**: Add comprehensive validation
|
|
|
|
```go
|
|
func validateSearchRequest(req *pb.SearchRequest) error {
|
|
if len(req.Query) == 0 {
|
|
return errors.New("query cannot be empty")
|
|
}
|
|
|
|
if len(req.Query) > 500 {
|
|
return errors.New("query too long (max 500 characters)")
|
|
}
|
|
|
|
if req.Limit < 1 || req.Limit > 50 {
|
|
return errors.New("limit must be between 1 and 50")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
```
|
|
|
|
### SQL Injection Prevention
|
|
|
|
**Parameterized Queries**: All queries use placeholders (good)
|
|
|
|
```go
|
|
err := s.db.QueryRow(ctx,
|
|
"SELECT * FROM users WHERE email = $1",
|
|
email,
|
|
).Scan(&user)
|
|
```
|
|
|
|
**No String Concatenation**: No SQL injection risk
|
|
|
|
### Secrets Management
|
|
|
|
**Environment Variables**: Secrets in plaintext `.env` files
|
|
|
|
**Recommendation**: Use secrets manager
|
|
|
|
```go
|
|
import "github.com/aws/aws-sdk-go/service/secretsmanager"
|
|
|
|
func getSecret(secretName string) (string, error) {
|
|
svc := secretsmanager.New(session.New())
|
|
|
|
result, err := svc.GetSecretValue(&secretsmanager.GetSecretValueInput{
|
|
SecretId: aws.String(secretName),
|
|
})
|
|
|
|
if err != nil {
|
|
return "", err
|
|
}
|
|
|
|
return *result.SecretString, nil
|
|
}
|
|
```
|
|
|
|
## Code Quality Metrics
|
|
|
|
### Cyclomatic Complexity
|
|
|
|
**High Complexity**: `main.go` (1329 lines, 23 methods)
|
|
|
|
**Recommendation**: Split into smaller files
|
|
|
|
### Code Duplication
|
|
|
|
**Provider Adapters**: Similar patterns across providers (acceptable)
|
|
|
|
**Search Methods**: Identical fan-out pattern (could be abstracted)
|
|
|
|
**Recommendation**: Extract common fan-out logic
|
|
|
|
```go
|
|
func (s *server) fanOutSearch(
|
|
ctx context.Context,
|
|
providers []trackProvider,
|
|
searchFunc func(trackProvider) ([]*pb.Track, error),
|
|
) ([]*pb.Track, []*pb.ProviderError, pb.ResponseStatus) {
|
|
var (
|
|
mu sync.Mutex
|
|
wg sync.WaitGroup
|
|
tracks []*pb.Track
|
|
errors []*pb.ProviderError
|
|
)
|
|
|
|
for _, provider := range providers {
|
|
wg.Add(1)
|
|
go func(p trackProvider) {
|
|
defer wg.Done()
|
|
|
|
results, err := searchFunc(p)
|
|
|
|
mu.Lock()
|
|
defer mu.Unlock()
|
|
|
|
if err != nil {
|
|
errors = append(errors, &pb.ProviderError{
|
|
Provider: p.Name(),
|
|
Message: err.Error(),
|
|
})
|
|
} else {
|
|
tracks = append(tracks, results...)
|
|
}
|
|
}(provider)
|
|
}
|
|
|
|
wg.Wait()
|
|
|
|
status := pb.ResponseStatus_OK
|
|
if len(errors) > 0 {
|
|
if len(tracks) == 0 {
|
|
status = pb.ResponseStatus_ERROR
|
|
} else {
|
|
status = pb.ResponseStatus_PARTIAL
|
|
}
|
|
}
|
|
|
|
return tracks, errors, status
|
|
}
|
|
```
|
|
|
|
### Documentation
|
|
|
|
**No Package Documentation**: Missing package-level comments
|
|
|
|
**Recommendation**: Add package docs
|
|
|
|
```go
|
|
// Package bedrock provides a unified music metadata and streaming API
|
|
// that aggregates data from multiple music platforms (Spotify, SoundCloud,
|
|
// Deezer, YouTube Music).
|
|
//
|
|
// The service exposes a gRPC interface with 23 methods for searching,
|
|
// retrieving, and streaming music content. It also provides an HTTP proxy
|
|
// for streaming audio and album art.
|
|
//
|
|
// Authentication is handled via JWT tokens with bcrypt password hashing.
|
|
// All provider queries are executed in parallel for optimal performance.
|
|
package main
|
|
```
|
|
|
|
## Recommendations for Metadata Aggregator
|
|
|
|
### Adopt
|
|
|
|
- Provider interface pattern (clean abstraction)
|
|
- Fan-out concurrency (parallel queries)
|
|
- Partial response handling (resilient to failures)
|
|
- gRPC interceptors (authentication)
|
|
- bcrypt password hashing (secure)
|
|
- Parameterized queries (SQL injection safe)
|
|
|
|
### Avoid
|
|
|
|
- Single 1300+ line file (split by domain)
|
|
- No unit tests (add mocked tests)
|
|
- No error wrapping (add context)
|
|
- Unbounded goroutines (use worker pool)
|
|
- No input validation (validate all inputs)
|
|
- Stub health checks (implement real checks)
|
|
|
|
### Enhance
|
|
|
|
- Add structured logging (zap, zerolog)
|
|
- Add metrics (Prometheus)
|
|
- Add caching (Redis)
|
|
- Add rate limiting (per-user, per-provider)
|
|
- Add circuit breakers (failing providers)
|
|
- Add retry logic (exponential backoff)
|
|
- Add comprehensive validation
|
|
- Add unit tests with mocks
|
|
- Add code coverage reporting
|
|
- Add API documentation (OpenAPI/Swagger for HTTP, gRPC reflection)
|