Add Week 9 Smart Features: collections, artwork, predictive prefetch
Smart Collections (musicfs-search/src/collections.rs): - CollectionStore with thread-safe Mutex<Connection> - CollectionQuery enum: Match, DateRange, RecentlyAdded/Played, MostPlayed, Genre, Compound - Builtin collections for Recently Added, 80s/90s Music Artwork Extraction & Caching: - ArtworkExtractor using symphonia Visual (musicfs-metadata) - ArtworkCache with CAS storage + on-demand resize (musicfs-cache) - ArtType: Front/Back/Other, ArtSize: Thumbnail/Medium/Full Predictive Prefetching: - PatternStore tracks access patterns with sequence prediction - PrefetchEngine listens to FileAccessed events, prefetches predictions - PrefetchOps exposes /.prefetch/ virtual directory with status/hints Oracle review fixes applied: - CollectionStore uses Mutex for thread safety - FileAccessed event now includes file_id for canonical correlation - JSON parse warnings in collection deserialization 130 tests pass (15 new tests added)
This commit is contained in:
Alexander
@@ -0,0 +1,315 @@
|
||||
# Smart Features API Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
MusicFS Week 9 introduces three intelligent features:
|
||||
1. **Smart Collections** - Dynamic playlists based on queries, time ranges, and listening patterns
|
||||
2. **Artwork Extraction & Caching** - Extract and serve album art in multiple sizes
|
||||
3. **Predictive Prefetching** - Learn listening patterns to preload likely-next tracks
|
||||
|
||||
---
|
||||
|
||||
## Smart Collections
|
||||
|
||||
### CollectionStore
|
||||
|
||||
Manages persistent smart collections using SQLite.
|
||||
|
||||
```rust
|
||||
pub struct CollectionStore {
|
||||
db: rusqlite::Connection,
|
||||
}
|
||||
|
||||
pub struct Collection {
|
||||
pub id: i64,
|
||||
pub name: String,
|
||||
pub query: CollectionQuery,
|
||||
pub created_at: SystemTime,
|
||||
pub updated_at: SystemTime,
|
||||
}
|
||||
```
|
||||
|
||||
### CollectionQuery Types
|
||||
|
||||
| Query Type | Description | Example |
|
||||
|------------|-------------|---------|
|
||||
| `Match(String)` | tantivy search query | `"artist:Metallica"` |
|
||||
| `DateRange { start, end }` | Files added within range | Last 30 days |
|
||||
| `RecentlyAdded(days)` | Files added in last N days | `RecentlyAdded(7)` |
|
||||
| `RecentlyPlayed(days)` | Files played in last N days | `RecentlyPlayed(30)` |
|
||||
| `MostPlayed(limit)` | Top N most played tracks | `MostPlayed(100)` |
|
||||
| `Genre(String)` | All tracks matching genre | `"Progressive Rock"` |
|
||||
| `Compound(Vec)` | AND combination of queries | Multiple conditions |
|
||||
|
||||
### API
|
||||
|
||||
```rust
|
||||
impl CollectionStore {
|
||||
fn create(&self, name: &str, query: CollectionQuery) -> Result<i64, CollectionError>;
|
||||
fn get(&self, id: i64) -> Result<Option<Collection>, CollectionError>;
|
||||
fn list(&self) -> Result<Vec<Collection>, CollectionError>;
|
||||
fn update(&self, id: i64, name: &str, query: CollectionQuery) -> Result<(), CollectionError>;
|
||||
fn delete(&self, id: i64) -> Result<(), CollectionError>;
|
||||
fn evaluate(&self, id: i64, index: &SearchIndex, patterns: &PatternStore) -> Result<Vec<FileId>, CollectionError>;
|
||||
}
|
||||
```
|
||||
|
||||
### FUSE Integration (Planned)
|
||||
|
||||
Collections will appear as virtual directories under `/.collections/`:
|
||||
|
||||
```bash
|
||||
$ ls /mnt/musicfs/.collections/
|
||||
Recent Additions/
|
||||
Most Played/
|
||||
80s Metal/
|
||||
|
||||
$ ls /mnt/musicfs/.collections/Most\ Played/
|
||||
001. Track1.flac -> /mnt/musicfs/Artist/Album/Track1.flac
|
||||
002. Track2.flac -> /mnt/musicfs/Artist/Album/Track2.flac
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Artwork Extraction & Caching
|
||||
|
||||
### ArtworkExtractor
|
||||
|
||||
Extracts embedded artwork from audio files.
|
||||
|
||||
```rust
|
||||
pub struct Artwork {
|
||||
pub data: Vec<u8>,
|
||||
pub mime_type: String,
|
||||
pub art_type: ArtType,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
}
|
||||
|
||||
pub enum ArtType {
|
||||
Front,
|
||||
Back,
|
||||
Other,
|
||||
}
|
||||
|
||||
pub enum ArtSize {
|
||||
Thumbnail, // 150x150 max
|
||||
Medium, // 300x300 max
|
||||
Full, // Original size
|
||||
}
|
||||
```
|
||||
|
||||
### API
|
||||
|
||||
```rust
|
||||
impl ArtworkExtractor {
|
||||
fn extract(&self, path: &Path) -> Result<Vec<Artwork>, ArtworkError>;
|
||||
fn extract_first(&self, path: &Path) -> Result<Option<Artwork>, ArtworkError>;
|
||||
fn resize(data: &[u8], size: ArtSize) -> Result<Vec<u8>, ArtworkError>;
|
||||
}
|
||||
```
|
||||
|
||||
### ArtworkCache
|
||||
|
||||
Caches artwork in CAS (Content-Addressable Storage).
|
||||
|
||||
```rust
|
||||
impl ArtworkCache {
|
||||
async fn store(&self, file_id: i64, artwork: &Artwork) -> Result<ChunkHash, ArtworkError>;
|
||||
async fn get(&self, file_id: i64, art_type: &str, size: ArtSize) -> Result<Option<Vec<u8>>, ArtworkError>;
|
||||
async fn has(&self, file_id: i64, art_type: &str) -> Result<bool, ArtworkError>;
|
||||
}
|
||||
```
|
||||
|
||||
### Size Specifications
|
||||
|
||||
| Size | Max Dimension | Use Case |
|
||||
|------|---------------|----------|
|
||||
| Thumbnail | 150px | List views, grids |
|
||||
| Medium | 300px | Detail panels |
|
||||
| Full | Original | High-res display |
|
||||
|
||||
### Caching Strategy
|
||||
|
||||
1. Original artwork stored in CAS with content hash
|
||||
2. SQLite maps `(file_id, art_type)` → `chunk_hash`
|
||||
3. Resizing performed on-demand, not cached (saves storage)
|
||||
4. Max input size: 10MB (reject larger images)
|
||||
|
||||
---
|
||||
|
||||
## Predictive Prefetching
|
||||
|
||||
### Access Patterns (PatternStore)
|
||||
|
||||
Tracks file access history to predict next tracks.
|
||||
|
||||
```rust
|
||||
pub struct AccessPattern {
|
||||
pub file_id: FileId,
|
||||
pub timestamp: SystemTime,
|
||||
pub context: AccessContext,
|
||||
pub hour_of_day: u8,
|
||||
}
|
||||
|
||||
pub struct AccessContext {
|
||||
pub album_id: Option<i64>,
|
||||
pub track_number: Option<u32>,
|
||||
pub artist: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern Learning
|
||||
|
||||
| Pattern Type | Description | Use Case |
|
||||
|--------------|-------------|----------|
|
||||
| Sequential | A → B → C transitions | Album playback |
|
||||
| Time-based | Hour-of-day preferences | Morning playlist |
|
||||
| Frequency | Most played tracks | Popular content |
|
||||
|
||||
### API
|
||||
|
||||
```rust
|
||||
impl PatternStore {
|
||||
fn record(&self, file_id: FileId, context: AccessContext) -> Result<(), PatternError>;
|
||||
fn predict_next(&self, current: FileId, limit: usize) -> Vec<FileId>;
|
||||
fn predict_for_time(&self, hour: u8, limit: usize) -> Vec<FileId>;
|
||||
fn recently_played(&self, days: u32) -> Result<Vec<FileId>, PatternError>;
|
||||
fn most_played(&self, limit: u32) -> Result<Vec<FileId>, PatternError>;
|
||||
}
|
||||
```
|
||||
|
||||
### PrefetchEngine
|
||||
|
||||
Background engine that listens for file access events and prefetches predicted content.
|
||||
|
||||
```rust
|
||||
pub struct PrefetchConfig {
|
||||
pub lookahead: usize, // How many tracks to prefetch (default: 3)
|
||||
pub max_concurrent: usize, // Concurrent prefetch limit (default: 2)
|
||||
pub cooldown: Duration, // Delay between prefetch bursts (default: 100ms)
|
||||
pub enabled: bool, // Master switch
|
||||
}
|
||||
```
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ EventBus │────▶│ PrefetchEngine │────▶│ ContentFetcher │
|
||||
│ (FileAccessed) │ │ (predictions) │ │ (CAS storage) │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ PatternStore │
|
||||
│ (SQLite DB) │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
### FUSE Interface
|
||||
|
||||
Virtual directory `/.prefetch/` exposes prefetch status and hints:
|
||||
|
||||
```bash
|
||||
$ cat /mnt/musicfs/.prefetch/status
|
||||
MusicFS Prefetch Status
|
||||
=======================
|
||||
running: true
|
||||
in_flight: 2
|
||||
most_played: [42, 57, 103, 89, 12]
|
||||
|
||||
$ ls /mnt/musicfs/.prefetch/
|
||||
status
|
||||
hint_0042
|
||||
hint_0057
|
||||
hint_0103
|
||||
|
||||
$ cat /mnt/musicfs/.prefetch/hint_0042
|
||||
57
|
||||
103
|
||||
89
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Targets
|
||||
|
||||
| Feature | Metric | Target |
|
||||
|---------|--------|--------|
|
||||
| Collection evaluation | Latency | <100ms for 100k files |
|
||||
| Artwork extraction | Throughput | >10 files/sec |
|
||||
| Artwork resize | Latency | <50ms per image |
|
||||
| Pattern prediction | Latency | <10ms |
|
||||
| Prefetch hit rate | Accuracy | >70% for sequential play |
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### CollectionError
|
||||
|
||||
| Error | Description |
|
||||
|-------|-------------|
|
||||
| `Database(rusqlite::Error)` | SQLite operation failed |
|
||||
| `NotFound` | Collection ID doesn't exist |
|
||||
| `InvalidQuery` | Query failed to serialize |
|
||||
| `Search(SearchError)` | tantivy query failed |
|
||||
| `Pattern(PatternError)` | Pattern lookup failed |
|
||||
|
||||
### ArtworkError
|
||||
|
||||
| Error | Description |
|
||||
|-------|-------------|
|
||||
| `Database(rusqlite::Error)` | Cache DB operation failed |
|
||||
| `Cas(CasError)` | CAS storage operation failed |
|
||||
| `InvalidHash` | Stored hash is malformed |
|
||||
| `NotFound` | Artwork not in cache |
|
||||
| `ImageTooLarge(usize)` | Input exceeds 10MB limit |
|
||||
| `InvalidImage` | Cannot decode image data |
|
||||
| `ResizeFailed` | Image resize operation failed |
|
||||
|
||||
### PatternError
|
||||
|
||||
| Error | Description |
|
||||
|-------|-------------|
|
||||
| `Database(rusqlite::Error)` | SQLite operation failed |
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Default Settings
|
||||
|
||||
```toml
|
||||
[prefetch]
|
||||
enabled = true
|
||||
lookahead = 3
|
||||
max_concurrent = 2
|
||||
cooldown_ms = 100
|
||||
|
||||
[artwork]
|
||||
max_input_size_mb = 10
|
||||
thumbnail_size = 150
|
||||
medium_size = 300
|
||||
|
||||
[patterns]
|
||||
max_history_days = 30
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tests
|
||||
|
||||
| Test | Type | Validates |
|
||||
|------|------|-----------|
|
||||
| `test_collection_crud` | Unit | Create, read, update, delete |
|
||||
| `test_collection_evaluate_match` | Unit | Match query evaluation |
|
||||
| `test_collection_persistence` | Unit | Collections survive restart |
|
||||
| `test_artwork_extract_flac` | Unit | FLAC artwork extraction |
|
||||
| `test_artwork_cache_store_get` | Unit | Cache round-trip |
|
||||
| `test_artwork_resize` | Unit | Resize produces valid output |
|
||||
| `test_pattern_prediction` | Unit | Sequential pattern learning |
|
||||
| `test_pattern_persistence` | Unit | Patterns survive restart |
|
||||
| `test_prefetch_config_defaults` | Unit | Default config values |
|
||||
| `test_prefetch_ops_*` | Unit | FUSE PrefetchOps integration |
|
||||
Generated
+193
@@ -2,6 +2,12 @@
|
||||
# It is not intended for manual editing.
|
||||
version = 4
|
||||
|
||||
[[package]]
|
||||
name = "adler2"
|
||||
version = "2.0.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
|
||||
|
||||
[[package]]
|
||||
name = "ahash"
|
||||
version = "0.8.12"
|
||||
@@ -29,6 +35,15 @@ version = "0.2.21"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
|
||||
|
||||
[[package]]
|
||||
name = "android_system_properties"
|
||||
version = "0.1.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311"
|
||||
dependencies = [
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "anstream"
|
||||
version = "1.0.0"
|
||||
@@ -265,6 +280,19 @@ version = "1.0.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
|
||||
|
||||
[[package]]
|
||||
name = "chrono"
|
||||
version = "0.4.44"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0"
|
||||
dependencies = [
|
||||
"iana-time-zone",
|
||||
"js-sys",
|
||||
"num-traits",
|
||||
"wasm-bindgen",
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "clap"
|
||||
version = "4.6.1"
|
||||
@@ -305,12 +333,24 @@ version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9"
|
||||
|
||||
[[package]]
|
||||
name = "color_quant"
|
||||
version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
|
||||
|
||||
[[package]]
|
||||
name = "colorchoice"
|
||||
version = "1.0.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
|
||||
|
||||
[[package]]
|
||||
name = "core-foundation-sys"
|
||||
version = "0.8.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
|
||||
|
||||
[[package]]
|
||||
name = "crc32fast"
|
||||
version = "1.5.0"
|
||||
@@ -477,6 +517,15 @@ version = "2.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6"
|
||||
|
||||
[[package]]
|
||||
name = "fdeflate"
|
||||
version = "0.3.7"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e6853b52649d4ac5c0bd02320cddc5ba956bdb407c4b75a2c6b75bf51500f8c"
|
||||
dependencies = [
|
||||
"simd-adler32",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "filetime"
|
||||
version = "0.2.28"
|
||||
@@ -499,6 +548,16 @@ version = "0.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0ce7134b9999ecaf8bcd65542e436736ef32ddca1b3e06094cb6ec5755203b80"
|
||||
|
||||
[[package]]
|
||||
name = "flate2"
|
||||
version = "1.1.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
|
||||
dependencies = [
|
||||
"crc32fast",
|
||||
"miniz_oxide",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fnv"
|
||||
version = "1.0.7"
|
||||
@@ -793,12 +852,50 @@ dependencies = [
|
||||
"tokio-io-timeout",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iana-time-zone"
|
||||
version = "0.1.65"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
|
||||
dependencies = [
|
||||
"android_system_properties",
|
||||
"core-foundation-sys",
|
||||
"iana-time-zone-haiku",
|
||||
"js-sys",
|
||||
"log",
|
||||
"wasm-bindgen",
|
||||
"windows-core",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iana-time-zone-haiku"
|
||||
version = "0.1.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
|
||||
dependencies = [
|
||||
"cc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "id-arena"
|
||||
version = "2.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
|
||||
|
||||
[[package]]
|
||||
name = "image"
|
||||
version = "0.24.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5690139d2f55868e080017335e4b94cb7414274c74f1669c84fb5feba2c9f69d"
|
||||
dependencies = [
|
||||
"bytemuck",
|
||||
"byteorder",
|
||||
"color_quant",
|
||||
"jpeg-decoder",
|
||||
"num-traits",
|
||||
"png",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "indexmap"
|
||||
version = "1.9.3"
|
||||
@@ -884,6 +981,12 @@ dependencies = [
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jpeg-decoder"
|
||||
version = "0.3.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "00810f1d8b74be64b13dbf3db89ac67740615d6c891f0e7b6179326533011a07"
|
||||
|
||||
[[package]]
|
||||
name = "js-sys"
|
||||
version = "0.3.98"
|
||||
@@ -1060,6 +1163,16 @@ version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
|
||||
|
||||
[[package]]
|
||||
name = "miniz_oxide"
|
||||
version = "0.8.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
|
||||
dependencies = [
|
||||
"adler2",
|
||||
"simd-adler32",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mio"
|
||||
version = "0.8.11"
|
||||
@@ -1116,8 +1229,12 @@ checksum = "2195bf6aa996a481483b29d62a7663eed3fe39600c460e323f8ff41e90bdd89b"
|
||||
name = "musicfs-cache"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"chrono",
|
||||
"image",
|
||||
"musicfs-cas",
|
||||
"musicfs-core",
|
||||
"musicfs-metadata",
|
||||
"parking_lot 0.12.5",
|
||||
"rmp-serde",
|
||||
"rusqlite",
|
||||
"serde",
|
||||
@@ -1217,6 +1334,7 @@ dependencies = [
|
||||
name = "musicfs-metadata"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"image",
|
||||
"musicfs-core",
|
||||
"symphonia",
|
||||
"thiserror",
|
||||
@@ -1248,6 +1366,9 @@ dependencies = [
|
||||
"moka",
|
||||
"musicfs-core",
|
||||
"parking_lot 0.12.5",
|
||||
"rusqlite",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"tantivy",
|
||||
"tempfile",
|
||||
"thiserror",
|
||||
@@ -1476,6 +1597,19 @@ version = "0.3.33"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e"
|
||||
|
||||
[[package]]
|
||||
name = "png"
|
||||
version = "0.17.16"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "82151a2fc869e011c153adc57cf2789ccb8d9906ce52c0b39a6b5697749d7526"
|
||||
dependencies = [
|
||||
"bitflags 1.3.2",
|
||||
"crc32fast",
|
||||
"fdeflate",
|
||||
"flate2",
|
||||
"miniz_oxide",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "portable-atomic"
|
||||
version = "1.13.1"
|
||||
@@ -1887,6 +2021,12 @@ dependencies = [
|
||||
"libc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "simd-adler32"
|
||||
version = "0.3.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
|
||||
|
||||
[[package]]
|
||||
name = "sketches-ddsketch"
|
||||
version = "0.2.2"
|
||||
@@ -2785,12 +2925,65 @@ version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
|
||||
|
||||
[[package]]
|
||||
name = "windows-core"
|
||||
version = "0.62.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
|
||||
dependencies = [
|
||||
"windows-implement",
|
||||
"windows-interface",
|
||||
"windows-link",
|
||||
"windows-result",
|
||||
"windows-strings",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-implement"
|
||||
version = "0.60.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-interface"
|
||||
version = "0.59.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-link"
|
||||
version = "0.2.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
|
||||
|
||||
[[package]]
|
||||
name = "windows-result"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-strings"
|
||||
version = "0.5.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.48.0"
|
||||
|
||||
@@ -75,5 +75,9 @@ tonic = "0.11"
|
||||
prost = "0.12"
|
||||
tokio-stream = "0.1"
|
||||
|
||||
# Smart Features (Week 9)
|
||||
image = { version = "0.24", default-features = false, features = ["jpeg", "png"] }
|
||||
chrono = "0.4"
|
||||
|
||||
[workspace.dependencies.tonic-build]
|
||||
version = "0.11"
|
||||
|
||||
@@ -6,6 +6,7 @@ edition.workspace = true
|
||||
[dependencies]
|
||||
musicfs-core = { path = "../musicfs-core" }
|
||||
musicfs-cas = { path = "../musicfs-cas" }
|
||||
musicfs-metadata = { path = "../musicfs-metadata" }
|
||||
rusqlite = { workspace = true, features = ["bundled"] }
|
||||
sled.workspace = true
|
||||
tokio.workspace = true
|
||||
@@ -13,6 +14,9 @@ tracing.workspace = true
|
||||
thiserror.workspace = true
|
||||
serde.workspace = true
|
||||
rmp-serde.workspace = true
|
||||
image.workspace = true
|
||||
parking_lot.workspace = true
|
||||
chrono.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile.workspace = true
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
use image::ImageFormat;
|
||||
use musicfs_cas::CasStore;
|
||||
use musicfs_core::ChunkHash;
|
||||
use musicfs_metadata::artwork::{ArtSize, ArtType, Artwork};
|
||||
use std::io::Cursor;
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
use tracing::debug;
|
||||
|
||||
const MAX_ARTWORK_INPUT_SIZE: usize = 10 * 1024 * 1024;
|
||||
|
||||
pub struct ArtworkCache {
|
||||
store: Arc<CasStore>,
|
||||
db_path: std::path::PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct CachedArtwork {
|
||||
pub file_id: i64,
|
||||
pub art_type: String,
|
||||
pub chunk_hash: ChunkHash,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
}
|
||||
|
||||
impl ArtworkCache {
|
||||
pub fn new(store: Arc<CasStore>, db_path: &Path) -> Result<Self, ArtworkError> {
|
||||
let db = rusqlite::Connection::open(db_path)?;
|
||||
|
||||
db.execute(
|
||||
"CREATE TABLE IF NOT EXISTS artwork (
|
||||
id INTEGER PRIMARY KEY,
|
||||
file_id INTEGER NOT NULL,
|
||||
art_type TEXT NOT NULL,
|
||||
chunk_hash TEXT NOT NULL,
|
||||
width INTEGER NOT NULL,
|
||||
height INTEGER NOT NULL,
|
||||
UNIQUE(file_id, art_type)
|
||||
)",
|
||||
[],
|
||||
)?;
|
||||
|
||||
Ok(Self {
|
||||
store,
|
||||
db_path: db_path.to_path_buf(),
|
||||
})
|
||||
}
|
||||
|
||||
pub async fn store(&self, file_id: i64, artwork: &Artwork) -> Result<ChunkHash, ArtworkError> {
|
||||
if artwork.data.len() > MAX_ARTWORK_INPUT_SIZE {
|
||||
return Err(ArtworkError::ImageTooLarge(artwork.data.len()));
|
||||
}
|
||||
|
||||
let hash = self.store.put(&artwork.data).await?;
|
||||
|
||||
let art_type_str = match artwork.art_type {
|
||||
ArtType::Front => "front",
|
||||
ArtType::Back => "back",
|
||||
ArtType::Other => "other",
|
||||
};
|
||||
|
||||
let db_path = self.db_path.clone();
|
||||
let art_type_clone = art_type_str.to_string();
|
||||
let hash_hex = hash.to_hex();
|
||||
let width = artwork.width;
|
||||
let height = artwork.height;
|
||||
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let db = rusqlite::Connection::open(&db_path)?;
|
||||
db.execute(
|
||||
"INSERT OR REPLACE INTO artwork
|
||||
(file_id, art_type, chunk_hash, width, height)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5)",
|
||||
rusqlite::params![file_id, art_type_clone, hash_hex, width, height],
|
||||
)?;
|
||||
Ok::<_, ArtworkError>(())
|
||||
})
|
||||
.await
|
||||
.map_err(|e| ArtworkError::SpawnBlocking(e.to_string()))??;
|
||||
|
||||
debug!("Cached artwork for file {}", file_id);
|
||||
Ok(hash)
|
||||
}
|
||||
|
||||
pub async fn get(
|
||||
&self,
|
||||
file_id: i64,
|
||||
art_type: &str,
|
||||
size: ArtSize,
|
||||
) -> Result<Option<Vec<u8>>, ArtworkError> {
|
||||
let db_path = self.db_path.clone();
|
||||
let art_type_clone = art_type.to_string();
|
||||
|
||||
let hash_hex: Option<String> = tokio::task::spawn_blocking(move || {
|
||||
let db = rusqlite::Connection::open(&db_path)?;
|
||||
db.query_row(
|
||||
"SELECT chunk_hash FROM artwork WHERE file_id = ?1 AND art_type = ?2",
|
||||
rusqlite::params![file_id, art_type_clone],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.ok()
|
||||
.ok_or(ArtworkError::NotFound)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| ArtworkError::SpawnBlocking(e.to_string()))?
|
||||
.ok();
|
||||
|
||||
match hash_hex {
|
||||
Some(hex) => {
|
||||
let hash = ChunkHash::from_hex(&hex).ok_or(ArtworkError::InvalidHash)?;
|
||||
let data = self.store.get(&hash).await?;
|
||||
|
||||
match size {
|
||||
ArtSize::Full => Ok(Some(data.to_vec())),
|
||||
ArtSize::Thumbnail | ArtSize::Medium => {
|
||||
let resized = self.resize_on_demand(&data, size)?;
|
||||
Ok(Some(resized))
|
||||
}
|
||||
}
|
||||
}
|
||||
None => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn has(&self, file_id: i64, art_type: &str) -> Result<bool, ArtworkError> {
|
||||
let db_path = self.db_path.clone();
|
||||
let art_type_clone = art_type.to_string();
|
||||
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let db = rusqlite::Connection::open(&db_path)?;
|
||||
let count: i64 = db.query_row(
|
||||
"SELECT COUNT(*) FROM artwork WHERE file_id = ?1 AND art_type = ?2",
|
||||
rusqlite::params![file_id, art_type_clone],
|
||||
|row| row.get(0),
|
||||
)?;
|
||||
Ok(count > 0)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| ArtworkError::SpawnBlocking(e.to_string()))?
|
||||
}
|
||||
|
||||
fn resize_on_demand(&self, data: &[u8], size: ArtSize) -> Result<Vec<u8>, ArtworkError> {
|
||||
let max_dim = size.max_dimension().unwrap_or(300);
|
||||
let img = image::load_from_memory(data).map_err(|_| ArtworkError::InvalidImage)?;
|
||||
|
||||
if img.width() <= max_dim && img.height() <= max_dim {
|
||||
return Ok(data.to_vec());
|
||||
}
|
||||
|
||||
let resized = img.thumbnail(max_dim, max_dim);
|
||||
let mut output = Vec::new();
|
||||
let mut cursor = Cursor::new(&mut output);
|
||||
resized
|
||||
.write_to(&mut cursor, ImageFormat::Jpeg)
|
||||
.map_err(|_| ArtworkError::ResizeFailed)?;
|
||||
|
||||
Ok(output)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum ArtworkError {
|
||||
#[error("database error: {0}")]
|
||||
Database(#[from] rusqlite::Error),
|
||||
|
||||
#[error("CAS error: {0}")]
|
||||
Cas(#[from] musicfs_cas::CasError),
|
||||
|
||||
#[error("invalid hash")]
|
||||
InvalidHash,
|
||||
|
||||
#[error("artwork not found")]
|
||||
NotFound,
|
||||
|
||||
#[error("image too large: {0} bytes (max 10MB)")]
|
||||
ImageTooLarge(usize),
|
||||
|
||||
#[error("invalid image data")]
|
||||
InvalidImage,
|
||||
|
||||
#[error("resize failed")]
|
||||
ResizeFailed,
|
||||
|
||||
#[error("spawn_blocking error: {0}")]
|
||||
SpawnBlocking(String),
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_max_artwork_size() {
|
||||
assert_eq!(MAX_ARTWORK_INPUT_SIZE, 10 * 1024 * 1024);
|
||||
}
|
||||
}
|
||||
@@ -1,11 +1,17 @@
|
||||
mod artwork;
|
||||
mod db;
|
||||
mod eviction;
|
||||
mod metadata;
|
||||
mod patterns;
|
||||
mod prefetch;
|
||||
mod tree;
|
||||
|
||||
pub use artwork::{ArtworkCache, ArtworkError, CachedArtwork};
|
||||
pub use db::Database;
|
||||
pub use eviction::{EvictionError, EvictionPolicy, LruEviction};
|
||||
pub use metadata::MetadataCache;
|
||||
pub use patterns::{AccessContext, AccessPattern, PatternError, PatternStore};
|
||||
pub use prefetch::{PrefetchConfig, PrefetchEngine, PrefetchHandle};
|
||||
pub use tree::{
|
||||
DirNode, FileNode, Inode, RefreshPolicy, TreeBuilder, VirtualNode, VirtualTree, ROOT_INODE,
|
||||
};
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
use musicfs_core::FileId;
|
||||
use parking_lot::{Mutex, RwLock};
|
||||
use std::collections::HashMap;
|
||||
use std::path::Path;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AccessPattern {
|
||||
pub file_id: FileId,
|
||||
pub timestamp: SystemTime,
|
||||
pub context: AccessContext,
|
||||
pub hour_of_day: u8,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct AccessContext {
|
||||
pub album_id: Option<i64>,
|
||||
pub track_number: Option<u32>,
|
||||
pub artist: Option<String>,
|
||||
}
|
||||
|
||||
pub struct PatternStore {
|
||||
db: Mutex<rusqlite::Connection>,
|
||||
sequence_counts: RwLock<HashMap<(FileId, FileId), u32>>,
|
||||
time_patterns: RwLock<HashMap<u8, Vec<FileId>>>,
|
||||
max_history: usize,
|
||||
}
|
||||
|
||||
impl PatternStore {
|
||||
pub fn new(db_path: &Path, max_history: usize) -> Result<Self, PatternError> {
|
||||
let db = rusqlite::Connection::open(db_path)?;
|
||||
|
||||
db.execute(
|
||||
"CREATE TABLE IF NOT EXISTS access_log (
|
||||
id INTEGER PRIMARY KEY,
|
||||
file_id INTEGER NOT NULL,
|
||||
access_time INTEGER NOT NULL,
|
||||
hour_of_day INTEGER NOT NULL
|
||||
)",
|
||||
[],
|
||||
)?;
|
||||
|
||||
db.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_access_log_file ON access_log(file_id)",
|
||||
[],
|
||||
)?;
|
||||
|
||||
db.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_access_log_time ON access_log(access_time)",
|
||||
[],
|
||||
)?;
|
||||
|
||||
db.execute(
|
||||
"CREATE TABLE IF NOT EXISTS sequence_counts (
|
||||
from_file_id INTEGER NOT NULL,
|
||||
to_file_id INTEGER NOT NULL,
|
||||
count INTEGER NOT NULL DEFAULT 1,
|
||||
PRIMARY KEY (from_file_id, to_file_id)
|
||||
)",
|
||||
[],
|
||||
)?;
|
||||
|
||||
let sequence_counts = {
|
||||
let mut map = HashMap::new();
|
||||
let mut stmt = db.prepare("SELECT from_file_id, to_file_id, count FROM sequence_counts")?;
|
||||
let rows = stmt.query_map([], |row| {
|
||||
Ok((
|
||||
(
|
||||
FileId(row.get::<_, i64>(0)?),
|
||||
FileId(row.get::<_, i64>(1)?),
|
||||
),
|
||||
row.get::<_, u32>(2)?,
|
||||
))
|
||||
})?;
|
||||
for row in rows {
|
||||
let (key, count) = row?;
|
||||
map.insert(key, count);
|
||||
}
|
||||
map
|
||||
};
|
||||
|
||||
Ok(Self {
|
||||
db: Mutex::new(db),
|
||||
sequence_counts: RwLock::new(sequence_counts),
|
||||
time_patterns: RwLock::new(HashMap::new()),
|
||||
max_history,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn record(&self, file_id: FileId, _context: AccessContext) -> Result<(), PatternError> {
|
||||
let now = SystemTime::now();
|
||||
let timestamp = now.duration_since(UNIX_EPOCH).unwrap().as_secs() as i64;
|
||||
let hour = (timestamp / 3600 % 24) as u8;
|
||||
|
||||
let db = self.db.lock();
|
||||
|
||||
db.execute(
|
||||
"INSERT INTO access_log (file_id, access_time, hour_of_day) VALUES (?1, ?2, ?3)",
|
||||
rusqlite::params![file_id.0, timestamp, hour],
|
||||
)?;
|
||||
|
||||
{
|
||||
let mut time_patterns = self.time_patterns.write();
|
||||
time_patterns.entry(hour).or_default().push(file_id);
|
||||
}
|
||||
|
||||
let prev_file_id: Option<i64> = db
|
||||
.query_row(
|
||||
"SELECT file_id FROM access_log WHERE id = (SELECT MAX(id) - 1 FROM access_log)",
|
||||
[],
|
||||
|row| row.get(0),
|
||||
)
|
||||
.ok();
|
||||
|
||||
if let Some(prev_id) = prev_file_id {
|
||||
let prev = FileId(prev_id);
|
||||
|
||||
{
|
||||
let mut sequences = self.sequence_counts.write();
|
||||
*sequences.entry((prev, file_id)).or_insert(0) += 1;
|
||||
}
|
||||
|
||||
db.execute(
|
||||
"INSERT INTO sequence_counts (from_file_id, to_file_id, count)
|
||||
VALUES (?1, ?2, 1)
|
||||
ON CONFLICT(from_file_id, to_file_id) DO UPDATE SET count = count + 1",
|
||||
rusqlite::params![prev_id, file_id.0],
|
||||
)?;
|
||||
}
|
||||
|
||||
let cutoff = timestamp - (self.max_history as i64 * 86400);
|
||||
db.execute("DELETE FROM access_log WHERE access_time < ?1", [cutoff])?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn predict_next(&self, current: FileId, limit: usize) -> Vec<FileId> {
|
||||
let sequences = self.sequence_counts.read();
|
||||
|
||||
let mut predictions: Vec<_> = sequences
|
||||
.iter()
|
||||
.filter(|((from, _), count)| *from == current && **count >= 2)
|
||||
.map(|((_, to), count)| (*to, *count))
|
||||
.collect();
|
||||
|
||||
predictions.sort_by(|a, b| b.1.cmp(&a.1));
|
||||
predictions
|
||||
.into_iter()
|
||||
.take(limit)
|
||||
.map(|(id, _)| id)
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn predict_for_time(&self, hour: u8, limit: usize) -> Vec<FileId> {
|
||||
let time_patterns = self.time_patterns.read();
|
||||
|
||||
time_patterns
|
||||
.get(&hour)
|
||||
.map(|files| files.iter().rev().take(limit).copied().collect())
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
pub fn recently_played(&self, days: u32) -> Result<Vec<FileId>, PatternError> {
|
||||
let cutoff = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap()
|
||||
.as_secs() as i64
|
||||
- (days as i64 * 86400);
|
||||
|
||||
let db = self.db.lock();
|
||||
let mut stmt = db.prepare(
|
||||
"SELECT DISTINCT file_id FROM access_log WHERE access_time >= ?1 ORDER BY access_time DESC",
|
||||
)?;
|
||||
|
||||
let files: Vec<FileId> = stmt
|
||||
.query_map([cutoff], |row| Ok(FileId(row.get(0)?)))?
|
||||
.filter_map(|r| r.ok())
|
||||
.collect();
|
||||
|
||||
Ok(files)
|
||||
}
|
||||
|
||||
pub fn most_played(&self, limit: u32) -> Result<Vec<FileId>, PatternError> {
|
||||
let db = self.db.lock();
|
||||
let mut stmt = db.prepare(
|
||||
"SELECT file_id, COUNT(*) as play_count FROM access_log
|
||||
GROUP BY file_id ORDER BY play_count DESC LIMIT ?1",
|
||||
)?;
|
||||
|
||||
let files: Vec<FileId> = stmt
|
||||
.query_map([limit], |row| Ok(FileId(row.get(0)?)))?
|
||||
.filter_map(|r| r.ok())
|
||||
.collect();
|
||||
|
||||
Ok(files)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum PatternError {
|
||||
#[error("database error: {0}")]
|
||||
Database(#[from] rusqlite::Error),
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
#[test]
|
||||
fn test_pattern_prediction() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let db_path = dir.path().join("patterns.db");
|
||||
let store = PatternStore::new(&db_path, 30).unwrap();
|
||||
let ctx = AccessContext::default();
|
||||
|
||||
for _ in 0..5 {
|
||||
store.record(FileId(1), ctx.clone()).unwrap();
|
||||
store.record(FileId(2), ctx.clone()).unwrap();
|
||||
store.record(FileId(3), ctx.clone()).unwrap();
|
||||
}
|
||||
|
||||
let predictions = store.predict_next(FileId(1), 3);
|
||||
assert!(!predictions.is_empty());
|
||||
assert_eq!(predictions[0], FileId(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pattern_persistence() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let db_path = dir.path().join("patterns.db");
|
||||
let ctx = AccessContext::default();
|
||||
|
||||
{
|
||||
let store = PatternStore::new(&db_path, 30).unwrap();
|
||||
for _ in 0..3 {
|
||||
store.record(FileId(1), ctx.clone()).unwrap();
|
||||
store.record(FileId(2), ctx.clone()).unwrap();
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let store = PatternStore::new(&db_path, 30).unwrap();
|
||||
let predictions = store.predict_next(FileId(1), 3);
|
||||
assert!(!predictions.is_empty());
|
||||
assert_eq!(predictions[0], FileId(2));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_recently_played() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let db_path = dir.path().join("patterns.db");
|
||||
let store = PatternStore::new(&db_path, 30).unwrap();
|
||||
let ctx = AccessContext::default();
|
||||
|
||||
store.record(FileId(100), ctx.clone()).unwrap();
|
||||
store.record(FileId(200), ctx.clone()).unwrap();
|
||||
|
||||
let recent = store.recently_played(7).unwrap();
|
||||
assert!(recent.contains(&FileId(100)));
|
||||
assert!(recent.contains(&FileId(200)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_most_played() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let db_path = dir.path().join("patterns.db");
|
||||
let store = PatternStore::new(&db_path, 30).unwrap();
|
||||
let ctx = AccessContext::default();
|
||||
|
||||
for _ in 0..5 {
|
||||
store.record(FileId(1), ctx.clone()).unwrap();
|
||||
}
|
||||
for _ in 0..2 {
|
||||
store.record(FileId(2), ctx.clone()).unwrap();
|
||||
}
|
||||
|
||||
let most = store.most_played(10).unwrap();
|
||||
assert_eq!(most[0], FileId(1));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
use crate::patterns::{AccessContext, PatternStore};
|
||||
use musicfs_cas::ContentFetcher;
|
||||
use musicfs_core::{Event, EventBus, FileId};
|
||||
use parking_lot::Mutex as ParkingMutex;
|
||||
use std::collections::HashSet;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::sync::Semaphore;
|
||||
use tokio::task::JoinHandle;
|
||||
use tracing::{debug, info, warn};
|
||||
|
||||
const DEFAULT_PREFETCH_LOOKAHEAD: usize = 3;
|
||||
const DEFAULT_MAX_CONCURRENT: usize = 2;
|
||||
const DEFAULT_COOLDOWN_MS: u64 = 100;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PrefetchConfig {
|
||||
pub lookahead: usize,
|
||||
pub max_concurrent: usize,
|
||||
pub cooldown: Duration,
|
||||
pub enabled: bool,
|
||||
}
|
||||
|
||||
impl Default for PrefetchConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
lookahead: DEFAULT_PREFETCH_LOOKAHEAD,
|
||||
max_concurrent: DEFAULT_MAX_CONCURRENT,
|
||||
cooldown: Duration::from_millis(DEFAULT_COOLDOWN_MS),
|
||||
enabled: true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct PrefetchEngine {
|
||||
config: PrefetchConfig,
|
||||
fetcher: Arc<ContentFetcher>,
|
||||
in_flight: Arc<ParkingMutex<HashSet<FileId>>>,
|
||||
semaphore: Arc<Semaphore>,
|
||||
running: Arc<AtomicBool>,
|
||||
}
|
||||
|
||||
pub struct PrefetchHandle {
|
||||
handle: JoinHandle<()>,
|
||||
running: Arc<AtomicBool>,
|
||||
}
|
||||
|
||||
impl PrefetchHandle {
|
||||
pub async fn stop(self) {
|
||||
self.running.store(false, Ordering::SeqCst);
|
||||
let _ = self.handle.await;
|
||||
}
|
||||
}
|
||||
|
||||
impl PrefetchEngine {
|
||||
pub fn new(
|
||||
config: PrefetchConfig,
|
||||
_pattern_store: Arc<PatternStore>,
|
||||
fetcher: Arc<ContentFetcher>,
|
||||
) -> Self {
|
||||
let semaphore = Arc::new(Semaphore::new(config.max_concurrent));
|
||||
|
||||
Self {
|
||||
config,
|
||||
fetcher,
|
||||
in_flight: Arc::new(ParkingMutex::new(HashSet::new())),
|
||||
semaphore,
|
||||
running: Arc::new(AtomicBool::new(false)),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn start(
|
||||
self: Arc<Self>,
|
||||
event_bus: Arc<EventBus>,
|
||||
pattern_store: Arc<PatternStore>,
|
||||
) -> PrefetchHandle {
|
||||
self.running.store(true, Ordering::SeqCst);
|
||||
let running = self.running.clone();
|
||||
|
||||
let config = self.config.clone();
|
||||
let fetcher = self.fetcher.clone();
|
||||
let in_flight = self.in_flight.clone();
|
||||
let semaphore = self.semaphore.clone();
|
||||
let running_inner = running.clone();
|
||||
|
||||
let handle = tokio::spawn(async move {
|
||||
let mut rx = event_bus.subscribe();
|
||||
|
||||
while running_inner.load(Ordering::SeqCst) {
|
||||
match tokio::time::timeout(Duration::from_secs(1), rx.recv()).await {
|
||||
Ok(Ok(event)) => {
|
||||
if let Event::FileAccessed { file_id, .. } = event {
|
||||
if config.enabled {
|
||||
let ctx = AccessContext::default();
|
||||
if let Err(e) = pattern_store.record(file_id, ctx) {
|
||||
warn!("Failed to record access pattern: {}", e);
|
||||
continue;
|
||||
}
|
||||
|
||||
let predictions =
|
||||
pattern_store.predict_next(file_id, config.lookahead);
|
||||
|
||||
for predicted_id in predictions {
|
||||
prefetch_file(
|
||||
predicted_id,
|
||||
&fetcher,
|
||||
&in_flight,
|
||||
&semaphore,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
tokio::time::sleep(config.cooldown).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(Err(_)) => break,
|
||||
Err(_) => continue,
|
||||
}
|
||||
}
|
||||
|
||||
info!("Prefetch engine stopped");
|
||||
});
|
||||
|
||||
PrefetchHandle { handle, running }
|
||||
}
|
||||
|
||||
pub fn is_running(&self) -> bool {
|
||||
self.running.load(Ordering::SeqCst)
|
||||
}
|
||||
|
||||
pub fn in_flight_count(&self) -> usize {
|
||||
self.in_flight.lock().len()
|
||||
}
|
||||
|
||||
pub fn update_config(&mut self, config: PrefetchConfig) {
|
||||
self.config = config;
|
||||
}
|
||||
}
|
||||
|
||||
async fn prefetch_file(
|
||||
file_id: FileId,
|
||||
fetcher: &Arc<ContentFetcher>,
|
||||
in_flight: &Arc<ParkingMutex<HashSet<FileId>>>,
|
||||
semaphore: &Arc<Semaphore>,
|
||||
) {
|
||||
{
|
||||
let mut guard = in_flight.lock();
|
||||
if guard.contains(&file_id) {
|
||||
debug!("Skipping prefetch for {:?} - already in flight", file_id);
|
||||
return;
|
||||
}
|
||||
guard.insert(file_id);
|
||||
}
|
||||
|
||||
let permit = match semaphore.clone().try_acquire_owned() {
|
||||
Ok(p) => p,
|
||||
Err(_) => {
|
||||
debug!("Skipping prefetch for {:?} - concurrency limit", file_id);
|
||||
in_flight.lock().remove(&file_id);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
let fetcher = fetcher.clone();
|
||||
let in_flight = in_flight.clone();
|
||||
|
||||
tokio::spawn(async move {
|
||||
debug!("Prefetching file {:?}", file_id);
|
||||
|
||||
match fetcher.ensure_cached(file_id).await {
|
||||
Ok(manifest) => {
|
||||
info!(
|
||||
"Prefetched {:?}: {} chunks, {} bytes",
|
||||
file_id,
|
||||
manifest.chunks.len(),
|
||||
manifest.total_size
|
||||
);
|
||||
}
|
||||
Err(e) => {
|
||||
debug!("Prefetch failed for {:?}: {}", file_id, e);
|
||||
}
|
||||
}
|
||||
|
||||
in_flight.lock().remove(&file_id);
|
||||
drop(permit);
|
||||
});
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_prefetch_config_defaults() {
|
||||
let config = PrefetchConfig::default();
|
||||
assert_eq!(config.lookahead, 3);
|
||||
assert_eq!(config.max_concurrent, 2);
|
||||
assert!(config.enabled);
|
||||
}
|
||||
}
|
||||
@@ -129,6 +129,7 @@ impl ContentFetcher {
|
||||
pub fn emit_access_event(&self, meta: &FileMeta, offset: u64, size: u32) {
|
||||
if let Some(bus) = &self.event_bus {
|
||||
bus.publish(Event::FileAccessed {
|
||||
file_id: meta.id,
|
||||
path: meta.virtual_path.clone(),
|
||||
origin_id: meta.real_path.origin_id.clone(),
|
||||
offset,
|
||||
|
||||
@@ -40,6 +40,7 @@ pub enum Event {
|
||||
path: VirtualPath,
|
||||
},
|
||||
FileAccessed {
|
||||
file_id: FileId,
|
||||
path: VirtualPath,
|
||||
origin_id: OriginId,
|
||||
offset: u64,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
mod prefetch;
|
||||
mod search;
|
||||
|
||||
pub use prefetch::PrefetchOps;
|
||||
pub use search::SearchOps;
|
||||
|
||||
@@ -0,0 +1,293 @@
|
||||
use fuser::{FileAttr, FileType, ReplyAttr, ReplyData, ReplyDirectory, ReplyEntry};
|
||||
use musicfs_cache::{PatternStore, PrefetchConfig, PrefetchEngine};
|
||||
use musicfs_cas::ContentFetcher;
|
||||
use musicfs_core::{EventBus, FileId};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, SystemTime};
|
||||
|
||||
const PREFETCH_DIR_INODE: u64 = 0xFFFF_FFFF_0000_0002;
|
||||
const PREFETCH_STATUS_INODE: u64 = 0xFFFF_FFFF_0000_0003;
|
||||
const PREFETCH_HINTS_BASE: u64 = 0xFFFF_FFFF_2000_0000;
|
||||
|
||||
pub struct PrefetchOps {
|
||||
pattern_store: Arc<PatternStore>,
|
||||
engine: Option<Arc<PrefetchEngine>>,
|
||||
uid: u32,
|
||||
gid: u32,
|
||||
}
|
||||
|
||||
impl PrefetchOps {
|
||||
pub fn new(pattern_store: Arc<PatternStore>, uid: u32, gid: u32) -> Self {
|
||||
Self {
|
||||
pattern_store,
|
||||
engine: None,
|
||||
uid,
|
||||
gid,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_engine(
|
||||
pattern_store: Arc<PatternStore>,
|
||||
fetcher: Arc<ContentFetcher>,
|
||||
config: PrefetchConfig,
|
||||
uid: u32,
|
||||
gid: u32,
|
||||
) -> Self {
|
||||
let engine = Arc::new(PrefetchEngine::new(config, pattern_store.clone(), fetcher));
|
||||
|
||||
Self {
|
||||
pattern_store,
|
||||
engine: Some(engine),
|
||||
uid,
|
||||
gid,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn start_engine(
|
||||
&self,
|
||||
event_bus: Arc<EventBus>,
|
||||
) -> Option<musicfs_cache::PrefetchHandle> {
|
||||
self.engine
|
||||
.as_ref()
|
||||
.map(|e| e.clone().start(event_bus, self.pattern_store.clone()))
|
||||
}
|
||||
|
||||
pub fn is_prefetch_dir_name(name: &str) -> bool {
|
||||
name == ".prefetch"
|
||||
}
|
||||
|
||||
pub fn is_prefetch_inode(inode: u64) -> bool {
|
||||
inode == PREFETCH_DIR_INODE
|
||||
|| inode == PREFETCH_STATUS_INODE
|
||||
|| inode >= PREFETCH_HINTS_BASE
|
||||
}
|
||||
|
||||
pub fn prefetch_dir_inode() -> u64 {
|
||||
PREFETCH_DIR_INODE
|
||||
}
|
||||
|
||||
pub fn lookup_prefetch_dir(&self, reply: ReplyEntry) {
|
||||
let attr = self.dir_attr(PREFETCH_DIR_INODE);
|
||||
reply.entry(&Duration::from_secs(60), &attr, 0);
|
||||
}
|
||||
|
||||
pub fn lookup_status(&self, reply: ReplyEntry) {
|
||||
let status = self.generate_status();
|
||||
let attr = self.file_attr(PREFETCH_STATUS_INODE, status.len() as u64);
|
||||
reply.entry(&Duration::from_secs(1), &attr, 0);
|
||||
}
|
||||
|
||||
pub fn lookup_hint(&self, name: &str, reply: ReplyEntry) {
|
||||
if let Some(inode) = self.hint_name_to_inode(name) {
|
||||
let attr = self.file_attr(inode, 256);
|
||||
reply.entry(&Duration::from_secs(1), &attr, 0);
|
||||
} else {
|
||||
reply.error(libc::ENOENT);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn getattr_prefetch_dir(&self, reply: ReplyAttr) {
|
||||
let attr = self.dir_attr(PREFETCH_DIR_INODE);
|
||||
reply.attr(&Duration::from_secs(60), &attr);
|
||||
}
|
||||
|
||||
pub fn getattr_status(&self, reply: ReplyAttr) {
|
||||
let status = self.generate_status();
|
||||
let attr = self.file_attr(PREFETCH_STATUS_INODE, status.len() as u64);
|
||||
reply.attr(&Duration::from_secs(1), &attr);
|
||||
}
|
||||
|
||||
pub fn getattr_hint(&self, inode: u64, reply: ReplyAttr) {
|
||||
let attr = self.file_attr(inode, 256);
|
||||
reply.attr(&Duration::from_secs(1), &attr);
|
||||
}
|
||||
|
||||
pub fn readdir_prefetch_root(&self, offset: i64, mut reply: ReplyDirectory) {
|
||||
let entries: Vec<(u64, FileType, &str)> = vec![
|
||||
(PREFETCH_DIR_INODE, FileType::Directory, "."),
|
||||
(1, FileType::Directory, ".."),
|
||||
(PREFETCH_STATUS_INODE, FileType::RegularFile, "status"),
|
||||
];
|
||||
|
||||
let recently_played = self.pattern_store.recently_played(7).unwrap_or_default();
|
||||
let predictions: Vec<(u64, FileType, String)> = recently_played
|
||||
.iter()
|
||||
.take(10)
|
||||
.enumerate()
|
||||
.map(|(i, file_id)| {
|
||||
let inode = PREFETCH_HINTS_BASE + i as u64;
|
||||
let name = format!("hint_{:04}", file_id.0);
|
||||
(inode, FileType::RegularFile, name)
|
||||
})
|
||||
.collect();
|
||||
|
||||
for (i, (inode, kind, name)) in entries.iter().enumerate().skip(offset as usize) {
|
||||
if reply.add(*inode, (i + 1) as i64, *kind, *name) {
|
||||
reply.ok();
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
let base_offset = entries.len();
|
||||
for (i, (inode, kind, name)) in predictions.iter().enumerate() {
|
||||
let entry_offset = base_offset + i;
|
||||
if entry_offset < offset as usize {
|
||||
continue;
|
||||
}
|
||||
if reply.add(*inode, (entry_offset + 1) as i64, *kind, name) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
reply.ok();
|
||||
}
|
||||
|
||||
pub fn read_status(&self, offset: i64, size: u32, reply: ReplyData) {
|
||||
let status = self.generate_status();
|
||||
let start = offset as usize;
|
||||
let end = std::cmp::min(start + size as usize, status.len());
|
||||
|
||||
if start >= status.len() {
|
||||
reply.data(&[]);
|
||||
} else {
|
||||
reply.data(&status.as_bytes()[start..end]);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn read_hint(&self, inode: u64, offset: i64, size: u32, reply: ReplyData) {
|
||||
let file_id = self.inode_to_file_id(inode);
|
||||
let predictions = self.pattern_store.predict_next(file_id, 5);
|
||||
|
||||
let content = predictions
|
||||
.iter()
|
||||
.map(|id| format!("{}", id.0))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n");
|
||||
|
||||
let start = offset as usize;
|
||||
let end = std::cmp::min(start + size as usize, content.len());
|
||||
|
||||
if start >= content.len() {
|
||||
reply.data(&[]);
|
||||
} else {
|
||||
reply.data(&content.as_bytes()[start..end]);
|
||||
}
|
||||
}
|
||||
|
||||
fn generate_status(&self) -> String {
|
||||
let engine_status = if let Some(engine) = &self.engine {
|
||||
format!(
|
||||
"running: {}\nin_flight: {}",
|
||||
engine.is_running(),
|
||||
engine.in_flight_count()
|
||||
)
|
||||
} else {
|
||||
"engine: disabled".to_string()
|
||||
};
|
||||
|
||||
let most_played = self
|
||||
.pattern_store
|
||||
.most_played(5)
|
||||
.unwrap_or_default()
|
||||
.iter()
|
||||
.map(|id| format!("{}", id.0))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
|
||||
format!(
|
||||
"MusicFS Prefetch Status\n\
|
||||
=======================\n\
|
||||
{}\n\
|
||||
most_played: [{}]\n",
|
||||
engine_status, most_played
|
||||
)
|
||||
}
|
||||
|
||||
fn hint_name_to_inode(&self, name: &str) -> Option<u64> {
|
||||
if name.starts_with("hint_") {
|
||||
let id_str = name.strip_prefix("hint_")?;
|
||||
let id: i64 = id_str.parse().ok()?;
|
||||
Some(PREFETCH_HINTS_BASE + id as u64)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
fn inode_to_file_id(&self, inode: u64) -> FileId {
|
||||
FileId((inode - PREFETCH_HINTS_BASE) as i64)
|
||||
}
|
||||
|
||||
fn dir_attr(&self, inode: u64) -> FileAttr {
|
||||
FileAttr {
|
||||
ino: inode,
|
||||
size: 0,
|
||||
blocks: 0,
|
||||
atime: SystemTime::UNIX_EPOCH,
|
||||
mtime: SystemTime::UNIX_EPOCH,
|
||||
ctime: SystemTime::UNIX_EPOCH,
|
||||
crtime: SystemTime::UNIX_EPOCH,
|
||||
kind: FileType::Directory,
|
||||
perm: 0o555,
|
||||
nlink: 2,
|
||||
uid: self.uid,
|
||||
gid: self.gid,
|
||||
rdev: 0,
|
||||
blksize: 512,
|
||||
flags: 0,
|
||||
}
|
||||
}
|
||||
|
||||
fn file_attr(&self, inode: u64, size: u64) -> FileAttr {
|
||||
FileAttr {
|
||||
ino: inode,
|
||||
size,
|
||||
blocks: (size + 511) / 512,
|
||||
atime: SystemTime::UNIX_EPOCH,
|
||||
mtime: SystemTime::UNIX_EPOCH,
|
||||
ctime: SystemTime::UNIX_EPOCH,
|
||||
crtime: SystemTime::UNIX_EPOCH,
|
||||
kind: FileType::RegularFile,
|
||||
perm: 0o444,
|
||||
nlink: 1,
|
||||
uid: self.uid,
|
||||
gid: self.gid,
|
||||
rdev: 0,
|
||||
blksize: 512,
|
||||
flags: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
#[test]
|
||||
fn test_prefetch_ops_new() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let pattern_store = Arc::new(PatternStore::new(&dir.path().join("patterns.db"), 30).unwrap());
|
||||
let _ops = PrefetchOps::new(pattern_store, 1000, 1000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_prefetch_inode() {
|
||||
assert!(PrefetchOps::is_prefetch_inode(PREFETCH_DIR_INODE));
|
||||
assert!(PrefetchOps::is_prefetch_inode(PREFETCH_STATUS_INODE));
|
||||
assert!(PrefetchOps::is_prefetch_inode(PREFETCH_HINTS_BASE));
|
||||
assert!(PrefetchOps::is_prefetch_inode(PREFETCH_HINTS_BASE + 100));
|
||||
assert!(!PrefetchOps::is_prefetch_inode(1));
|
||||
assert!(!PrefetchOps::is_prefetch_inode(1000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_hint_name_to_inode() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let pattern_store = Arc::new(PatternStore::new(&dir.path().join("patterns.db"), 30).unwrap());
|
||||
let ops = PrefetchOps::new(pattern_store, 1000, 1000);
|
||||
|
||||
assert_eq!(ops.hint_name_to_inode("hint_0001"), Some(PREFETCH_HINTS_BASE + 1));
|
||||
assert_eq!(ops.hint_name_to_inode("hint_9999"), Some(PREFETCH_HINTS_BASE + 9999));
|
||||
assert_eq!(ops.hint_name_to_inode("invalid"), None);
|
||||
}
|
||||
}
|
||||
@@ -8,3 +8,4 @@ musicfs-core = { path = "../musicfs-core" }
|
||||
symphonia.workspace = true
|
||||
thiserror.workspace = true
|
||||
tracing.workspace = true
|
||||
image.workspace = true
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
use image::ImageFormat;
|
||||
use std::io::Cursor;
|
||||
use symphonia::core::meta::Visual;
|
||||
use tracing::debug;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Artwork {
|
||||
pub art_type: ArtType,
|
||||
pub mime_type: String,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub data: Vec<u8>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum ArtType {
|
||||
Front,
|
||||
Back,
|
||||
Other,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum ArtSize {
|
||||
Thumbnail,
|
||||
Medium,
|
||||
Full,
|
||||
}
|
||||
|
||||
impl ArtSize {
|
||||
pub fn max_dimension(&self) -> Option<u32> {
|
||||
match self {
|
||||
ArtSize::Thumbnail => Some(150),
|
||||
ArtSize::Medium => Some(300),
|
||||
ArtSize::Full => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct ArtworkExtractor;
|
||||
|
||||
impl ArtworkExtractor {
|
||||
pub fn extract_from_visual(visual: &Visual) -> Option<Artwork> {
|
||||
let data = visual.data.to_vec();
|
||||
|
||||
let img = image::load_from_memory(&data).ok()?;
|
||||
|
||||
let art_type = match visual.usage {
|
||||
Some(symphonia::core::meta::StandardVisualKey::FrontCover) => ArtType::Front,
|
||||
Some(symphonia::core::meta::StandardVisualKey::BackCover) => ArtType::Back,
|
||||
_ => ArtType::Other,
|
||||
};
|
||||
|
||||
let mime_type = if visual.media_type.is_empty() {
|
||||
"image/jpeg".to_string()
|
||||
} else {
|
||||
visual.media_type.clone()
|
||||
};
|
||||
|
||||
Some(Artwork {
|
||||
art_type,
|
||||
mime_type,
|
||||
width: img.width(),
|
||||
height: img.height(),
|
||||
data,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn resize(artwork: &Artwork, size: ArtSize) -> Option<Artwork> {
|
||||
let max_dim = size.max_dimension()?;
|
||||
|
||||
if artwork.width <= max_dim && artwork.height <= max_dim {
|
||||
return Some(artwork.clone());
|
||||
}
|
||||
|
||||
let img = image::load_from_memory(&artwork.data).ok()?;
|
||||
let resized = img.thumbnail(max_dim, max_dim);
|
||||
|
||||
let mut output = Vec::new();
|
||||
let mut cursor = Cursor::new(&mut output);
|
||||
resized.write_to(&mut cursor, ImageFormat::Jpeg).ok()?;
|
||||
|
||||
debug!(
|
||||
"Resized artwork from {}x{} to {}x{}",
|
||||
artwork.width,
|
||||
artwork.height,
|
||||
resized.width(),
|
||||
resized.height()
|
||||
);
|
||||
|
||||
Some(Artwork {
|
||||
art_type: artwork.art_type,
|
||||
mime_type: "image/jpeg".to_string(),
|
||||
width: resized.width(),
|
||||
height: resized.height(),
|
||||
data: output,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_art_size_dimensions() {
|
||||
assert_eq!(ArtSize::Thumbnail.max_dimension(), Some(150));
|
||||
assert_eq!(ArtSize::Medium.max_dimension(), Some(300));
|
||||
assert_eq!(ArtSize::Full.max_dimension(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_art_type_equality() {
|
||||
assert_eq!(ArtType::Front, ArtType::Front);
|
||||
assert_ne!(ArtType::Front, ArtType::Back);
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,5 @@
|
||||
pub mod artwork;
|
||||
mod parser;
|
||||
|
||||
pub use artwork::{ArtSize, ArtType, Artwork, ArtworkExtractor};
|
||||
pub use parser::MetadataParser;
|
||||
|
||||
@@ -12,6 +12,9 @@ parking_lot.workspace = true
|
||||
tokio = { workspace = true, features = ["sync", "time"] }
|
||||
tracing.workspace = true
|
||||
thiserror.workspace = true
|
||||
rusqlite.workspace = true
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile.workspace = true
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
use parking_lot::Mutex;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::Path;
|
||||
use std::time::{Duration, SystemTime};
|
||||
use tracing::warn;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct SmartCollection {
|
||||
pub id: i64,
|
||||
pub name: String,
|
||||
pub query: CollectionQuery,
|
||||
pub created_at: SystemTime,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(tag = "type")]
|
||||
pub enum CollectionQuery {
|
||||
Match {
|
||||
field: String,
|
||||
pattern: String,
|
||||
},
|
||||
DateRange {
|
||||
field: String,
|
||||
start: i32,
|
||||
end: i32,
|
||||
},
|
||||
RecentlyAdded {
|
||||
days: u32,
|
||||
},
|
||||
RecentlyPlayed {
|
||||
days: u32,
|
||||
},
|
||||
MostPlayed {
|
||||
limit: u32,
|
||||
},
|
||||
Genre {
|
||||
genre: String,
|
||||
},
|
||||
Compound {
|
||||
op: BoolOp,
|
||||
children: Vec<CollectionQuery>,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
|
||||
pub enum BoolOp {
|
||||
And,
|
||||
Or,
|
||||
}
|
||||
|
||||
impl CollectionQuery {
|
||||
pub fn to_tantivy_query(&self) -> String {
|
||||
match self {
|
||||
CollectionQuery::Match { field, pattern } => {
|
||||
format!("{}:{}", field, pattern)
|
||||
}
|
||||
CollectionQuery::DateRange { field, start, end } => {
|
||||
format!("{}:[{} TO {}]", field, start, end)
|
||||
}
|
||||
CollectionQuery::Genre { genre } => {
|
||||
format!("genre:{}", genre)
|
||||
}
|
||||
CollectionQuery::Compound { op, children } => {
|
||||
let sep = match op {
|
||||
BoolOp::And => " AND ",
|
||||
BoolOp::Or => " OR ",
|
||||
};
|
||||
let parts: Vec<_> = children
|
||||
.iter()
|
||||
.map(|c| format!("({})", c.to_tantivy_query()))
|
||||
.collect();
|
||||
parts.join(sep)
|
||||
}
|
||||
_ => String::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_dynamic(&self) -> bool {
|
||||
matches!(
|
||||
self,
|
||||
CollectionQuery::RecentlyAdded { .. }
|
||||
| CollectionQuery::RecentlyPlayed { .. }
|
||||
| CollectionQuery::MostPlayed { .. }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct CollectionStore {
|
||||
db: Mutex<rusqlite::Connection>,
|
||||
}
|
||||
|
||||
impl CollectionStore {
|
||||
pub fn new(db_path: &Path) -> Result<Self, CollectionError> {
|
||||
let db = rusqlite::Connection::open(db_path)?;
|
||||
|
||||
db.execute(
|
||||
"CREATE TABLE IF NOT EXISTS collections (
|
||||
id INTEGER PRIMARY KEY,
|
||||
name TEXT UNIQUE NOT NULL,
|
||||
query_json TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL
|
||||
)",
|
||||
[],
|
||||
)?;
|
||||
|
||||
Ok(Self { db: Mutex::new(db) })
|
||||
}
|
||||
|
||||
pub fn create(
|
||||
&self,
|
||||
name: &str,
|
||||
query: CollectionQuery,
|
||||
) -> Result<SmartCollection, CollectionError> {
|
||||
let query_json = serde_json::to_string(&query)?;
|
||||
let now = SystemTime::now()
|
||||
.duration_since(SystemTime::UNIX_EPOCH)
|
||||
.unwrap()
|
||||
.as_secs() as i64;
|
||||
|
||||
let db = self.db.lock();
|
||||
db.execute(
|
||||
"INSERT INTO collections (name, query_json, created_at) VALUES (?1, ?2, ?3)",
|
||||
rusqlite::params![name, query_json, now],
|
||||
)?;
|
||||
|
||||
let id = db.last_insert_rowid();
|
||||
|
||||
Ok(SmartCollection {
|
||||
id,
|
||||
name: name.to_string(),
|
||||
query,
|
||||
created_at: SystemTime::UNIX_EPOCH + Duration::from_secs(now as u64),
|
||||
})
|
||||
}
|
||||
|
||||
pub fn list(&self) -> Result<Vec<SmartCollection>, CollectionError> {
|
||||
let db = self.db.lock();
|
||||
let mut stmt = db.prepare("SELECT id, name, query_json, created_at FROM collections")?;
|
||||
|
||||
let collections = stmt.query_map([], |row| {
|
||||
let query_json: String = row.get(2)?;
|
||||
let created_secs: i64 = row.get(3)?;
|
||||
|
||||
let query = match serde_json::from_str(&query_json) {
|
||||
Ok(q) => q,
|
||||
Err(e) => {
|
||||
warn!("Failed to parse collection query JSON: {}", e);
|
||||
CollectionQuery::Match {
|
||||
field: "title".to_string(),
|
||||
pattern: "*".to_string(),
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
Ok(SmartCollection {
|
||||
id: row.get(0)?,
|
||||
name: row.get(1)?,
|
||||
query,
|
||||
created_at: SystemTime::UNIX_EPOCH + Duration::from_secs(created_secs as u64),
|
||||
})
|
||||
})?;
|
||||
|
||||
collections
|
||||
.collect::<Result<Vec<_>, _>>()
|
||||
.map_err(CollectionError::from)
|
||||
}
|
||||
|
||||
pub fn get(&self, name: &str) -> Result<Option<SmartCollection>, CollectionError> {
|
||||
let db = self.db.lock();
|
||||
let mut stmt =
|
||||
db.prepare("SELECT id, name, query_json, created_at FROM collections WHERE name = ?1")?;
|
||||
|
||||
let result = stmt
|
||||
.query_row([name], |row| {
|
||||
let query_json: String = row.get(2)?;
|
||||
let created_secs: i64 = row.get(3)?;
|
||||
|
||||
let query = match serde_json::from_str(&query_json) {
|
||||
Ok(q) => q,
|
||||
Err(e) => {
|
||||
warn!("Failed to parse collection query JSON: {}", e);
|
||||
CollectionQuery::Match {
|
||||
field: "title".to_string(),
|
||||
pattern: "*".to_string(),
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
Ok(SmartCollection {
|
||||
id: row.get(0)?,
|
||||
name: row.get(1)?,
|
||||
query,
|
||||
created_at: SystemTime::UNIX_EPOCH + Duration::from_secs(created_secs as u64),
|
||||
})
|
||||
})
|
||||
.ok();
|
||||
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
pub fn delete(&self, name: &str) -> Result<(), CollectionError> {
|
||||
let db = self.db.lock();
|
||||
db.execute("DELETE FROM collections WHERE name = ?1", [name])?;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
pub fn builtin_collections() -> Vec<SmartCollection> {
|
||||
vec![
|
||||
SmartCollection {
|
||||
id: -1,
|
||||
name: "Recently Added".to_string(),
|
||||
query: CollectionQuery::RecentlyAdded { days: 30 },
|
||||
created_at: SystemTime::UNIX_EPOCH,
|
||||
},
|
||||
SmartCollection {
|
||||
id: -2,
|
||||
name: "80s Music".to_string(),
|
||||
query: CollectionQuery::DateRange {
|
||||
field: "year".to_string(),
|
||||
start: 1980,
|
||||
end: 1989,
|
||||
},
|
||||
created_at: SystemTime::UNIX_EPOCH,
|
||||
},
|
||||
SmartCollection {
|
||||
id: -3,
|
||||
name: "90s Music".to_string(),
|
||||
query: CollectionQuery::DateRange {
|
||||
field: "year".to_string(),
|
||||
start: 1990,
|
||||
end: 1999,
|
||||
},
|
||||
created_at: SystemTime::UNIX_EPOCH,
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum CollectionError {
|
||||
#[error("database error: {0}")]
|
||||
Database(#[from] rusqlite::Error),
|
||||
|
||||
#[error("serialization error: {0}")]
|
||||
Serialization(#[from] serde_json::Error),
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tempfile::TempDir;
|
||||
|
||||
#[test]
|
||||
fn test_collection_crud() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let db_path = dir.path().join("collections.db");
|
||||
let store = CollectionStore::new(&db_path).unwrap();
|
||||
|
||||
let collection = store
|
||||
.create("Jazz", CollectionQuery::Genre { genre: "Jazz".to_string() })
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(collection.name, "Jazz");
|
||||
|
||||
let collections = store.list().unwrap();
|
||||
assert_eq!(collections.len(), 1);
|
||||
|
||||
store.delete("Jazz").unwrap();
|
||||
let collections = store.list().unwrap();
|
||||
assert_eq!(collections.len(), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_compound_query() {
|
||||
let query = CollectionQuery::Compound {
|
||||
op: BoolOp::And,
|
||||
children: vec![
|
||||
CollectionQuery::Genre { genre: "Metal".to_string() },
|
||||
CollectionQuery::DateRange {
|
||||
field: "year".to_string(),
|
||||
start: 1980,
|
||||
end: 1989,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
let tantivy_query = query.to_tantivy_query();
|
||||
assert!(tantivy_query.contains("genre:Metal"));
|
||||
assert!(tantivy_query.contains("year:[1980 TO 1989]"));
|
||||
assert!(tantivy_query.contains(" AND "));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_builtin_collections() {
|
||||
let builtins = builtin_collections();
|
||||
assert_eq!(builtins.len(), 3);
|
||||
assert!(builtins.iter().any(|c| c.name == "Recently Added"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_dynamic_query_detection() {
|
||||
assert!(CollectionQuery::RecentlyAdded { days: 30 }.is_dynamic());
|
||||
assert!(CollectionQuery::RecentlyPlayed { days: 7 }.is_dynamic());
|
||||
assert!(CollectionQuery::MostPlayed { limit: 100 }.is_dynamic());
|
||||
assert!(!CollectionQuery::Genre { genre: "Rock".to_string() }.is_dynamic());
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,12 @@
|
||||
mod collections;
|
||||
mod index;
|
||||
mod indexer;
|
||||
mod query;
|
||||
|
||||
pub use collections::{
|
||||
builtin_collections, BoolOp, CollectionError, CollectionQuery, CollectionStore,
|
||||
SmartCollection,
|
||||
};
|
||||
pub use index::{SearchError, SearchHit, SearchIndex};
|
||||
pub use indexer::{Indexer, IndexerHandle, MetadataLookup};
|
||||
pub use query::SearchQueryBuilder;
|
||||
|
||||
Reference in New Issue
Block a user