Compare commits

...

17 Commits

Author SHA1 Message Date
Alexander 18024dbc62 fix(cli): wire OverlayReader into mount command
The metadata overlay feature was implemented but not connected to the
CLI daemon. Files were being served with original metadata instead of
synthesized headers from the database.

- Import OverlayReader from musicfs-cache
- Create OverlayReader with db, format_registry, and reader
- Call .with_overlay() on MusicFs builder

Tested: ffprobe now shows modified metadata from database updates.
2026-05-17 18:23:15 +02:00
Alexander b0c41e3fa0 feat(cli): add metadata subcommands for overlay management
- Add 6 subcommands: get, set, clear, diff, import, export
- Connect to gRPC MetadataService
- Support JSON and CSV formats
- All subcommands functional, help output correct
2026-05-17 17:59:35 +02:00
Alexander 1a7f70ae1c feat(grpc): implement MetadataService handlers
- Implement all 5 RPCs (Get, Update, Clear, Batch, Import)
- Add MetadataServiceImpl with database integration
- Add 10 comprehensive unit tests
- All 19 tests pass, full workspace compiles
2026-05-17 17:53:44 +02:00
Alexander 391f556286 feat(grpc): add MetadataService proto definition
- Add MetadataService with 5 RPCs (Get, Update, Clear, Batch, Import)
- Add 11 message types for requests/responses
- Use optional fields and map for custom_tags
- Proto codegen successful, all tests pass
2026-05-17 17:46:53 +02:00
Alexander 9623644263 feat(fuse): integrate OverlayReader in read path
- Update read() to use OverlayReader when available
- Map OverlayError to libc error codes
- Maintain 30s timeout and backward compatibility
- Fallback to FileReader for non-overlay files
- All tests pass, full workspace compiles
2026-05-17 17:44:29 +02:00
Alexander 487b119935 feat(fuse): return virtual size in getattr for overlay files
- Add overlay_reader field to MusicFs struct
- Add with_overlay() builder method
- Update getattr() to call estimate_virtual_size()
- Graceful fallback to original size
- All tests pass, backward compatible
2026-05-17 17:41:50 +02:00
Alexander c826bcf35f feat(cache): implement OverlayReader for header/audio splice
- Implement three-region splice logic (header, audio, boundary)
- Add passthrough mode for files without format_layout
- Add estimate_virtual_size() for getattr
- Create OverlayError enum with proper error conversions
- Add 8 comprehensive unit tests
- All tests pass, LSP diagnostics clean
2026-05-17 17:38:03 +02:00
Alexander ebf4044a01 feat(sync): populate format_layout during origin scan
- Create FormatHandlerRegistry in CLI initialization
- Register Id3v2Handler and FlacHandler
- Add analyze_format_layout() helper to read file headers
- Update scan functions to call handler.analyze()
- Use upsert_file_with_layout() when format_layout available
- Graceful degradation for unsupported formats
- Full workspace compiles successfully
2026-05-17 17:31:34 +02:00
Alexander 4f4a4169f8 feat(cache): update database layer for expanded metadata
- Update upsert_file() to include all 26 new AudioMeta fields
- Update get_file_by_virtual_path() to read all new columns
- Add get_file_metadata_row() for overlay synthesis
- Add update_metadata() for partial metadata updates
- Add clear_overlay() to reset metadata to NULL
- Handle format_layout BLOB with msgpack serialization
- Handle custom_tags JSON with serde_json
- Add 8 comprehensive unit tests
- All 92 tests pass, LSP diagnostics clean
2026-05-17 17:27:24 +02:00
Alexander 84bbd8f630 feat(cache): implement FlacHandler for FLAC metadata synthesis
- Implement all 8 FormatHandler trait methods
- Parse FLAC metadata blocks and extract STREAMINFO
- Preserve original STREAMINFO in synthesized headers
- Map all 36 AudioMeta fields to Vorbis comment tags
- Binary serialization of Vorbis comments with little-endian lengths
- Add 16 comprehensive unit tests including STREAMINFO preservation
- All tests pass, LSP diagnostics clean
2026-05-17 17:21:11 +02:00
Alexander 128a6e079e feat(cache): implement Id3v2Handler for MP3 metadata synthesis
- Implement all 8 FormatHandler trait methods
- Use lofty 0.24 for ID3v2.4 tag creation/parsing
- Map all 36 AudioMeta fields to ID3v2 frames
- Handle ID3v2 header parsing for audio_start
- Detect ID3v1 tags at EOF for audio_end
- Add 13 comprehensive unit tests
- Fix test-utils AudioMeta construction with ..Default::default()
- All tests pass, LSP diagnostics clean
2026-05-17 17:14:23 +02:00
Alexander 693b4f067b chore: add .sisyphus/ to gitignore 2026-05-17 15:44:31 +02:00
Alexander 66cd4e945c feat(fuse): implement rm with virtual .trash/ directory
- Add trashed/original_path/trashed_at columns to files table
- Implement FUSE unlink: moves files to /.trash/ preserving path structure
- Implement FUSE rmdir: removes empty directories
- Add trash CLI commands: list, restore, empty
- Add SIGHUP handler for CLI-triggered restore
- Fix upsert_file returning 0 on UPDATE (query actual ID)
- Auto-clear trashed flag when moving files out of /.trash/
2026-05-17 15:44:31 +02:00
Alexander 9d74f1a7a3 feat(fuse): implement mkdir and mv with persistence
Add mkdir and mv (rename) FUSE operations to the virtual filesystem:

- mkdir: Create directories that persist across remounts via SQLite
- mv: Move/rename files and directories with database persistence

Changes:
- Add directories table to schema for user-created empty dirs
- Add tree operations: mkdir, rename_file, rename_directory
- Add DB methods for path updates and directory CRUD
- Remove MountOption::RO to allow write syscalls
- Load stored virtual_path from DB instead of regenerating
- Restore user directories on mount from directories table
- Upsert files to DB during origin scan

POSIX compliant: mv fails with ENOENT if parent doesn't exist
(use mkdir first, shell handles -p flag and brace expansion)
2026-05-17 15:44:27 +02:00
Alexander 6e20ffe939 Make mount point optional when config file provides it
- CLI mountpoint argument is now Option<PathBuf>
- Falls back to config.mount_point when --config is provided
- CLI mountpoint still overrides config if both are given
- Expanded config.example.toml with all available options
2026-05-17 13:55:41 +02:00
Alexander daffd518d1 Update flake 2026-05-17 13:44:20 +02:00
Alexander a705d4d3b9 Add opencode 2026-05-17 13:43:12 +02:00
31 changed files with 8214 additions and 93 deletions
+3
View File
@@ -47,3 +47,6 @@ rustc-ice-*.txt
# and can be added to the global gitignore or merged into this file. For a more nuclear # and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder. # option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/ #.idea/
dev/
.sisyphus/
Generated
+72
View File
@@ -616,6 +616,27 @@ dependencies = [
"typenum", "typenum",
] ]
[[package]]
name = "csv"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52cd9d68cf7efc6ddfaaee42e7288d3a99d613d4b50f76ce9827ae0c6e14f938"
dependencies = [
"csv-core",
"itoa",
"ryu",
"serde_core",
]
[[package]]
name = "csv-core"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "704a3c26996a80471189265814dbc2c257598b96b8a7feae2d31ace646bb9782"
dependencies = [
"memchr",
]
[[package]] [[package]]
name = "dashmap" name = "dashmap"
version = "5.5.3" version = "5.5.3"
@@ -629,6 +650,12 @@ dependencies = [
"parking_lot_core 0.9.12", "parking_lot_core 0.9.12",
] ]
[[package]]
name = "data-encoding"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4ae5f15dda3c708c0ade84bfee31ccab44a3da4f88015ed22f63732abe300c8"
[[package]] [[package]]
name = "debugid" name = "debugid"
version = "0.8.0" version = "0.8.0"
@@ -1691,6 +1718,32 @@ dependencies = [
"scopeguard", "scopeguard",
] ]
[[package]]
name = "lofty"
version = "0.24.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dec4feeff6c7d75093278133a06e827d7af6d2bfe20b0f331f9d10338a5ec7ca"
dependencies = [
"byteorder",
"data-encoding",
"flate2",
"lofty_attr",
"log",
"ogg_pager",
"paste",
]
[[package]]
name = "lofty_attr"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "458ace39169e4b83c4f77ae3d42d5d1d11c422feef590219a97c973d3b524557"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]] [[package]]
name = "log" name = "log"
version = "0.4.29" version = "0.4.29"
@@ -1883,8 +1936,10 @@ checksum = "2195bf6aa996a481483b29d62a7663eed3fe39600c460e323f8ff41e90bdd89b"
name = "musicfs-cache" name = "musicfs-cache"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"bytes",
"chrono", "chrono",
"image", "image",
"lofty",
"musicfs-cas", "musicfs-cas",
"musicfs-core", "musicfs-core",
"musicfs-metadata", "musicfs-metadata",
@@ -1892,6 +1947,7 @@ dependencies = [
"rmp-serde", "rmp-serde",
"rusqlite", "rusqlite",
"serde", "serde",
"serde_json",
"sled", "sled",
"tempfile", "tempfile",
"thiserror 1.0.69", "thiserror 1.0.69",
@@ -1934,13 +1990,18 @@ dependencies = [
"musicfs-cas", "musicfs-cas",
"musicfs-core", "musicfs-core",
"musicfs-fuse", "musicfs-fuse",
"musicfs-grpc",
"musicfs-metadata", "musicfs-metadata",
"musicfs-origins", "musicfs-origins",
"parking_lot 0.12.5", "parking_lot 0.12.5",
"sd-notify", "sd-notify",
"serde",
"serde_json",
"tokio", "tokio",
"tokio-stream",
"tokio-util 0.7.18", "tokio-util 0.7.18",
"toml", "toml",
"tonic",
"tracing", "tracing",
"tracing-appender", "tracing-appender",
"tracing-journald", "tracing-journald",
@@ -1985,8 +2046,10 @@ name = "musicfs-grpc"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"chrono", "chrono",
"csv",
"hex", "hex",
"hmac", "hmac",
"musicfs-cache",
"musicfs-core", "musicfs-core",
"musicfs-search", "musicfs-search",
"prost", "prost",
@@ -2256,6 +2319,15 @@ dependencies = [
"memchr", "memchr",
] ]
[[package]]
name = "ogg_pager"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d36b1d6964c3ac92b7aea701057e02b6b91143d70d83b20abf75a231a3c0216"
dependencies = [
"byteorder",
]
[[package]] [[package]]
name = "once_cell" name = "once_cell"
version = "1.21.4" version = "1.21.4"
+84 -2
View File
@@ -1,25 +1,107 @@
# MusicFS Configuration
# Copy to /etc/musicfs/config.toml or ~/.config/musicfs/config.toml
# Required: where to mount the virtual filesystem
mount_point = "/mnt/music" mount_point = "/mnt/music"
# Required: directory for cache data (CAS chunks, metadata, search index)
cache_dir = "/var/cache/musicfs" cache_dir = "/var/cache/musicfs"
# ------------------------------------------------------------------------------
# Origins - music sources (at least one required)
# Supported types: local, nfs, smb, s3, sftp
# Lower priority number = preferred source for failover
# ------------------------------------------------------------------------------
[[origins]] [[origins]]
id = "local-storage" id = "local-music"
origin_type = "local" origin_type = "local"
priority = 1 priority = 1
enabled = true enabled = true
path = "/path/to/local/music" path = "/home/user/Music"
[[origins]]
id = "nas-nfs"
origin_type = "nfs"
priority = 2
enabled = true
path = "/mnt/nas/music"
[[origins]]
id = "nas-smb"
origin_type = "smb"
priority = 3
enabled = false
path = "/mnt/smb/music"
[[origins]]
id = "cloud-backup"
origin_type = "s3"
priority = 10
enabled = false
bucket = "my-music-backup"
region = "us-east-1"
[[origins]]
id = "remote-server"
origin_type = "sftp"
priority = 10
enabled = false
host = "music.example.com"
port = 22
user = "musicfs"
path = "/srv/music"
# ------------------------------------------------------------------------------
# Cache settings
# ------------------------------------------------------------------------------
[cache] [cache]
# In-memory metadata cache size (artist/album/track info)
metadata_cache_mb = 100 metadata_cache_mb = 100
# On-disk content cache size (audio chunks)
content_cache_gb = 10 content_cache_gb = 10
# ------------------------------------------------------------------------------
# Health monitoring for origin failover
# ------------------------------------------------------------------------------
[health] [health]
# How often to check origin health
check_interval_secs = 30 check_interval_secs = 30
# Timeout for health check probes
timeout_ms = 5000 timeout_ms = 5000
# Consecutive failures before marking origin unhealthy
unhealthy_threshold = 3 unhealthy_threshold = 3
# Per-origin type thresholds (overrides unhealthy_threshold)
[health.per_origin_thresholds]
local = 1
nfs = 3
smb = 3
s3 = 3
sftp = 3
# ------------------------------------------------------------------------------
# Logging
# ------------------------------------------------------------------------------
[logging] [logging]
# Directory for log files
log_dir = "/var/log/musicfs" log_dir = "/var/log/musicfs"
# Output logs as JSON (for log aggregators)
json_output = false json_output = false
# Send logs to systemd journal
journald = true journald = true
# Log level filter (tracing format)
# Examples: "info", "debug", "musicfs=debug,warn", "musicfs_fuse=trace"
level = "musicfs=info,warn" level = "musicfs=info,warn"
# Trace sampling rate for performance tracing (0.0 to 1.0)
trace_sample_rate = 1.0 trace_sample_rate = 1.0
+3
View File
@@ -7,6 +7,7 @@ edition.workspace = true
musicfs-core = { path = "../musicfs-core" } musicfs-core = { path = "../musicfs-core" }
musicfs-cas = { path = "../musicfs-cas" } musicfs-cas = { path = "../musicfs-cas" }
musicfs-metadata = { path = "../musicfs-metadata" } musicfs-metadata = { path = "../musicfs-metadata" }
bytes.workspace = true
rusqlite = { workspace = true, features = ["bundled"] } rusqlite = { workspace = true, features = ["bundled"] }
sled.workspace = true sled.workspace = true
tokio.workspace = true tokio.workspace = true
@@ -14,7 +15,9 @@ tracing.workspace = true
thiserror.workspace = true thiserror.workspace = true
serde.workspace = true serde.workspace = true
rmp-serde.workspace = true rmp-serde.workspace = true
serde_json.workspace = true
image.workspace = true image.workspace = true
lofty = "0.24"
parking_lot.workspace = true parking_lot.workspace = true
chrono.workspace = true chrono.workspace = true
File diff suppressed because it is too large Load Diff
+103
View File
@@ -0,0 +1,103 @@
use crate::FormatLayout;
use musicfs_core::AudioMeta;
use std::collections::HashMap;
use std::sync::Arc;
/// Error types for format handling operations
#[derive(Debug, thiserror::Error)]
pub enum FormatError {
#[error("Unsupported format")]
UnsupportedFormat,
#[error("Invalid data: {0}")]
InvalidData(String),
#[error("Synthesis failed: {0}")]
SynthesisFailed(String),
}
/// Trait for format-specific metadata handling.
///
/// Implementations handle:
/// 1. Analyzing original files to find audio boundaries
/// 2. Synthesizing new headers from database metadata
pub trait FormatHandler: Send + Sync + 'static {
/// Unique identifier for this handler
fn id(&self) -> &'static str;
/// Human-readable name
fn name(&self) -> &'static str;
/// File extensions this handler supports
fn extensions(&self) -> &[&'static str];
/// MIME types this handler supports
fn mime_types(&self) -> &[&'static str];
/// Analyze file bytes to determine audio layout
fn analyze(
&self,
data: &[u8],
file_size: u64,
) -> std::result::Result<FormatLayout, FormatError>;
/// Synthesize header bytes from metadata. Called on every read().
fn synthesize(
&self,
metadata: &AudioMeta,
layout: &FormatLayout,
) -> std::result::Result<Vec<u8>, FormatError>;
/// Extract metadata from header bytes (for initial ingest)
fn extract(&self, data: &[u8]) -> std::result::Result<AudioMeta, FormatError>;
/// Estimate header size without full synthesis (for getattr)
fn estimate_header_size(&self, _metadata: &AudioMeta) -> usize {
10 * 1024 // 10KB default
}
}
/// Registry for format handlers
pub struct FormatHandlerRegistry {
handlers: HashMap<String, Arc<dyn FormatHandler>>,
extension_map: HashMap<String, String>,
}
impl FormatHandlerRegistry {
/// Create empty registry
pub fn new() -> Self {
Self {
handlers: HashMap::new(),
extension_map: HashMap::new(),
}
}
/// Register a format handler
pub fn register(&mut self, handler: Arc<dyn FormatHandler>) {
let id = handler.id().to_string();
// Map extensions to handler ID
for ext in handler.extensions() {
self.extension_map.insert(ext.to_string(), id.clone());
}
self.handlers.insert(id, handler);
}
/// Get handler by file extension
pub fn get_by_extension(&self, ext: &str) -> Option<Arc<dyn FormatHandler>> {
let id = self.extension_map.get(ext)?;
self.handlers.get(id).cloned()
}
/// Get handler by format ID
pub fn get_by_format(&self, format: &str) -> Option<Arc<dyn FormatHandler>> {
self.handlers.get(format).cloned()
}
}
impl Default for FormatHandlerRegistry {
fn default() -> Self {
Self::new()
}
}
+22
View File
@@ -0,0 +1,22 @@
use musicfs_core::AudioFormat;
use serde::{Deserialize, Serialize};
/// Describes the byte layout of an audio file for overlay splicing.
///
/// This struct tracks where the audio data begins and ends in the origin file,
/// allowing the OverlayReader to splice synthetic headers with original audio.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FormatLayout {
/// Byte offset where audio data begins in the origin file
pub audio_start: u64,
/// Byte offset where audio data ends in the origin file
pub audio_end: u64,
/// Audio format (from musicfs-core)
pub format: AudioFormat,
/// Format-specific data (e.g., FLAC STREAMINFO block, MP4 stco offsets)
/// Stored as raw bytes, interpreted by format handlers
pub format_data: Option<Vec<u8>>,
}
+886
View File
@@ -0,0 +1,886 @@
//! FLAC format handler for metadata synthesis.
//!
//! FLAC files use Vorbis comments for metadata. The file structure is:
//! - "fLaC" marker (4 bytes)
//! - STREAMINFO block (mandatory, 38 bytes total: 4 header + 34 data)
//! - Optional metadata blocks (VORBIS_COMMENT, PICTURE, PADDING, etc.)
//! - Audio frames
//!
//! CRITICAL: STREAMINFO must be preserved from the original file as it contains
//! MD5 checksum, sample count, and audio properties that must match the audio data.
use crate::{FormatError, FormatHandler, FormatLayout};
use lofty::config::ParseOptions;
use lofty::file::AudioFile;
use lofty::flac::FlacFile;
use lofty::ogg::VorbisComments;
use lofty::tag::Accessor;
use musicfs_core::{AudioFormat, AudioMeta};
use std::borrow::Cow;
use std::io::Cursor;
/// FLAC stream marker: "fLaC" in ASCII
const FLAC_MARKER: &[u8; 4] = b"fLaC";
/// FLAC metadata block types
const BLOCK_TYPE_STREAMINFO: u8 = 0;
const BLOCK_TYPE_VORBIS_COMMENT: u8 = 4;
/// STREAMINFO block data size (always 34 bytes)
const STREAMINFO_DATA_SIZE: usize = 34;
/// Metadata block header size (1 byte type/flags + 3 bytes length)
const BLOCK_HEADER_SIZE: usize = 4;
/// Full STREAMINFO block size (header + data)
const STREAMINFO_BLOCK_SIZE: usize = BLOCK_HEADER_SIZE + STREAMINFO_DATA_SIZE;
pub struct FlacHandler;
impl FlacHandler {
pub fn new() -> Self {
Self
}
/// Parse FLAC metadata block header.
/// Returns (is_last, block_type, block_size).
fn parse_block_header(data: &[u8]) -> Option<(bool, u8, usize)> {
if data.len() < BLOCK_HEADER_SIZE {
return None;
}
let is_last = (data[0] & 0x80) != 0;
let block_type = data[0] & 0x7F;
let block_size =
((data[1] as usize) << 16) | ((data[2] as usize) << 8) | (data[3] as usize);
Some((is_last, block_type, block_size))
}
/// Write a metadata block header.
fn write_block_header(is_last: bool, block_type: u8, size: usize) -> [u8; 4] {
let type_byte = if is_last {
block_type | 0x80
} else {
block_type
};
[
type_byte,
((size >> 16) & 0xFF) as u8,
((size >> 8) & 0xFF) as u8,
(size & 0xFF) as u8,
]
}
/// Build Vorbis comments from AudioMeta.
fn build_vorbis_comments(metadata: &AudioMeta) -> VorbisComments {
let mut tag = VorbisComments::default();
// Basic fields (using Accessor trait)
if let Some(ref title) = metadata.title {
tag.set_title(title.clone());
}
if let Some(ref artist) = metadata.artist {
tag.set_artist(artist.clone());
}
if let Some(ref album) = metadata.album {
tag.set_album(album.clone());
}
if let Some(ref genre) = metadata.genre {
tag.set_genre(genre.clone());
}
// Album artist
if let Some(ref album_artist) = metadata.album_artist {
tag.insert("ALBUMARTIST".to_string(), album_artist.clone());
}
// Year/Date
if let Some(ref date) = metadata.date {
tag.insert("DATE".to_string(), date.clone());
} else if let Some(year) = metadata.year {
tag.insert("DATE".to_string(), year.to_string());
}
// Track/Disc numbers
if let Some(track) = metadata.track {
tag.insert("TRACKNUMBER".to_string(), track.to_string());
}
if let Some(track_total) = metadata.track_total {
tag.insert("TRACKTOTAL".to_string(), track_total.to_string());
}
if let Some(disc) = metadata.disc {
tag.insert("DISCNUMBER".to_string(), disc.to_string());
}
if let Some(disc_total) = metadata.disc_total {
tag.insert("DISCTOTAL".to_string(), disc_total.to_string());
}
// Extended metadata
if let Some(ref composer) = metadata.composer {
tag.insert("COMPOSER".to_string(), composer.clone());
}
if let Some(ref comment) = metadata.comment {
tag.insert("COMMENT".to_string(), comment.clone());
}
if let Some(ref lyrics) = metadata.lyrics {
tag.insert("LYRICS".to_string(), lyrics.clone());
}
if let Some(ref copyright) = metadata.copyright {
tag.insert("COPYRIGHT".to_string(), copyright.clone());
}
if let Some(compilation) = metadata.compilation {
tag.insert(
"COMPILATION".to_string(),
if compilation { "1" } else { "0" }.to_string(),
);
}
// Sort fields
if let Some(ref title_sort) = metadata.title_sort {
tag.insert("TITLESORT".to_string(), title_sort.clone());
}
if let Some(ref artist_sort) = metadata.artist_sort {
tag.insert("ARTISTSORT".to_string(), artist_sort.clone());
}
if let Some(ref album_sort) = metadata.album_sort {
tag.insert("ALBUMSORT".to_string(), album_sort.clone());
}
if let Some(ref album_artist_sort) = metadata.album_artist_sort {
tag.insert("ALBUMARTISTSORT".to_string(), album_artist_sort.clone());
}
// MusicBrainz IDs
if let Some(ref mb_recording_id) = metadata.mb_recording_id {
tag.insert("MUSICBRAINZ_TRACKID".to_string(), mb_recording_id.clone());
}
if let Some(ref mb_album_id) = metadata.mb_album_id {
tag.insert("MUSICBRAINZ_ALBUMID".to_string(), mb_album_id.clone());
}
if let Some(ref mb_artist_id) = metadata.mb_artist_id {
tag.insert("MUSICBRAINZ_ARTISTID".to_string(), mb_artist_id.clone());
}
if let Some(ref mb_album_artist_id) = metadata.mb_album_artist_id {
tag.insert(
"MUSICBRAINZ_ALBUMARTISTID".to_string(),
mb_album_artist_id.clone(),
);
}
if let Some(ref mb_release_group_id) = metadata.mb_release_group_id {
tag.insert(
"MUSICBRAINZ_RELEASEGROUPID".to_string(),
mb_release_group_id.clone(),
);
}
// ReplayGain
if let Some(gain) = metadata.replaygain_track_gain {
tag.insert(
"REPLAYGAIN_TRACK_GAIN".to_string(),
format!("{:.2} dB", gain),
);
}
if let Some(peak) = metadata.replaygain_track_peak {
tag.insert("REPLAYGAIN_TRACK_PEAK".to_string(), format!("{:.6}", peak));
}
if let Some(gain) = metadata.replaygain_album_gain {
tag.insert(
"REPLAYGAIN_ALBUM_GAIN".to_string(),
format!("{:.2} dB", gain),
);
}
if let Some(peak) = metadata.replaygain_album_peak {
tag.insert("REPLAYGAIN_ALBUM_PEAK".to_string(), format!("{:.6}", peak));
}
// Encoder
if let Some(ref encoder) = metadata.encoder {
tag.insert("ENCODER".to_string(), encoder.clone());
}
tag
}
/// Serialize Vorbis comments to bytes (without block header).
/// Format: vendor_length (4 LE) + vendor + comment_count (4 LE) + comments
fn serialize_vorbis_comments(tag: &VorbisComments) -> Vec<u8> {
let vendor = tag.vendor();
let vendor = if vendor.is_empty() { "musicfs" } else { vendor };
let mut data = Vec::new();
// Vendor string (little-endian length + UTF-8 string)
let vendor_bytes = vendor.as_bytes();
data.extend_from_slice(&(vendor_bytes.len() as u32).to_le_bytes());
data.extend_from_slice(vendor_bytes);
// Collect all comments
let comments: Vec<_> = tag.items().collect();
data.extend_from_slice(&(comments.len() as u32).to_le_bytes());
for (key, value) in comments {
let comment = format!("{}={}", key, value);
let comment_bytes = comment.as_bytes();
data.extend_from_slice(&(comment_bytes.len() as u32).to_le_bytes());
data.extend_from_slice(comment_bytes);
}
data
}
/// Extract metadata from Vorbis comments tag.
fn extract_from_vorbis_comments(tag: &VorbisComments) -> AudioMeta {
let mut meta = AudioMeta::default();
meta.format = AudioFormat::Flac;
// Basic fields (using Accessor trait)
meta.title = tag.title().map(|c: Cow<'_, str>| c.into_owned());
meta.artist = tag.artist().map(|c: Cow<'_, str>| c.into_owned());
meta.album = tag.album().map(|c: Cow<'_, str>| c.into_owned());
meta.genre = tag.genre().map(|c: Cow<'_, str>| c.into_owned());
// Album artist
meta.album_artist = tag.get("ALBUMARTIST").map(String::from);
// Date/Year
meta.date = tag.get("DATE").map(String::from);
if let Some(ref date) = meta.date {
if let Some(year_str) = date.split('-').next() {
meta.year = year_str.parse().ok();
}
}
// Track/Disc numbers
meta.track = tag.get("TRACKNUMBER").and_then(|s| s.parse().ok());
meta.track_total = tag.get("TRACKTOTAL").and_then(|s| s.parse().ok());
meta.disc = tag.get("DISCNUMBER").and_then(|s| s.parse().ok());
meta.disc_total = tag.get("DISCTOTAL").and_then(|s| s.parse().ok());
// Extended metadata
meta.composer = tag.get("COMPOSER").map(String::from);
meta.comment = tag.get("COMMENT").map(String::from);
meta.lyrics = tag.get("LYRICS").map(String::from);
meta.copyright = tag.get("COPYRIGHT").map(String::from);
meta.compilation = tag
.get("COMPILATION")
.map(|s| s == "1" || s.eq_ignore_ascii_case("true"));
// Sort fields
meta.title_sort = tag.get("TITLESORT").map(String::from);
meta.artist_sort = tag.get("ARTISTSORT").map(String::from);
meta.album_sort = tag.get("ALBUMSORT").map(String::from);
meta.album_artist_sort = tag.get("ALBUMARTISTSORT").map(String::from);
// MusicBrainz IDs
meta.mb_recording_id = tag.get("MUSICBRAINZ_TRACKID").map(String::from);
meta.mb_album_id = tag.get("MUSICBRAINZ_ALBUMID").map(String::from);
meta.mb_artist_id = tag.get("MUSICBRAINZ_ARTISTID").map(String::from);
meta.mb_album_artist_id = tag.get("MUSICBRAINZ_ALBUMARTISTID").map(String::from);
meta.mb_release_group_id = tag.get("MUSICBRAINZ_RELEASEGROUPID").map(String::from);
// ReplayGain
meta.replaygain_track_gain = tag
.get("REPLAYGAIN_TRACK_GAIN")
.and_then(|s| Self::parse_replaygain_value(s));
meta.replaygain_track_peak = tag
.get("REPLAYGAIN_TRACK_PEAK")
.and_then(|s| s.parse().ok());
meta.replaygain_album_gain = tag
.get("REPLAYGAIN_ALBUM_GAIN")
.and_then(|s| Self::parse_replaygain_value(s));
meta.replaygain_album_peak = tag
.get("REPLAYGAIN_ALBUM_PEAK")
.and_then(|s| s.parse().ok());
// Encoder
meta.encoder = tag.get("ENCODER").map(String::from);
meta
}
/// Parse ReplayGain value, stripping optional "dB" suffix.
fn parse_replaygain_value(value: &str) -> Option<f32> {
value
.trim()
.trim_end_matches(" dB")
.trim_end_matches("dB")
.parse()
.ok()
}
}
impl Default for FlacHandler {
fn default() -> Self {
Self::new()
}
}
impl FormatHandler for FlacHandler {
fn id(&self) -> &'static str {
"flac"
}
fn name(&self) -> &'static str {
"FLAC"
}
fn extensions(&self) -> &[&'static str] {
&["flac"]
}
fn mime_types(&self) -> &[&'static str] {
&["audio/flac", "audio/x-flac"]
}
fn analyze(&self, data: &[u8], file_size: u64) -> Result<FormatLayout, FormatError> {
// Verify FLAC marker
if data.len() < FLAC_MARKER.len() || &data[0..4] != FLAC_MARKER {
return Err(FormatError::InvalidData("Not a FLAC file".to_string()));
}
let mut offset = FLAC_MARKER.len();
let mut streaminfo_data: Option<Vec<u8>> = None;
// Parse metadata blocks to find audio_start and extract STREAMINFO
loop {
if offset + BLOCK_HEADER_SIZE > data.len() {
return Err(FormatError::InvalidData(
"Truncated FLAC metadata".to_string(),
));
}
let (is_last, block_type, block_size) = Self::parse_block_header(&data[offset..])
.ok_or_else(|| FormatError::InvalidData("Invalid block header".to_string()))?;
// Extract STREAMINFO block data (without header)
if block_type == BLOCK_TYPE_STREAMINFO {
if block_size != STREAMINFO_DATA_SIZE {
return Err(FormatError::InvalidData(format!(
"Invalid STREAMINFO size: {} (expected {})",
block_size, STREAMINFO_DATA_SIZE
)));
}
let data_start = offset + BLOCK_HEADER_SIZE;
let data_end = data_start + block_size;
if data_end > data.len() {
return Err(FormatError::InvalidData(
"Truncated STREAMINFO block".to_string(),
));
}
streaminfo_data = Some(data[data_start..data_end].to_vec());
}
offset += BLOCK_HEADER_SIZE + block_size;
if is_last {
break;
}
}
let streaminfo = streaminfo_data
.ok_or_else(|| FormatError::InvalidData("Missing STREAMINFO block".to_string()))?;
Ok(FormatLayout {
audio_start: offset as u64,
audio_end: file_size,
format: AudioFormat::Flac,
format_data: Some(streaminfo),
})
}
fn synthesize(
&self,
metadata: &AudioMeta,
layout: &FormatLayout,
) -> Result<Vec<u8>, FormatError> {
// STREAMINFO must be preserved from original
let streaminfo_data = layout.format_data.as_ref().ok_or_else(|| {
FormatError::SynthesisFailed("Missing STREAMINFO data in layout".to_string())
})?;
if streaminfo_data.len() != STREAMINFO_DATA_SIZE {
return Err(FormatError::SynthesisFailed(format!(
"Invalid STREAMINFO size: {} (expected {})",
streaminfo_data.len(),
STREAMINFO_DATA_SIZE
)));
}
// Build Vorbis comments
let vorbis_tag = Self::build_vorbis_comments(metadata);
let vorbis_data = Self::serialize_vorbis_comments(&vorbis_tag);
// Calculate total header size
let total_size =
FLAC_MARKER.len() + STREAMINFO_BLOCK_SIZE + BLOCK_HEADER_SIZE + vorbis_data.len();
let mut buffer = Vec::with_capacity(total_size);
// Write FLAC marker
buffer.extend_from_slice(FLAC_MARKER);
// Write STREAMINFO block (not last)
let streaminfo_header =
Self::write_block_header(false, BLOCK_TYPE_STREAMINFO, STREAMINFO_DATA_SIZE);
buffer.extend_from_slice(&streaminfo_header);
buffer.extend_from_slice(streaminfo_data);
// Write VORBIS_COMMENT block (last)
let vorbis_header =
Self::write_block_header(true, BLOCK_TYPE_VORBIS_COMMENT, vorbis_data.len());
buffer.extend_from_slice(&vorbis_header);
buffer.extend_from_slice(&vorbis_data);
Ok(buffer)
}
fn extract(&self, data: &[u8]) -> Result<AudioMeta, FormatError> {
let mut cursor = Cursor::new(data);
let flac_file = FlacFile::read_from(&mut cursor, ParseOptions::new())
.map_err(|e| FormatError::InvalidData(e.to_string()))?;
let tag = flac_file
.vorbis_comments()
.ok_or_else(|| FormatError::InvalidData("No Vorbis comments found".to_string()))?;
Ok(Self::extract_from_vorbis_comments(tag))
}
fn estimate_header_size(&self, _metadata: &AudioMeta) -> usize {
// fLaC (4) + STREAMINFO (38) + VORBIS_COMMENT header (4) + typical comments (~4KB)
8192
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_test_meta() -> AudioMeta {
AudioMeta {
title: Some("Test Title".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
album_artist: Some("Test Album Artist".to_string()),
genre: Some("Rock".to_string()),
year: Some(2024),
track: Some(5),
track_total: Some(12),
disc: Some(1),
disc_total: Some(2),
format: AudioFormat::Flac,
date: Some("2024-03-15".to_string()),
composer: Some("Test Composer".to_string()),
comment: Some("Test Comment".to_string()),
lyrics: Some("Test Lyrics\nLine 2".to_string()),
copyright: Some("2024 Test Copyright".to_string()),
compilation: Some(false),
title_sort: Some("Title, Test".to_string()),
artist_sort: Some("Artist, Test".to_string()),
album_sort: Some("Album, Test".to_string()),
album_artist_sort: Some("Album Artist, Test".to_string()),
mb_recording_id: Some("rec-12345".to_string()),
mb_album_id: Some("alb-12345".to_string()),
mb_artist_id: Some("art-12345".to_string()),
mb_album_artist_id: Some("albart-12345".to_string()),
mb_release_group_id: Some("rg-12345".to_string()),
replaygain_track_gain: Some(-6.5),
replaygain_track_peak: Some(0.987654),
replaygain_album_gain: Some(-5.2),
replaygain_album_peak: Some(0.999999),
encoder: Some("FLAC 1.4.0".to_string()),
..Default::default()
}
}
/// Create a minimal valid FLAC file header for testing.
fn make_minimal_flac_header() -> Vec<u8> {
let mut data = Vec::new();
// FLAC marker
data.extend_from_slice(b"fLaC");
// STREAMINFO block (last=true for minimal file)
// Header: type=0 (STREAMINFO), last=1, size=34
data.push(0x80); // 0x80 = last flag set, type 0
data.push(0x00);
data.push(0x00);
data.push(0x22); // 34 bytes
// STREAMINFO data (34 bytes) - minimal valid values
// min_block_size (16 bits) = 4096
data.push(0x10);
data.push(0x00);
// max_block_size (16 bits) = 4096
data.push(0x10);
data.push(0x00);
// min_frame_size (24 bits) = 0 (unknown)
data.push(0x00);
data.push(0x00);
data.push(0x00);
// max_frame_size (24 bits) = 0 (unknown)
data.push(0x00);
data.push(0x00);
data.push(0x00);
// sample_rate (20 bits) = 44100, channels-1 (3 bits) = 1, bits-1 (5 bits) = 15
// 44100 = 0xAC44, channels=2 (1), bits=16 (15)
// Packed: SSSS SSSS SSSS SSSS SSSS CCCC CBBB BB
// 0xAC44 << 12 | (1 << 9) | (15 << 4) = ...
// Let's use simpler encoding:
// Byte 0-1: sample_rate high 16 bits of 20
// Byte 2: sample_rate low 4 bits | channels 3 bits | bits high 1 bit
// Byte 3: bits low 4 bits | total_samples high 4 bits
// Actually the format is:
// 20 bits sample rate, 3 bits channels-1, 5 bits bits-1, 36 bits total samples
// 44100 = 0x0AC44
data.push(0x0A); // sample_rate bits 19-12
data.push(0xC4); // sample_rate bits 11-4
data.push(0x42); // sample_rate bits 3-0 (0x4), channels-1 (0x1=stereo), bits-1 high bit (0)
data.push(0xF0); // bits-1 low 4 bits (0xF=15, so 16 bits), total_samples high 4 bits (0)
// total_samples (remaining 32 bits) = 0
data.push(0x00);
data.push(0x00);
data.push(0x00);
data.push(0x00);
// MD5 signature (128 bits = 16 bytes)
data.extend_from_slice(&[0u8; 16]);
data
}
/// Create a FLAC header with Vorbis comments for testing extract().
fn make_flac_with_vorbis_comments() -> Vec<u8> {
let mut data = Vec::new();
// FLAC marker
data.extend_from_slice(b"fLaC");
// STREAMINFO block (not last)
data.push(0x00); // type=0, last=0
data.push(0x00);
data.push(0x00);
data.push(0x22); // 34 bytes
// STREAMINFO data (34 bytes)
data.push(0x10);
data.push(0x00);
data.push(0x10);
data.push(0x00);
data.extend_from_slice(&[0u8; 6]); // frame sizes
data.push(0x0A);
data.push(0xC4);
data.push(0x42);
data.push(0xF0);
data.extend_from_slice(&[0u8; 4]); // total samples
data.extend_from_slice(&[0u8; 16]); // MD5
// VORBIS_COMMENT block (last)
// Vendor: "test"
// Comments: TITLE=Test Song, ARTIST=Test Artist
let vendor = b"test";
let comments = [
b"TITLE=Test Song".as_slice(),
b"ARTIST=Test Artist".as_slice(),
b"ALBUM=Test Album".as_slice(),
b"TRACKNUMBER=3".as_slice(),
b"REPLAYGAIN_TRACK_GAIN=-5.50 dB".as_slice(),
];
let mut vorbis_data = Vec::new();
// Vendor length (LE)
vorbis_data.extend_from_slice(&(vendor.len() as u32).to_le_bytes());
vorbis_data.extend_from_slice(vendor);
// Comment count (LE)
vorbis_data.extend_from_slice(&(comments.len() as u32).to_le_bytes());
for comment in &comments {
vorbis_data.extend_from_slice(&(comment.len() as u32).to_le_bytes());
vorbis_data.extend_from_slice(*comment);
}
// VORBIS_COMMENT header
data.push(0x84); // type=4, last=1
data.push(((vorbis_data.len() >> 16) & 0xFF) as u8);
data.push(((vorbis_data.len() >> 8) & 0xFF) as u8);
data.push((vorbis_data.len() & 0xFF) as u8);
data.extend_from_slice(&vorbis_data);
data
}
#[test]
fn test_id_and_name() {
let handler = FlacHandler::new();
assert_eq!(handler.id(), "flac");
assert_eq!(handler.name(), "FLAC");
}
#[test]
fn test_extensions_and_mime_types() {
let handler = FlacHandler::new();
assert_eq!(handler.extensions(), &["flac"]);
assert_eq!(handler.mime_types(), &["audio/flac", "audio/x-flac"]);
}
#[test]
fn test_estimate_header_size() {
let handler = FlacHandler::new();
let meta = AudioMeta::default();
assert_eq!(handler.estimate_header_size(&meta), 8192);
}
#[test]
fn test_analyze_valid_flac() {
let handler = FlacHandler::new();
let data = make_minimal_flac_header();
let file_size = data.len() as u64 + 1000; // Pretend there's audio data
let result = handler.analyze(&data, file_size);
assert!(result.is_ok(), "analyze failed: {:?}", result.err());
let layout = result.unwrap();
assert_eq!(layout.audio_start, 42); // 4 (marker) + 38 (STREAMINFO)
assert_eq!(layout.audio_end, file_size);
assert_eq!(layout.format, AudioFormat::Flac);
assert!(layout.format_data.is_some());
assert_eq!(
layout.format_data.as_ref().unwrap().len(),
STREAMINFO_DATA_SIZE
);
}
#[test]
fn test_analyze_invalid_marker() {
let handler = FlacHandler::new();
let data = b"ID3\x04\x00\x00"; // MP3 header, not FLAC
let result = handler.analyze(data, 1000);
assert!(result.is_err());
assert!(matches!(result.unwrap_err(), FormatError::InvalidData(_)));
}
#[test]
fn test_analyze_truncated() {
let handler = FlacHandler::new();
let data = b"fLaC"; // Just the marker, no blocks
let result = handler.analyze(data, 4);
assert!(result.is_err());
}
#[test]
fn test_synthesize_creates_valid_flac_header() {
let handler = FlacHandler::new();
let meta = make_test_meta();
// Create layout with STREAMINFO
let original_data = make_minimal_flac_header();
let layout = handler
.analyze(&original_data, original_data.len() as u64)
.unwrap();
let result = handler.synthesize(&meta, &layout);
assert!(result.is_ok(), "synthesize failed: {:?}", result.err());
let bytes = result.unwrap();
// Verify FLAC marker
assert!(bytes.len() >= 4);
assert_eq!(&bytes[0..4], b"fLaC");
// Verify STREAMINFO block header
assert_eq!(bytes[4] & 0x7F, BLOCK_TYPE_STREAMINFO); // Type 0
assert_eq!(bytes[4] & 0x80, 0); // Not last
// Verify STREAMINFO size
let streaminfo_size =
((bytes[5] as usize) << 16) | ((bytes[6] as usize) << 8) | (bytes[7] as usize);
assert_eq!(streaminfo_size, STREAMINFO_DATA_SIZE);
// Verify VORBIS_COMMENT block follows
let vorbis_offset = 4 + 4 + STREAMINFO_DATA_SIZE;
assert_eq!(bytes[vorbis_offset] & 0x7F, BLOCK_TYPE_VORBIS_COMMENT);
assert_eq!(bytes[vorbis_offset] & 0x80, 0x80); // Is last
}
#[test]
fn test_synthesize_preserves_streaminfo() {
let handler = FlacHandler::new();
let meta = AudioMeta::default();
// Create layout with specific STREAMINFO
let original_data = make_minimal_flac_header();
let layout = handler
.analyze(&original_data, original_data.len() as u64)
.unwrap();
let original_streaminfo = layout.format_data.as_ref().unwrap().clone();
let synthesized = handler.synthesize(&meta, &layout).unwrap();
// Extract STREAMINFO from synthesized header
let streaminfo_start = 4 + 4; // After marker and header
let streaminfo_end = streaminfo_start + STREAMINFO_DATA_SIZE;
let synthesized_streaminfo = &synthesized[streaminfo_start..streaminfo_end];
assert_eq!(synthesized_streaminfo, original_streaminfo.as_slice());
}
#[test]
fn test_synthesize_missing_streaminfo() {
let handler = FlacHandler::new();
let meta = AudioMeta::default();
let layout = FormatLayout {
audio_start: 42,
audio_end: 1000,
format: AudioFormat::Flac,
format_data: None, // Missing STREAMINFO
};
let result = handler.synthesize(&meta, &layout);
assert!(result.is_err());
assert!(matches!(
result.unwrap_err(),
FormatError::SynthesisFailed(_)
));
}
#[test]
fn test_extract_from_flac() {
let handler = FlacHandler::new();
let data = make_flac_with_vorbis_comments();
let result = handler.extract(&data);
assert!(result.is_ok(), "extract failed: {:?}", result.err());
let meta = result.unwrap();
assert_eq!(meta.title, Some("Test Song".to_string()));
assert_eq!(meta.artist, Some("Test Artist".to_string()));
assert_eq!(meta.album, Some("Test Album".to_string()));
assert_eq!(meta.track, Some(3));
assert_eq!(meta.format, AudioFormat::Flac);
// Check ReplayGain parsing
let gain = meta.replaygain_track_gain.unwrap();
assert!((gain - (-5.5)).abs() < 0.01);
}
#[test]
fn test_build_and_extract_vorbis_comments() {
let original_meta = make_test_meta();
let tag = FlacHandler::build_vorbis_comments(&original_meta);
let extracted = FlacHandler::extract_from_vorbis_comments(&tag);
assert_eq!(extracted.title, original_meta.title);
assert_eq!(extracted.artist, original_meta.artist);
assert_eq!(extracted.album, original_meta.album);
assert_eq!(extracted.album_artist, original_meta.album_artist);
assert_eq!(extracted.genre, original_meta.genre);
assert_eq!(extracted.track, original_meta.track);
assert_eq!(extracted.track_total, original_meta.track_total);
assert_eq!(extracted.disc, original_meta.disc);
assert_eq!(extracted.disc_total, original_meta.disc_total);
assert_eq!(extracted.composer, original_meta.composer);
assert_eq!(extracted.comment, original_meta.comment);
assert_eq!(extracted.lyrics, original_meta.lyrics);
assert_eq!(extracted.copyright, original_meta.copyright);
assert_eq!(extracted.compilation, original_meta.compilation);
assert_eq!(extracted.title_sort, original_meta.title_sort);
assert_eq!(extracted.artist_sort, original_meta.artist_sort);
assert_eq!(extracted.album_sort, original_meta.album_sort);
assert_eq!(extracted.album_artist_sort, original_meta.album_artist_sort);
assert_eq!(extracted.mb_recording_id, original_meta.mb_recording_id);
assert_eq!(extracted.mb_album_id, original_meta.mb_album_id);
assert_eq!(extracted.mb_artist_id, original_meta.mb_artist_id);
assert_eq!(
extracted.mb_album_artist_id,
original_meta.mb_album_artist_id
);
assert_eq!(
extracted.mb_release_group_id,
original_meta.mb_release_group_id
);
assert_eq!(extracted.encoder, original_meta.encoder);
// ReplayGain values (with tolerance for formatting)
let orig_track_gain = original_meta.replaygain_track_gain.unwrap();
let ext_track_gain = extracted.replaygain_track_gain.unwrap();
assert!((orig_track_gain - ext_track_gain).abs() < 0.01);
let orig_track_peak = original_meta.replaygain_track_peak.unwrap();
let ext_track_peak = extracted.replaygain_track_peak.unwrap();
assert!((orig_track_peak - ext_track_peak).abs() < 0.0001);
}
#[test]
fn test_parse_replaygain_value() {
assert_eq!(FlacHandler::parse_replaygain_value("-6.50 dB"), Some(-6.50));
assert_eq!(FlacHandler::parse_replaygain_value("-6.50dB"), Some(-6.50));
assert_eq!(FlacHandler::parse_replaygain_value("-6.50"), Some(-6.50));
assert_eq!(FlacHandler::parse_replaygain_value(" 3.2 dB "), Some(3.2));
assert_eq!(FlacHandler::parse_replaygain_value("invalid"), None);
}
#[test]
fn test_parse_block_header() {
// Not last, type 0, size 34
let header = [0x00, 0x00, 0x00, 0x22];
let (is_last, block_type, size) = FlacHandler::parse_block_header(&header).unwrap();
assert!(!is_last);
assert_eq!(block_type, 0);
assert_eq!(size, 34);
// Last, type 4, size 256
let header = [0x84, 0x00, 0x01, 0x00];
let (is_last, block_type, size) = FlacHandler::parse_block_header(&header).unwrap();
assert!(is_last);
assert_eq!(block_type, 4);
assert_eq!(size, 256);
}
#[test]
fn test_write_block_header() {
let header = FlacHandler::write_block_header(false, 0, 34);
assert_eq!(header, [0x00, 0x00, 0x00, 0x22]);
let header = FlacHandler::write_block_header(true, 4, 256);
assert_eq!(header, [0x84, 0x00, 0x01, 0x00]);
}
#[test]
fn test_empty_metadata_produces_minimal_vorbis() {
let handler = FlacHandler::new();
let meta = AudioMeta::default();
let original_data = make_minimal_flac_header();
let layout = handler
.analyze(&original_data, original_data.len() as u64)
.unwrap();
let result = handler.synthesize(&meta, &layout);
assert!(result.is_ok());
let bytes = result.unwrap();
// Should have: fLaC (4) + STREAMINFO (38) + VORBIS_COMMENT (header + minimal data)
assert!(bytes.len() >= 42 + 4 + 8); // At least vendor string overhead
}
#[test]
fn test_round_trip_synthesize_analyze() {
let handler = FlacHandler::new();
let meta = make_test_meta();
// Create initial layout
let original_data = make_minimal_flac_header();
let layout = handler
.analyze(&original_data, original_data.len() as u64)
.unwrap();
// Synthesize new header
let synthesized = handler.synthesize(&meta, &layout).unwrap();
// Analyze synthesized header
let new_layout = handler
.analyze(&synthesized, synthesized.len() as u64)
.unwrap();
// STREAMINFO should be preserved
assert_eq!(new_layout.format_data, layout.format_data);
assert_eq!(new_layout.format, AudioFormat::Flac);
}
}
+631
View File
@@ -0,0 +1,631 @@
use crate::{FormatError, FormatHandler, FormatLayout};
use lofty::config::{ParseOptions, WriteOptions};
use lofty::file::AudioFile;
use lofty::id3::v2::{
CommentFrame, Frame, FrameId, Id3v2Tag, TextInformationFrame, UnsynchronizedTextFrame,
};
use lofty::mpeg::MpegFile;
use lofty::tag::{Accessor, TagExt};
use lofty::TextEncoding;
use musicfs_core::{AudioFormat, AudioMeta};
use std::borrow::Cow;
use std::io::Cursor;
const ID3V2_HEADER_SIZE: usize = 10;
const ID3V1_TAG_SIZE: usize = 128;
pub struct Id3v2Handler;
impl Id3v2Handler {
pub fn new() -> Self {
Self
}
fn parse_id3v2_header(data: &[u8]) -> Option<usize> {
if data.len() < ID3V2_HEADER_SIZE {
return None;
}
if &data[0..3] != b"ID3" {
return None;
}
let size = syncsafe_decode(&data[6..10]);
Some(ID3V2_HEADER_SIZE + size)
}
fn has_id3v1_tag(data: &[u8], file_size: u64) -> bool {
if file_size < ID3V1_TAG_SIZE as u64 {
return false;
}
let tag_start = (file_size as usize).saturating_sub(ID3V1_TAG_SIZE);
if tag_start >= data.len() {
return false;
}
&data[tag_start..tag_start + 3] == b"TAG"
}
fn set_text_frame(tag: &mut Id3v2Tag, frame_id: &'static str, value: &str) {
let id = FrameId::Valid(Cow::Borrowed(frame_id));
let frame = Frame::Text(TextInformationFrame::new(
id,
TextEncoding::UTF8,
value.to_string(),
));
tag.insert(frame);
}
fn set_track_disc_frame(
tag: &mut Id3v2Tag,
frame_id: &'static str,
num: u32,
total: Option<u32>,
) {
let value = match total {
Some(t) => format!("{}/{}", num, t),
None => num.to_string(),
};
Self::set_text_frame(tag, frame_id, &value);
}
fn set_comment_frame(tag: &mut Id3v2Tag, value: &str) {
let frame = Frame::Comment(CommentFrame::new(
TextEncoding::UTF8,
*b"eng",
String::new(),
value.to_string(),
));
tag.insert(frame);
}
fn set_lyrics_frame(tag: &mut Id3v2Tag, value: &str) {
let frame = Frame::UnsynchronizedText(UnsynchronizedTextFrame::new(
TextEncoding::UTF8,
*b"eng",
String::new(),
value.to_string(),
));
tag.insert(frame);
}
fn build_tag_from_meta(metadata: &AudioMeta) -> Id3v2Tag {
let mut tag = Id3v2Tag::new();
if let Some(ref title) = metadata.title {
tag.set_title(title.clone());
}
if let Some(ref artist) = metadata.artist {
tag.set_artist(artist.clone());
}
if let Some(ref album) = metadata.album {
tag.set_album(album.clone());
}
if let Some(ref album_artist) = metadata.album_artist {
Self::set_text_frame(&mut tag, "TPE2", album_artist);
}
if let Some(year) = metadata.year {
Self::set_text_frame(&mut tag, "TDRC", &year.to_string());
}
if let Some(ref genre) = metadata.genre {
tag.set_genre(genre.clone());
}
if let Some(track) = metadata.track {
Self::set_track_disc_frame(&mut tag, "TRCK", track, metadata.track_total);
}
if let Some(disc) = metadata.disc {
Self::set_track_disc_frame(&mut tag, "TPOS", disc, metadata.disc_total);
}
if let Some(ref date) = metadata.date {
Self::set_text_frame(&mut tag, "TDRC", date);
}
if let Some(ref composer) = metadata.composer {
Self::set_text_frame(&mut tag, "TCOM", composer);
}
if let Some(ref comment) = metadata.comment {
Self::set_comment_frame(&mut tag, comment);
}
if let Some(ref lyrics) = metadata.lyrics {
Self::set_lyrics_frame(&mut tag, lyrics);
}
if let Some(ref copyright) = metadata.copyright {
Self::set_text_frame(&mut tag, "TCOP", copyright);
}
if let Some(compilation) = metadata.compilation {
Self::set_text_frame(&mut tag, "TCMP", if compilation { "1" } else { "0" });
}
if let Some(ref title_sort) = metadata.title_sort {
Self::set_text_frame(&mut tag, "TSOT", title_sort);
}
if let Some(ref artist_sort) = metadata.artist_sort {
Self::set_text_frame(&mut tag, "TSOP", artist_sort);
}
if let Some(ref album_sort) = metadata.album_sort {
Self::set_text_frame(&mut tag, "TSOA", album_sort);
}
if let Some(ref album_artist_sort) = metadata.album_artist_sort {
Self::set_text_frame(&mut tag, "TSO2", album_artist_sort);
}
if let Some(ref mb_recording_id) = metadata.mb_recording_id {
tag.insert_user_text(
"MusicBrainz Recording Id".to_string(),
mb_recording_id.clone(),
);
}
if let Some(ref mb_album_id) = metadata.mb_album_id {
tag.insert_user_text("MusicBrainz Album Id".to_string(), mb_album_id.clone());
}
if let Some(ref mb_artist_id) = metadata.mb_artist_id {
tag.insert_user_text("MusicBrainz Artist Id".to_string(), mb_artist_id.clone());
}
if let Some(ref mb_album_artist_id) = metadata.mb_album_artist_id {
tag.insert_user_text(
"MusicBrainz Album Artist Id".to_string(),
mb_album_artist_id.clone(),
);
}
if let Some(ref mb_release_group_id) = metadata.mb_release_group_id {
tag.insert_user_text(
"MusicBrainz Release Group Id".to_string(),
mb_release_group_id.clone(),
);
}
if let Some(gain) = metadata.replaygain_track_gain {
tag.insert_user_text(
"REPLAYGAIN_TRACK_GAIN".to_string(),
format!("{:.2} dB", gain),
);
}
if let Some(peak) = metadata.replaygain_track_peak {
tag.insert_user_text("REPLAYGAIN_TRACK_PEAK".to_string(), format!("{:.6}", peak));
}
if let Some(gain) = metadata.replaygain_album_gain {
tag.insert_user_text(
"REPLAYGAIN_ALBUM_GAIN".to_string(),
format!("{:.2} dB", gain),
);
}
if let Some(peak) = metadata.replaygain_album_peak {
tag.insert_user_text("REPLAYGAIN_ALBUM_PEAK".to_string(), format!("{:.6}", peak));
}
if let Some(ref encoder) = metadata.encoder {
Self::set_text_frame(&mut tag, "TSSE", encoder);
}
tag
}
fn extract_text_frame(tag: &Id3v2Tag, frame_id: &str) -> Option<String> {
let id = FrameId::new(frame_id).ok()?;
tag.get_text(&id).map(|s| s.to_string())
}
fn parse_track_disc(value: &str) -> (Option<u32>, Option<u32>) {
let parts: Vec<&str> = value.split('/').collect();
let num = parts.first().and_then(|s| s.parse().ok());
let total = parts.get(1).and_then(|s| s.parse().ok());
(num, total)
}
fn parse_replaygain_value(value: &str) -> Option<f32> {
value
.trim()
.trim_end_matches(" dB")
.trim_end_matches("dB")
.parse()
.ok()
}
fn extract_from_tag(tag: &Id3v2Tag) -> AudioMeta {
let mut meta = AudioMeta::default();
meta.format = AudioFormat::Mp3;
meta.title = tag.title().map(|c: Cow<'_, str>| c.into_owned());
meta.artist = tag.artist().map(|c: Cow<'_, str>| c.into_owned());
meta.album = tag.album().map(|c: Cow<'_, str>| c.into_owned());
meta.album_artist = Self::extract_text_frame(tag, "TPE2");
meta.genre = tag.genre().map(|c: Cow<'_, str>| c.into_owned());
if let Some(track_str) = Self::extract_text_frame(tag, "TRCK") {
let (track, track_total) = Self::parse_track_disc(&track_str);
meta.track = track;
meta.track_total = track_total;
} else {
meta.track = tag.track();
meta.track_total = tag.track_total();
}
if let Some(disc_str) = Self::extract_text_frame(tag, "TPOS") {
let (disc, disc_total) = Self::parse_track_disc(&disc_str);
meta.disc = disc;
meta.disc_total = disc_total;
} else {
meta.disc = tag.disk();
meta.disc_total = tag.disk_total();
}
meta.date = Self::extract_text_frame(tag, "TDRC");
if let Some(ref date) = meta.date {
if let Some(year_str) = date.split('-').next() {
meta.year = year_str.parse().ok();
}
}
meta.composer = Self::extract_text_frame(tag, "TCOM");
meta.comment = tag.comment().map(|c: Cow<'_, str>| c.into_owned());
if let Some(uslt) = tag.unsync_text().next() {
meta.lyrics = Some(uslt.content.to_string());
}
meta.copyright = Self::extract_text_frame(tag, "TCOP");
if let Some(tcmp) = Self::extract_text_frame(tag, "TCMP") {
meta.compilation = Some(tcmp == "1");
}
meta.title_sort = Self::extract_text_frame(tag, "TSOT");
meta.artist_sort = Self::extract_text_frame(tag, "TSOP");
meta.album_sort = Self::extract_text_frame(tag, "TSOA");
meta.album_artist_sort = Self::extract_text_frame(tag, "TSO2");
meta.mb_recording_id = tag
.get_user_text("MusicBrainz Recording Id")
.map(String::from);
meta.mb_album_id = tag.get_user_text("MusicBrainz Album Id").map(String::from);
meta.mb_artist_id = tag.get_user_text("MusicBrainz Artist Id").map(String::from);
meta.mb_album_artist_id = tag
.get_user_text("MusicBrainz Album Artist Id")
.map(String::from);
meta.mb_release_group_id = tag
.get_user_text("MusicBrainz Release Group Id")
.map(String::from);
if let Some(gain_str) = tag.get_user_text("REPLAYGAIN_TRACK_GAIN") {
meta.replaygain_track_gain = Self::parse_replaygain_value(gain_str);
}
if let Some(peak_str) = tag.get_user_text("REPLAYGAIN_TRACK_PEAK") {
meta.replaygain_track_peak = peak_str.parse::<f32>().ok();
}
if let Some(gain_str) = tag.get_user_text("REPLAYGAIN_ALBUM_GAIN") {
meta.replaygain_album_gain = Self::parse_replaygain_value(gain_str);
}
if let Some(peak_str) = tag.get_user_text("REPLAYGAIN_ALBUM_PEAK") {
meta.replaygain_album_peak = peak_str.parse::<f32>().ok();
}
meta.encoder = Self::extract_text_frame(tag, "TSSE");
meta
}
}
impl Default for Id3v2Handler {
fn default() -> Self {
Self::new()
}
}
impl FormatHandler for Id3v2Handler {
fn id(&self) -> &'static str {
"id3v2"
}
fn name(&self) -> &'static str {
"ID3v2 (MP3)"
}
fn extensions(&self) -> &[&'static str] {
&["mp3"]
}
fn mime_types(&self) -> &[&'static str] {
&["audio/mpeg"]
}
fn analyze(&self, data: &[u8], file_size: u64) -> Result<FormatLayout, FormatError> {
let audio_start = Self::parse_id3v2_header(data).unwrap_or(0) as u64;
let audio_end = if Self::has_id3v1_tag(data, file_size) {
file_size - ID3V1_TAG_SIZE as u64
} else {
file_size
};
Ok(FormatLayout {
audio_start,
audio_end,
format: AudioFormat::Mp3,
format_data: None,
})
}
fn synthesize(
&self,
metadata: &AudioMeta,
_layout: &FormatLayout,
) -> Result<Vec<u8>, FormatError> {
let tag = Self::build_tag_from_meta(metadata);
let mut buffer = Cursor::new(Vec::new());
let write_options = WriteOptions::new().preferred_padding(1024);
tag.dump_to(&mut buffer, write_options)
.map_err(|e| FormatError::SynthesisFailed(e.to_string()))?;
Ok(buffer.into_inner())
}
fn extract(&self, data: &[u8]) -> Result<AudioMeta, FormatError> {
let mut cursor = Cursor::new(data);
let mpeg_file = MpegFile::read_from(&mut cursor, ParseOptions::new())
.map_err(|e| FormatError::InvalidData(e.to_string()))?;
let tag = mpeg_file
.id3v2()
.ok_or_else(|| FormatError::InvalidData("No ID3v2 tag found".to_string()))?;
Ok(Self::extract_from_tag(tag))
}
fn estimate_header_size(&self, _metadata: &AudioMeta) -> usize {
4096 + 1024
}
}
fn syncsafe_decode(bytes: &[u8]) -> usize {
((bytes[0] as usize) << 21)
| ((bytes[1] as usize) << 14)
| ((bytes[2] as usize) << 7)
| (bytes[3] as usize)
}
#[cfg(test)]
mod tests {
use super::*;
fn make_test_meta() -> AudioMeta {
AudioMeta {
title: Some("Test Title".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
album_artist: Some("Test Album Artist".to_string()),
genre: Some("Rock".to_string()),
year: Some(2024),
track: Some(5),
track_total: Some(12),
disc: Some(1),
disc_total: Some(2),
format: AudioFormat::Mp3,
date: Some("2024-03-15".to_string()),
composer: Some("Test Composer".to_string()),
comment: Some("Test Comment".to_string()),
lyrics: Some("Test Lyrics\nLine 2".to_string()),
copyright: Some("2024 Test Copyright".to_string()),
compilation: Some(false),
title_sort: Some("Title, Test".to_string()),
artist_sort: Some("Artist, Test".to_string()),
album_sort: Some("Album, Test".to_string()),
album_artist_sort: Some("Album Artist, Test".to_string()),
mb_recording_id: Some("rec-12345".to_string()),
mb_album_id: Some("alb-12345".to_string()),
mb_artist_id: Some("art-12345".to_string()),
mb_album_artist_id: Some("albart-12345".to_string()),
mb_release_group_id: Some("rg-12345".to_string()),
replaygain_track_gain: Some(-6.5),
replaygain_track_peak: Some(0.987654),
replaygain_album_gain: Some(-5.2),
replaygain_album_peak: Some(0.999999),
encoder: Some("LAME 3.100".to_string()),
..Default::default()
}
}
#[test]
fn test_id_and_name() {
let handler = Id3v2Handler::new();
assert_eq!(handler.id(), "id3v2");
assert_eq!(handler.name(), "ID3v2 (MP3)");
}
#[test]
fn test_extensions_and_mime_types() {
let handler = Id3v2Handler::new();
assert_eq!(handler.extensions(), &["mp3"]);
assert_eq!(handler.mime_types(), &["audio/mpeg"]);
}
#[test]
fn test_estimate_header_size() {
let handler = Id3v2Handler::new();
let meta = AudioMeta::default();
assert_eq!(handler.estimate_header_size(&meta), 5120);
}
#[test]
fn test_synthesize_creates_valid_id3v2() {
let handler = Id3v2Handler::new();
let meta = make_test_meta();
let layout = FormatLayout {
audio_start: 0,
audio_end: 1000,
format: AudioFormat::Mp3,
format_data: None,
};
let result = handler.synthesize(&meta, &layout);
assert!(result.is_ok());
let bytes = result.unwrap();
assert!(bytes.len() >= 10);
assert_eq!(&bytes[0..3], b"ID3");
assert_eq!(bytes[3], 0x04);
}
#[test]
fn test_analyze_no_id3v2() {
let handler = Id3v2Handler::new();
let data = vec![0xFF, 0xFB, 0x90, 0x00];
let file_size = 1000;
let result = handler.analyze(&data, file_size);
assert!(result.is_ok());
let layout = result.unwrap();
assert_eq!(layout.audio_start, 0);
assert_eq!(layout.audio_end, 1000);
assert_eq!(layout.format, AudioFormat::Mp3);
}
#[test]
fn test_analyze_with_id3v2() {
let handler = Id3v2Handler::new();
let mut data = vec![b'I', b'D', b'3', 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x64];
data.extend(vec![0u8; 100]);
let file_size = data.len() as u64;
let result = handler.analyze(&data, file_size);
assert!(result.is_ok());
let layout = result.unwrap();
assert_eq!(layout.audio_start, 110);
assert_eq!(layout.audio_end, file_size);
}
#[test]
fn test_analyze_with_id3v1() {
let handler = Id3v2Handler::new();
let mut data = vec![0xFF, 0xFB, 0x90, 0x00];
data.extend(vec![0u8; 100]);
data.extend(b"TAG");
data.extend(vec![0u8; 125]);
let file_size = data.len() as u64;
let result = handler.analyze(&data, file_size);
assert!(result.is_ok());
let layout = result.unwrap();
assert_eq!(layout.audio_start, 0);
assert_eq!(layout.audio_end, file_size - 128);
}
#[test]
fn test_syncsafe_decode() {
assert_eq!(syncsafe_decode(&[0x00, 0x00, 0x00, 0x7F]), 127);
assert_eq!(syncsafe_decode(&[0x00, 0x00, 0x01, 0x00]), 128);
assert_eq!(syncsafe_decode(&[0x00, 0x00, 0x00, 0x64]), 100);
}
#[test]
fn test_parse_track_disc() {
assert_eq!(Id3v2Handler::parse_track_disc("5/12"), (Some(5), Some(12)));
assert_eq!(Id3v2Handler::parse_track_disc("5"), (Some(5), None));
assert_eq!(Id3v2Handler::parse_track_disc(""), (None, None));
}
#[test]
fn test_parse_replaygain_value() {
assert_eq!(
Id3v2Handler::parse_replaygain_value("-6.50 dB"),
Some(-6.50)
);
assert_eq!(Id3v2Handler::parse_replaygain_value("-6.50dB"), Some(-6.50));
assert_eq!(Id3v2Handler::parse_replaygain_value("-6.50"), Some(-6.50));
assert_eq!(Id3v2Handler::parse_replaygain_value("invalid"), None);
}
#[test]
fn test_empty_metadata_produces_empty_tag() {
let handler = Id3v2Handler::new();
let meta = AudioMeta::default();
let layout = FormatLayout {
audio_start: 0,
audio_end: 1000,
format: AudioFormat::Mp3,
format_data: None,
};
let result = handler.synthesize(&meta, &layout);
assert!(result.is_ok());
let bytes = result.unwrap();
assert!(bytes.is_empty());
}
#[test]
fn test_minimal_metadata_produces_valid_tag() {
let handler = Id3v2Handler::new();
let mut meta = AudioMeta::default();
meta.title = Some("Test".to_string());
let layout = FormatLayout {
audio_start: 0,
audio_end: 1000,
format: AudioFormat::Mp3,
format_data: None,
};
let result = handler.synthesize(&meta, &layout);
assert!(result.is_ok());
let bytes = result.unwrap();
assert!(bytes.len() >= 10);
assert_eq!(&bytes[0..3], b"ID3");
assert_eq!(bytes[3], 0x04);
}
#[test]
fn test_build_and_extract_tag() {
let original_meta = make_test_meta();
let tag = Id3v2Handler::build_tag_from_meta(&original_meta);
let extracted = Id3v2Handler::extract_from_tag(&tag);
assert_eq!(extracted.title, original_meta.title);
assert_eq!(extracted.artist, original_meta.artist);
assert_eq!(extracted.album, original_meta.album);
assert_eq!(extracted.album_artist, original_meta.album_artist);
assert_eq!(extracted.genre, original_meta.genre);
assert_eq!(extracted.track, original_meta.track);
assert_eq!(extracted.track_total, original_meta.track_total);
assert_eq!(extracted.disc, original_meta.disc);
assert_eq!(extracted.disc_total, original_meta.disc_total);
assert_eq!(extracted.composer, original_meta.composer);
assert_eq!(extracted.comment, original_meta.comment);
assert_eq!(extracted.lyrics, original_meta.lyrics);
assert_eq!(extracted.copyright, original_meta.copyright);
assert_eq!(extracted.compilation, original_meta.compilation);
assert_eq!(extracted.title_sort, original_meta.title_sort);
assert_eq!(extracted.artist_sort, original_meta.artist_sort);
assert_eq!(extracted.album_sort, original_meta.album_sort);
assert_eq!(extracted.album_artist_sort, original_meta.album_artist_sort);
assert_eq!(extracted.mb_recording_id, original_meta.mb_recording_id);
assert_eq!(extracted.mb_album_id, original_meta.mb_album_id);
assert_eq!(extracted.mb_artist_id, original_meta.mb_artist_id);
assert_eq!(
extracted.mb_album_artist_id,
original_meta.mb_album_artist_id
);
assert_eq!(
extracted.mb_release_group_id,
original_meta.mb_release_group_id
);
assert_eq!(extracted.encoder, original_meta.encoder);
let orig_track_gain = original_meta.replaygain_track_gain.unwrap();
let ext_track_gain = extracted.replaygain_track_gain.unwrap();
assert!((orig_track_gain - ext_track_gain).abs() < 0.01);
let orig_track_peak = original_meta.replaygain_track_peak.unwrap();
let ext_track_peak = extracted.replaygain_track_peak.unwrap();
assert!((orig_track_peak - ext_track_peak).abs() < 0.0001);
}
}
+12
View File
@@ -0,0 +1,12 @@
//! Format-specific metadata handlers for audio file synthesis.
//!
//! Each handler implements the `FormatHandler` trait to support:
//! - Analyzing original files to find audio boundaries
//! - Synthesizing new headers from database metadata
//! - Extracting metadata from existing files
mod flac;
mod id3v2;
pub use flac::FlacHandler;
pub use id3v2::Id3v2Handler;
+11 -2
View File
@@ -1,17 +1,26 @@
mod artwork; mod artwork;
mod db; mod db;
mod eviction; mod eviction;
mod format_handler;
mod format_layout;
pub mod handlers;
mod metadata; mod metadata;
mod overlay;
mod patterns; mod patterns;
mod prefetch; mod prefetch;
mod tree; mod tree;
pub use artwork::{ArtworkCache, ArtworkError, CachedArtwork}; pub use artwork::{ArtworkCache, ArtworkError, CachedArtwork};
pub use db::Database; pub use db::{Database, TrashedFile, TrashedFilter};
pub use eviction::{EvictionError, EvictionPolicy, LruEviction}; pub use eviction::{EvictionError, EvictionPolicy, LruEviction};
pub use format_handler::{FormatError, FormatHandler, FormatHandlerRegistry};
pub use format_layout::FormatLayout;
pub use handlers::{FlacHandler, Id3v2Handler};
pub use metadata::MetadataCache; pub use metadata::MetadataCache;
pub use overlay::{OverlayError, OverlayReader};
pub use patterns::{AccessContext, AccessPattern, PatternError, PatternStore}; pub use patterns::{AccessContext, AccessPattern, PatternError, PatternStore};
pub use prefetch::{PrefetchConfig, PrefetchEngine, PrefetchHandle}; pub use prefetch::{PrefetchConfig, PrefetchEngine, PrefetchHandle};
pub use tree::{ pub use tree::{
DirNode, FileNode, Inode, RefreshPolicy, TreeBuilder, VirtualNode, VirtualTree, ROOT_INODE, DirNode, FileNode, Inode, RefreshPolicy, RemoveError, RenameError, TreeBuilder, VirtualNode,
VirtualTree, ROOT_INODE,
}; };
+467
View File
@@ -0,0 +1,467 @@
//! OverlayReader: On-the-fly metadata overlay with header/audio splice logic.
//!
//! This module provides the core read path for metadata overlay. It synthesizes
//! headers on-the-fly from database metadata and splices them with original audio
//! data from the CAS.
use crate::{Database, FormatError, FormatHandlerRegistry};
use bytes::{Bytes, BytesMut};
use musicfs_cas::{FileReader, ReaderError};
use musicfs_core::{AudioFormat, FileId};
use std::sync::Arc;
use tracing::{debug, trace};
/// Error types for overlay operations
#[derive(Debug, thiserror::Error)]
pub enum OverlayError {
#[error("Database error: {0}")]
Database(#[from] musicfs_core::Error),
#[error("Format handler error: {0}")]
Handler(#[from] FormatError),
#[error("CAS error: {0}")]
Cas(#[from] ReaderError),
#[error("File not found: {0:?}")]
NotFound(FileId),
#[error("No handler for format: {0:?}")]
NoHandler(AudioFormat),
}
/// OverlayReader provides on-the-fly metadata overlay for audio files.
///
/// It synthesizes headers from database metadata and splices them with
/// original audio data from the CAS, presenting a virtual file that
/// reflects the current metadata state.
pub struct OverlayReader {
db: Arc<Database>,
registry: Arc<FormatHandlerRegistry>,
cas_reader: Arc<FileReader>,
}
impl OverlayReader {
/// Create a new OverlayReader with the given dependencies.
pub fn new(
db: Arc<Database>,
registry: Arc<FormatHandlerRegistry>,
cas_reader: Arc<FileReader>,
) -> Self {
Self {
db,
registry,
cas_reader,
}
}
/// Read bytes from a virtual file with metadata overlay.
///
/// This method implements the three-region splice logic:
/// - Region 1: Synthetic header (offset < header_len)
/// - Region 2: Audio data from CAS (offset >= header_len)
/// - Region 3: Boundary crossing (spans header/audio)
///
/// If no format_layout exists for the file, delegates directly to CAS reader.
pub async fn read(
&self,
file_id: FileId,
offset: u64,
size: u32,
) -> Result<Bytes, OverlayError> {
// Get format layout - if None, passthrough to CAS
let layout = match self.db.get_format_layout(file_id)? {
Some(layout) => layout,
None => {
trace!(file_id = ?file_id, "No format_layout, passthrough to CAS");
return Ok(self.cas_reader.read(file_id, offset, size).await?);
}
};
// Get metadata for synthesis
let metadata = self.db.get_file_metadata_row(file_id)?;
// Get handler for this format (handler IDs are lowercase)
let format_id = format!("{:?}", layout.format).to_lowercase();
let handler = self
.registry
.get_by_format(&format_id)
.ok_or_else(|| OverlayError::NoHandler(layout.format))?;
// Synthesize header on-the-fly
let header = handler.synthesize(&metadata, &layout)?;
let header_len = header.len() as u64;
let audio_len = layout.audio_end - layout.audio_start;
let virtual_size = header_len + audio_len;
trace!(
file_id = ?file_id,
header_len,
audio_len,
virtual_size,
offset,
size,
"Overlay read"
);
// Handle EOF
if offset >= virtual_size {
return Ok(Bytes::new());
}
let virtual_end = (offset + size as u64).min(virtual_size);
let mut result = BytesMut::with_capacity((virtual_end - offset) as usize);
// Region 1: Synthetic header
if offset < header_len {
let end = virtual_end.min(header_len);
result.extend_from_slice(&header[offset as usize..end as usize]);
trace!(
file_id = ?file_id,
start = offset,
end,
bytes = end - offset,
"Read from synthetic header"
);
}
// Region 2: Origin audio data (from CAS)
if virtual_end > header_len {
let audio_start_in_virtual = header_len.max(offset);
let audio_offset_in_origin = layout.audio_start + (audio_start_in_virtual - header_len);
let audio_bytes_needed = (virtual_end - audio_start_in_virtual) as u32;
trace!(
file_id = ?file_id,
audio_offset_in_origin,
audio_bytes_needed,
"Read from CAS audio"
);
let audio = self
.cas_reader
.read(file_id, audio_offset_in_origin, audio_bytes_needed)
.await?;
result.extend_from_slice(&audio);
}
debug!(
file_id = ?file_id,
offset,
size,
returned = result.len(),
"Overlay read complete"
);
Ok(result.freeze())
}
/// Estimate the virtual size of a file for getattr.
///
/// Returns the estimated size based on format layout. If no layout exists,
/// returns None to indicate the caller should use the original file size.
pub fn estimate_virtual_size(&self, file_id: FileId) -> Result<Option<u64>, OverlayError> {
// Get format layout - if None, return None to indicate passthrough
let layout = match self.db.get_format_layout(file_id)? {
Some(layout) => layout,
None => return Ok(None),
};
// Get metadata for header size estimation
let metadata = self.db.get_file_metadata_row(file_id)?;
let format_id = format!("{:?}", layout.format).to_lowercase();
let handler = self
.registry
.get_by_format(&format_id)
.ok_or_else(|| OverlayError::NoHandler(layout.format))?;
// Estimate header size
let estimated_header = handler.estimate_header_size(&metadata) as u64;
let audio_len = layout.audio_end - layout.audio_start;
let virtual_size = estimated_header + audio_len;
trace!(
file_id = ?file_id,
estimated_header,
audio_len,
virtual_size,
"Estimated virtual size"
);
Ok(Some(virtual_size))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::handlers::FlacHandler;
use crate::FormatLayout;
use musicfs_cas::{CasConfig, CasStore, ChunkManifest, ChunkRef};
use musicfs_core::{AudioFormat, AudioMeta, OriginId, VirtualPath};
use std::path::Path;
use std::time::UNIX_EPOCH;
use tempfile::TempDir;
fn make_test_metadata() -> AudioMeta {
AudioMeta {
title: Some("Test Track".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
track: Some(1),
format: AudioFormat::Flac,
sample_rate: Some(44100),
bits_per_sample: Some(16),
channels: Some(2),
..Default::default()
}
}
fn make_test_layout() -> FormatLayout {
// Simulate a file with minimal FLAC header, audio from 42 to 102442 (100KB audio)
// STREAMINFO data (34 bytes) - minimal valid values for FLAC synthesis
let streaminfo_data = vec![
0x10, 0x00, // min_block_size = 4096
0x10, 0x00, // max_block_size = 4096
0x00, 0x00, 0x00, // min_frame_size = 0
0x00, 0x00, 0x00, // max_frame_size = 0
0x0A, 0xC4, 0x42, 0xF0, // sample_rate=44100, channels=2, bits=16
0x00, 0x00, 0x00, 0x00, // total_samples
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // MD5 (16 bytes)
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
];
FormatLayout {
audio_start: 42, // fLaC (4) + STREAMINFO block (38)
audio_end: 42 + 100 * 1024, // 100KB audio
format: AudioFormat::Flac,
format_data: Some(streaminfo_data),
}
}
async fn setup_test_env() -> (
TempDir,
Arc<Database>,
Arc<FormatHandlerRegistry>,
Arc<FileReader>,
FileId,
) {
let dir = TempDir::new().unwrap();
// Setup database
let db = Arc::new(Database::open_memory().unwrap());
// Setup registry with FLAC handler
let mut registry = FormatHandlerRegistry::new();
registry.register(Arc::new(FlacHandler::new()));
let registry = Arc::new(registry);
// Setup CAS store and reader
let cas_config = CasConfig {
chunks_dir: dir.path().join("chunks"),
..Default::default()
};
let store = Arc::new(CasStore::open(cas_config).await.unwrap());
// Create test audio data (simulating 100KB of audio)
let audio_data: Vec<u8> = (0..100 * 1024).map(|i| (i % 256) as u8).collect();
let hash = store.put(&audio_data).await.unwrap();
let reader = Arc::new(FileReader::new(store));
// Register manifest for the test file
// The manifest represents the ORIGINAL file in CAS, with audio starting at offset 42
reader.register_manifest(ChunkManifest {
file_id: FileId(1),
total_size: 42 + 100 * 1024, // Original file size (42 byte header + 100KB audio)
mtime: 0,
chunks: vec![ChunkRef {
hash,
offset: 42, // Audio starts at offset 42 in the original file
size: audio_data.len() as u32,
}],
});
let file_id = db
.upsert_file_with_layout(
&OriginId::from("test"),
Path::new("/test.flac"),
&VirtualPath::new("/Test Artist/Test Album/01 - Test Track.flac"),
&make_test_metadata(),
UNIX_EPOCH,
42 + 100 * 1024,
Some(&make_test_layout()),
None,
)
.unwrap();
(dir, db, registry, reader, file_id)
}
#[tokio::test]
async fn test_read_header_region() {
let (_dir, db, registry, reader, file_id) = setup_test_env().await;
let overlay = OverlayReader::new(db, registry, reader);
// Read first 100 bytes (should be from synthetic header)
let result = overlay.read(file_id, 0, 100).await.unwrap();
// Should return data (synthetic header)
assert!(!result.is_empty());
assert!(result.len() <= 100);
// FLAC files start with "fLaC" magic
assert_eq!(&result[0..4], b"fLaC");
}
#[tokio::test]
async fn test_read_audio_region() {
let (_dir, db, registry, reader, file_id) = setup_test_env().await;
let overlay = OverlayReader::new(db.clone(), registry.clone(), reader.clone());
// First, get the actual header size by reading it
let _header_result = overlay.read(file_id, 0, 64 * 1024).await.unwrap();
// Get the layout to know where audio starts in virtual file
let layout = db.get_format_layout(file_id).unwrap().unwrap();
let metadata = db.get_file_metadata_row(file_id).unwrap();
let handler = registry.get_by_format("flac").unwrap();
let header = handler.synthesize(&metadata, &layout).unwrap();
let header_len = header.len() as u64;
// Read from well into the audio region
let audio_offset = header_len + 1000;
let result = overlay.read(file_id, audio_offset, 1000).await.unwrap();
// Should return audio data
assert!(!result.is_empty());
}
#[tokio::test]
async fn test_read_boundary() {
let (_dir, db, registry, reader, file_id) = setup_test_env().await;
let overlay = OverlayReader::new(db.clone(), registry.clone(), reader.clone());
// Get the actual header size
let layout = db.get_format_layout(file_id).unwrap().unwrap();
let metadata = db.get_file_metadata_row(file_id).unwrap();
let handler = registry.get_by_format("flac").unwrap();
let header = handler.synthesize(&metadata, &layout).unwrap();
let header_len = header.len() as u64;
// Read across the header/audio boundary
let boundary_offset = header_len - 50;
let result = overlay.read(file_id, boundary_offset, 100).await.unwrap();
// Should return 100 bytes spanning both regions
assert_eq!(result.len(), 100);
// First 50 bytes should be from header
assert_eq!(&result[0..50], &header[(header_len - 50) as usize..]);
}
#[tokio::test]
async fn test_passthrough() {
let dir = TempDir::new().unwrap();
let db = Arc::new(Database::open_memory().unwrap());
let registry = Arc::new(FormatHandlerRegistry::new());
let cas_config = CasConfig {
chunks_dir: dir.path().join("chunks"),
..Default::default()
};
let store = Arc::new(CasStore::open(cas_config).await.unwrap());
let test_data = b"Hello, World! This is test data.";
let hash = store.put(test_data).await.unwrap();
// Insert file WITHOUT format_layout first to get the file_id
let file_id = db
.upsert_file(
&OriginId::from("test"),
Path::new("/test.txt"),
&VirtualPath::new("/test.txt"),
&AudioMeta::default(),
UNIX_EPOCH,
test_data.len() as u64,
)
.unwrap();
let reader = Arc::new(FileReader::new(store));
// Register manifest with the actual file_id from database
reader.register_manifest(ChunkManifest {
file_id,
total_size: test_data.len() as u64,
mtime: 0,
chunks: vec![ChunkRef {
hash,
offset: 0,
size: test_data.len() as u32,
}],
});
let overlay = OverlayReader::new(db, registry, reader);
let result = overlay
.read(file_id, 0, test_data.len() as u32)
.await
.unwrap();
assert_eq!(&result[..], test_data);
}
#[tokio::test]
async fn test_estimate_virtual_size() {
let (_dir, db, registry, reader, file_id) = setup_test_env().await;
let overlay = OverlayReader::new(db, registry, reader);
// Should return estimated size
let size = overlay.estimate_virtual_size(file_id).unwrap();
assert!(size.is_some());
let virtual_size = size.unwrap();
// Virtual size should be header + audio (100KB audio)
assert!(virtual_size > 100 * 1024);
}
#[tokio::test]
async fn test_estimate_virtual_size_passthrough() {
let dir = TempDir::new().unwrap();
let db = Arc::new(Database::open_memory().unwrap());
let registry = Arc::new(FormatHandlerRegistry::new());
let cas_config = CasConfig {
chunks_dir: dir.path().join("chunks"),
..Default::default()
};
let store = Arc::new(CasStore::open(cas_config).await.unwrap());
let reader = Arc::new(FileReader::new(store));
// Insert file WITHOUT format_layout
let file_id = db
.upsert_file(
&OriginId::from("test"),
Path::new("/test.txt"),
&VirtualPath::new("/test.txt"),
&AudioMeta::default(),
UNIX_EPOCH,
1000,
)
.unwrap();
let overlay = OverlayReader::new(db, registry, reader);
// Should return None for passthrough
let size = overlay.estimate_virtual_size(file_id).unwrap();
assert!(size.is_none());
}
#[tokio::test]
async fn test_read_eof() {
let (_dir, db, registry, reader, file_id) = setup_test_env().await;
let overlay = OverlayReader::new(db, registry, reader);
// Read past EOF
let result = overlay.read(file_id, 1_000_000, 100).await.unwrap();
assert!(result.is_empty());
}
}
+44
View File
@@ -20,6 +20,32 @@ CREATE TABLE IF NOT EXISTS files (
bitrate INTEGER, bitrate INTEGER,
sample_rate INTEGER, sample_rate INTEGER,
format TEXT, format TEXT,
track_total INTEGER,
disc_total INTEGER,
date TEXT,
composer TEXT,
comment TEXT,
lyrics TEXT,
copyright TEXT,
compilation INTEGER,
artist_sort TEXT,
album_artist_sort TEXT,
album_sort TEXT,
title_sort TEXT,
mb_recording_id TEXT,
mb_album_id TEXT,
mb_artist_id TEXT,
mb_album_artist_id TEXT,
mb_release_group_id TEXT,
replaygain_track_gain REAL,
replaygain_track_peak REAL,
replaygain_album_gain REAL,
replaygain_album_peak REAL,
channels INTEGER,
bits_per_sample INTEGER,
encoder TEXT,
custom_tags TEXT,
format_layout BLOB,
origin_mtime INTEGER NOT NULL, origin_mtime INTEGER NOT NULL,
origin_size INTEGER NOT NULL, origin_size INTEGER NOT NULL,
@@ -27,6 +53,10 @@ CREATE TABLE IF NOT EXISTS files (
chunk_manifest BLOB, chunk_manifest BLOB,
last_sync INTEGER NOT NULL DEFAULT (strftime('%s', 'now')), last_sync INTEGER NOT NULL DEFAULT (strftime('%s', 'now')),
trashed INTEGER NOT NULL DEFAULT 0,
original_path TEXT,
trashed_at INTEGER,
UNIQUE(origin_id, real_path) UNIQUE(origin_id, real_path)
); );
@@ -55,4 +85,18 @@ CREATE INDEX IF NOT EXISTS idx_files_content_hash ON files(content_hash);
CREATE INDEX IF NOT EXISTS idx_files_real ON files(origin_id, real_path); CREATE INDEX IF NOT EXISTS idx_files_real ON files(origin_id, real_path);
CREATE INDEX IF NOT EXISTS idx_files_origin ON files(origin_id); CREATE INDEX IF NOT EXISTS idx_files_origin ON files(origin_id);
CREATE INDEX IF NOT EXISTS idx_files_last_sync ON files(last_sync); CREATE INDEX IF NOT EXISTS idx_files_last_sync ON files(last_sync);
CREATE INDEX IF NOT EXISTS idx_files_mb_album ON files(mb_album_id);
CREATE INDEX IF NOT EXISTS idx_files_mb_artist ON files(mb_artist_id);
CREATE INDEX IF NOT EXISTS idx_files_genre ON files(genre);
CREATE INDEX IF NOT EXISTS idx_files_year ON files(year);
CREATE INDEX IF NOT EXISTS idx_files_composer ON files(composer);
CREATE INDEX IF NOT EXISTS idx_artwork_file ON artwork(file_id); CREATE INDEX IF NOT EXISTS idx_artwork_file ON artwork(file_id);
CREATE TABLE IF NOT EXISTS directories (
id INTEGER PRIMARY KEY,
path TEXT NOT NULL UNIQUE,
created_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))
);
CREATE INDEX IF NOT EXISTS idx_directories_path ON directories(path);
CREATE INDEX IF NOT EXISTS idx_files_trashed ON files(trashed) WHERE trashed = 1;
+776
View File
@@ -310,6 +310,477 @@ impl VirtualTree {
pub fn refresh_policy(&self) -> &RefreshPolicy { pub fn refresh_policy(&self) -> &RefreshPolicy {
&self.refresh_policy &self.refresh_policy
} }
pub fn path_to_inode_iter(&self) -> impl Iterator<Item = (&VirtualPath, &Inode)> {
self.path_to_inode.iter()
}
pub fn mkdir(&mut self, path: &VirtualPath) -> std::result::Result<Inode, RenameError> {
if self.path_to_inode.contains_key(path) {
return Err(RenameError::TargetExists);
}
let parent_path = std::path::Path::new(path.as_str())
.parent()
.map(|p| {
let s = p.to_string_lossy();
if s.is_empty() {
VirtualPath::new("/")
} else {
VirtualPath::new(s.into_owned())
}
})
.unwrap_or_else(|| VirtualPath::new("/"));
let parent_inode = self
.path_to_inode
.get(&parent_path)
.copied()
.ok_or(RenameError::ParentNotFound)?;
if !self
.nodes
.get(&parent_inode)
.map(|n| n.is_dir())
.unwrap_or(false)
{
return Err(RenameError::ParentNotFound);
}
let inode = self.alloc_inode();
let name = std::path::Path::new(path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
let dir_node = DirNode {
inode,
parent: parent_inode,
name: name.clone(),
children: BTreeMap::new(),
mtime: SystemTime::now(),
};
self.nodes.insert(inode, VirtualNode::Directory(dir_node));
self.path_to_inode.insert(path.clone(), inode);
if let Some(VirtualNode::Directory(parent)) = self.nodes.get_mut(&parent_inode) {
parent.children.insert(name, inode);
}
debug!(path = path.as_str(), inode, "created directory");
Ok(inode)
}
pub fn rename_file(
&mut self,
old_path: &VirtualPath,
new_path: &VirtualPath,
) -> std::result::Result<(), RenameError> {
let old_inode = self
.path_to_inode
.get(old_path)
.copied()
.ok_or(RenameError::SourceNotFound)?;
if self.path_to_inode.contains_key(new_path) {
return Err(RenameError::TargetExists);
}
let node = self
.nodes
.get(&old_inode)
.ok_or(RenameError::SourceNotFound)?;
if node.is_dir() {
return Err(RenameError::IsDirectory);
}
let new_parent_path = std::path::Path::new(new_path.as_str())
.parent()
.map(|p| {
let s = p.to_string_lossy();
if s.is_empty() {
VirtualPath::new("/")
} else {
VirtualPath::new(s.into_owned())
}
})
.unwrap_or_else(|| VirtualPath::new("/"));
let new_parent_inode = self
.path_to_inode
.get(&new_parent_path)
.copied()
.ok_or(RenameError::ParentNotFound)?;
if !self
.nodes
.get(&new_parent_inode)
.map(|n| n.is_dir())
.unwrap_or(false)
{
return Err(RenameError::ParentNotFound);
}
self.path_to_inode.remove(old_path);
let old_parent_path = std::path::Path::new(old_path.as_str())
.parent()
.map(|p| VirtualPath::new(p.to_string_lossy().into_owned()))
.unwrap_or_else(|| VirtualPath::new("/"));
if let Some(&old_parent_inode) = self.path_to_inode.get(&old_parent_path) {
if let Some(VirtualNode::Directory(dir)) = self.nodes.get_mut(&old_parent_inode) {
let old_name = std::path::Path::new(old_path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
dir.children.remove(&old_name);
}
}
let new_name = std::path::Path::new(new_path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
if let Some(VirtualNode::File(file)) = self.nodes.get_mut(&old_inode) {
file.name = new_name.clone();
}
if let Some(VirtualNode::Directory(dir)) = self.nodes.get_mut(&new_parent_inode) {
dir.children.insert(new_name, old_inode);
}
self.path_to_inode.insert(new_path.clone(), old_inode);
debug!(
old = old_path.as_str(),
new = new_path.as_str(),
inode = old_inode,
"renamed file"
);
Ok(())
}
pub fn rename_directory(
&mut self,
old_path: &VirtualPath,
new_path: &VirtualPath,
) -> std::result::Result<u64, RenameError> {
let old_inode = self
.path_to_inode
.get(old_path)
.copied()
.ok_or(RenameError::SourceNotFound)?;
if !self
.nodes
.get(&old_inode)
.map(|n| n.is_dir())
.unwrap_or(false)
{
return Err(RenameError::NotDirectory);
}
if self.path_to_inode.contains_key(new_path) {
return Err(RenameError::TargetExists);
}
let new_parent_path = std::path::Path::new(new_path.as_str())
.parent()
.map(|p| {
let s = p.to_string_lossy();
if s.is_empty() {
VirtualPath::new("/")
} else {
VirtualPath::new(s.into_owned())
}
})
.unwrap_or_else(|| VirtualPath::new("/"));
let new_parent_inode = self
.path_to_inode
.get(&new_parent_path)
.copied()
.ok_or(RenameError::ParentNotFound)?;
if !self
.nodes
.get(&new_parent_inode)
.map(|n| n.is_dir())
.unwrap_or(false)
{
return Err(RenameError::ParentNotFound);
}
let old_prefix = old_path.as_str();
let new_prefix = new_path.as_str();
let paths_to_update: Vec<(VirtualPath, Inode)> = self
.path_to_inode
.iter()
.filter(|(p, _)| p.as_str().starts_with(old_prefix))
.map(|(p, &i)| (p.clone(), i))
.collect();
let count = paths_to_update.len() as u64;
for (old_p, inode) in paths_to_update {
self.path_to_inode.remove(&old_p);
let new_p_str = format!("{}{}", new_prefix, &old_p.as_str()[old_prefix.len()..]);
let new_p = VirtualPath::new(&new_p_str);
self.path_to_inode.insert(new_p, inode);
}
let old_parent_path = std::path::Path::new(old_path.as_str())
.parent()
.map(|p| VirtualPath::new(p.to_string_lossy().into_owned()))
.unwrap_or_else(|| VirtualPath::new("/"));
if let Some(&old_parent_inode) = self.path_to_inode.get(&old_parent_path) {
if let Some(VirtualNode::Directory(dir)) = self.nodes.get_mut(&old_parent_inode) {
let old_name = std::path::Path::new(old_path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
dir.children.remove(&old_name);
}
}
let new_name = std::path::Path::new(new_path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
if let Some(VirtualNode::Directory(dir)) = self.nodes.get_mut(&old_inode) {
dir.name = new_name.clone();
dir.parent = new_parent_inode;
}
if let Some(VirtualNode::Directory(dir)) = self.nodes.get_mut(&new_parent_inode) {
dir.children.insert(new_name, old_inode);
}
debug!(
old = old_path.as_str(),
new = new_path.as_str(),
count,
"renamed directory"
);
Ok(count)
}
pub fn is_trash_path(path: &VirtualPath) -> bool {
path.as_str().starts_with("/.trash") || path.as_str() == "/.trash"
}
pub fn ensure_trash_dir(&mut self) -> Inode {
let trash_path = VirtualPath::new("/.trash");
if let Some(&inode) = self.path_to_inode.get(&trash_path) {
return inode;
}
let inode = self.alloc_inode();
let dir_node = DirNode {
inode,
parent: ROOT_INODE,
name: OsString::from(".trash"),
children: BTreeMap::new(),
mtime: SystemTime::now(),
};
self.nodes.insert(inode, VirtualNode::Directory(dir_node));
self.path_to_inode.insert(trash_path, inode);
if let Some(VirtualNode::Directory(root)) = self.nodes.get_mut(&ROOT_INODE) {
root.children.insert(OsString::from(".trash"), inode);
}
debug!(inode, "created .trash directory");
inode
}
pub fn mkdir_p(&mut self, path: &VirtualPath) -> std::result::Result<Inode, RenameError> {
if let Some(&existing) = self.path_to_inode.get(path) {
if self
.nodes
.get(&existing)
.map(|n| n.is_dir())
.unwrap_or(false)
{
return Ok(existing);
}
return Err(RenameError::TargetExists);
}
let components: Vec<&str> = path
.as_str()
.trim_start_matches('/')
.split('/')
.filter(|s| !s.is_empty())
.collect();
let mut current_inode = ROOT_INODE;
let mut current_path = String::from("/");
for component in &components {
if !current_path.ends_with('/') {
current_path.push('/');
}
current_path.push_str(component);
let vpath = VirtualPath::new(&current_path);
if let Some(&existing) = self.path_to_inode.get(&vpath) {
current_inode = existing;
} else {
let new_inode = self.alloc_inode();
let name = OsString::from(*component);
let dir_node = DirNode {
inode: new_inode,
parent: current_inode,
name: name.clone(),
children: BTreeMap::new(),
mtime: SystemTime::now(),
};
self.nodes
.insert(new_inode, VirtualNode::Directory(dir_node));
self.path_to_inode.insert(vpath, new_inode);
if let Some(VirtualNode::Directory(parent)) = self.nodes.get_mut(&current_inode) {
parent.children.insert(name, new_inode);
}
current_inode = new_inode;
}
}
Ok(current_inode)
}
pub fn remove_directory(&mut self, path: &VirtualPath) -> std::result::Result<(), RemoveError> {
let inode = self
.path_to_inode
.get(path)
.copied()
.ok_or(RemoveError::NotFound)?;
let node = self.nodes.get(&inode).ok_or(RemoveError::NotFound)?;
match node {
VirtualNode::File(_) => return Err(RemoveError::NotDirectory),
VirtualNode::Directory(dir) => {
if !dir.children.is_empty() {
return Err(RemoveError::NotEmpty);
}
}
}
let parent_path = std::path::Path::new(path.as_str())
.parent()
.map(|p| VirtualPath::new(p.to_string_lossy().into_owned()))
.unwrap_or_else(|| VirtualPath::new("/"));
if let Some(&parent_inode) = self.path_to_inode.get(&parent_path) {
if let Some(VirtualNode::Directory(parent)) = self.nodes.get_mut(&parent_inode) {
let name = std::path::Path::new(path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
parent.children.remove(&name);
}
}
self.path_to_inode.remove(path);
self.nodes.remove(&inode);
debug!(path = path.as_str(), inode, "removed directory");
Ok(())
}
pub fn remove_directory_recursive(
&mut self,
path: &VirtualPath,
) -> std::result::Result<Vec<FileId>, RemoveError> {
let inode = self
.path_to_inode
.get(path)
.copied()
.ok_or(RemoveError::NotFound)?;
if !self.nodes.get(&inode).map(|n| n.is_dir()).unwrap_or(false) {
return Err(RemoveError::NotDirectory);
}
let prefix = path.as_str();
let paths_to_remove: Vec<(VirtualPath, Inode)> = self
.path_to_inode
.iter()
.filter(|(p, _)| p.as_str().starts_with(prefix))
.map(|(p, &i)| (p.clone(), i))
.collect();
let mut removed_files = Vec::new();
for (p, ino) in &paths_to_remove {
if let Some(VirtualNode::File(f)) = self.nodes.get(ino) {
removed_files.push(f.file_id);
}
self.path_to_inode.remove(p);
self.nodes.remove(ino);
}
let parent_path = std::path::Path::new(path.as_str())
.parent()
.map(|p| VirtualPath::new(p.to_string_lossy().into_owned()))
.unwrap_or_else(|| VirtualPath::new("/"));
if let Some(&parent_inode) = self.path_to_inode.get(&parent_path) {
if let Some(VirtualNode::Directory(parent)) = self.nodes.get_mut(&parent_inode) {
let name = std::path::Path::new(path.as_str())
.file_name()
.map(|n| n.to_os_string())
.unwrap_or_default();
parent.children.remove(&name);
}
}
debug!(
path = path.as_str(),
file_count = removed_files.len(),
"removed directory recursively"
);
Ok(removed_files)
}
pub fn is_directory_empty(&self, path: &VirtualPath) -> Option<bool> {
let inode = self.path_to_inode.get(path)?;
if let Some(VirtualNode::Directory(dir)) = self.nodes.get(inode) {
Some(dir.children.is_empty())
} else {
None
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RemoveError {
NotFound,
NotEmpty,
NotDirectory,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RenameError {
SourceNotFound,
TargetExists,
ParentNotFound,
IsDirectory,
NotDirectory,
} }
impl Default for VirtualTree { impl Default for VirtualTree {
@@ -445,4 +916,309 @@ mod tests {
let tree = builder.build(); let tree = builder.build();
assert_eq!(tree.file_count(), 2); assert_eq!(tree.file_count(), 2);
} }
#[test]
fn test_rename_file() {
let mut tree = VirtualTree::new();
let old_path = VirtualPath::new("/Artist/Album/Track.flac");
let new_path = VirtualPath::new("/Artist/Album/Renamed.flac");
tree.insert_file(&make_file_meta(1, old_path.as_str()));
assert!(tree.get_by_path(&old_path).is_some());
tree.rename_file(&old_path, &new_path).unwrap();
assert!(tree.get_by_path(&old_path).is_none());
assert!(tree.get_by_path(&new_path).is_some());
}
#[test]
fn test_rename_file_to_new_dir() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Album/Track.flac"));
tree.mkdir(&VirtualPath::new("/New Artist")).unwrap();
tree.mkdir(&VirtualPath::new("/New Artist/New Album"))
.unwrap();
let result = tree.rename_file(
&VirtualPath::new("/Artist/Album/Track.flac"),
&VirtualPath::new("/New Artist/New Album/Track.flac"),
);
assert!(result.is_ok());
assert!(tree
.get_by_path(&VirtualPath::new("/New Artist/New Album/Track.flac"))
.is_some());
}
#[test]
fn test_rename_file_parent_not_found() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Album/Track.flac"));
let result = tree.rename_file(
&VirtualPath::new("/Artist/Album/Track.flac"),
&VirtualPath::new("/NonExistent/Album/Track.flac"),
);
assert_eq!(result, Err(RenameError::ParentNotFound));
}
#[test]
fn test_rename_file_target_exists() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/A/Track1.flac"));
tree.insert_file(&make_file_meta(2, "/A/Track2.flac"));
let result = tree.rename_file(
&VirtualPath::new("/A/Track1.flac"),
&VirtualPath::new("/A/Track2.flac"),
);
assert_eq!(result, Err(RenameError::TargetExists));
}
#[test]
fn test_rename_file_source_not_found() {
let mut tree = VirtualTree::new();
let result = tree.rename_file(
&VirtualPath::new("/Nonexistent.flac"),
&VirtualPath::new("/New.flac"),
);
assert_eq!(result, Err(RenameError::SourceNotFound));
}
#[test]
fn test_rename_directory() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Album/Track1.flac"));
tree.insert_file(&make_file_meta(2, "/Artist/Album/Track2.flac"));
tree.insert_file(&make_file_meta(3, "/Artist/Other/Track3.flac"));
let count = tree
.rename_directory(
&VirtualPath::new("/Artist"),
&VirtualPath::new("/Renamed Artist"),
)
.unwrap();
assert_eq!(count, 6);
assert!(tree.get_by_path(&VirtualPath::new("/Artist")).is_none());
assert!(tree
.get_by_path(&VirtualPath::new("/Renamed Artist"))
.is_some());
assert!(tree
.get_by_path(&VirtualPath::new("/Renamed Artist/Album/Track1.flac"))
.is_some());
assert!(tree
.get_by_path(&VirtualPath::new("/Renamed Artist/Album/Track2.flac"))
.is_some());
assert!(tree
.get_by_path(&VirtualPath::new("/Renamed Artist/Other/Track3.flac"))
.is_some());
}
#[test]
fn test_rename_directory_parent_not_found() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Album/Track.flac"));
let result = tree.rename_directory(
&VirtualPath::new("/Artist"),
&VirtualPath::new("/NonExistent/Renamed"),
);
assert_eq!(result, Err(RenameError::ParentNotFound));
}
#[test]
fn test_rename_directory_not_directory() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Track.flac"));
let result = tree.rename_directory(
&VirtualPath::new("/Artist/Track.flac"),
&VirtualPath::new("/New"),
);
assert_eq!(result, Err(RenameError::NotDirectory));
}
#[test]
fn test_mkdir() {
let mut tree = VirtualTree::new();
let inode = tree.mkdir(&VirtualPath::new("/NewDir")).unwrap();
assert!(inode > ROOT_INODE);
assert!(tree.get_by_path(&VirtualPath::new("/NewDir")).is_some());
assert!(tree
.get_by_path(&VirtualPath::new("/NewDir"))
.unwrap()
.is_dir());
}
#[test]
fn test_mkdir_nested() {
let mut tree = VirtualTree::new();
tree.mkdir(&VirtualPath::new("/A")).unwrap();
tree.mkdir(&VirtualPath::new("/A/B")).unwrap();
tree.mkdir(&VirtualPath::new("/A/B/C")).unwrap();
assert!(tree.get_by_path(&VirtualPath::new("/A/B/C")).is_some());
}
#[test]
fn test_mkdir_parent_not_found() {
let mut tree = VirtualTree::new();
let result = tree.mkdir(&VirtualPath::new("/A/B/C"));
assert_eq!(result, Err(RenameError::ParentNotFound));
}
#[test]
fn test_mkdir_already_exists() {
let mut tree = VirtualTree::new();
tree.mkdir(&VirtualPath::new("/Existing")).unwrap();
let result = tree.mkdir(&VirtualPath::new("/Existing"));
assert_eq!(result, Err(RenameError::TargetExists));
}
#[test]
fn test_is_trash_path() {
assert!(VirtualTree::is_trash_path(&VirtualPath::new("/.trash")));
assert!(VirtualTree::is_trash_path(&VirtualPath::new(
"/.trash/Artist/Track.flac"
)));
assert!(!VirtualTree::is_trash_path(&VirtualPath::new(
"/Artist/Track.flac"
)));
assert!(!VirtualTree::is_trash_path(&VirtualPath::new(
"/trash/Artist/Track.flac"
)));
}
#[test]
fn test_ensure_trash_dir() {
let mut tree = VirtualTree::new();
assert!(tree.get_by_path(&VirtualPath::new("/.trash")).is_none());
let inode = tree.ensure_trash_dir();
assert!(inode > ROOT_INODE);
let node = tree.get_by_path(&VirtualPath::new("/.trash"));
assert!(node.is_some());
assert!(node.unwrap().is_dir());
let inode2 = tree.ensure_trash_dir();
assert_eq!(inode, inode2);
}
#[test]
fn test_mkdir_p() {
let mut tree = VirtualTree::new();
tree.mkdir_p(&VirtualPath::new("/A/B/C/D")).unwrap();
assert!(tree.get_by_path(&VirtualPath::new("/A")).is_some());
assert!(tree.get_by_path(&VirtualPath::new("/A/B")).is_some());
assert!(tree.get_by_path(&VirtualPath::new("/A/B/C")).is_some());
assert!(tree.get_by_path(&VirtualPath::new("/A/B/C/D")).is_some());
}
#[test]
fn test_mkdir_p_partial_exists() {
let mut tree = VirtualTree::new();
tree.mkdir(&VirtualPath::new("/A")).unwrap();
tree.mkdir(&VirtualPath::new("/A/B")).unwrap();
tree.mkdir_p(&VirtualPath::new("/A/B/C/D")).unwrap();
assert!(tree.get_by_path(&VirtualPath::new("/A/B/C")).is_some());
assert!(tree.get_by_path(&VirtualPath::new("/A/B/C/D")).is_some());
}
#[test]
fn test_remove_directory_empty() {
let mut tree = VirtualTree::new();
tree.mkdir(&VirtualPath::new("/EmptyDir")).unwrap();
assert!(tree.get_by_path(&VirtualPath::new("/EmptyDir")).is_some());
tree.remove_directory(&VirtualPath::new("/EmptyDir"))
.unwrap();
assert!(tree.get_by_path(&VirtualPath::new("/EmptyDir")).is_none());
}
#[test]
fn test_remove_directory_not_empty() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Track.flac"));
let result = tree.remove_directory(&VirtualPath::new("/Artist"));
assert_eq!(result, Err(RemoveError::NotEmpty));
}
#[test]
fn test_remove_directory_not_found() {
let mut tree = VirtualTree::new();
let result = tree.remove_directory(&VirtualPath::new("/NonExistent"));
assert_eq!(result, Err(RemoveError::NotFound));
}
#[test]
fn test_remove_directory_is_file() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Track.flac"));
let result = tree.remove_directory(&VirtualPath::new("/Track.flac"));
assert_eq!(result, Err(RemoveError::NotDirectory));
}
#[test]
fn test_remove_directory_recursive() {
let mut tree = VirtualTree::new();
tree.insert_file(&make_file_meta(1, "/Artist/Album/Track1.flac"));
tree.insert_file(&make_file_meta(2, "/Artist/Album/Track2.flac"));
tree.insert_file(&make_file_meta(3, "/Artist/Other/Track3.flac"));
let removed = tree
.remove_directory_recursive(&VirtualPath::new("/Artist"))
.unwrap();
assert_eq!(removed.len(), 3);
assert!(tree.get_by_path(&VirtualPath::new("/Artist")).is_none());
}
#[test]
fn test_is_directory_empty() {
let mut tree = VirtualTree::new();
tree.mkdir(&VirtualPath::new("/Empty")).unwrap();
assert_eq!(
tree.is_directory_empty(&VirtualPath::new("/Empty")),
Some(true)
);
tree.insert_file(&make_file_meta(1, "/NonEmpty/Track.flac"));
assert_eq!(
tree.is_directory_empty(&VirtualPath::new("/NonEmpty")),
Some(false)
);
assert_eq!(
tree.is_directory_empty(&VirtualPath::new("/NonExistent")),
None
);
}
} }
+5
View File
@@ -14,10 +14,13 @@ musicfs-cache.path = "../musicfs-cache"
musicfs-cas.path = "../musicfs-cas" musicfs-cas.path = "../musicfs-cas"
musicfs-fuse.path = "../musicfs-fuse" musicfs-fuse.path = "../musicfs-fuse"
musicfs-metadata.path = "../musicfs-metadata" musicfs-metadata.path = "../musicfs-metadata"
musicfs-grpc.path = "../musicfs-grpc"
clap.workspace = true clap.workspace = true
tokio.workspace = true tokio.workspace = true
tokio-util.workspace = true tokio-util.workspace = true
tokio-stream.workspace = true
tonic.workspace = true
tracing.workspace = true tracing.workspace = true
tracing-subscriber.workspace = true tracing-subscriber.workspace = true
tracing-appender.workspace = true tracing-appender.workspace = true
@@ -26,6 +29,8 @@ dirs.workspace = true
toml.workspace = true toml.workspace = true
parking_lot.workspace = true parking_lot.workspace = true
libc.workspace = true libc.workspace = true
serde.workspace = true
serde_json.workspace = true
[target.'cfg(target_os = "linux")'.dependencies] [target.'cfg(target_os = "linux")'.dependencies]
tracing-journald.workspace = true tracing-journald.workspace = true
+476 -22
View File
@@ -1,6 +1,12 @@
mod metadata;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use clap::{Parser, Subcommand}; use clap::{Parser, Subcommand};
use musicfs_cache::TreeBuilder; use metadata::MetadataCommand;
use musicfs_cache::{
Database, FlacHandler, FormatHandlerRegistry, FormatLayout, Id3v2Handler, OverlayReader,
RenameError, TrashedFilter, TreeBuilder, VirtualTree,
};
use musicfs_cas::{CasConfig, CasStore, ContentFetcher, FileReader}; use musicfs_cas::{CasConfig, CasStore, ContentFetcher, FileReader};
use musicfs_core::{FileId, FileMeta, LoggingConfig, OriginId, RealPath, VirtualPath}; use musicfs_core::{FileId, FileMeta, LoggingConfig, OriginId, RealPath, VirtualPath};
use musicfs_fuse::MusicFs; use musicfs_fuse::MusicFs;
@@ -9,7 +15,7 @@ use musicfs_origins::{LocalOrigin, Origin};
use parking_lot::RwLock; use parking_lot::RwLock;
use std::collections::HashMap; use std::collections::HashMap;
use std::fs::File; use std::fs::File;
use std::io::Write; use std::io::{Read as _, Write};
use std::os::unix::io::AsRawFd; use std::os::unix::io::AsRawFd;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::Arc; use std::sync::Arc;
@@ -35,8 +41,8 @@ enum Commands {
Mount { Mount {
#[arg(short, long, help = "Config file path")] #[arg(short, long, help = "Config file path")]
config: Option<PathBuf>, config: Option<PathBuf>,
#[arg(help = "Mount point")] #[arg(help = "Mount point (optional if provided in config file)")]
mountpoint: PathBuf, mountpoint: Option<PathBuf>,
#[arg(short, long, help = "Source music directory")] #[arg(short, long, help = "Source music directory")]
origin: Option<PathBuf>, origin: Option<PathBuf>,
#[arg(short = 'd', long, help = "Cache directory")] #[arg(short = 'd', long, help = "Cache directory")]
@@ -66,6 +72,20 @@ enum Commands {
#[arg(short, long, default_value = "30")] #[arg(short, long, default_value = "30")]
timeout: u32, timeout: u32,
}, },
Trash {
#[arg(short, long, help = "Config file path")]
config: Option<PathBuf>,
#[arg(short = 'd', long, help = "Cache directory")]
cache_dir: Option<PathBuf>,
#[command(subcommand)]
command: TrashCommands,
},
Metadata {
#[arg(long, default_value = "http://[::1]:50051", help = "gRPC endpoint")]
endpoint: String,
#[command(subcommand)]
command: MetadataCommand,
},
} }
#[derive(Subcommand)] #[derive(Subcommand)]
@@ -88,6 +108,30 @@ enum OriginCommands {
Rescan { origin_id: String }, Rescan { origin_id: String },
} }
#[derive(Subcommand)]
enum TrashCommands {
List {
#[arg(long, help = "Filter by origin")]
origin: Option<String>,
#[arg(long, help = "Show files deleted within duration (e.g., 7d, 24h)")]
since: Option<String>,
#[arg(long, help = "Filter by path prefix")]
path: Option<String>,
},
Restore {
#[arg(help = "Path to restore (restores folder recursively)")]
path: Option<String>,
#[arg(long, help = "Restore all deleted files")]
all: bool,
},
Empty {
#[arg(long, help = "Delete files older than duration (e.g., 30d)")]
older_than: Option<String>,
#[arg(long, help = "Delete files matching pattern")]
pattern: Option<String>,
},
}
struct LockFile { struct LockFile {
_file: File, _file: File,
} }
@@ -127,6 +171,9 @@ fn main() -> Result<()> {
} else { } else {
let origin_path = origin let origin_path = origin
.context("--origin is required for mount if no config file is provided")?; .context("--origin is required for mount if no config file is provided")?;
let mp = mountpoint
.clone()
.context("mount point is required if no config file is provided")?;
let cache_dir = cache_dir.clone().unwrap_or_else(|| { let cache_dir = cache_dir.clone().unwrap_or_else(|| {
dirs::cache_dir() dirs::cache_dir()
.unwrap_or_else(|| PathBuf::from("/tmp")) .unwrap_or_else(|| PathBuf::from("/tmp"))
@@ -140,7 +187,7 @@ fn main() -> Result<()> {
); );
musicfs_core::Config { musicfs_core::Config {
mount_point: mountpoint.clone(), mount_point: mp,
cache_dir: cache_dir.clone(), cache_dir: cache_dir.clone(),
origins: vec![musicfs_core::OriginConfig { origins: vec![musicfs_core::OriginConfig {
id: "local".to_string(), id: "local".to_string(),
@@ -161,7 +208,9 @@ fn main() -> Result<()> {
if let Some(c_dir) = cache_dir { if let Some(c_dir) = cache_dir {
config.cache_dir = c_dir; config.cache_dir = c_dir;
} }
config.mount_point = mountpoint; if let Some(cli_mountpoint) = mountpoint {
config.mount_point = cli_mountpoint;
}
let _guard = init_logging(&config.logging)?; let _guard = init_logging(&config.logging)?;
run_mount(config) run_mount(config)
@@ -190,20 +239,41 @@ fn main() -> Result<()> {
init_basic_logging(&cli.log_level); init_basic_logging(&cli.log_level);
run_shutdown(graceful, timeout) run_shutdown(graceful, timeout)
} }
Commands::Trash {
config,
cache_dir,
command,
} => {
init_basic_logging(&cli.log_level);
run_trash(config, cache_dir, command)
} }
Commands::Metadata { endpoint, command } => {
init_basic_logging(&cli.log_level);
run_metadata(endpoint, command)
}
}
}
fn run_metadata(endpoint: String, command: MetadataCommand) -> Result<()> {
let runtime = tokio::runtime::Runtime::new().context("Failed to create Tokio runtime")?;
runtime.block_on(metadata::run_metadata(command, &endpoint))
} }
fn run_mount(config: musicfs_core::Config) -> Result<()> { fn run_mount(config: musicfs_core::Config) -> Result<()> {
let runtime = tokio::runtime::Runtime::new().context("Failed to create Tokio runtime")?; let runtime = tokio::runtime::Runtime::new().context("Failed to create Tokio runtime")?;
let handle = runtime.handle().clone(); let handle = runtime.handle().clone();
let (tree, reader) = runtime.block_on(async { let (tree, reader, db, overlay_reader) = runtime.block_on(async {
info!(mountpoint = ?config.mount_point, "Mount configuration"); info!(mountpoint = ?config.mount_point, "Mount configuration");
info!("Cache directory: {:?}", config.cache_dir); info!("Cache directory: {:?}", config.cache_dir);
std::fs::create_dir_all(&config.cache_dir).context("Failed to create cache directory")?; std::fs::create_dir_all(&config.cache_dir).context("Failed to create cache directory")?;
std::fs::create_dir_all(&config.mount_point).context("Failed to create mountpoint")?; std::fs::create_dir_all(&config.mount_point).context("Failed to create mountpoint")?;
let db_path = config.cache_dir.join("musicfs.db");
let db = Arc::new(Database::open(&db_path).context("Failed to open metadata database")?);
info!("Metadata database opened at {:?}", db_path);
let cas_config = CasConfig { let cas_config = CasConfig {
chunks_dir: config.cache_dir.join("chunks"), chunks_dir: config.cache_dir.join("chunks"),
..Default::default() ..Default::default()
@@ -218,6 +288,12 @@ fn run_mount(config: musicfs_core::Config) -> Result<()> {
let fetcher = Arc::new(ContentFetcher::new(store.clone())); let fetcher = Arc::new(ContentFetcher::new(store.clone()));
let mut files = Vec::new(); let mut files = Vec::new();
let mut format_registry = FormatHandlerRegistry::new();
format_registry.register(Arc::new(Id3v2Handler::new()));
format_registry.register(Arc::new(FlacHandler::new()));
let format_registry = Arc::new(format_registry);
info!("Format handler registry initialized (MP3, FLAC)");
for origin_cfg in &config.origins { for origin_cfg in &config.origins {
if !origin_cfg.enabled { if !origin_cfg.enabled {
continue; continue;
@@ -253,9 +329,11 @@ fn run_mount(config: musicfs_core::Config) -> Result<()> {
.unwrap(); .unwrap();
let origin_path = PathBuf::from(path_str); let origin_path = PathBuf::from(path_str);
info!("Scanning music files for origin {}...", origin_cfg.id); info!("Scanning music files for origin {}...", origin_cfg.id);
let origin_files = scan_music_files(&origin_path, &origin_id).await?; let origin_files =
scan_music_files(&origin_path, &origin_id, db.as_ref(), &format_registry)
.await?;
info!( info!(
"Fount {} music files for origin {}", "Found {} music files for origin {}",
origin_files.len(), origin_files.len(),
origin_cfg.id origin_cfg.id
); );
@@ -268,12 +346,34 @@ fn run_mount(config: musicfs_core::Config) -> Result<()> {
builder.add_file(file); builder.add_file(file);
fetcher.register_file(file.clone()); fetcher.register_file(file.clone());
} }
let tree = Arc::new(RwLock::new(builder.build())); let mut tree = builder.build();
info!("Virtual tree built");
let reader = Arc::new(FileReader::with_fetcher(store, fetcher)); let dirs = db.list_directories().unwrap_or_default();
for dir_path in &dirs {
if tree.get_by_path(dir_path).is_none() {
if let Err(e) = tree.mkdir(dir_path) {
debug!("Could not restore directory {:?}: {:?}", dir_path, e);
}
}
}
info!(
"Virtual tree built ({} files, {} user directories)",
tree.file_count(),
dirs.len()
);
Ok::<_, anyhow::Error>((tree, reader)) let tree = Arc::new(RwLock::new(tree));
let reader = Arc::new(FileReader::with_fetcher(store.clone(), fetcher));
// Create overlay reader for metadata synthesis
let overlay_reader = Arc::new(OverlayReader::new(
db.clone(),
format_registry,
reader.clone(),
));
Ok::<_, anyhow::Error>((tree, reader, db, overlay_reader))
})?; })?;
check_stale_mount(&config.mount_point)?; check_stale_mount(&config.mount_point)?;
@@ -283,7 +383,17 @@ fn run_mount(config: musicfs_core::Config) -> Result<()> {
.context("Failed to acquire lock — is another instance running?")?; .context("Failed to acquire lock — is another instance running?")?;
info!(lock_path = ?lock_path, "Lock acquired"); info!(lock_path = ?lock_path, "Lock acquired");
let fs = MusicFs::with_reader(tree, reader, handle.clone()); let pid_path = config.cache_dir.join("musicfs.pid");
std::fs::write(&pid_path, std::process::id().to_string())
.context("Failed to write PID file")?;
info!(pid_path = ?pid_path, "PID file written");
let tree_for_restore = tree.clone();
let db_for_restore = db.clone();
let fs = MusicFs::with_reader(tree, reader, handle.clone())
.with_db(db)
.with_overlay(overlay_reader);
info!("Mounting filesystem at {:?}", config.mount_point); info!("Mounting filesystem at {:?}", config.mount_point);
@@ -305,13 +415,22 @@ fn run_mount(config: musicfs_core::Config) -> Result<()> {
let mut sigterm = let mut sigterm =
tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?; tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?;
let mut sigint = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::interrupt())?; let mut sigint = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::interrupt())?;
let mut sighup = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::hangup())?;
loop {
tokio::select! { tokio::select! {
_ = sigterm.recv() => { _ = sigterm.recv() => {
info!("Received SIGTERM, shutting down"); info!("Received SIGTERM, shutting down");
break;
} }
_ = sigint.recv() => { _ = sigint.recv() => {
info!("Received SIGINT, shutting down"); info!("Received SIGINT, shutting down");
break;
}
_ = sighup.recv() => {
info!("Received SIGHUP, processing pending restores");
process_pending_restores(&tree_for_restore, &db_for_restore);
}
} }
} }
@@ -331,6 +450,8 @@ fn run_mount(config: musicfs_core::Config) -> Result<()> {
} }
info!("Unmounting filesystem"); info!("Unmounting filesystem");
drop(session); drop(session);
let _ = std::fs::remove_file(&pid_path);
info!("Shutdown complete"); info!("Shutdown complete");
Ok(()) Ok(())
@@ -398,6 +519,254 @@ fn run_shutdown(graceful: bool, timeout: u32) -> Result<()> {
Ok(()) Ok(())
} }
fn run_trash(
config: Option<PathBuf>,
cache_dir: Option<PathBuf>,
command: TrashCommands,
) -> Result<()> {
let cache_dir = if let Some(dir) = cache_dir {
dir
} else if let Some(cfg_path) = config {
let content = std::fs::read_to_string(&cfg_path).context("Failed to read config file")?;
let config: Value = toml::from_str(&content).context("Failed to parse config file")?;
PathBuf::from(
config
.get("cache_dir")
.and_then(|v| v.as_str())
.context("cache_dir not found in config")?,
)
} else {
return Err(anyhow::anyhow!(
"Either --config or --cache-dir must be provided"
));
};
let db_path = cache_dir.join("musicfs.db");
let db = Database::open(&db_path).context("Failed to open database")?;
match command {
TrashCommands::List {
origin,
since,
path,
} => {
let filter = TrashedFilter {
origin_id: origin.map(|s| OriginId::from(s.as_str())),
path_prefix: path,
since: since.and_then(|s| parse_duration(&s)),
};
let trashed = db.list_trashed(&filter)?;
if trashed.is_empty() {
println!("No deleted files found.");
return Ok(());
}
println!("{:<6} {:<20} PATH", "IDX", "DELETED");
println!("{}", "-".repeat(80));
for (i, file) in trashed.iter().enumerate() {
let ago = format_time_ago(file.trashed_at);
println!("{:<6} {:<20} {}", i, ago, file.original_path.as_str());
}
println!("\nTotal: {} deleted files", trashed.len());
}
TrashCommands::Restore { path, all } => {
let trashed = if all {
db.list_trashed(&TrashedFilter::default())?
} else if let Some(ref p) = path {
db.get_trashed_by_prefix(p)?
} else {
return Err(anyhow::anyhow!("Either --all or a path must be provided"));
};
if trashed.is_empty() {
println!("No files to restore.");
return Ok(());
}
let restore_file = cache_dir.join("pending_restore.txt");
let paths: Vec<String> = trashed
.iter()
.map(|f| f.original_path.as_str().to_string())
.collect();
std::fs::write(&restore_file, paths.join("\n"))?;
let pid_path = cache_dir.join("musicfs.pid");
if pid_path.exists() {
let pid_str = std::fs::read_to_string(&pid_path)?;
let pid: i32 = pid_str.trim().parse().context("Invalid PID in pid file")?;
std::env::set_var("MUSICFS_RESTORE_FILE", &restore_file);
unsafe {
libc::kill(pid, libc::SIGHUP);
}
println!("Restore signal sent for {} files.", trashed.len());
println!("Files will appear at their original locations.");
} else {
println!(
"Daemon not running. Marked {} files for restore.",
trashed.len()
);
println!("Start the daemon to complete restore, or restore manually with 'mv'.");
}
}
TrashCommands::Empty {
older_than,
pattern,
} => {
let filter = TrashedFilter {
since: older_than.and_then(|s| parse_duration(&s)),
path_prefix: pattern,
..Default::default()
};
let count = db.purge_trashed(&filter)?;
println!("Permanently deleted {} files from trash.", count);
}
}
Ok(())
}
fn parse_duration(s: &str) -> Option<std::time::Duration> {
let s = s.trim();
if s.is_empty() {
return None;
}
let (num_str, unit) = if s.ends_with('d') {
(&s[..s.len() - 1], 'd')
} else if s.ends_with('h') {
(&s[..s.len() - 1], 'h')
} else if s.ends_with('m') {
(&s[..s.len() - 1], 'm')
} else if s.ends_with('s') {
(&s[..s.len() - 1], 's')
} else {
return None;
};
let num: u64 = num_str.parse().ok()?;
let secs = match unit {
'd' => num * 86400,
'h' => num * 3600,
'm' => num * 60,
's' => num,
_ => return None,
};
Some(std::time::Duration::from_secs(secs))
}
fn format_time_ago(timestamp: i64) -> String {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
let diff = now - timestamp;
if diff < 60 {
format!("{}s ago", diff)
} else if diff < 3600 {
format!("{}m ago", diff / 60)
} else if diff < 86400 {
format!("{}h ago", diff / 3600)
} else {
format!("{}d ago", diff / 86400)
}
}
fn process_pending_restores(tree: &Arc<RwLock<VirtualTree>>, db: &Arc<Database>) {
let restore_file = match std::env::var("MUSICFS_RESTORE_FILE") {
Ok(path) => PathBuf::from(path),
Err(_) => {
debug!("MUSICFS_RESTORE_FILE not set, no restores to process");
return;
}
};
let restore_paths: Vec<String> = match std::fs::read_to_string(&restore_file) {
Ok(content) => content.lines().map(|s| s.to_string()).collect(),
Err(e) => {
warn!(error = %e, path = ?restore_file, "failed to read restore file");
return;
}
};
if restore_paths.is_empty() {
debug!("no paths to restore");
return;
}
let trashed = match db.list_trashed(&TrashedFilter::default()) {
Ok(files) => files,
Err(e) => {
warn!(error = %e, "failed to list trashed files");
return;
}
};
let mut restored = 0;
for original_path_str in &restore_paths {
let matching: Vec<_> = trashed
.iter()
.filter(|f| {
f.original_path.as_str() == original_path_str
|| f.original_path
.as_str()
.starts_with(&format!("{}/", original_path_str))
})
.collect();
for file in matching {
let parent_path = std::path::Path::new(file.original_path.as_str())
.parent()
.map(|p| {
let s = p.to_string_lossy();
if s.is_empty() {
VirtualPath::new("/")
} else {
VirtualPath::new(s.into_owned())
}
})
.unwrap_or_else(|| VirtualPath::new("/"));
let mut tree_guard = tree.write();
if let Err(e) = tree_guard.mkdir_p(&parent_path) {
if !matches!(e, RenameError::TargetExists) {
warn!(error = ?e, path = %parent_path.as_str(), "failed to create parent for restore");
continue;
}
}
if let Err(e) = tree_guard.rename_file(&file.current_path, &file.original_path) {
warn!(error = ?e, from = %file.current_path.as_str(), to = %file.original_path.as_str(), "failed to restore file");
continue;
}
drop(tree_guard);
if let Err(e) = db.update_virtual_path(file.file_id, &file.original_path) {
warn!(error = %e, "failed to update virtual path after restore");
}
if let Err(e) = db.unmark_trashed(file.file_id) {
warn!(error = %e, "failed to unmark trashed after restore");
}
restored += 1;
info!(path = %file.original_path.as_str(), "restored file from trash");
}
}
let _ = std::fs::remove_file(&restore_file);
info!(count = restored, "restore complete");
}
fn init_logging(config: &LoggingConfig) -> Result<WorkerGuard> { fn init_logging(config: &LoggingConfig) -> Result<WorkerGuard> {
std::fs::create_dir_all(&config.log_dir)?; std::fs::create_dir_all(&config.log_dir)?;
@@ -454,7 +823,12 @@ fn init_basic_logging(level: &str) {
.init(); .init();
} }
async fn scan_music_files(dir: &Path, origin_id: &OriginId) -> Result<Vec<FileMeta>> { async fn scan_music_files(
dir: &Path,
origin_id: &OriginId,
db: &Database,
format_registry: &Arc<FormatHandlerRegistry>,
) -> Result<Vec<FileMeta>> {
let parser = MetadataParser::new(); let parser = MetadataParser::new();
let mut files = Vec::new(); let mut files = Vec::new();
let mut file_id_counter = 1i64; let mut file_id_counter = 1i64;
@@ -464,6 +838,8 @@ async fn scan_music_files(dir: &Path, origin_id: &OriginId) -> Result<Vec<FileMe
dir, dir,
origin_id, origin_id,
&parser, &parser,
db,
format_registry,
&mut files, &mut files,
&mut file_id_counter, &mut file_id_counter,
) )
@@ -477,6 +853,8 @@ async fn scan_dir_recursive(
dir: &Path, dir: &Path,
origin_id: &OriginId, origin_id: &OriginId,
parser: &MetadataParser, parser: &MetadataParser,
db: &Database,
format_registry: &Arc<FormatHandlerRegistry>,
files: &mut Vec<FileMeta>, files: &mut Vec<FileMeta>,
id_counter: &mut i64, id_counter: &mut i64,
) -> Result<()> { ) -> Result<()> {
@@ -488,11 +866,19 @@ async fn scan_dir_recursive(
if metadata.is_dir() { if metadata.is_dir() {
Box::pin(scan_dir_recursive( Box::pin(scan_dir_recursive(
base, &path, origin_id, parser, files, id_counter, base,
&path,
origin_id,
parser,
db,
format_registry,
files,
id_counter,
)) ))
.await?; .await?;
} else if is_audio_file(&path) { } else if is_audio_file(&path) {
let relative_path = path.strip_prefix(base).unwrap_or(&path); let relative_path = path.strip_prefix(base).unwrap_or(&path);
let real_path_for_db = PathBuf::from("/").join(relative_path);
let audio_meta = match parser.parse_file(&path) { let audio_meta = match parser.parse_file(&path) {
Ok(meta) => Some(meta), Ok(meta) => Some(meta),
@@ -502,15 +888,41 @@ async fn scan_dir_recursive(
} }
}; };
let virtual_path = build_virtual_path(&path, audio_meta.as_ref()); let virtual_path = if let Ok(Some(stored_path)) =
db.get_file_by_real_path(origin_id, &real_path_for_db)
{
stored_path
} else {
build_virtual_path(&path, audio_meta.as_ref())
};
let real_path = RealPath {
origin_id: origin_id.clone(),
path: real_path_for_db.clone(),
};
let format_layout = analyze_format_layout(&path, metadata.len(), format_registry);
let file_id = db
.upsert_file_with_layout(
origin_id,
&real_path.path,
&virtual_path,
audio_meta.as_ref().unwrap_or(&Default::default()),
metadata.modified().unwrap_or(SystemTime::UNIX_EPOCH),
metadata.len(),
format_layout.as_ref(),
None,
)
.unwrap_or_else(|e| {
debug!("Failed to upsert file to DB: {}", e);
FileId(*id_counter)
});
let file_meta = FileMeta { let file_meta = FileMeta {
id: FileId(*id_counter), id: file_id,
virtual_path, virtual_path,
real_path: RealPath { real_path,
origin_id: origin_id.clone(),
path: PathBuf::from("/").join(relative_path),
},
size: metadata.len(), size: metadata.len(),
mtime: metadata.modified().unwrap_or(SystemTime::UNIX_EPOCH), mtime: metadata.modified().unwrap_or(SystemTime::UNIX_EPOCH),
content_hash: None, content_hash: None,
@@ -539,6 +951,48 @@ fn is_audio_file(path: &Path) -> bool {
) )
} }
const HEADER_READ_SIZE: usize = 65536;
fn analyze_format_layout(
path: &Path,
file_size: u64,
registry: &FormatHandlerRegistry,
) -> Option<FormatLayout> {
let ext = path.extension().and_then(|e| e.to_str())?;
let handler = registry.get_by_extension(&ext.to_lowercase())?;
let mut file = match std::fs::File::open(path) {
Ok(f) => f,
Err(e) => {
warn!("Failed to open file for format analysis {:?}: {}", path, e);
return None;
}
};
let mut buffer = vec![0u8; HEADER_READ_SIZE.min(file_size as usize)];
if let Err(e) = file.read_exact(&mut buffer) {
warn!(
"Failed to read header for format analysis {:?}: {}",
path, e
);
return None;
}
match handler.analyze(&buffer, file_size) {
Ok(layout) => {
debug!(
"Format layout analyzed for {:?}: audio_start={}, audio_end={}",
path, layout.audio_start, layout.audio_end
);
Some(layout)
}
Err(e) => {
debug!("Format analysis failed for {:?}: {}", path, e);
None
}
}
}
fn build_virtual_path(path: &Path, audio: Option<&musicfs_core::AudioMeta>) -> VirtualPath { fn build_virtual_path(path: &Path, audio: Option<&musicfs_core::AudioMeta>) -> VirtualPath {
if let Some(meta) = audio { if let Some(meta) = audio {
let artist = meta.artist.as_deref().unwrap_or("Unknown Artist"); let artist = meta.artist.as_deref().unwrap_or("Unknown Artist");
+632
View File
@@ -0,0 +1,632 @@
//! CLI subcommands for metadata overlay management.
use anyhow::{Context, Result};
use clap::Subcommand;
use musicfs_grpc::proto::musicfs::v1::{
metadata_service_client::MetadataServiceClient, ClearOverlayRequest, GetMetadataRequest,
ImportMetadataRequest, UpdateMetadataRequest,
};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::path::PathBuf;
use tokio_stream::StreamExt;
use tonic::transport::Channel;
use tracing::{debug, info};
/// Metadata overlay management subcommands.
#[derive(Subcommand)]
pub enum MetadataCommand {
/// Get metadata for a file (prints as JSON)
Get {
/// Virtual path of the file
path: String,
/// Print only a specific field
#[arg(long)]
field: Option<String>,
},
/// Set metadata fields for a file
Set {
/// Virtual path of the file
path: String,
/// Track title
#[arg(long)]
title: Option<String>,
/// Artist name
#[arg(long)]
artist: Option<String>,
/// Album name
#[arg(long)]
album: Option<String>,
/// Album artist
#[arg(long)]
album_artist: Option<String>,
/// Track number
#[arg(long)]
track: Option<u32>,
/// Disc number
#[arg(long)]
disc: Option<u32>,
/// Genre
#[arg(long)]
genre: Option<String>,
/// Date (YYYY-MM-DD or YYYY)
#[arg(long)]
date: Option<String>,
/// Composer
#[arg(long)]
composer: Option<String>,
/// Comment
#[arg(long)]
comment: Option<String>,
/// Set metadata from JSON string
#[arg(long, conflicts_with_all = ["title", "artist", "album", "album_artist", "track", "disc", "genre", "date", "composer", "comment"])]
json: Option<String>,
},
/// Clear metadata overlay (revert to original)
Clear {
/// Virtual path of the file
path: String,
},
/// Show difference between current and original metadata
Diff {
/// Virtual path of the file
path: String,
},
/// Import metadata from CSV or JSON file
Import {
/// Import file path
file: PathBuf,
/// File format (csv or json, auto-detected if not specified)
#[arg(long)]
format: Option<String>,
},
/// Export metadata to file
Export {
/// Output file path
#[arg(long, short)]
output: PathBuf,
/// Filter by search query
#[arg(long)]
query: Option<String>,
/// Output format (csv or json, auto-detected from extension)
#[arg(long)]
format: Option<String>,
},
}
/// Metadata fields for JSON serialization.
#[derive(Debug, Clone, Serialize, Deserialize, Default)]
pub struct MetadataFields {
#[serde(skip_serializing_if = "Option::is_none")]
pub file_id: Option<i64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub artist: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub album: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub album_artist: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub year: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub track: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub disc: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub genre: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub format: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub duration_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub bitrate: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub track_total: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub disc_total: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub date: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub composer: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub comment: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub lyrics: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub copyright: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub compilation: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub artist_sort: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub album_artist_sort: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub album_sort: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub title_sort: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mb_recording_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mb_album_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mb_artist_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mb_album_artist_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub mb_release_group_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub replaygain_track_gain: Option<f32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub replaygain_track_peak: Option<f32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub replaygain_album_gain: Option<f32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub replaygain_album_peak: Option<f32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub channels: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub bits_per_sample: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
pub encoder: Option<String>,
#[serde(skip_serializing_if = "HashMap::is_empty", default)]
pub custom_tags: HashMap<String, String>,
}
/// Execute a metadata subcommand.
pub async fn run_metadata(command: MetadataCommand, endpoint: &str) -> Result<()> {
match command {
MetadataCommand::Get { path, field } => run_get(endpoint, &path, field.as_deref()).await,
MetadataCommand::Set {
path,
title,
artist,
album,
album_artist,
track,
disc,
genre,
date,
composer,
comment,
json,
} => {
run_set(
endpoint,
&path,
title,
artist,
album,
album_artist,
track,
disc,
genre,
date,
composer,
comment,
json,
)
.await
}
MetadataCommand::Clear { path } => run_clear(endpoint, &path).await,
MetadataCommand::Diff { path } => run_diff(endpoint, &path).await,
MetadataCommand::Import { file, format } => run_import(endpoint, &file, format).await,
MetadataCommand::Export {
output,
query,
format,
} => run_export(endpoint, &output, query, format).await,
}
}
async fn connect(endpoint: &str) -> Result<MetadataServiceClient<Channel>> {
MetadataServiceClient::connect(endpoint.to_string())
.await
.context("Failed to connect to gRPC server")
}
async fn run_get(endpoint: &str, path: &str, field: Option<&str>) -> Result<()> {
let mut client = connect(endpoint).await?;
let response = client
.get_metadata(GetMetadataRequest {
virtual_path: path.to_string(),
})
.await
.context("GetMetadata RPC failed")?;
let meta = response.into_inner();
let fields = MetadataFields {
file_id: Some(meta.file_id),
title: meta.title,
artist: meta.artist,
album: meta.album,
album_artist: meta.album_artist,
year: meta.year,
track: meta.track,
disc: meta.disc,
genre: meta.genre,
format: meta.format,
duration_ms: meta.duration_ms,
bitrate: meta.bitrate,
track_total: meta.track_total,
disc_total: meta.disc_total,
date: meta.date,
composer: meta.composer,
comment: meta.comment,
lyrics: meta.lyrics,
copyright: meta.copyright,
compilation: meta.compilation,
artist_sort: meta.artist_sort,
album_artist_sort: meta.album_artist_sort,
album_sort: meta.album_sort,
title_sort: meta.title_sort,
mb_recording_id: meta.mb_recording_id,
mb_album_id: meta.mb_album_id,
mb_artist_id: meta.mb_artist_id,
mb_album_artist_id: meta.mb_album_artist_id,
mb_release_group_id: meta.mb_release_group_id,
replaygain_track_gain: meta.replaygain_track_gain,
replaygain_track_peak: meta.replaygain_track_peak,
replaygain_album_gain: meta.replaygain_album_gain,
replaygain_album_peak: meta.replaygain_album_peak,
channels: meta.channels,
bits_per_sample: meta.bits_per_sample,
encoder: meta.encoder,
custom_tags: meta.custom_tags,
};
if let Some(field_name) = field {
let value = get_field_value(&fields, field_name)?;
println!("{}", value);
} else {
let json = serde_json::to_string_pretty(&fields)?;
println!("{}", json);
}
Ok(())
}
fn get_field_value(fields: &MetadataFields, field_name: &str) -> Result<String> {
let value = match field_name {
"file_id" => fields.file_id.map(|v| v.to_string()),
"title" => fields.title.clone(),
"artist" => fields.artist.clone(),
"album" => fields.album.clone(),
"album_artist" => fields.album_artist.clone(),
"year" => fields.year.map(|v| v.to_string()),
"track" => fields.track.map(|v| v.to_string()),
"disc" => fields.disc.map(|v| v.to_string()),
"genre" => fields.genre.clone(),
"format" => fields.format.clone(),
"duration_ms" => fields.duration_ms.map(|v| v.to_string()),
"bitrate" => fields.bitrate.map(|v| v.to_string()),
"track_total" => fields.track_total.map(|v| v.to_string()),
"disc_total" => fields.disc_total.map(|v| v.to_string()),
"date" => fields.date.clone(),
"composer" => fields.composer.clone(),
"comment" => fields.comment.clone(),
"lyrics" => fields.lyrics.clone(),
"copyright" => fields.copyright.clone(),
"compilation" => fields.compilation.map(|v| v.to_string()),
"artist_sort" => fields.artist_sort.clone(),
"album_artist_sort" => fields.album_artist_sort.clone(),
"album_sort" => fields.album_sort.clone(),
"title_sort" => fields.title_sort.clone(),
"mb_recording_id" => fields.mb_recording_id.clone(),
"mb_album_id" => fields.mb_album_id.clone(),
"mb_artist_id" => fields.mb_artist_id.clone(),
"mb_album_artist_id" => fields.mb_album_artist_id.clone(),
"mb_release_group_id" => fields.mb_release_group_id.clone(),
"replaygain_track_gain" => fields.replaygain_track_gain.map(|v| v.to_string()),
"replaygain_track_peak" => fields.replaygain_track_peak.map(|v| v.to_string()),
"replaygain_album_gain" => fields.replaygain_album_gain.map(|v| v.to_string()),
"replaygain_album_peak" => fields.replaygain_album_peak.map(|v| v.to_string()),
"channels" => fields.channels.map(|v| v.to_string()),
"bits_per_sample" => fields.bits_per_sample.map(|v| v.to_string()),
"encoder" => fields.encoder.clone(),
_ => return Err(anyhow::anyhow!("Unknown field: {}", field_name)),
};
Ok(value.unwrap_or_else(|| "null".to_string()))
}
#[allow(clippy::too_many_arguments)]
async fn run_set(
endpoint: &str,
path: &str,
title: Option<String>,
artist: Option<String>,
album: Option<String>,
album_artist: Option<String>,
track: Option<u32>,
disc: Option<u32>,
genre: Option<String>,
date: Option<String>,
composer: Option<String>,
comment: Option<String>,
json: Option<String>,
) -> Result<()> {
let mut client = connect(endpoint).await?;
let get_response = client
.get_metadata(GetMetadataRequest {
virtual_path: path.to_string(),
})
.await
.context("Failed to get file metadata")?;
let file_id = get_response.into_inner().file_id;
let request = if let Some(json_str) = json {
let fields: MetadataFields =
serde_json::from_str(&json_str).context("Failed to parse JSON metadata")?;
UpdateMetadataRequest {
file_id,
title: fields.title,
artist: fields.artist,
album: fields.album,
album_artist: fields.album_artist,
track_number: fields.track,
disc_number: fields.disc,
genre: fields.genre,
date: fields.date,
composer: fields.composer,
comment: fields.comment,
lyrics: fields.lyrics,
copyright: fields.copyright,
compilation: fields.compilation,
artist_sort: fields.artist_sort,
album_artist_sort: fields.album_artist_sort,
album_sort: fields.album_sort,
title_sort: fields.title_sort,
mb_recording_id: fields.mb_recording_id,
mb_album_id: fields.mb_album_id,
mb_artist_id: fields.mb_artist_id,
replaygain_track_gain: fields.replaygain_track_gain,
replaygain_track_peak: fields.replaygain_track_peak,
replaygain_album_gain: fields.replaygain_album_gain,
replaygain_album_peak: fields.replaygain_album_peak,
custom_tags: fields.custom_tags,
}
} else {
UpdateMetadataRequest {
file_id,
title,
artist,
album,
album_artist,
track_number: track,
disc_number: disc,
genre,
date,
composer,
comment,
lyrics: None,
copyright: None,
compilation: None,
artist_sort: None,
album_artist_sort: None,
album_sort: None,
title_sort: None,
mb_recording_id: None,
mb_album_id: None,
mb_artist_id: None,
replaygain_track_gain: None,
replaygain_track_peak: None,
replaygain_album_gain: None,
replaygain_album_peak: None,
custom_tags: HashMap::new(),
}
};
let response = client
.update_metadata(request)
.await
.context("UpdateMetadata RPC failed")?;
let result = response.into_inner();
if result.success {
info!(file_id = result.file_id, "Metadata updated successfully");
println!("Metadata updated for file_id={}", result.file_id);
} else {
let msg = result
.error_message
.unwrap_or_else(|| "Unknown error".to_string());
anyhow::bail!("Failed to update metadata: {}", msg);
}
Ok(())
}
async fn run_clear(endpoint: &str, path: &str) -> Result<()> {
let mut client = connect(endpoint).await?;
let get_response = client
.get_metadata(GetMetadataRequest {
virtual_path: path.to_string(),
})
.await
.context("Failed to get file metadata")?;
let file_id = get_response.into_inner().file_id;
let response = client
.clear_overlay(ClearOverlayRequest { file_id })
.await
.context("ClearOverlay RPC failed")?;
let result = response.into_inner();
if result.success {
info!(file_id = result.file_id, "Overlay cleared successfully");
println!("Metadata overlay cleared for file_id={}", result.file_id);
} else {
let msg = result
.error_message
.unwrap_or_else(|| "Unknown error".to_string());
anyhow::bail!("Failed to clear overlay: {}", msg);
}
Ok(())
}
async fn run_diff(endpoint: &str, path: &str) -> Result<()> {
let mut client = connect(endpoint).await?;
let response = client
.get_metadata(GetMetadataRequest {
virtual_path: path.to_string(),
})
.await
.context("GetMetadata RPC failed")?;
let meta = response.into_inner();
debug!(file_id = meta.file_id, "Retrieved metadata for diff");
println!("Current metadata for: {}", path);
println!("---");
let fields = MetadataFields {
file_id: Some(meta.file_id),
title: meta.title,
artist: meta.artist,
album: meta.album,
album_artist: meta.album_artist,
year: meta.year,
track: meta.track,
disc: meta.disc,
genre: meta.genre,
format: meta.format,
duration_ms: meta.duration_ms,
bitrate: meta.bitrate,
track_total: meta.track_total,
disc_total: meta.disc_total,
date: meta.date,
composer: meta.composer,
comment: meta.comment,
lyrics: meta.lyrics,
copyright: meta.copyright,
compilation: meta.compilation,
artist_sort: meta.artist_sort,
album_artist_sort: meta.album_artist_sort,
album_sort: meta.album_sort,
title_sort: meta.title_sort,
mb_recording_id: meta.mb_recording_id,
mb_album_id: meta.mb_album_id,
mb_artist_id: meta.mb_artist_id,
mb_album_artist_id: meta.mb_album_artist_id,
mb_release_group_id: meta.mb_release_group_id,
replaygain_track_gain: meta.replaygain_track_gain,
replaygain_track_peak: meta.replaygain_track_peak,
replaygain_album_gain: meta.replaygain_album_gain,
replaygain_album_peak: meta.replaygain_album_peak,
channels: meta.channels,
bits_per_sample: meta.bits_per_sample,
encoder: meta.encoder,
custom_tags: meta.custom_tags,
};
let json = serde_json::to_string_pretty(&fields)?;
println!("{}", json);
println!("---");
println!("Note: Original metadata comparison requires re-parsing the source file.");
println!("Use 'musicfs metadata clear <path>' to revert to original metadata.");
Ok(())
}
async fn run_import(endpoint: &str, file: &PathBuf, format: Option<String>) -> Result<()> {
let mut client = connect(endpoint).await?;
let file_format = format.or_else(|| {
file.extension()
.and_then(|e| e.to_str())
.map(|s| s.to_lowercase())
});
let source_path = file
.canonicalize()
.unwrap_or_else(|_| file.clone())
.to_string_lossy()
.to_string();
info!(source_path = %source_path, format = ?file_format, "Starting metadata import");
let response = client
.import_metadata(ImportMetadataRequest {
source_path,
format: file_format,
})
.await
.context("ImportMetadata RPC failed")?;
let mut stream = response.into_inner();
let mut last_imported = 0u32;
let mut last_total = 0u32;
let mut errors = Vec::new();
while let Some(progress) = stream.next().await {
let progress = progress.context("Stream error")?;
last_imported = progress.imported;
last_total = progress.total;
if let Some(ref err) = progress.error_message {
let file = progress.current_file.as_deref().unwrap_or("unknown");
errors.push(format!("{}: {}", file, err));
}
if let Some(ref current) = progress.current_file {
print!(
"\rImporting: {}/{} - {}",
progress.imported, progress.total, current
);
std::io::Write::flush(&mut std::io::stdout())?;
}
}
println!();
println!(
"Import complete: {}/{} files imported",
last_imported, last_total
);
if !errors.is_empty() {
println!("\nErrors ({}):", errors.len());
for err in errors.iter().take(10) {
println!(" - {}", err);
}
if errors.len() > 10 {
println!(" ... and {} more", errors.len() - 10);
}
}
Ok(())
}
async fn run_export(
_endpoint: &str,
output: &PathBuf,
query: Option<String>,
format: Option<String>,
) -> Result<()> {
let output_format = format.or_else(|| {
output
.extension()
.and_then(|e| e.to_str())
.map(|s| s.to_lowercase())
});
println!("Export metadata to: {}", output.display());
if let Some(ref q) = query {
println!("Filter query: {}", q);
}
println!("Format: {}", output_format.as_deref().unwrap_or("json"));
println!();
println!("Note: Export requires file listing capability.");
println!("This feature requires integration with the Search service.");
println!(
"Use 'musicfs search <query>' to find files, then 'musicfs metadata get <path>' for each."
);
Ok(())
}
+24
View File
@@ -132,6 +132,30 @@ pub struct AudioMeta {
pub bitrate: Option<u32>, pub bitrate: Option<u32>,
pub sample_rate: Option<u32>, pub sample_rate: Option<u32>,
pub format: AudioFormat, pub format: AudioFormat,
pub track_total: Option<u32>,
pub disc_total: Option<u32>,
pub date: Option<String>,
pub composer: Option<String>,
pub comment: Option<String>,
pub lyrics: Option<String>,
pub copyright: Option<String>,
pub compilation: Option<bool>,
pub artist_sort: Option<String>,
pub album_artist_sort: Option<String>,
pub album_sort: Option<String>,
pub title_sort: Option<String>,
pub mb_recording_id: Option<String>,
pub mb_album_id: Option<String>,
pub mb_artist_id: Option<String>,
pub mb_album_artist_id: Option<String>,
pub mb_release_group_id: Option<String>,
pub replaygain_track_gain: Option<f32>,
pub replaygain_track_peak: Option<f32>,
pub replaygain_album_gain: Option<f32>,
pub replaygain_album_peak: Option<f32>,
pub channels: Option<u32>,
pub bits_per_sample: Option<u32>,
pub encoder: Option<String>,
} }
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
+359 -18
View File
@@ -3,9 +3,12 @@ use fuser::{
FileAttr, FileType, Filesystem, ReplyAttr, ReplyData, ReplyDirectory, ReplyEntry, ReplyOpen, FileAttr, FileType, Filesystem, ReplyAttr, ReplyData, ReplyDirectory, ReplyEntry, ReplyOpen,
Request, Request,
}; };
use musicfs_cache::{VirtualNode, VirtualTree, ROOT_INODE}; use musicfs_cache::{
Database, OverlayError, OverlayReader, RemoveError, RenameError, VirtualNode, VirtualTree,
ROOT_INODE,
};
use musicfs_cas::FileReader; use musicfs_cas::FileReader;
use musicfs_core::Result; use musicfs_core::{Result, VirtualPath};
use parking_lot::RwLock; use parking_lot::RwLock;
use std::collections::HashMap; use std::collections::HashMap;
use std::ffi::OsStr; use std::ffi::OsStr;
@@ -22,6 +25,8 @@ const SEARCH_QUERY_INODE_BASE: u64 = 0xFFFF_FFFF_0000_0100;
pub struct MusicFs { pub struct MusicFs {
tree: Arc<RwLock<VirtualTree>>, tree: Arc<RwLock<VirtualTree>>,
reader: Option<Arc<FileReader>>, reader: Option<Arc<FileReader>>,
db: Option<Arc<Database>>,
overlay_reader: Option<Arc<OverlayReader>>,
runtime_handle: Handle, runtime_handle: Handle,
search_ops: Option<SearchOps>, search_ops: Option<SearchOps>,
query_inodes: RwLock<HashMap<String, u64>>, query_inodes: RwLock<HashMap<String, u64>>,
@@ -36,6 +41,8 @@ impl MusicFs {
Self { Self {
tree, tree,
reader: None, reader: None,
db: None,
overlay_reader: None,
runtime_handle, runtime_handle,
search_ops: None, search_ops: None,
query_inodes: RwLock::new(HashMap::new()), query_inodes: RwLock::new(HashMap::new()),
@@ -54,6 +61,8 @@ impl MusicFs {
Self { Self {
tree, tree,
reader: Some(reader), reader: Some(reader),
db: None,
overlay_reader: None,
runtime_handle, runtime_handle,
search_ops: None, search_ops: None,
query_inodes: RwLock::new(HashMap::new()), query_inodes: RwLock::new(HashMap::new()),
@@ -64,11 +73,42 @@ impl MusicFs {
} }
} }
pub fn with_db(mut self, db: Arc<Database>) -> Self {
self.db = Some(db);
self
}
pub fn with_overlay(mut self, overlay: Arc<OverlayReader>) -> Self {
self.overlay_reader = Some(overlay);
self
}
pub fn with_search(mut self, search_ops: SearchOps) -> Self { pub fn with_search(mut self, search_ops: SearchOps) -> Self {
self.search_ops = Some(search_ops); self.search_ops = Some(search_ops);
self self
} }
fn resolve_path(&self, parent_inode: u64, name: &OsStr) -> Option<VirtualPath> {
let tree = self.tree.read();
let parent_path = self.inode_to_path_inner(&tree, parent_inode)?;
let name_str = name.to_string_lossy();
let full_path = if parent_path == "/" {
format!("/{}", name_str)
} else {
format!("{}/{}", parent_path, name_str)
};
Some(VirtualPath::new(full_path))
}
fn inode_to_path_inner(&self, tree: &VirtualTree, inode: u64) -> Option<String> {
for (path, &ino) in tree.path_to_inode_iter() {
if ino == inode {
return Some(path.as_str().to_string());
}
}
None
}
fn get_or_create_query_inode(&self, query: &str) -> u64 { fn get_or_create_query_inode(&self, query: &str) -> u64 {
let query_inodes = self.query_inodes.read(); let query_inodes = self.query_inodes.read();
if let Some(&inode) = query_inodes.get(query) { if let Some(&inode) = query_inodes.get(query) {
@@ -99,7 +139,6 @@ impl MusicFs {
info!("Mounting MusicFS at {:?}", mountpoint); info!("Mounting MusicFS at {:?}", mountpoint);
let options = vec![ let options = vec![
fuser::MountOption::RO,
fuser::MountOption::FSName("musicfs".to_string()), fuser::MountOption::FSName("musicfs".to_string()),
fuser::MountOption::AutoUnmount, fuser::MountOption::AutoUnmount,
fuser::MountOption::AllowOther, fuser::MountOption::AllowOther,
@@ -114,7 +153,6 @@ impl MusicFs {
info!("Mounting MusicFS at {:?}", mountpoint); info!("Mounting MusicFS at {:?}", mountpoint);
let options = vec![ let options = vec![
fuser::MountOption::RO,
fuser::MountOption::FSName("musicfs".to_string()), fuser::MountOption::FSName("musicfs".to_string()),
fuser::MountOption::AutoUnmount, fuser::MountOption::AutoUnmount,
fuser::MountOption::AllowOther, fuser::MountOption::AllowOther,
@@ -255,7 +293,27 @@ impl Filesystem for MusicFs {
if let Some(node) = tree.get(ino) { if let Some(node) = tree.get(ino) {
trace!(ino, "inode found in tree"); trace!(ino, "inode found in tree");
let attr = self.node_to_attr(node); let mut attr = self.node_to_attr(node);
if let VirtualNode::File(file) = node {
if let Some(ref overlay) = self.overlay_reader {
match overlay.estimate_virtual_size(file.file_id) {
Ok(Some(virtual_size)) => {
trace!(ino, file_id = ?file.file_id, virtual_size, "using overlay virtual size");
attr.size = virtual_size;
attr.blocks =
(virtual_size + BLOCK_SIZE as u64 - 1) / BLOCK_SIZE as u64;
}
Ok(None) => {
trace!(ino, file_id = ?file.file_id, "no overlay, using original size");
}
Err(e) => {
warn!(ino, file_id = ?file.file_id, error = %e, "overlay size estimation failed, using original");
}
}
}
}
reply.attr(&TTL, &attr); reply.attr(&TTL, &attr);
} else { } else {
trace!(ino, "inode not found"); trace!(ino, "inode not found");
@@ -385,6 +443,53 @@ impl Filesystem for MusicFs {
} }
}; };
let handle = self.runtime_handle.clone();
if let Some(ref overlay) = self.overlay_reader {
let overlay = overlay.clone();
let result = std::thread::scope(|_| {
handle.block_on(async {
tokio::time::timeout(
Duration::from_secs(30),
overlay.read(file_id, offset as u64, size),
)
.await
})
});
match result {
Ok(Ok(data)) => {
trace!(
ino,
offset,
size_bytes = size,
bytes_read = data.len(),
"overlay read successful"
);
reply.data(&data);
}
Ok(Err(e)) => {
let errno = match &e {
OverlayError::NotFound(_) => libc::ENOENT,
OverlayError::Database(_) => libc::EIO,
OverlayError::Handler(_) => libc::EIO,
OverlayError::Cas(_) => libc::EIO,
OverlayError::NoHandler(_) => libc::EIO,
};
warn!(ino, offset, size_bytes = size, error = %e, "overlay read failed");
reply.error(errno);
}
Err(_timeout) => {
warn!(
ino,
offset,
size_bytes = size,
"overlay read timed out after 30s"
);
reply.error(libc::EIO);
}
}
} else {
let Some(reader) = &self.reader else { let Some(reader) = &self.reader else {
trace!(ino, "no reader available"); trace!(ino, "no reader available");
reply.data(&[]); reply.data(&[]);
@@ -392,7 +497,6 @@ impl Filesystem for MusicFs {
}; };
let reader = reader.clone(); let reader = reader.clone();
let handle = self.runtime_handle.clone();
let result = std::thread::scope(|_| { let result = std::thread::scope(|_| {
handle.block_on(async { handle.block_on(async {
tokio::time::timeout( tokio::time::timeout(
@@ -424,6 +528,7 @@ impl Filesystem for MusicFs {
} }
} }
} }
}
#[instrument(level = "debug", skip(self, reply))] #[instrument(level = "debug", skip(self, reply))]
fn release( fn release(
@@ -471,34 +576,270 @@ impl Filesystem for MusicFs {
fn mkdir( fn mkdir(
&mut self, &mut self,
_req: &Request, _req: &Request,
_parent: u64, parent: u64,
_name: &OsStr, name: &OsStr,
_mode: u32, _mode: u32,
_umask: u32, _umask: u32,
reply: ReplyEntry, reply: ReplyEntry,
) { ) {
reply.error(libc::EROFS); let path = match self.resolve_path(parent, name) {
Some(p) => p,
None => {
reply.error(libc::ENOENT);
return;
}
};
let mut tree = self.tree.write();
match tree.mkdir(&path) {
Ok(inode) => {
if let Some(ref db) = self.db {
if let Err(e) = db.insert_directory(&path) {
warn!(error = %e, "failed to persist directory to database");
}
}
let attr = FileAttr {
ino: inode,
size: 0,
blocks: 0,
atime: SystemTime::now(),
mtime: SystemTime::now(),
ctime: SystemTime::now(),
crtime: SystemTime::now(),
kind: FileType::Directory,
perm: 0o755,
nlink: 2,
uid: self.uid,
gid: self.gid,
rdev: 0,
blksize: BLOCK_SIZE,
flags: 0,
};
debug!(path = %path.as_str(), inode, "mkdir successful");
reply.entry(&TTL, &attr, 0);
}
Err(RenameError::TargetExists) => reply.error(libc::EEXIST),
Err(RenameError::ParentNotFound) => reply.error(libc::ENOENT),
Err(_) => reply.error(libc::EIO),
}
} }
fn unlink(&mut self, _req: &Request, _parent: u64, _name: &OsStr, reply: fuser::ReplyEmpty) { fn unlink(&mut self, _req: &Request, parent: u64, name: &OsStr, reply: fuser::ReplyEmpty) {
reply.error(libc::EROFS); let path = match self.resolve_path(parent, name) {
Some(p) => p,
None => {
reply.error(libc::ENOENT);
return;
}
};
let (file_id, is_dir) = {
let tree = self.tree.read();
match tree.get_by_path(&path) {
Some(VirtualNode::File(f)) => (Some(f.file_id), false),
Some(VirtualNode::Directory(_)) => (None, true),
None => {
reply.error(libc::ENOENT);
return;
}
}
};
if is_dir {
reply.error(libc::EISDIR);
return;
} }
fn rmdir(&mut self, _req: &Request, _parent: u64, _name: &OsStr, reply: fuser::ReplyEmpty) { let trash_path = VirtualPath::new(format!("/.trash{}", path.as_str()));
reply.error(libc::EROFS);
{
let mut tree = self.tree.write();
tree.ensure_trash_dir();
let trash_parent = std::path::Path::new(trash_path.as_str())
.parent()
.map(|p| VirtualPath::new(p.to_string_lossy().into_owned()))
.unwrap_or_else(|| VirtualPath::new("/.trash"));
if let Err(e) = tree.mkdir_p(&trash_parent) {
if !matches!(e, RenameError::TargetExists) {
warn!(error = ?e, "failed to create trash parent directories");
reply.error(libc::EIO);
return;
}
}
if let Err(e) = tree.rename_file(&path, &trash_path) {
match e {
RenameError::SourceNotFound => reply.error(libc::ENOENT),
RenameError::TargetExists => reply.error(libc::EEXIST),
_ => reply.error(libc::EIO),
}
return;
}
}
if let (Some(ref db), Some(id)) = (&self.db, file_id) {
if let Err(e) = db.update_virtual_path(id, &trash_path) {
warn!(error = %e, "failed to update virtual path in database");
}
if let Err(e) = db.mark_trashed(id, &path) {
warn!(error = %e, "failed to mark file as trashed in database");
}
}
debug!(path = %path.as_str(), trash = %trash_path.as_str(), "file moved to trash");
reply.ok();
}
fn rmdir(&mut self, _req: &Request, parent: u64, name: &OsStr, reply: fuser::ReplyEmpty) {
let path = match self.resolve_path(parent, name) {
Some(p) => p,
None => {
reply.error(libc::ENOENT);
return;
}
};
if VirtualTree::is_trash_path(&path) {
reply.error(libc::EPERM);
return;
}
{
let mut tree = self.tree.write();
match tree.remove_directory(&path) {
Ok(()) => {}
Err(RemoveError::NotFound) => {
reply.error(libc::ENOENT);
return;
}
Err(RemoveError::NotEmpty) => {
reply.error(libc::ENOTEMPTY);
return;
}
Err(RemoveError::NotDirectory) => {
reply.error(libc::ENOTDIR);
return;
}
}
}
if let Some(ref db) = self.db {
if let Err(e) = db.delete_directory(&path) {
warn!(error = %e, "failed to delete directory from database");
}
}
debug!(path = %path.as_str(), "directory removed");
reply.ok();
} }
fn rename( fn rename(
&mut self, &mut self,
_req: &Request, _req: &Request,
_parent: u64, parent: u64,
_name: &OsStr, name: &OsStr,
_newparent: u64, newparent: u64,
_newname: &OsStr, newname: &OsStr,
_flags: u32, _flags: u32,
reply: fuser::ReplyEmpty, reply: fuser::ReplyEmpty,
) { ) {
reply.error(libc::EROFS); let old_path = match self.resolve_path(parent, name) {
Some(p) => p,
None => {
reply.error(libc::ENOENT);
return;
}
};
let new_path = match self.resolve_path(newparent, newname) {
Some(p) => p,
None => {
reply.error(libc::ENOENT);
return;
}
};
if old_path.as_str() == new_path.as_str() {
reply.ok();
return;
}
let is_dir = {
let tree = self.tree.read();
tree.get_by_path(&old_path)
.map(|n| n.is_dir())
.unwrap_or(false)
};
let result = if is_dir {
let mut tree = self.tree.write();
match tree.rename_directory(&old_path, &new_path) {
Ok(count) => {
if let Some(ref db) = self.db {
let old_prefix = if old_path.as_str().ends_with('/') {
old_path.as_str().to_string()
} else {
format!("{}/", old_path.as_str())
};
let new_prefix = if new_path.as_str().ends_with('/') {
new_path.as_str().to_string()
} else {
format!("{}/", new_path.as_str())
};
if let Err(e) = db.rename_directory(&old_prefix, &new_prefix) {
warn!(error = %e, "failed to persist file path rename to database");
}
if let Err(e) = db.rename_directories(&old_prefix, &new_prefix) {
warn!(error = %e, "failed to persist directory rename to database");
}
}
debug!(old = %old_path.as_str(), new = %new_path.as_str(), count, "directory renamed");
Ok(())
}
Err(e) => Err(e),
}
} else {
let file_id = {
let tree = self.tree.read();
match tree.get_by_path(&old_path) {
Some(VirtualNode::File(f)) => Some(f.file_id),
_ => None,
}
};
let mut tree = self.tree.write();
match tree.rename_file(&old_path, &new_path) {
Ok(()) => {
if let (Some(ref db), Some(id)) = (&self.db, file_id) {
if let Err(e) = db.update_virtual_path(id, &new_path) {
warn!(error = %e, "failed to persist file rename to database");
}
let was_in_trash = VirtualTree::is_trash_path(&old_path);
let now_in_trash = VirtualTree::is_trash_path(&new_path);
if was_in_trash && !now_in_trash {
if let Err(e) = db.unmark_trashed(id) {
warn!(error = %e, "failed to unmark trashed after restore");
}
debug!(path = %new_path.as_str(), "file restored from trash");
}
}
debug!(old = %old_path.as_str(), new = %new_path.as_str(), "file renamed");
Ok(())
}
Err(e) => Err(e),
}
};
match result {
Ok(()) => reply.ok(),
Err(RenameError::SourceNotFound) => reply.error(libc::ENOENT),
Err(RenameError::TargetExists) => reply.error(libc::EEXIST),
Err(RenameError::ParentNotFound) => reply.error(libc::ENOENT),
Err(RenameError::IsDirectory) => reply.error(libc::EISDIR),
Err(RenameError::NotDirectory) => reply.error(libc::ENOTDIR),
}
} }
fn create( fn create(
+2
View File
@@ -4,6 +4,7 @@ version.workspace = true
edition.workspace = true edition.workspace = true
[dependencies] [dependencies]
musicfs-cache = { path = "../musicfs-cache" }
musicfs-search = { path = "../musicfs-search" } musicfs-search = { path = "../musicfs-search" }
musicfs-core = { path = "../musicfs-core" } musicfs-core = { path = "../musicfs-core" }
tonic.workspace = true tonic.workspace = true
@@ -15,6 +16,7 @@ thiserror.workspace = true
serde.workspace = true serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
chrono.workspace = true chrono.workspace = true
csv = "1.3"
reqwest = { version = "0.11", features = ["json"] } reqwest = { version = "0.11", features = ["json"] }
hmac = "0.12" hmac = "0.12"
sha2 = "0.10" sha2 = "0.10"
+127
View File
@@ -16,6 +16,14 @@ service MusicFS {
rpc SubscribeEvents(EventFilter) returns (stream Event); rpc SubscribeEvents(EventFilter) returns (stream Event);
} }
service MetadataService {
rpc GetMetadata(GetMetadataRequest) returns (MetadataResponse);
rpc UpdateMetadata(UpdateMetadataRequest) returns (UpdateMetadataResponse);
rpc ClearOverlay(ClearOverlayRequest) returns (ClearOverlayResponse);
rpc BatchUpdateMetadata(BatchUpdateRequest) returns (stream BatchUpdateProgress);
rpc ImportMetadata(ImportMetadataRequest) returns (stream ImportProgress);
}
message Empty {} message Empty {}
message SearchRequest { message SearchRequest {
@@ -174,3 +182,122 @@ message Event {
optional int64 file_id = 5; optional int64 file_id = 5;
map<string, string> metadata = 6; map<string, string> metadata = 6;
} }
// MetadataService messages
message GetMetadataRequest {
string virtual_path = 1;
}
message MetadataResponse {
int64 file_id = 1;
optional string title = 2;
optional string artist = 3;
optional string album = 4;
optional string album_artist = 5;
optional uint32 year = 6;
optional uint32 track = 7;
optional uint32 disc = 8;
optional string genre = 9;
optional string format = 10;
optional uint64 duration_ms = 11;
optional uint64 bitrate = 12;
optional uint32 track_total = 13;
optional uint32 disc_total = 14;
optional string date = 15;
optional string composer = 16;
optional string comment = 17;
optional string lyrics = 18;
optional string copyright = 19;
optional bool compilation = 20;
optional string artist_sort = 21;
optional string album_artist_sort = 22;
optional string album_sort = 23;
optional string title_sort = 24;
optional string mb_recording_id = 25;
optional string mb_album_id = 26;
optional string mb_artist_id = 27;
optional string mb_album_artist_id = 28;
optional string mb_release_group_id = 29;
optional float replaygain_track_gain = 30;
optional float replaygain_track_peak = 31;
optional float replaygain_album_gain = 32;
optional float replaygain_album_peak = 33;
optional uint32 channels = 34;
optional uint32 bits_per_sample = 35;
optional string encoder = 36;
map<string, string> custom_tags = 50;
}
message UpdateMetadataRequest {
int64 file_id = 1;
optional string title = 2;
optional string artist = 3;
optional string album = 4;
optional string album_artist = 5;
optional uint32 track_number = 6;
optional uint32 disc_number = 7;
optional string date = 8;
optional string genre = 9;
optional string composer = 10;
optional string comment = 11;
optional string lyrics = 12;
optional string copyright = 13;
optional bool compilation = 14;
optional string artist_sort = 15;
optional string album_artist_sort = 16;
optional string album_sort = 17;
optional string title_sort = 18;
optional string mb_recording_id = 20;
optional string mb_album_id = 21;
optional string mb_artist_id = 22;
optional float replaygain_track_gain = 30;
optional float replaygain_track_peak = 31;
optional float replaygain_album_gain = 32;
optional float replaygain_album_peak = 33;
map<string, string> custom_tags = 50;
}
message UpdateMetadataResponse {
int64 file_id = 1;
bool success = 2;
optional string error_message = 3;
}
message ClearOverlayRequest {
int64 file_id = 1;
}
message ClearOverlayResponse {
int64 file_id = 1;
bool success = 2;
optional string error_message = 3;
}
message BatchUpdateRequest {
repeated BatchUpdateItem items = 1;
}
message BatchUpdateItem {
int64 file_id = 1;
UpdateMetadataRequest metadata = 2;
}
message BatchUpdateProgress {
uint32 completed = 1;
uint32 total = 2;
optional int64 current_file_id = 3;
optional string error_message = 4;
}
message ImportMetadataRequest {
string source_path = 1;
optional string format = 2;
}
message ImportProgress {
uint32 imported = 1;
uint32 total = 2;
optional string current_file = 3;
optional string error_message = 4;
}
+3
View File
@@ -6,10 +6,13 @@ pub mod proto {
} }
} }
mod metadata;
mod search_service; mod search_service;
mod server; mod server;
mod webhook; mod webhook;
pub use metadata::MetadataServiceImpl;
pub use proto::musicfs::v1::metadata_service_server::MetadataServiceServer;
pub use proto::musicfs::v1::music_fs_server::{MusicFs, MusicFsServer as MusicFsGrpcServer}; pub use proto::musicfs::v1::music_fs_server::{MusicFs, MusicFsServer as MusicFsGrpcServer};
pub use proto::musicfs::v1::*; pub use proto::musicfs::v1::*;
pub use search_service::SearchService; pub use search_service::SearchService;
+754
View File
@@ -0,0 +1,754 @@
//! MetadataService gRPC handlers for metadata overlay operations.
use crate::proto::musicfs::v1::{
metadata_service_server::MetadataService, BatchUpdateProgress, BatchUpdateRequest,
ClearOverlayRequest, ClearOverlayResponse, GetMetadataRequest, ImportMetadataRequest,
ImportProgress, MetadataResponse, UpdateMetadataRequest, UpdateMetadataResponse,
};
use musicfs_cache::Database;
use musicfs_core::{AudioMeta, FileId, VirtualPath};
use std::sync::Arc;
use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
use tonic::{Request, Response, Status};
use tracing::{debug, info, instrument, warn};
/// gRPC service implementation for metadata operations.
pub struct MetadataServiceImpl {
db: Arc<Database>,
}
impl MetadataServiceImpl {
/// Create a new MetadataServiceImpl with the given database.
pub fn new(db: Arc<Database>) -> Self {
Self { db }
}
/// Convert AudioMeta to MetadataResponse proto message.
fn audio_meta_to_response(file_id: FileId, meta: &AudioMeta) -> MetadataResponse {
MetadataResponse {
file_id: file_id.0,
title: meta.title.clone(),
artist: meta.artist.clone(),
album: meta.album.clone(),
album_artist: meta.album_artist.clone(),
year: meta.year,
track: meta.track,
disc: meta.disc,
genre: meta.genre.clone(),
format: Some(format!("{:?}", meta.format)),
duration_ms: meta.duration_ms,
bitrate: meta.bitrate.map(|b| b as u64),
track_total: meta.track_total,
disc_total: meta.disc_total,
date: meta.date.clone(),
composer: meta.composer.clone(),
comment: meta.comment.clone(),
lyrics: meta.lyrics.clone(),
copyright: meta.copyright.clone(),
compilation: meta.compilation,
artist_sort: meta.artist_sort.clone(),
album_artist_sort: meta.album_artist_sort.clone(),
album_sort: meta.album_sort.clone(),
title_sort: meta.title_sort.clone(),
mb_recording_id: meta.mb_recording_id.clone(),
mb_album_id: meta.mb_album_id.clone(),
mb_artist_id: meta.mb_artist_id.clone(),
mb_album_artist_id: meta.mb_album_artist_id.clone(),
mb_release_group_id: meta.mb_release_group_id.clone(),
replaygain_track_gain: meta.replaygain_track_gain,
replaygain_track_peak: meta.replaygain_track_peak,
replaygain_album_gain: meta.replaygain_album_gain,
replaygain_album_peak: meta.replaygain_album_peak,
channels: meta.channels,
bits_per_sample: meta.bits_per_sample,
encoder: meta.encoder.clone(),
custom_tags: Default::default(),
}
}
/// Convert UpdateMetadataRequest to AudioMeta for database update.
fn request_to_audio_meta(req: &UpdateMetadataRequest) -> AudioMeta {
AudioMeta {
title: req.title.clone(),
artist: req.artist.clone(),
album: req.album.clone(),
album_artist: req.album_artist.clone(),
genre: req.genre.clone(),
year: None,
track: req.track_number,
disc: req.disc_number,
duration_ms: None,
bitrate: None,
sample_rate: None,
format: musicfs_core::AudioFormat::Unknown,
track_total: None,
disc_total: None,
date: req.date.clone(),
composer: req.composer.clone(),
comment: req.comment.clone(),
lyrics: req.lyrics.clone(),
copyright: req.copyright.clone(),
compilation: req.compilation,
artist_sort: req.artist_sort.clone(),
album_artist_sort: req.album_artist_sort.clone(),
album_sort: req.album_sort.clone(),
title_sort: req.title_sort.clone(),
mb_recording_id: req.mb_recording_id.clone(),
mb_album_id: req.mb_album_id.clone(),
mb_artist_id: req.mb_artist_id.clone(),
mb_album_artist_id: None,
mb_release_group_id: None,
replaygain_track_gain: req.replaygain_track_gain,
replaygain_track_peak: req.replaygain_track_peak,
replaygain_album_gain: req.replaygain_album_gain,
replaygain_album_peak: req.replaygain_album_peak,
channels: None,
bits_per_sample: None,
encoder: None,
}
}
}
#[tonic::async_trait]
impl MetadataService for MetadataServiceImpl {
#[instrument(level = "debug", skip(self, request), fields(method = "get_metadata"))]
async fn get_metadata(
&self,
request: Request<GetMetadataRequest>,
) -> Result<Response<MetadataResponse>, Status> {
let req = request.into_inner();
debug!(virtual_path = %req.virtual_path, "GetMetadata request");
if req.virtual_path.is_empty() {
return Err(Status::invalid_argument("virtual_path cannot be empty"));
}
let vpath = VirtualPath::new(&req.virtual_path);
let file_meta = self
.db
.get_file_by_virtual_path(&vpath)
.map_err(|e| Status::internal(format!("Database error: {}", e)))?
.ok_or_else(|| Status::not_found(format!("File not found: {}", req.virtual_path)))?;
let audio_meta = self
.db
.get_file_metadata_row(file_meta.id)
.map_err(|e| Status::internal(format!("Failed to get metadata: {}", e)))?;
let response = Self::audio_meta_to_response(file_meta.id, &audio_meta);
Ok(Response::new(response))
}
#[instrument(
level = "info",
skip(self, request),
fields(method = "update_metadata")
)]
async fn update_metadata(
&self,
request: Request<UpdateMetadataRequest>,
) -> Result<Response<UpdateMetadataResponse>, Status> {
let req = request.into_inner();
let file_id = FileId(req.file_id);
info!(file_id = req.file_id, "UpdateMetadata request");
if req.file_id <= 0 {
return Err(Status::invalid_argument("file_id must be positive"));
}
let audio_meta = Self::request_to_audio_meta(&req);
match self.db.update_metadata(file_id, &audio_meta) {
Ok(()) => {
debug!(file_id = req.file_id, "Metadata updated successfully");
Ok(Response::new(UpdateMetadataResponse {
file_id: req.file_id,
success: true,
error_message: None,
}))
}
Err(e) => {
warn!(file_id = req.file_id, error = %e, "Failed to update metadata");
Ok(Response::new(UpdateMetadataResponse {
file_id: req.file_id,
success: false,
error_message: Some(e.to_string()),
}))
}
}
}
#[instrument(level = "info", skip(self, request), fields(method = "clear_overlay"))]
async fn clear_overlay(
&self,
request: Request<ClearOverlayRequest>,
) -> Result<Response<ClearOverlayResponse>, Status> {
let req = request.into_inner();
let file_id = FileId(req.file_id);
info!(file_id = req.file_id, "ClearOverlay request");
if req.file_id <= 0 {
return Err(Status::invalid_argument("file_id must be positive"));
}
match self.db.clear_overlay(file_id) {
Ok(()) => {
debug!(file_id = req.file_id, "Overlay cleared successfully");
Ok(Response::new(ClearOverlayResponse {
file_id: req.file_id,
success: true,
error_message: None,
}))
}
Err(e) => {
warn!(file_id = req.file_id, error = %e, "Failed to clear overlay");
Ok(Response::new(ClearOverlayResponse {
file_id: req.file_id,
success: false,
error_message: Some(e.to_string()),
}))
}
}
}
type BatchUpdateMetadataStream = ReceiverStream<Result<BatchUpdateProgress, Status>>;
#[instrument(
level = "info",
skip(self, request),
fields(method = "batch_update_metadata")
)]
async fn batch_update_metadata(
&self,
request: Request<BatchUpdateRequest>,
) -> Result<Response<Self::BatchUpdateMetadataStream>, Status> {
let req = request.into_inner();
let total = req.items.len() as u32;
info!(item_count = total, "BatchUpdateMetadata request");
let (tx, rx) = mpsc::channel(32);
let db = Arc::clone(&self.db);
tokio::spawn(async move {
for (i, item) in req.items.into_iter().enumerate() {
let file_id = FileId(item.file_id);
let completed = (i + 1) as u32;
let error_message = if let Some(ref metadata_req) = item.metadata {
let audio_meta = MetadataServiceImpl::request_to_audio_meta(metadata_req);
match db.update_metadata(file_id, &audio_meta) {
Ok(()) => None,
Err(e) => Some(e.to_string()),
}
} else {
Some("Missing metadata in batch item".to_string())
};
let progress = BatchUpdateProgress {
completed,
total,
current_file_id: Some(item.file_id),
error_message,
};
if tx.send(Ok(progress)).await.is_err() {
break;
}
}
});
Ok(Response::new(ReceiverStream::new(rx)))
}
type ImportMetadataStream = ReceiverStream<Result<ImportProgress, Status>>;
#[instrument(
level = "info",
skip(self, request),
fields(method = "import_metadata")
)]
async fn import_metadata(
&self,
request: Request<ImportMetadataRequest>,
) -> Result<Response<Self::ImportMetadataStream>, Status> {
let req = request.into_inner();
info!(source_path = %req.source_path, format = ?req.format, "ImportMetadata request");
if req.source_path.is_empty() {
return Err(Status::invalid_argument("source_path cannot be empty"));
}
let (tx, rx) = mpsc::channel(32);
let db = Arc::clone(&self.db);
let source_path = req.source_path.clone();
let format = req.format.clone();
tokio::spawn(async move {
let file_format = format.as_deref().unwrap_or_else(|| {
if source_path.ends_with(".csv") {
"csv"
} else if source_path.ends_with(".json") {
"json"
} else {
"unknown"
}
});
let content = match tokio::fs::read_to_string(&source_path).await {
Ok(c) => c,
Err(e) => {
let _ = tx
.send(Ok(ImportProgress {
imported: 0,
total: 0,
current_file: None,
error_message: Some(format!("Failed to read file: {}", e)),
}))
.await;
return;
}
};
let entries: Vec<ImportEntry> = match file_format {
"json" => match serde_json::from_str::<Vec<ImportEntry>>(&content) {
Ok(e) => e,
Err(e) => {
let _ = tx
.send(Ok(ImportProgress {
imported: 0,
total: 0,
current_file: None,
error_message: Some(format!("Failed to parse JSON: {}", e)),
}))
.await;
return;
}
},
"csv" => match parse_csv_entries(&content) {
Ok(e) => e,
Err(e) => {
let _ = tx
.send(Ok(ImportProgress {
imported: 0,
total: 0,
current_file: None,
error_message: Some(format!("Failed to parse CSV: {}", e)),
}))
.await;
return;
}
},
_ => {
let _ = tx
.send(Ok(ImportProgress {
imported: 0,
total: 0,
current_file: None,
error_message: Some(format!("Unsupported format: {}", file_format)),
}))
.await;
return;
}
};
let total = entries.len() as u32;
let mut imported = 0u32;
for entry in entries {
let vpath = VirtualPath::new(&entry.virtual_path);
let file_meta = match db.get_file_by_virtual_path(&vpath) {
Ok(Some(f)) => f,
Ok(None) => {
let progress = ImportProgress {
imported,
total,
current_file: Some(entry.virtual_path.clone()),
error_message: Some(format!("File not found: {}", entry.virtual_path)),
};
if tx.send(Ok(progress)).await.is_err() {
break;
}
continue;
}
Err(e) => {
let progress = ImportProgress {
imported,
total,
current_file: Some(entry.virtual_path.clone()),
error_message: Some(format!("Database error: {}", e)),
};
if tx.send(Ok(progress)).await.is_err() {
break;
}
continue;
}
};
let audio_meta = entry.to_audio_meta();
let error_message = match db.update_metadata(file_meta.id, &audio_meta) {
Ok(()) => {
imported += 1;
None
}
Err(e) => Some(e.to_string()),
};
let progress = ImportProgress {
imported,
total,
current_file: Some(entry.virtual_path),
error_message,
};
if tx.send(Ok(progress)).await.is_err() {
break;
}
}
});
Ok(Response::new(ReceiverStream::new(rx)))
}
}
/// Entry from import file (CSV or JSON).
#[derive(Debug, Clone, serde::Deserialize)]
struct ImportEntry {
virtual_path: String,
#[serde(default)]
title: Option<String>,
#[serde(default)]
artist: Option<String>,
#[serde(default)]
album: Option<String>,
#[serde(default)]
album_artist: Option<String>,
#[serde(default)]
genre: Option<String>,
#[serde(default)]
year: Option<u32>,
#[serde(default)]
track: Option<u32>,
#[serde(default)]
disc: Option<u32>,
#[serde(default)]
date: Option<String>,
#[serde(default)]
composer: Option<String>,
#[serde(default)]
comment: Option<String>,
}
impl ImportEntry {
fn to_audio_meta(&self) -> AudioMeta {
AudioMeta {
title: self.title.clone(),
artist: self.artist.clone(),
album: self.album.clone(),
album_artist: self.album_artist.clone(),
genre: self.genre.clone(),
year: self.year,
track: self.track,
disc: self.disc,
duration_ms: None,
bitrate: None,
sample_rate: None,
format: musicfs_core::AudioFormat::Unknown,
track_total: None,
disc_total: None,
date: self.date.clone(),
composer: self.composer.clone(),
comment: self.comment.clone(),
lyrics: None,
copyright: None,
compilation: None,
artist_sort: None,
album_artist_sort: None,
album_sort: None,
title_sort: None,
mb_recording_id: None,
mb_album_id: None,
mb_artist_id: None,
mb_album_artist_id: None,
mb_release_group_id: None,
replaygain_track_gain: None,
replaygain_track_peak: None,
replaygain_album_gain: None,
replaygain_album_peak: None,
channels: None,
bits_per_sample: None,
encoder: None,
}
}
}
/// Parse CSV content into ImportEntry list.
fn parse_csv_entries(content: &str) -> Result<Vec<ImportEntry>, String> {
let mut reader = csv::Reader::from_reader(content.as_bytes());
let mut entries = Vec::new();
for result in reader.deserialize() {
let entry: ImportEntry = result.map_err(|e| format!("CSV parse error: {}", e))?;
entries.push(entry);
}
Ok(entries)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::proto::musicfs::v1::BatchUpdateItem;
use musicfs_core::{AudioFormat, OriginId};
use std::path::Path;
use std::time::UNIX_EPOCH;
use tempfile::TempDir;
use tokio_stream::StreamExt;
fn create_test_db() -> (TempDir, Arc<Database>) {
let dir = TempDir::new().unwrap();
let db = Arc::new(Database::open_memory().unwrap());
(dir, db)
}
fn insert_test_file(db: &Database, vpath: &str) -> FileId {
let real_path = format!("/music{}", vpath);
db.upsert_file(
&OriginId::from("local"),
Path::new(&real_path),
&VirtualPath::new(vpath),
&AudioMeta {
title: Some("Test Track".to_string()),
artist: Some("Test Artist".to_string()),
album: Some("Test Album".to_string()),
format: AudioFormat::Flac,
..Default::default()
},
UNIX_EPOCH,
1000,
)
.unwrap()
}
#[tokio::test]
async fn test_get_metadata_success() {
let (_dir, db) = create_test_db();
let vpath = "/Artist/Album/Track.flac";
insert_test_file(&db, vpath);
let service = MetadataServiceImpl::new(db);
let request = Request::new(GetMetadataRequest {
virtual_path: vpath.to_string(),
});
let response = service.get_metadata(request).await.unwrap();
let meta = response.into_inner();
assert_eq!(meta.title, Some("Test Track".to_string()));
assert_eq!(meta.artist, Some("Test Artist".to_string()));
assert_eq!(meta.album, Some("Test Album".to_string()));
}
#[tokio::test]
async fn test_get_metadata_not_found() {
let (_dir, db) = create_test_db();
let service = MetadataServiceImpl::new(db);
let request = Request::new(GetMetadataRequest {
virtual_path: "/nonexistent.flac".to_string(),
});
let result = service.get_metadata(request).await;
assert!(result.is_err());
assert_eq!(result.unwrap_err().code(), tonic::Code::NotFound);
}
#[tokio::test]
async fn test_get_metadata_empty_path() {
let (_dir, db) = create_test_db();
let service = MetadataServiceImpl::new(db);
let request = Request::new(GetMetadataRequest {
virtual_path: String::new(),
});
let result = service.get_metadata(request).await;
assert!(result.is_err());
assert_eq!(result.unwrap_err().code(), tonic::Code::InvalidArgument);
}
#[tokio::test]
async fn test_update_metadata_success() {
let (_dir, db) = create_test_db();
let vpath = "/Artist/Album/Track.flac";
let file_id = insert_test_file(&db, vpath);
let service = MetadataServiceImpl::new(db.clone());
let request = Request::new(UpdateMetadataRequest {
file_id: file_id.0,
title: Some("Updated Title".to_string()),
artist: Some("Updated Artist".to_string()),
..Default::default()
});
let response = service.update_metadata(request).await.unwrap();
let result = response.into_inner();
assert!(result.success);
assert!(result.error_message.is_none());
let meta = db.get_file_metadata_row(file_id).unwrap();
assert_eq!(meta.title, Some("Updated Title".to_string()));
assert_eq!(meta.artist, Some("Updated Artist".to_string()));
}
#[tokio::test]
async fn test_update_metadata_invalid_id() {
let (_dir, db) = create_test_db();
let service = MetadataServiceImpl::new(db);
let request = Request::new(UpdateMetadataRequest {
file_id: 0,
title: Some("Title".to_string()),
..Default::default()
});
let result = service.update_metadata(request).await;
assert!(result.is_err());
assert_eq!(result.unwrap_err().code(), tonic::Code::InvalidArgument);
}
#[tokio::test]
async fn test_clear_overlay_success() {
let (_dir, db) = create_test_db();
let vpath = "/Artist/Album/Track.flac";
let file_id = insert_test_file(&db, vpath);
let service = MetadataServiceImpl::new(db.clone());
let request = Request::new(ClearOverlayRequest { file_id: file_id.0 });
let response = service.clear_overlay(request).await.unwrap();
let result = response.into_inner();
assert!(result.success);
assert!(result.error_message.is_none());
let meta = db.get_file_metadata_row(file_id).unwrap();
assert!(meta.title.is_none());
assert!(meta.artist.is_none());
}
#[tokio::test]
async fn test_clear_overlay_invalid_id() {
let (_dir, db) = create_test_db();
let service = MetadataServiceImpl::new(db);
let request = Request::new(ClearOverlayRequest { file_id: -1 });
let result = service.clear_overlay(request).await;
assert!(result.is_err());
assert_eq!(result.unwrap_err().code(), tonic::Code::InvalidArgument);
}
#[tokio::test]
async fn test_batch_update_metadata() {
let (_dir, db) = create_test_db();
let file_id1 = insert_test_file(&db, "/Track1.flac");
let file_id2 = insert_test_file(&db, "/Track2.flac");
let service = MetadataServiceImpl::new(db.clone());
let request = Request::new(BatchUpdateRequest {
items: vec![
BatchUpdateItem {
file_id: file_id1.0,
metadata: Some(UpdateMetadataRequest {
file_id: file_id1.0,
title: Some("Batch Title 1".to_string()),
..Default::default()
}),
},
BatchUpdateItem {
file_id: file_id2.0,
metadata: Some(UpdateMetadataRequest {
file_id: file_id2.0,
title: Some("Batch Title 2".to_string()),
..Default::default()
}),
},
],
});
let response = service.batch_update_metadata(request).await.unwrap();
let mut stream = response.into_inner();
let mut progress_count = 0;
while let Some(Ok(result)) = stream.next().await {
progress_count += 1;
assert!(result.error_message.is_none());
}
assert_eq!(progress_count, 2);
let meta1 = db.get_file_metadata_row(file_id1).unwrap();
assert_eq!(meta1.title, Some("Batch Title 1".to_string()));
let meta2 = db.get_file_metadata_row(file_id2).unwrap();
assert_eq!(meta2.title, Some("Batch Title 2".to_string()));
}
#[tokio::test]
async fn test_import_metadata_empty_path() {
let (_dir, db) = create_test_db();
let service = MetadataServiceImpl::new(db);
let request = Request::new(ImportMetadataRequest {
source_path: String::new(),
format: None,
});
let result = service.import_metadata(request).await;
assert!(result.is_err());
assert_eq!(result.unwrap_err().code(), tonic::Code::InvalidArgument);
}
#[test]
fn test_parse_csv_entries() {
let csv_content = r#"virtual_path,title,artist,album
/Track1.flac,Title 1,Artist 1,Album 1
/Track2.flac,Title 2,Artist 2,Album 2"#;
let entries = parse_csv_entries(csv_content).unwrap();
assert_eq!(entries.len(), 2);
assert_eq!(entries[0].virtual_path, "/Track1.flac");
assert_eq!(entries[0].title, Some("Title 1".to_string()));
assert_eq!(entries[1].virtual_path, "/Track2.flac");
assert_eq!(entries[1].artist, Some("Artist 2".to_string()));
}
#[test]
fn test_import_entry_to_audio_meta() {
let entry = ImportEntry {
virtual_path: "/test.flac".to_string(),
title: Some("Test".to_string()),
artist: Some("Artist".to_string()),
album: None,
album_artist: None,
genre: Some("Rock".to_string()),
year: Some(2024),
track: Some(1),
disc: None,
date: None,
composer: None,
comment: None,
};
let meta = entry.to_audio_meta();
assert_eq!(meta.title, Some("Test".to_string()));
assert_eq!(meta.artist, Some("Artist".to_string()));
assert_eq!(meta.genre, Some("Rock".to_string()));
assert_eq!(meta.year, Some(2024));
assert_eq!(meta.track, Some(1));
}
}
+79 -2
View File
@@ -57,7 +57,12 @@ impl MetadataParser {
} }
} }
if let Some(channels) = params.channels {
audio_meta.channels = Some(channels.count() as u32);
}
if let Some(bits_per_sample) = params.bits_per_sample { if let Some(bits_per_sample) = params.bits_per_sample {
audio_meta.bits_per_sample = Some(bits_per_sample);
if let Some(sample_rate) = params.sample_rate { if let Some(sample_rate) = params.sample_rate {
if let Some(channels) = params.channels { if let Some(channels) = params.channels {
audio_meta.bitrate = audio_meta.bitrate =
@@ -82,20 +87,82 @@ impl MetadataParser {
if let Some(std_key) = tag.std_key { if let Some(std_key) = tag.std_key {
let value = tag.value.to_string(); let value = tag.value.to_string();
match std_key { match std_key {
// Basic metadata
StandardTagKey::TrackTitle => meta.title = Some(value), StandardTagKey::TrackTitle => meta.title = Some(value),
StandardTagKey::Artist => meta.artist = Some(value), StandardTagKey::Artist => meta.artist = Some(value),
StandardTagKey::Album => meta.album = Some(value), StandardTagKey::Album => meta.album = Some(value),
StandardTagKey::AlbumArtist => meta.album_artist = Some(value), StandardTagKey::AlbumArtist => meta.album_artist = Some(value),
StandardTagKey::Genre => meta.genre = Some(value), StandardTagKey::Genre => meta.genre = Some(value),
// Track/disc with totals (parse "X/Y" format)
StandardTagKey::TrackNumber => { StandardTagKey::TrackNumber => {
meta.track = value.split('/').next().and_then(|s| s.parse().ok()); let parts: Vec<&str> = value.split('/').collect();
meta.track = parts.first().and_then(|s| s.trim().parse().ok());
if parts.len() > 1 {
meta.track_total = parts.get(1).and_then(|s| s.trim().parse().ok());
}
} }
StandardTagKey::DiscNumber => { StandardTagKey::DiscNumber => {
meta.disc = value.split('/').next().and_then(|s| s.parse().ok()); let parts: Vec<&str> = value.split('/').collect();
meta.disc = parts.first().and_then(|s| s.trim().parse().ok());
if parts.len() > 1 {
meta.disc_total = parts.get(1).and_then(|s| s.trim().parse().ok());
} }
}
StandardTagKey::TrackTotal => {
meta.track_total = value.trim().parse().ok();
}
StandardTagKey::DiscTotal => {
meta.disc_total = value.trim().parse().ok();
}
// Date handling: store full date string, extract year
StandardTagKey::Date | StandardTagKey::ReleaseDate => { StandardTagKey::Date | StandardTagKey::ReleaseDate => {
meta.date = Some(value.clone());
meta.year = value.chars().take(4).collect::<String>().parse().ok(); meta.year = value.chars().take(4).collect::<String>().parse().ok();
} }
// Additional metadata
StandardTagKey::Composer => meta.composer = Some(value),
StandardTagKey::Comment => meta.comment = Some(value),
StandardTagKey::Lyrics => meta.lyrics = Some(value),
StandardTagKey::Copyright => meta.copyright = Some(value),
StandardTagKey::Compilation => {
meta.compilation = Some(value == "1" || value.eq_ignore_ascii_case("true"));
}
StandardTagKey::Encoder => meta.encoder = Some(value),
// Sort keys
StandardTagKey::SortTrackTitle => meta.title_sort = Some(value),
StandardTagKey::SortArtist => meta.artist_sort = Some(value),
StandardTagKey::SortAlbum => meta.album_sort = Some(value),
StandardTagKey::SortAlbumArtist => meta.album_artist_sort = Some(value),
// MusicBrainz IDs
StandardTagKey::MusicBrainzRecordingId => meta.mb_recording_id = Some(value),
StandardTagKey::MusicBrainzAlbumId => meta.mb_album_id = Some(value),
StandardTagKey::MusicBrainzArtistId => meta.mb_artist_id = Some(value),
StandardTagKey::MusicBrainzAlbumArtistId => {
meta.mb_album_artist_id = Some(value)
}
StandardTagKey::MusicBrainzReleaseGroupId => {
meta.mb_release_group_id = Some(value)
}
// ReplayGain (parse as f32, values may have "dB" suffix)
StandardTagKey::ReplayGainTrackGain => {
meta.replaygain_track_gain = parse_replaygain(&value);
}
StandardTagKey::ReplayGainTrackPeak => {
meta.replaygain_track_peak = value.trim().parse().ok();
}
StandardTagKey::ReplayGainAlbumGain => {
meta.replaygain_album_gain = parse_replaygain(&value);
}
StandardTagKey::ReplayGainAlbumPeak => {
meta.replaygain_album_peak = value.trim().parse().ok();
}
_ => {} _ => {}
} }
} }
@@ -103,6 +170,16 @@ impl MetadataParser {
} }
} }
/// Parse ReplayGain value, stripping optional "dB" suffix
fn parse_replaygain(value: &str) -> Option<f32> {
let trimmed = value.trim();
let without_db = trimmed
.strip_suffix("dB")
.or_else(|| trimmed.strip_suffix(" dB"))
.unwrap_or(trimmed);
without_db.trim().parse().ok()
}
impl Default for MetadataParser { impl Default for MetadataParser {
fn default() -> Self { fn default() -> Self {
Self::new() Self::new()
@@ -50,6 +50,7 @@ pub fn make_audio_meta(artist: &str, album: &str, title: &str) -> AudioMeta {
bitrate: Some(320), bitrate: Some(320),
sample_rate: Some(44100), sample_rate: Some(44100),
format: AudioFormat::Flac, format: AudioFormat::Flac,
..Default::default()
} }
} }
File diff suppressed because it is too large Load Diff
+105
View File
@@ -0,0 +1,105 @@
**Date**: 2026-05-17
**Status**: Shipped
# Feature: Create Directory (mkdir)
## Overview
MusicFS supports creating directories in the virtual filesystem. This enables organizing files into custom folder structures beyond the auto-generated metadata-based layout.
## Behavior
### Basic Usage
```bash
mkdir "/mnt/music/New Artist"
mkdir "/mnt/music/New Artist/New Album"
```
- Creates empty directory at specified path
- Parent directory must exist
- Standard POSIX semantics
### Nested Directories
```bash
# This works (shell handles -p)
mkdir -p "/mnt/music/A/B/C"
# Equivalent to:
mkdir "/mnt/music/A"
mkdir "/mnt/music/A/B"
mkdir "/mnt/music/A/B/C"
```
The `-p` flag is handled by the shell, which makes multiple `mkdir` syscalls.
### Brace Expansion
```bash
# Shell expands this to multiple mkdir calls
mkdir "/mnt/music/Artist/{Album1,Album2,Album3}"
# Equivalent to:
mkdir "/mnt/music/Artist/Album1"
mkdir "/mnt/music/Artist/Album2"
mkdir "/mnt/music/Artist/Album3"
```
Brace expansion is shell functionality, not filesystem.
## Error Codes
| Condition | Error |
|-----------|-------|
| Parent doesn't exist | `ENOENT` |
| Path already exists | `EEXIST` |
## Persistence
**Empty directories persist across remounts.**
- User-created directories are stored in the `directories` table
- On mount, directories are restored from database
- Directories survive even when empty
## Use Cases
### Organizing Downloads
```bash
# Create structure
mkdir "/mnt/music/Unsorted"
mkdir "/mnt/music/Unsorted/2026"
# Move untagged files
mv "/mnt/music/Unknown Artist/Unknown Album/"*.flac "/mnt/music/Unsorted/2026/"
```
### Custom Collections
```bash
# Create playlist-like structure
mkdir "/mnt/music/_Playlists"
mkdir "/mnt/music/_Playlists/Road Trip"
# Move tracks (they'll still be in original location too - wait, no they won't)
# Note: mv moves, doesn't copy
```
## Implementation
| Component | File |
|-----------|------|
| Tree | `crates/musicfs-cache/src/tree.rs` |
| FUSE | `crates/musicfs-fuse/src/filesystem.rs` |
### Key Functions
- `VirtualTree::mkdir()` - Create directory node in tree
- `Filesystem::mkdir()` - FUSE operation handler
## Limitations
- **No permissions**: Mode/umask parameters are ignored (always 0755)
- **No ownership**: UID/GID set to mounting user
+94
View File
@@ -0,0 +1,94 @@
**Date**: 2026-05-17
**Status**: Shipped
# Feature: Move/Rename (mv)
## Overview
MusicFS supports moving and renaming files and directories within the virtual filesystem. Moves are persisted to the SQLite database and survive remounts.
## Behavior
### File Rename
```bash
mv "/mnt/music/Artist/Album/old.flac" "/mnt/music/Artist/Album/new.flac"
```
- Renames file within same directory
- Updates `virtual_path` in database
- Original file on origin is unchanged
### File Move
```bash
mv "/mnt/music/Artist/Album/track.flac" "/mnt/music/Other Artist/Other Album/track.flac"
```
- Moves file to different directory
- **Requires target directory to exist** (use `mkdir` first)
- Returns `ENOENT` if target parent doesn't exist
### Directory Rename
```bash
mv "/mnt/music/Old Artist" "/mnt/music/New Artist"
```
- Renames directory and all descendants
- All files under the directory have their `virtual_path` updated in DB
- Single atomic operation
### Directory Move
```bash
mv "/mnt/music/Artist/Album" "/mnt/music/Other Artist/Album"
```
- Moves directory subtree to new parent
- **Requires target parent to exist**
- Returns `ENOENT` if target parent doesn't exist
## Error Codes
| Condition | Error |
|-----------|-------|
| Source doesn't exist | `ENOENT` |
| Target already exists | `EEXIST` |
| Target parent doesn't exist | `ENOENT` |
| Source is file but treated as dir | `EISDIR` |
| Source is dir but treated as file | `ENOTDIR` |
## Persistence
- File moves: `virtual_path` column updated in `files` table
- Directory moves: All matching `virtual_path` entries updated with new prefix
- User directories: Tracked in separate `directories` table
- Changes persist across unmount/remount cycles
On mount, the CLI:
1. Scans origin files
2. For each file, checks DB for stored `virtual_path` (by origin_id + real_path)
3. Uses stored path if found, otherwise generates from metadata
4. Restores user-created directories from `directories` table
## Limitations
- **Read-only content**: File contents cannot be modified, only paths
- **No cross-origin moves**: All files remain on their original origin
- **No overwrite**: Moving to existing path fails (no implicit delete)
## Implementation
| Component | File |
|-----------|------|
| Database | `crates/musicfs-cache/src/db.rs` |
| Tree | `crates/musicfs-cache/src/tree.rs` |
| FUSE | `crates/musicfs-fuse/src/filesystem.rs` |
### Key Functions
- `Database::update_virtual_path()` - Update single file path
- `Database::rename_directory()` - Bulk update paths with prefix
- `VirtualTree::rename_file()` - Move file node in tree
- `VirtualTree::rename_directory()` - Move directory subtree
+166
View File
@@ -0,0 +1,166 @@
**Date**: 2026-05-17
**Status**: Shipped
# Feature: Remove (rm)
## Overview
MusicFS supports removing files and directories. Deleted files are moved to a virtual `/.trash/` directory and can be restored. The trash is browsable — users can manually move files out.
## Behavior
### Remove File
```bash
rm "/mnt/music/Artist/Album/track.flac"
```
- File moves to `/.trash/Artist/Album/track.flac`
- Original directory structure preserved in trash
- File still accessible via `/.trash/` path
- Database marks file as `trashed=1` with original path stored
### Remove Empty Directory
```bash
rmdir "/mnt/music/Empty Folder"
```
- Removes empty directory from tree
- Removes from `directories` table if user-created
- Fails with `ENOTEMPTY` if directory has children
### Remove Directory Recursively
```bash
rm -rf "/mnt/music/Artist"
```
- Shell handles recursion (depth-first unlink + rmdir)
- All files moved to `/.trash/Artist/...`
- Empty directories removed after files are trashed
## The `.trash/` Directory
Deleted files live in `/.trash/` with their original path structure:
```
/.trash/
├── Artist/
│ └── Album/
│ ├── track1.flac
│ └── track2.flac
└── Other Artist/
└── song.flac
```
### Browse Trash
```bash
ls "/.trash/"
ls "/.trash/Artist/Album/"
```
### Manual Restore
```bash
# Move file back manually - trashed flag is automatically cleared
mv "/.trash/Artist/Album/track.flac" "/Artist/Album/"
```
When moving a file out of `/.trash/`, the database `trashed` flag is automatically cleared.
## CLI Commands
All trash commands require either `--config` or `--cache-dir`:
```bash
musicfs trash -c config.toml <command>
musicfs trash --cache-dir ./dev/cache/musicfs <command>
```
### List Deleted Files
```bash
musicfs trash -c config.toml list
musicfs trash -c config.toml list --origin local-storage
musicfs trash -c config.toml list --since 7d
musicfs trash -c config.toml list --path "/Artist"
```
Output shows index, deletion time, and original path.
### Restore Files
```bash
# Restore single file or folder
musicfs trash -c config.toml restore "/Artist/Album/track.flac"
# Restore entire folder recursively
musicfs trash -c config.toml restore "/Artist"
# Restore everything
musicfs trash -c config.toml restore --all
```
CLI restore writes paths to a pending restore file and sends SIGHUP to the daemon.
The daemon processes pending restores and moves files back from `/.trash/`.
### Empty Trash
```bash
# Permanently delete all trashed files
musicfs trash -c config.toml empty
# Delete old items only
musicfs trash -c config.toml empty --older-than 30d
# Delete by path pattern
musicfs trash -c config.toml empty --pattern "/Artist"
```
**Warning:** Empty permanently removes files from MusicFS database. Origin files are unaffected.
## Error Codes
| Condition | Error |
|-----------|-------|
| Path doesn't exist | `ENOENT` |
| `rm` on directory (without `-r`) | `EISDIR` |
| `rmdir` on file | `ENOTDIR` |
| `rmdir` on non-empty directory | `ENOTEMPTY` |
| `rmdir` on `/.trash/` | `EPERM` |
## Database Schema
Files table extended with trash columns:
```sql
trashed INTEGER NOT NULL DEFAULT 0,
original_path TEXT,
trashed_at INTEGER
```
Partial index for efficient trash queries:
```sql
CREATE INDEX idx_files_trashed ON files(trashed) WHERE trashed = 1;
```
## How It Works
1. **Delete (`rm`)**: FUSE `unlink` moves file to `/.trash/`, marks `trashed=1` in DB
2. **Manual restore (`mv`)**: Moving out of `/.trash/` automatically clears `trashed` flag
3. **CLI restore**: Writes pending paths, sends SIGHUP to daemon, daemon processes restores
4. **Empty**: Deletes matching records from database
## Persistence
- Trashed files persist across remounts (stored in `/.trash/` subtree)
- Files marked with `trashed=1`, `original_path`, `trashed_at` in database
- PID file at `{cache_dir}/musicfs.pid` for CLI→daemon communication
## Limitations
- **No hard delete of remote files**: Origin content is never modified
- **Trash uses virtual space**: Files still in tree under `/.trash/` until emptied
- **CLI restore requires running daemon**: Manual `mv` works without daemon
Generated
+3 -3
View File
@@ -93,11 +93,11 @@
}, },
"nixpkgs_2": { "nixpkgs_2": {
"locked": { "locked": {
"lastModified": 1778443072, "lastModified": 1778869304,
"narHash": "sha256-zi7/fsqM/kFdNuED//4WOCUtezGtKKqRNORjMvfwjnA=", "narHash": "sha256-30sZNZoA1cqF5JNO9fVX+wgiQYjB7HJqqJ4ztCDeBZE=",
"owner": "NixOS", "owner": "NixOS",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "da5ad661ba4e5ef59ba743f0d112cbc30e474f32", "rev": "d233902339c02a9c334e7e593de68855ad26c4cb",
"type": "github" "type": "github"
}, },
"original": { "original": {
+1
View File
@@ -51,6 +51,7 @@
gitleaks gitleaks
just just
opencode
pkg-config pkg-config
fuse3 fuse3