- Add overlay_reader field to MusicFs struct
- Add with_overlay() builder method
- Update getattr() to call estimate_virtual_size()
- Graceful fallback to original size
- All tests pass, backward compatible
- Create FormatHandlerRegistry in CLI initialization
- Register Id3v2Handler and FlacHandler
- Add analyze_format_layout() helper to read file headers
- Update scan functions to call handler.analyze()
- Use upsert_file_with_layout() when format_layout available
- Graceful degradation for unsupported formats
- Full workspace compiles successfully
- Update upsert_file() to include all 26 new AudioMeta fields
- Update get_file_by_virtual_path() to read all new columns
- Add get_file_metadata_row() for overlay synthesis
- Add update_metadata() for partial metadata updates
- Add clear_overlay() to reset metadata to NULL
- Handle format_layout BLOB with msgpack serialization
- Handle custom_tags JSON with serde_json
- Add 8 comprehensive unit tests
- All 92 tests pass, LSP diagnostics clean
- Implement all 8 FormatHandler trait methods
- Use lofty 0.24 for ID3v2.4 tag creation/parsing
- Map all 36 AudioMeta fields to ID3v2 frames
- Handle ID3v2 header parsing for audio_start
- Detect ID3v1 tags at EOF for audio_end
- Add 13 comprehensive unit tests
- Fix test-utils AudioMeta construction with ..Default::default()
- All tests pass, LSP diagnostics clean
Add mkdir and mv (rename) FUSE operations to the virtual filesystem:
- mkdir: Create directories that persist across remounts via SQLite
- mv: Move/rename files and directories with database persistence
Changes:
- Add directories table to schema for user-created empty dirs
- Add tree operations: mkdir, rename_file, rename_directory
- Add DB methods for path updates and directory CRUD
- Remove MountOption::RO to allow write syscalls
- Load stored virtual_path from DB instead of regenerating
- Restore user directories on mount from directories table
- Upsert files to DB during origin scan
POSIX compliant: mv fails with ENOENT if parent doesn't exist
(use mkdir first, shell handles -p flag and brace expansion)
- CLI mountpoint argument is now Option<PathBuf>
- Falls back to config.mount_point when --config is provided
- CLI mountpoint still overrides config if both are given
- Expanded config.example.toml with all available options
Decision: SQLite (Option A) — existing schema, CRUD, row mapping,
and chunk_manifest column are already built but not wired into mount.
8-day plan to transform mount from O(N×origin_latency) to O(N×SQLite_read):
1. Database bulk load + manifest CRUD methods
2. Rewrite run_mount() with DB-load vs first-mount-scan paths
3. Persist chunk manifests via ManifestCached event
4. Wire tantivy + PatternStore + CollectionStore into mount
5. Background delta sync (origin vs DB reconciliation)
6. Shutdown WAL checkpoint
7-8. Integration testing + buffer
Implements all 6 critical resilience fixes from phase-a-stop-dying.md:
- Issue 2.9: Migrate std::sync::RwLock → parking_lot::RwLock (7 files)
Prevents lock poisoning cascade on writer panic
- Issue 2.2: Add install_panic_hook() to log panics via tracing
Ensures panics are captured in logs/journald before process death
- Issue 3.7: Add ExecStopPost to systemd service
Cleans up stale FUSE mounts on service stop
- Issue 2.7: Add check_stale_mount() detection on startup
Auto-cleans leftover mounts from previous crashes
- Issue 2.10: Integrate sd_notify for systemd lifecycle
Sends READY=1 after mount, STOPPING on shutdown
- Issue 2.1: Add signal handling with spawn_mount
Catches SIGTERM/SIGINT for clean shutdown instead of instant death
All 7 Phase A tests pass:
- test_poisoned_tree_lock_returns_eio_not_panic
- test_parking_lot_rwlock_survives_panic
- test_panic_hook_logs_to_tracing
- test_systemd_service_has_execstoppost
- test_stale_mount_check_function_exists
- test_sd_notify_ready_sent
- test_sigterm_triggers_shutdown
Phase 1 of resilience testing design doc implementation:
- New musicfs-test-utils crate with FaultyOrigin, FaultyCasStore, fixtures
- Failpoints instrumented in musicfs-cas/store.rs
- 16 resilience tests (13 RED for missing features, 3 GREEN for existing)
- 3 Docker/Toxiproxy network tests (RED until docker-compose up)
- docker-compose.yml for Toxiproxy + MinIO + SFTP test infrastructure
Tests properly fail-first (TDD): check_all() sequential, no health timeout,
missing corruption detection, no passthrough mode, etc.
Replace original resilience-testing.md with BlueDoc-structured version.
All code examples from original preserved in Appendix A (17 sections).
Added: Abstract, Background, Goals/Non-Goals, Cross-Cutting Concerns,
Alternatives Considered (Jepsen, proptest, loom, mockall), phased
implementation plan with rollout order. Removed v2 suffix.
Covers: Nix dev shell, build/test commands, 11-crate workspace layout,
architecture essentials, code conventions, golden documents, documentation
rules (BlueDoc/GreenDoc templates + when to propose new templates),
current state with known critical issues, and key dependencies.
Comprehensive fault tolerance analysis covering 34 issues across 6 phases:
signal handling, crash recovery, cache corruption, network failures,
resource exhaustion, and the critical finding that no persistent state
is used on mount (every restart is a full origin rescan).
Persistent state plan covers storage engine options, mount flow redesign,
background delta sync, and the in-memory state inventory.
- Add tracing-appender and tracing-journald for production logging
- Add LoggingConfig with trace_sample_rate, json_output, journald options
- Expand init_logging() with file rotation, journald, and stderr layers
- Add sanitize_path() helper for PII protection in logs
- Instrument FUSE operations with #[instrument] and trace decision points
- Instrument gRPC handlers (10 methods) with span correlation
- Add spawn instrumentation for health monitor, indexer, watcher tasks
- Add broadcast lag handling (RecvError::Lagged) in event subscribers
- Fix webhook.rs expect() calls with proper error handling
- Add logging to patterns.rs, collections.rs, artwork.rs database ops
- Add Drop impl logging for PluginManager and WatchHandle
- Update systemd service with rate limiting and journal output
- Add logrotate config and example config.toml with logging section
Week 10 - Plugin System (FR-19):
- Plugin traits: Plugin, OriginPlugin, MetadataPlugin, FormatPlugin
- NativePluginHost with libloading for dynamic loading
- WasmPluginHost (feature-gated) with wasmtime runtime
- PluginManager coordinating both hosts with version checks
- OriginInstance::watch() with WatchHandle, WatchEvent for live updates
- FormatPlugin::synthesize_header() for metadata overlay
Week 11 - Control API & Production (FR-17, FR-18, NFR-6, NFR-10):
- gRPC server with full MusicFS service (status, cache, origins, events)
- Proto extended: MountState enum, TierStats, full StatusResponse/CacheStats
- WebhookHandler with HMAC-SHA256 signing and exponential retry
- Metrics with latency histograms (p50/p95/p99) and origin health gauges
- CLI with mount, status, cache, search, origin, events, shutdown commands
- E2E player compatibility tests (mpv, VLC, file manager)
- systemd service, PKGBUILD, RPM spec for packaging
Plans added for Weeks 10-14 covering P1 features.
All 154 tests passing.
- Proto definitions for SearchRequest/SearchResponse/SearchResult
- SearchService implementing MusicFs trait
- Query validation (empty check, 256 char max, limit cap at 10k)
- Streaming support via tokio mpsc channel
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/claude-agent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- SearchOps with moka LRU cache for results and inode mapping
- 256 char query length limit with truncation
- Safe symlink targets with path normalization
- Integrates with filesystem.rs for lookup/getattr/readdir/readlink
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/claude-agent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- SearchIndex with Arc<RwLock<IndexWriter>> pattern
- Fuzzy query support via custom term~N parsing
- Indexer with EventBus integration and MetadataLookup trait
- SearchQueryBuilder for programmatic query construction
- remove_by_path fallback for FileRemoved handling
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/claude-agent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- Add CdcChunker using FastCDC v3 (16KB/64KB/256KB chunks)
- Add DeltaDetector with scan_origin() returning ScannedFile (no FileId assignment)
- Add OriginWatcher with inotify and 200ms debounce using tokio::spawn
- Fix LocalOrigin::read() to loop until all bytes read
- Add read_full() method to Origin trait
- Add mtime field to ChunkManifest
- Update ContentFetcher to use CDC chunking
- Update bandwidth reduction test to assert >90% (NFR-6.4)
Tests: 71 pass (+11 new)
- Create week-04b-origin-connector.md with ContentFetcher design
- Update development-plan.md: Phase 1 now includes Week 4b
- Update architecture.md: Phase 1 table includes Week 4b
- Plan includes EventBus integration per FR-18.1 (Oracle-verified)
- Add musicfs-cas crate: CasStore, ChunkHash, FileReader, ChunkManifest
- Add LruEviction policy to musicfs-cache for cache size management
- Integrate FileReader into FUSE filesystem for actual file reads
- Use xxHash64 for content hashing, sled for index, msgpack serialization
- Default cache path: ~/.cache/musicfs/chunks/ with 256 subdirs sharding
- 20 new tests (14 CAS unit + 3 integration + 3 eviction), 54 total
Alexander