Alexander
f0a83df190
- README.md: Overview, core concept diagram, component summary - architecture.md: System design, initialization flow, memory model - components.md: Deep dive on all classes and functions - data-flow.md: Complete read/write operation flows with diagrams - analysis.md: Performance analysis (latency, memory footprint, I/O) - drawbacks.md: 27 identified issues and limitations catalog - modernization.md: Python 3 migration guide with effort estimates
14 KiB
14 KiB
beetfs Architecture
System Overview
beetfs implements a metadata overlay filesystem using FUSE. The key innovation is separating metadata storage (in beets SQLite database) from audio data storage (original files on disk).
┌─────────────────────────────────────────────────────────────────────────────┐
│ USER SPACE │
│ ┌─────────────┐ ┌─────────────────────────────────────────────────────┐ │
│ │ Application │ │ beetfs │ │
│ │ (VLC, etc) │ │ ┌─────────────┐ ┌──────────────┐ ┌────────────┐ │ │
│ │ │◄───┼──┤beetFileSystem│──│ FileHandler │──│ Interpol. │ │ │
│ │ │ │ │ (FUSE) │ │ │ │ FLAC/ID3 │ │ │
│ └─────────────┘ │ └─────────────┘ └──────────────┘ └────────────┘ │ │
│ │ │ │ │ │ │
│ │ ▼ ▼ ▼ │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌────────────┐ │ │
│ │ │ FSNode │ │ Beets │ │ Original │ │ │
│ │ │ (dir tree) │ │ Database │ │ Files │ │ │
│ │ └─────────────┘ └──────────────┘ └────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────────────┤
│ KERNEL SPACE │
│ ┌───────────────┐ │
│ │ FUSE VFS │ │
│ └───────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
Component Architecture
1. Plugin Layer
class beetFs(BeetsPlugin):
"""Beets plugin hook - registers the 'mount' subcommand"""
def commands(self):
return [beetFs_command]
beetFs_command = Subcommand('mount', help='Mount a beets filesystem')
beetFs_command.func = mount
2. Initialization Flow
beet mount /mountpoint
│
▼
┌───────────────────────────────────────────────────────────────┐
│ mount() function │
│ 1. Parse PATH_FORMAT template │
│ 2. Create FSNode root (directory_structure) │
│ 3. Iterate all items in beets library │
│ 4. For each item: │
│ - Build template substitution map │
│ - Add directories to FSNode tree │
│ - Add file entry (filename → item.id mapping) │
│ 5. Create beetFileSystem FUSE server │
│ 6. server.main() - enter FUSE event loop │
└───────────────────────────────────────────────────────────────┘
3. Virtual Directory Structure
The default path template:
PATH_FORMAT = "$artist/$album ($year) [$format_upper]/$track - $artist - $title.$format"
Results in structure like:
/mountpoint/
├── Pink Floyd/
│ └── The Wall (1979) [FLAC]/
│ ├── 01 - Pink Floyd - In The Flesh?.flac
│ └── 02 - Pink Floyd - The Thin Ice.flac
└── Led Zeppelin/
└── IV (1971) [FLAC]/
└── 01 - Led Zeppelin - Black Dog.flac
4. FSNode Tree Structure
class FSNode:
dirs: Dict[str, FSNode] # subdirectories
files: Dict[str, int] # filename → beets item ID
# Example tree:
FSNode(
dirs={
"Pink Floyd": FSNode(
dirs={
"The Wall (1979) [FLAC]": FSNode(
dirs={},
files={
"01 - Pink Floyd - In The Flesh?.flac": 42,
"02 - Pink Floyd - The Thin Ice.flac": 43
}
)
},
files={}
)
},
files={}
)
Core Data Flow
Read Operation
Application: read("/mount/Artist/Album/track.flac", offset=0, size=4096)
│
▼
┌───────────────────────┐
│ beetFileSystem.read() │
│ Lines 1077-1106 │
└───────────┬───────────┘
│
┌───────────────┴───────────────┐
│ Get/Create FileHandler │
│ for this path │
└───────────────┬───────────────┘
│
┌───────────┴───────────┐
│ FileHandler.read() │
│ Lines 497-517 │
└───────────┬───────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ offset < bound │ │ offset >= bound │
│ (in header area) │ │ (in audio area) │
└──────────┬──────────┘ └──────────┬──────────┘
│ │
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ Return modified │ │ Return original │
│ header from DB │ │ audio from file │
│ │ │ │
│ self.header[...] │ │ self.music_data[...]│
└─────────────────────┘ └─────────────────────┘
Write Operation
Application: write("/mount/Artist/Album/track.flac", data, offset=100)
│
▼
┌───────────────────────┐
│ beetFileSystem.write()│
│ Lines 1108-1135 │
└───────────┬───────────┘
│
┌───────────┴───────────┐
│ FileHandler.write() │
│ Lines 519-565 │
└───────────┬───────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ offset < bound │ │ offset >= bound │
│ (in header area) │ │ (in audio area) │
└──────────┬──────────┘ └──────────┬──────────┘
│ │
▼ ▼
┌─────────────────────┐ ┌─────────────────────┐
│ 1. Patch header │ │ DISCARD │
│ 2. Parse new tags │ │ (audio writes │
│ 3. Extract values │ │ not allowed) │
│ 4. Update beets DB │ │ │
│ 5. Regenerate header│ │ │
└─────────────────────┘ └─────────────────────┘
Memory Model
FileHandler State
class FileHandler:
# Paths
path: str # Virtual path in FUSE mount
real_path: str # Actual file on disk
# Beets integration
item: Item # Beets library item
lib: Library # Beets library reference
# File data
file_object: File # File handle (closed after init)
music_data: bytes # Audio data cached in memory
# Metadata
format: str # "flac" or "mp3"
inf: FLAC/ID3 # Interpolated metadata object
header: bytes # Generated header with DB metadata
bound: int # Byte offset where header ends
music_offset: int # Byte offset where audio starts in original
# Reference counting
instance_count: int # Number of open handles
Memory Layout
Virtual File (as seen by application):
┌────────────────────────────────────────────────────────────────┐
│ HEADER (from DB) │ AUDIO (from file) │
│ [0 ... bound) │ [bound ... EOF) │
│ │ │
│ Generated by InterpolatedFLAC │ Cached in music_data │
│ Contains: title, artist, album, │ Original audio frames │
│ genre from beets DB │ Unchanged │
└────────────────────────────────────────────────────────────────┘
▲ ▲
│ │
self.header self.music_data
Original File (on disk):
┌────────────────────────────────────────────────────────────────┐
│ ORIGINAL HEADER │ AUDIO DATA │
│ [0 ... music_offset) │ [music_offset ... EOF) │
│ │ │
│ May have different │ Same as virtual file │
│ tag values │ │
└────────────────────────────────────────────────────────────────┘
Threading Model
server.multithreaded = 0 # Single-threaded mode
beetfs runs in single-threaded mode to avoid concurrency issues with:
- Shared
filesdictionary - Beets library access
- File handle reference counting
Global State
# Module-level globals (set during mount)
structure_split: List[str] # PATH_FORMAT split by "/"
structure_depth: int # Number of path components
library: Library # Beets library instance
directory_structure: FSNode # Root of virtual directory tree
Error Handling
| Situation | Response |
|---|---|
| File not found | Return -errno.ENOENT |
| Permission denied | Return -errno.EACCES |
| Operation not supported | Return -errno.EOPNOTSUPP |
| Parse error | Log and return -errno.ENOENT |
Limitations
- Format Support: Only FLAC fully implemented; MP3 support is incomplete
- Memory Usage: Entire audio portion cached in memory per open file
- Single-threaded: No concurrent access optimization
- No Streaming: Full file must be read into memory
- Python 2: Uses deprecated language features
- fuse-python: Old FUSE bindings, not maintained