docs/v1/ - Original beetfs documentation:
- analysis.md, components.md, data-flow.md, drawbacks.md
- features.md, modernization.md, rust-migration.md
- benchmark-plan.md, benchmark-results.md, e2e-test-plan.md
- README.md
docs/v2/ - New MusicFS architecture:
- requirements.md: Full requirements spec (FR-1 to FR-25, NFR-1 to NFR-14)
- P0: Multi-origin, plugins, CAS, control API
- P1: Search, album art, prefetch, metadata sources
- P3: HA, 10M+ files scalability
- architecture.md: Google BlueDoc style design document
- PlantUML diagrams for all components
- Design requirements with quantitative targets
- Alternatives considered, implementation plan
19 KiB
beetfs Components Deep Dive
Component Overview
┌─────────────────────────────────────────────────────────────────────────┐
│ beetFs.py │
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ PLUGIN LAYER ││
│ │ beetFs (BeetsPlugin) beetFs_command (Subcommand) ││
│ │ mount() template_mapping() ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ VIRTUAL FILESYSTEM ││
│ │ FSNode beetFileSystem (fuse.Fuse) ││
│ │ Stat ││
│ └─────────────────────────────────────────────────────────────────────┘│
│ ┌─────────────────────────────────────────────────────────────────────┐│
│ │ METADATA INTERPOLATION ││
│ │ FileHandler InterpolatedFLAC ││
│ │ InterpolatedID3 ││
│ └─────────────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────────────┘
1. Plugin Layer
1.1 beetFs (BeetsPlugin)
Location: Lines 188-191
class beetFs(BeetsPlugin):
""" The beets plugin hook."""
def commands(self):
return [beetFs_command]
Purpose: Registers beetfs as a beets plugin, exposing the mount subcommand.
1.2 beetFs_command
Location: Lines 47, 185
beetFs_command = Subcommand('mount', help='Mount a beets filesystem')
beetFs_command.func = mount
Purpose: CLI subcommand definition for beet mount.
1.3 mount() Function
Location: Lines 119-183
def mount(lib, config, opts, args):
# 1. Validate arguments
if not args:
raise beets.ui.UserError('no mountpoint specified')
# 2. Parse path template
global structure_split
structure_split = PATH_FORMAT.split("/")
global structure_depth
structure_depth = len(structure_split)
# 3. Store library reference
global library
library = lib
# 4. Build virtual directory tree
global directory_structure
directory_structure = FSNode({}, {})
# 5. Iterate all library items
for item in lib.items():
mapping = template_mapping(lib, item)
# ... build tree ...
directory_structure.addfile(sub_elements, filename, item.id)
# 6. Create and run FUSE server
server = beetFileSystem(...)
server.main()
Key Variables Set:
| Variable | Type | Purpose |
|---|---|---|
structure_split |
List[str] |
Path template components |
structure_depth |
int |
Number of path levels |
library |
Library |
Beets library reference |
directory_structure |
FSNode |
Root of virtual tree |
1.4 template_mapping() Function
Location: Lines 82-116
def template_mapping(lib, item):
"""Builds a template substitution map from beets item."""
mapping = {}
for key in METADATA_KEYS:
value = getattr(item, key)
# Sanitize value for filesystem paths
if isinstance(value, basestring):
value = re.sub(r'[\\/:]|^\.', '_', value)
elif key in ('track', 'tracktotal', 'disc', 'disctotal'):
value = '%02i' % value # Zero-pad numbers
mapping[key] = value
# Add format info
format_ = os.path.splitext(item.path)[1][1:]
mapping['format'] = format_
mapping['format_upper'] = format_.upper()
# Default values for missing fields
if mapping['artist'] == '':
mapping['artist'] = 'Unknown Artist'
# ... etc
return mapping
Template Variables Available:
| Variable | Source | Example |
|---|---|---|
$artist |
item.artist |
"Pink Floyd" |
$album |
item.album |
"The Wall" |
$title |
item.title |
"Comfortably Numb" |
$year |
item.year |
"1979" |
$track |
item.track |
"06" |
$format |
file extension | "flac" |
$format_upper |
file extension | "FLAC" |
2. Virtual Filesystem Layer
2.1 FSNode Class
Location: Lines 390-436
class FSNode(object):
"""A directory node in the virtual filesystem tree."""
def __init__(self, dirs, files):
self.dirs = dirs # Dict[str, FSNode] - subdirectories
self.files = files # Dict[str, int] - filename → beets item ID
Methods:
| Method | Purpose | Signature |
|---|---|---|
getnode() |
Navigate to nested node | getnode(elements, root=None) → FSNode |
adddir() |
Add a directory | adddir(elements, directory, root=None) |
addfile() |
Add a file entry | addfile(elements, filename, id, root=None) |
listdir() |
List contents | listdir(elements, directories, root=None) → List[str] |
Example Tree Navigation:
# Path: /Artist/Album/track.flac
# structure_split = ["$artist", "$album ($year) [$format_upper]", "$track - $artist - $title.$format"]
elements = ["Artist", "Album (2020) [FLAC]"]
node = directory_structure.getnode(elements)
# node.files = {"01 - Artist - Track.flac": 42, ...}
item_id = node.files["01 - Artist - Track.flac"]
# item_id = 42
2.2 Stat Class
Location: Lines 568-619
class Stat(fuse.Stat):
DIRSIZE = 4096
def __init__(self, st_mode, st_size, st_nlink=1, st_uid=None, st_gid=None,
dt_atime=None, dt_mtime=None, dt_ctime=None):
self.st_mode = st_mode
self.st_ino = 0
self.st_dev = 0
self.st_nlink = st_nlink
self.st_uid = st_uid or os.getuid()
self.st_gid = st_gid or os.getgid()
self.st_size = st_size
# ... timestamps ...
Purpose: Represents file/directory metadata for FUSE stat operations.
2.3 beetFileSystem Class
Location: Lines 622-1144
class beetFileSystem(fuse.Fuse):
"""Main FUSE filesystem implementation."""
def __init__(self, *args, **kwargs):
logging.basicConfig(filename="LOG", level=logging.INFO)
super(beetFileSystem, self).__init__(*args, **kwargs)
def fsinit(self):
"""Called after filesystem is mounted."""
self.lib = library
self.files = {} # Dict[path, FileHandler]
FUSE Operations Implemented:
| Operation | Lines | Purpose |
|---|---|---|
fsinit() |
630-636 | Post-mount initialization |
fsdestroy() |
638-639 | Pre-unmount cleanup |
statfs() |
641-646 | Filesystem statistics |
getattr() |
648-707 | Get file/dir attributes |
access() |
723-756 | Check permissions |
readdir() |
931-975 | List directory contents |
open() |
988-1021 | Open file |
read() |
1077-1106 | Read file data |
write() |
1108-1135 | Write file data |
release() |
1049-1059 | Close file |
Not Implemented (return EOPNOTSUPP):
mknod(),mkdir(),unlink(),rmdir()symlink(),link(),rename()chmod(),chown(),truncate()
3. Metadata Interpolation Layer
3.1 FileHandler Class
Location: Lines 439-565
This is the core component that implements metadata overlay.
class FileHandler(object):
def __init__(self, path, lib):
self.path = path # Virtual path
self.lib = lib # Beets library
# Resolve virtual path to real file
pathsplit = path[1:].split('/')
self.item = self.lib.get_item(id=directory_structure
.getnode(pathsplit[0:structure_depth-1])
.files[pathsplit[structure_depth-1]])
self.real_path = self.item.path
# Open real file
self.file_object = open(self.real_path, 'r+')
self.instance_count = 1
# Determine format
self.format = os.path.splitext(path)[1][1:].lower()
if self.format == "flac":
# Load file into interpolated FLAC object
self.inf = InterpolatedFLAC(self.file_object.read())
# INJECT DATABASE METADATA
self.inf["title"] = self.item.title
self.inf["album"] = self.item.album
self.inf["artist"] = self.item.artist
self.inf["genre"] = self.item.genre
# Generate new header with DB metadata
self.header = self.inf.get_header(self.real_path)
self.bound = len(self.header)
self.music_offset = self.inf.offset()
elif self.format == "mp3":
self.bound = 0 # MP3 interpolation disabled
self.music_offset = 0
# Cache audio data
self.file_object.seek(self.music_offset)
self.music_data = self.file_object.read()
self.file_object.close()
Key Attributes:
| Attribute | Type | Purpose |
|---|---|---|
path |
str |
Virtual path (e.g., /Artist/Album/track.flac) |
real_path |
str |
Actual file path on disk |
item |
Item |
Beets library item (has DB metadata) |
format |
str |
File format ("flac", "mp3") |
inf |
InterpolatedFLAC |
Mutagen object with injected metadata |
header |
bytes |
Generated header with DB tags |
bound |
int |
Byte offset where header ends |
music_offset |
int |
Byte offset in original file where audio starts |
music_data |
bytes |
Cached audio data |
instance_count |
int |
Reference count for file handles |
3.2 FileHandler.read() Method
Location: Lines 497-517
def read(self, size, offset):
# Case 1: Reading within header boundary
if offset < self.bound:
if offset + size < len(self.header):
# Entire read is within header
return self.header[offset:offset+size]
else:
# Read spans header and audio
ret = self.header[offset:len(self.header)]
ret = ret + self.music_data[0:size - (len(self.header) - offset)]
return ret
# Case 2: Reading audio data only
return self.music_data[offset - len(self.header):offset - len(self.header) + size]
Read Logic Diagram:
Virtual File Layout:
┌────────────────────────────────────────────────────────────────┐
│ 0 bound EOF │
│ ├─────────┼────────────────────────────────────────────────┤ │
│ │ HEADER │ AUDIO DATA │ │
│ │ (from │ (from self.music_data) │ │
│ │ self. │ │ │
│ │ header) │ │ │
│ └─────────┴────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────┘
Read scenarios:
1. offset=0, size=100, bound=500 → Return header[0:100]
2. offset=400, size=200, bound=500 → Return header[400:500] + music[0:100]
3. offset=600, size=100, bound=500 → Return music[100:200]
3.3 FileHandler.write() Method
Location: Lines 519-565
def write(self, offset, buf):
# Only handle writes to header area
if offset < self.bound:
# Reconstruct full file in memory
filedata = self.header + self.music_data
# Patch in new data
filedata = filedata[0:offset] + buf + filedata[offset + len(buf):]
if self.format == "flac":
# Parse the patched data
self.inf = InterpolatedFLAC(filedata)
# EXTRACT new tag values and save to DB
self.item.title = str(self.inf["title"][0]).encode('utf-8')
self.item.album = str(self.inf["album"][0]).encode('utf-8')
self.item.artist = str(self.inf["artist"][0]).encode('utf-8')
self.item.genre = str(self.inf["genre"][0]).encode('utf-8')
# Persist to beets database
self.lib.store(self.item)
self.lib.save()
# Regenerate header with updated values
self.inf["title"] = self.item.title
self.inf["album"] = self.item.album
self.inf["artist"] = self.item.artist
self.inf["genre"] = self.item.genre
self.header = self.inf.get_header(self.real_path)
self.bound = len(self.header)
return len(buf)
Write Flow:
1. App writes new tag data to header region
│
▼
2. Patch header + music_data with new bytes
│
▼
3. Parse patched data as FLAC
│
▼
4. Extract tag values from parsed FLAC
│
▼
5. Update beets Item with new values
│
▼
6. lib.store(item) + lib.save() → SQLite
│
▼
7. Regenerate header for subsequent reads
3.4 InterpolatedFLAC Class
Location: Lines 274-388
class InterpolatedFLAC(FLAC):
"""Custom FLAC handler that can load from bytes and generate headers."""
def load(self, filedata):
"""Load FLAC from byte string instead of file."""
self.metadata_blocks = []
self.tags = None
self.filedata = filedata
self.fileobj = BytesIO(filedata)
self.__check_header(self.fileobj)
while self.__read_metadata_block(self.fileobj):
pass
# Verify audio frame starts correctly
if self.fileobj.read(2) not in ["\xff\xf8", "\xff\xf9"]:
raise FLACNoHeaderError("End of metadata did not start audio")
def get_header(self, filename=None):
"""Generate FLAC header with current metadata."""
# Add padding block
self.metadata_blocks.append(Padding('\x00' * 1020))
MetadataBlock.group_padding(self.metadata_blocks)
# Calculate available space
header = self.__check_header(self.fileobj)
available = self.__find_audio_offset(self.fileobj) - header
data = MetadataBlock.writeblocks(self.metadata_blocks)
# Adjust padding to match available space
if len(data) > available:
# Reduce padding
padding = self.metadata_blocks[-1]
padding.length -= (len(data) - available)
data = MetadataBlock.writeblocks(self.metadata_blocks)
elif len(data) < available:
# Increase padding
self.metadata_blocks[-1].length += (available - len(data))
data = MetadataBlock.writeblocks(self.metadata_blocks)
self.__offset = len("fLaC" + data)
return "fLaC" + data
def offset(self):
"""Return byte offset where audio data starts."""
return self.__offset
FLAC Structure:
┌──────────────────────────────────────────────────────────────────┐
│ "fLaC" │ STREAMINFO │ VORBIS_COMMENT │ ... │ PADDING │ AUDIO... │
│ (4B) │ block │ block │ │ block │ │
└──────────────────────────────────────────────────────────────────┘
│◄──────── metadata_blocks ─────────►│
│ │
└──── get_header() returns this ─────┘
3.5 InterpolatedID3 Class
Location: Lines 200-271
class InterpolatedID3(ID3):
"""Custom ID3 handler for MP3 files."""
def save(self, filename=None, v1=0):
"""Save ID3 tags to file."""
# Sort frames by importance
order = ["TIT2", "TPE1", "TRCK", "TALB", "TPOS", "TDRC", "TCON"]
# ... write header ...
Note: MP3 support is incomplete in the current implementation. The FileHandler.__init__ sets self.bound = 0 for MP3, effectively disabling interpolation.
4. Supported Metadata Fields
Location: Lines 55-77
METADATA_RW_FIELDS = [
('title', 'text'),
('artist', 'text'),
('album', 'text'),
('genre', 'text'),
('composer', 'text'),
('grouping', 'text'),
('year', 'int'),
('month', 'int'),
('day', 'int'),
('track', 'int'),
('tracktotal', 'int'),
('disc', 'int'),
('disctotal', 'int'),
('lyrics', 'text'),
('comments', 'text'),
('bpm', 'int'),
('comp', 'bool'),
]
Actually Implemented (in FileHandler):
| Field | Read | Write |
|---|---|---|
title |
✅ | ✅ |
artist |
✅ | ✅ |
album |
✅ | ✅ |
genre |
✅ | ✅ |
| Others | ❌ | ❌ |
5. Error Handling
Error Codes Used:
| Code | Constant | Usage |
|---|---|---|
| 2 | ENOENT |
File/directory not found |
| 13 | EACCES |
Permission denied |
| 1 | EPERM |
Operation not permitted |
| 95 | EOPNOTSUPP |
Operation not supported |
Exception Handling Pattern:
def getattr(self, path):
try:
# ... logic ...
except Exception as e:
logging.error(e)
return -errno.ENOENT