musicfs

How it works
Layout
Running the client
Running the server in an Incus VM
- Poke at it (Nushell)
Development

A read-only FUSE filesystem that presents a music library reorganised by metadata rather than by however the files happen to sit on disk. Point it at a messy ~/Music tree (or a remote musicfs-server) and it mounts a clean Artist/Album/Track view, driven entirely by the tags parsed out of the audio files themselves.

mountpoint/
├── ДДТ/
│   └── Творчество в пустоте/
│       ├── 01 Intro.flac
│       └── 02 ...
└── Some Artist/
    └── Some Album/
        └── 01 Track.flac

The source files are never moved or modified. The directory hierarchy is virtual: synthesised from album_artist~/~album tags, with stable synthetic inodes so the layout survives remounts.

How it works

Two origins feed the same FUSE frontend:

LocalOrigin — walks a directory on the host, parses tags, builds the snapshot. A notify watcher keeps it live as files change.
NetworkOrigin — talks gRPC to a remote musicfs-server. Metadata and file bytes arrive over the wire; a Postgres table caches both. Reconciliation is hash-based: the client sends the (inode, hash) pairs it has, the server replies with only what changed or was deleted, so almost nothing crosses the network on a steady-state refresh.

Which one is used is chosen automatically from the --source argument: an http(s):// URL means network, anything else is treated as a local path.

State lives in Postgres — parsed metadata, the virtual-path layout, and (for the network origin) a lazy byte cache populated only for files that were actually read.

Layout

Crate	Responsibility
`musicfs-proto`	Protobuf/gRPC definitions (`proto/musicfs.proto`), generated bindings
`musicfs-core`	Shared logic: tag parsing (FLAC/MP3 via symphonia), hashing, logging
`musicfs-client`	FUSE mount, origins, Postgres cache/sync — the `musicfs` binary
`musicfs-server`	gRPC server sharing a library — the `musicfs-server` binary

The gRPC surface (see proto/musicfs.proto): Reconcile, GetMetadata, GetManifest, GetFile (whole-file or byte-range streaming), and SubscribeEvents (change wake-ups). The client also serves a ClientStatus RPC for introspection.

Running the client

Everything runs inside the devenv shell, which provides the Rust toolchain, Postgres, FUSE tooling, and gRPC utilities.

devenv shell
devenv up --profile local     # starts Postgres + the FUSE mount

Predefined profiles in devenv.nix:

Profile	Source	Purpose
`local`	a local directory	mount a host music folder
`remote`	`http://…:50051`	mount a remote musicfs-server
`e2e`	`http://127.0.0.1:50061`	end-to-end test harness

Or run the binary directly:

cargo run -p musicfs-client -- \
  --source /home/you/Music \
  --mountpoint /tmp/musicfs \
  --database "postgresql://you@localhost/musicfs?host=$PGHOST"

Key flags: --source (local path or http(s):// server URL), --mountpoint, --database (Postgres URL), --log-dir (daily-rotated logs, default ./logs), --listen (address for the client's status/health RPCs, default 127.0.0.1:50052).

Running the server in an Incus VM

musicfs-server (gRPC) runs inside an Incus VM, live-sharing the host's ~/Music as a read-only virtiofs mount. scripts/vm.sh builds the binary on the host, bundles its nix glibc so it runs unmodified in the VM, ships it in, and runs it under systemd.

devenv shell        # cargo, incus, patchelf, grpcurl
scripts/vm.sh up    # create VM, share ~/Music, build, ship, start server

Options (env): VM_NAME, IMAGE, MUSIC_SOURCE, LISTEN_PORT, RUST_LOG.

Command	Action
`scripts/vm.sh up`	create + build + ship + start
`scripts/vm.sh redeploy`	rebuild after a code change
`scripts/vm.sh logs`	tail server logs
`scripts/vm.sh status`	VM + service status
`scripts/vm.sh shell`	shell inside the VM
`scripts/vm.sh down`	stop the VM
`scripts/vm.sh destroy`	delete the VM

Reach the server at the VM's bridge IP (scripts/vm.sh status prints it).

Poke at it (Nushell)

let VMIP = (incus list musicfs -f csv -c 4 | lines | first | split row " " | first)
let ADDR = $"($VMIP):50051"

# liveness
^nc -z -w 3 $VMIP 50051
if $env.LAST_EXIT_CODE == 0 { print "alive" } else { print "dead" }

# gRPC: stream the manifest, count entries
grpcurl -plaintext -import-path proto -proto musicfs.proto -d "{}" $ADDR musicfs.MusicFs/GetManifest | lines | find relPath | length

# files the server scans
incus exec musicfs -- find /music -type f | lines | length

# play a track straight out of the VM's shared /music
incus exec musicfs -- cat "/music/DDT/ДДТ - Творчество в пустоте – 2 - 01 Intro.flac" | ^mpv -

Install mpv with nix profile install nixpkgs#mpv, or use ^ffplay - instead.

In Nushell, always put | lines between an external command's stdout and a builtin (find, first, length, …); external-to-external pipes (cat | mpv) are byte-stable and don't need it.

Development

just build            # cargo build
cargo test            # unit tests
just e2e              # end-to-end suite (scripts/e2e/run.sh)
just e2e-resilience   # resilience suite

Formatting and lint (clippy, treefmt with rustfmt + nixfmt) run as git hooks via devenv. Requires Rust edition 2024.

6.0 KiB Raw Blame History