MusicFS/README.org

#+title: musicfs

A read-only FUSE filesystem that presents a music library reorganised by
*metadata* rather than by however the files happen to sit on disk. Point it at a
messy ~~/Music~ tree (or a remote ~musicfs-server~) and it mounts a clean
=Artist/Album/Track= view, driven entirely by the tags parsed out of the audio
files themselves.

#+begin_example
mountpoint/
├── ДДТ/
│   └── Творчество в пустоте/
│       ├── 01 Intro.flac
│       └── 02 ...
└── Some Artist/
    └── Some Album/
        └── 01 Track.flac
#+end_example

The source files are never moved or modified. The directory hierarchy is
*virtual*: synthesised from ~album_artist~/~album~ tags, with stable synthetic
inodes so the layout survives remounts.

* How it works

Two *origins* feed the same FUSE frontend:

- *LocalOrigin* — walks a directory on the host, parses tags, builds the
  snapshot. A =notify= watcher keeps it live as files change.
- *NetworkOrigin* — talks gRPC to a remote ~musicfs-server~. Metadata and file
  bytes arrive over the wire; a Postgres table caches both. Reconciliation is
  hash-based: the client sends the ~(inode, hash)~ pairs it has, the server
  replies with only what changed or was deleted, so almost nothing crosses the
  network on a steady-state refresh.

Which one is used is chosen automatically from the ~--source~ argument: an
=http(s)://= URL means network, anything else is treated as a local path.

State lives in Postgres — parsed metadata, the virtual-path layout, and (for the
network origin) a lazy byte cache populated only for files that were actually
read.

* Layout

| Crate            | Responsibility                                                        |
|------------------+-----------------------------------------------------------------------|
| =musicfs-proto=  | Protobuf/gRPC definitions (=proto/musicfs.proto=), generated bindings |
| =musicfs-core=   | Shared logic: tag parsing (FLAC/MP3 via symphonia), hashing, logging  |
| =musicfs-client= | FUSE mount, origins, Postgres cache/sync — the =musicfs= binary       |
| =musicfs-server= | gRPC server sharing a library — the =musicfs-server= binary           |

The gRPC surface (see =proto/musicfs.proto=): ~Reconcile~, ~GetMetadata~,
~GetManifest~, ~GetFile~ (whole-file or byte-range streaming), and
~SubscribeEvents~ (change wake-ups). The client also serves a ~ClientStatus~ RPC
for introspection.

* Running the client

Everything runs inside the =devenv= shell, which provides the Rust toolchain,
Postgres, FUSE tooling, and gRPC utilities.

#+begin_src bash
devenv shell
devenv up --profile local     # starts Postgres + the FUSE mount
#+end_src

Predefined profiles in =devenv.nix=:

| Profile   | Source                          | Purpose                       |
|-----------+---------------------------------+-------------------------------|
| =local=   | a local directory               | mount a host music folder     |
| =remote=  | =http://…:50051=                | mount a remote musicfs-server |
| =e2e=     | =http://127.0.0.1:50061=        | end-to-end test harness       |

Or run the binary directly:

#+begin_src bash
cargo run -p musicfs-client -- \
  --source /home/you/Music \
  --mountpoint /tmp/musicfs \
  --database "postgresql://you@localhost/musicfs?host=$PGHOST"
#+end_src

Key flags: =--source= (local path or ~http(s)://~ server URL), =--mountpoint=,
=--database= (Postgres URL), =--log-dir= (daily-rotated logs, default =./logs=),
=--listen= (address for the client's status/health RPCs, default
=127.0.0.1:50052=).

* Running the server in an Incus VM

~musicfs-server~ (gRPC) runs inside an Incus VM, live-sharing the host's
~~/Music~ as a read-only virtiofs mount. =scripts/vm.sh= builds the binary on
the host, bundles its nix ~glibc~ so it runs unmodified in the VM, ships it in,
and runs it under systemd.

#+begin_src bash
devenv shell        # cargo, incus, patchelf, grpcurl
scripts/vm.sh up    # create VM, share ~/Music, build, ship, start server
#+end_src

Options (env): =VM_NAME=, =IMAGE=, =MUSIC_SOURCE=, =LISTEN_PORT=, =RUST_LOG=.

| Command                | Action                        |
|------------------------+-------------------------------|
| =scripts/vm.sh up=       | create + build + ship + start |
| =scripts/vm.sh redeploy= | rebuild after a code change   |
| =scripts/vm.sh logs=     | tail server logs              |
| =scripts/vm.sh status=   | VM + service status           |
| =scripts/vm.sh shell=    | shell inside the VM           |
| =scripts/vm.sh down=     | stop the VM                   |
| =scripts/vm.sh destroy=  | delete the VM                 |

Reach the server at the VM's bridge IP (=scripts/vm.sh status= prints it).

** Poke at it (Nushell)

#+begin_src nushell
let VMIP = (incus list musicfs -f csv -c 4 | lines | first | split row " " | first)
let ADDR = $"($VMIP):50051"

# liveness
^nc -z -w 3 $VMIP 50051
if $env.LAST_EXIT_CODE == 0 { print "alive" } else { print "dead" }

# gRPC: stream the manifest, count entries
grpcurl -plaintext -import-path proto -proto musicfs.proto -d "{}" $ADDR musicfs.MusicFs/GetManifest | lines | find relPath | length

# files the server scans
incus exec musicfs -- find /music -type f | lines | length

# play a track straight out of the VM's shared /music
incus exec musicfs -- cat "/music/DDT/ДДТ - Творчество в пустоте – 2 - 01 Intro.flac" | ^mpv -
#+end_src

Install mpv with =nix profile install nixpkgs#mpv=, or use =^ffplay -= instead.

In Nushell, always put =| lines= between an external command's stdout and a
builtin (=find=, =first=, =length=, …); external-to-external pipes (=cat | mpv=)
are byte-stable and don't need it.

* Development

#+begin_src bash
just build            # cargo build
cargo test            # unit tests
just e2e              # end-to-end suite (scripts/e2e/run.sh)
just e2e-resilience   # resilience suite
#+end_src

Formatting and lint (clippy, treefmt with rustfmt + nixfmt) run as git hooks via
=devenv=. Requires Rust edition 2024.