backlog/doc/architecture/architecture.md

Personal Backlog — Architecture

1. Philosophy

One deliverable. The app ships as a single self-contained HTML file (webapp/index-style-v2.html). It talks to the filesystem either through a minimal localhost Python server or directly via the browser's File System Access API — no cloud, no accounts, no external CDNs at runtime.

The user chooses the mode by how they open the app: run python3 server/server.py for universal browser support, or open the HTML file directly in Chrome/Edge for a zero-install experience.

The source code lives in web/ as React 18 + JSX files and is compiled to the single-file artifact by web/bundle.js. The build step is only needed when changing source; users only ever touch the output.

2. Deployment Topology

Mode A — API Server (universal browsers)

┌─────────────────────────────────────┐
│  Browser (Chrome / Firefox / Safari)│
│  ─────────────────────────────────  │
│  Single self-contained HTML file    │
│  ├─ inline CSS                      │
│  ├─ inline React 18 + app JS        │
│  └─ inline SVG icons                │
└──────────────┬──────────────────────┘
               │ HTTP (localhost)
┌──────────────▼──────────────────────┐
│  Python File Server (~420 LoC)      │
│  Python 3 stdlib, zero pip deps     │
│  Serves HTML + REST API for disk    │
└──────────────┬──────────────────────┘
               │ read / write / watch
    ┌──────────┼──────────┐
    ▼          ▼          ▼
backlog.md  backups/   stats.jsonl
(master)    (rotating) (append-only)

Mode B — Direct File Access (Chrome/Edge, zero server)

┌────────────────────────────────────────┐
│  Browser (Chrome / Edge)               │
│  ────────────────────────────────────  │
│  HTML file opened via file:// or       │
│  served by any static file server      │
│  ├─ inline CSS                         │
│  ├─ inline React 18 + app JS           │
│  └─ inline SVG icons                   │
└──────────────┬─────────────────────────┘
               │ File System Access API
               │ (showDirectoryPicker)
    ┌──────────┼──────────┐
    ▼          ▼          ▼
backlog.md  backups/   stats.jsonl
(master)    (rotating) (append-only)

Note: Firefox and Safari do not support the File System Access API. Opening the HTML file directly in those browsers starts the app in read-only mode with a warning banner. Use Mode A (Python server) for full functionality in any browser.

3. Technology Stack

Layer	Choice	Rationale
Frontend runtime	React 18 (production UMD, inlined)	Component model for complex UI; inlined so there are no runtime CDN requests.
Frontend source	JSX + CSS in web/	Multi-file development ergonomics; compiled to single HTML by bundler.
Build tooling	bundle.js + @babel/core	Pre-compiles JSX, inlines prod React builds, removes Babel. Dev-only dependency.
Styling	External styles.css (inlined at build)	Edited as plain CSS; inlined into the bundle so the output file is self-contained.
Icons / graphics	Inline SVG	Scalable, styleable with CSS, no network requests.
Server (API mode)	Python 3 http.server	Ships with macOS/Linux/Windows; zero pip installs.
Server (Direct mode)	File System Access API	Native browser API for Chrome/Edge; zero install. Not available in Firefox/Safari.
Master storage	Markdown file (backlog.md)	Human-readable, diff-friendly, matches requirements exactly.
Backup storage	Filesystem directory (backups/)	Simple rotation via filename sorting.
Stats storage	JSONL file (stats.jsonl)	Append-only, no locking complexity, human-readable, trivial to parse.
Transport	HTTP/1.1 + JSON	Universally supported, trivial to debug with curl.

4. Frontend Architecture

The source code is split across files in web/ and compiled to a single self-contained HTML file in webapp/. Logical modules are organized into three layers so that UI components and business logic never know which storage backend is active.

4.1 Layered Module Map

┌─────────────────────────────────────────────────────────────┐
│  PRESENTATION LAYER                                         │
│  ─────────────────                                          │
│  <App>, <TreeItem>, <AdminPage>, <FilterPanel>,             │
│  <TweaksPanel>, <ItemDialog>, <ConfirmDialog>,              │
│  <ImportExportDialog>                                       │
│  React components — only read/write data through props      │
│  and callbacks passed down from App.                        │
├─────────────────────────────────────────────────────────────┤
│  DOMAIN / BUSINESS LOGIC LAYER                              │
│  ────────────────────────────                               │
│  Parser, buildDataFromStorage, filter logic in App          │
│  Parse markdown → data tree; serialize data tree → markdown │
│  Compute stats, enforce Parent Visibility Rule.             │
├─────────────────────────────────────────────────────────────┤
│  INFRASTRUCTURE LAYER                                       │
│  ────────────────────                                       │
│  Storage, ApiBackend, DirectBackend, SyncPoller             │
│  Route reads/writes to disk (API or File System API).       │
└─────────────────────────────────────────────────────────────┘

4.2 Source File Reference

File	Layer	Responsibility
app.jsx	App	Root <App> component. All state lives here via React hooks. Storage init, save/load lifecycle, filter state, debounced autosave.
storage.jsx	Infrastructure + Domain	Parser (parse/serialize markdown), ApiBackend, DirectBackend, Storage (detect + delegate), SyncPoller, buildDataFromStorage.
tree.jsx	Presentation	Recursive <TreeItem> component. Expand/collapse, drag-and-drop reorder, status glyphs, progress bars.
admin.jsx	Presentation	<AdminPage> — health card, storage stats, backup browser, stats chart, manual actions.
filter-panel.jsx	Presentation	Left sidebar: status/priority/tag/date filters with multi-select.
tweaks-panel.jsx	Presentation	Settings slide-out: density, accent hue, status glyph style.
dialogs.jsx	Presentation	<ItemDialog>, <ConfirmDialog>, <ImportExportDialog>.
helpers.jsx	Domain	walkTree, findItem, countAll, countByStatus, useTweaks hook, formatting utilities.

4.3 State Management

All state lives in the <App> component via React hooks. No external state library.

const [data, setData]               = useState(buildEmptyData);
// data = { entries, history, meta, health, stats, backups }

const [storageMode, setStorageMode] = useState('local'); // 'api' | 'direct' | 'local'
const [filters, setFilters]         = useState({ statuses, priorities, ... });
const [expandedMap, setExpandedMap] = useState({});   // id → bool, persisted to localStorage
const [saveState, setSaveState]     = useState({ status, lastSaved });

Only expandedMap is persisted to localStorage. All backlog data is always sourced from backlog.md.

4.4 Rendering Strategy

React 18 reconciliation. The full tree re-renders on data or filter changes; React diffs and patches only changed DOM nodes. Filters are applied in the render path — the canonical data.entries is never mutated by filtering.

Responsive breakpoints:

• Desktop: sidebar filter panel + main tree.
• Mobile (< 768 px): filter panel becomes a collapsible drawer.

4.5 Dual-Mode Storage

Storage is a thin router that picks the active backend at boot time.

Detection order:

1. Probe GET http://localhost:8080/api/health. If it responds within 500 ms → use ApiBackend.
2. Else if window.showDirectoryPicker is available → use DirectBackend.
3. Else show a message: "Please run python3 server.py or open this page in Chrome/Edge."

ApiBackend

• load() → GET /api/backlog
• save(content) → POST /api/backlog
• listBackups() → GET /api/backups
• restoreBackup(name) → POST /api/backups/restore
• getStats() → GET /api/stats
• appendStats(event) → POST /api/stats

DirectBackend

• On first launch the user is prompted to pick a directory via showDirectoryPicker().
• The directory handle is stored in IndexedDB (pb-storage-v2 → handles → root).
• On reload the handle is retrieved from IndexedDB; queryPermission() checks silently — no user gesture needed if permission is still active. If it has lapsed, a connect button is shown.
• load() → dirHandle.getFileHandle('backlog.md').getFile() → read text.
• save(content) → dirHandle.getFileHandle('backlog.md', { create: true }).createWritable() → write → close.
• listBackups() → iterate dirHandle.getDirectoryHandle('backups', { create: true }).
• restoreBackup(name) → copy backup file handle content over master.
• getStats() / appendStats() → read/write stats.jsonl via the same directory handle.
• Backup rotation runs in the frontend (same algorithm as server) after every successful save.

5. Backend Architecture

5.1 Server (server.py)

A single Python file (~420 LoC) extending http.server.BaseHTTPRequestHandler. Zero external dependencies — stdlib only.

Responsibilities:

1. Serve index-style-v2.html for GET /.
2. Handle REST API routes.
3. Perform atomic file writes.
4. Manage backup rotation.
5. Append stats events.
6. Compute SHA-256 integrity markers on write; verify on read for warning purposes only (never block loading).

5.2 REST API

Method	Path	Description
GET	/	Serve index-style-v2.html
GET	/api/backlog?checksum=<optional>	Return current markdown content + checksum. If checksum matches, return 304 Not Modified. Checksum mismatch never blocks loading.
POST	/api/backlog	Accept JSON { content }. Write atomically, create backup, verify integrity marker is syntactically valid, return new checksum.
GET	/api/backups	List backups: [{ name, size, timestamp, valid }].
GET	/api/backups/<name>	Download a backup file.
POST	/api/backups/restore	Body { name }. Copy backup over master after checking backup is readable.
POST	/api/export	Body { format: "json" }. Return structured dump.
POST	/api/import	Multipart upload. Validate, then replace master.
POST	/api/stats	Body { event, payload }. Append one line to stats.jsonl.
GET	/api/stats?from=&to=&aggregate=	Read stats events and/or roll-ups.
GET	/api/health	{ status, lastSave, lastBackup, masterSize, backupCount }.

5.3 Atomic Write Sequence

Client POSTs new markdown content
         │
         ▼
Server writes to backlog.md.tmp
         │
         ▼
Server parses tmp file, verifies integrity marker is syntactically valid
         │
         ▼
Server copies tmp → backups/backlog_YYYY-MM-DD_HH-mm-ss.md
         │
         ▼
Server runs backup rotation (prune old files)
         │
         ▼
Server renames tmp → backlog.md
         │
         ▼
Server appends "save_completed" event to stats.jsonl
         │
         ▼
Server returns { ok: true, checksum, saved }

5.4 Backup Rotation Algorithm

def rotate_backups(backups_dir):
    files = sorted(glob("backlog_*.md"), key=extract_timestamp)
    # Keep everything for 7 days
    for f in files:
        if age(f) <= 7 days:
            continue
        # After 7 days: keep only the newest per calendar day
        day = extract_calendar_day(f)
        if f != newest_file_for_day(day):
            os.remove(f)
    # Hard cap: never delete the single most recent backup

5.5 External Change Detection

• API mode: polls GET /api/backlog?checksum=<local> every 5 seconds.
  • 304 Not Modified → nothing to do.
  • 200 + new content → file changed externally.
• Direct mode: calls Storage.load() every 5 seconds and compares checksums.

In both modes:

• If Store.dirty === false: auto-reload, show toast notification.
• If Store.dirty === true: show conflict modal with options:
  • Overwrite local (discard unsaved edits, load disk version).
  • Force save (overwrite disk with local version).
  • Download both (save local as file, then reload disk version).

6. Data Layer Details

6.1 Master File (backlog.md)

Exactly as specified in Requirements §3.2. The server and client both know how to parse and generate the three sections (ENTRIES, HISTORY, INTEGRITY).

Checksum algorithm:

sha256( utf8_bytes( entries_section + "\n" + history_section ) )

The integrity marker comment is excluded from the hash.

Checksum policy:

• On save, the writer computes the hash and appends the marker. This proves the file was written completely.
• On load, the reader computes the hash and compares it to the stored marker.
  • If they match → green status, no banner.
  • If they mismatch → yellow warning banner ("Checksum mismatch — file was edited outside the app"). The file still loads normally. The next save will overwrite the marker with the correct value.
  • If the marker is missing → same yellow warning.

6.2 Stats File (stats.jsonl)

One JSON object per line, newline-delimited:

{"t":"2025-05-10T14:32:01Z","e":"item_created","d":{"id":"task-1","level":3,"project":"proj-1"}}
{"t":"2025-05-10T14:35:00Z","e":"save_completed","d":{"size":12400,"ms":45}}

The server exposes a simple aggregator that reads the last 90 days of lines and computes roll-ups on demand. For large files, a memory-mapped or seek-from-end strategy can be used.

7. Safety Considerations

This is a single-user local tool, not a production service. The only real threats are data loss and file corruption.

Concern	Mitigation
Data loss	Atomic writes (temp file + rename) + automatic backup on every save + rotating backup retention.
File corruption	Integrity marker proves the file was written completely; parser validates structure before accepting.
Path traversal	Backup names are generated by the server, never from user input. Restore validates the file exists and is readable.
Network exposure	Server binds to 0.0.0.0 for convenience (access from phone/tablet on same LAN). No auth — this is intentional for a personal tool.

8. Build & Distribution

The repository contains:

personal-backlog/
├── server/
│   └── server.py           # Python REST API server (stdlib only)
├── web/                    # V2 source (JSX + CSS + bundler)
├── webapp/
│   └── index-style-v2.html # Self-contained SPA (built artifact)
├── backlog.md              # master data file (created on first save)
├── backups/                # created automatically
└── stats.jsonl             # created automatically

8.1 Running the App

Mode A — API Server (all browsers):

python3 server/server.py --port 8080 --dir ~/my-backlog

Then open http://localhost:8080.

Mode B — Direct Access (Chrome/Edge, zero server):

open webapp/index-style-v2.html

The app will prompt you to pick a directory. Create a folder, select it, and the app will create backlog.md, backups/, and stats.jsonl inside it.

Building the bundle from source:

cd web && npm install
node bundle.js index.html ../webapp/index-style-v2.html

8.2 First-Time Setup

• API mode: If backlog.md does not exist in the server --dir, the server creates a blank template with an empty ENTRIES section, an empty HISTORY table, and a valid INTEGRITY marker.
• Direct mode: If the selected directory does not contain backlog.md, the frontend creates the same blank template via DirectBackend.

9. Performance Budget

Target	Limit	How
Initial load	< 200 KB transferred	Single HTML, no external assets.
Startup time	< 1 s for ≤ 1 MB backlog	Parse markdown in one pass; lazy-load admin charts.
Save latency	< 300 ms	Atomic rename is instant; SHA-256 of 1 MB is ~5 ms.
Render tree	< 50 ms for 500 items	Reuse DOM nodes where possible; virtual scrolling optional future enhancement.
Poll overhead	Negligible	304 responses are empty body; checksum is cached server-side.

10. Future Extensibility (No-Regret Decisions)

1. Dark mode — CSS custom properties (:root variables) are used for all colors, so a theme switch is a one-line class toggle.
2. Multi-user — The server already validates checksums; adding a simple session token or Basic Auth header would be trivial if needed later.
3. Firefox/Safari direct access — If those browsers ever gain write access via File System Access API, DirectBackend will work without code changes.