fujin/backlog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial version

Browse Source
This commit is contained in:
Ilia Sharin
2026-05-05 18:42:06 -04:00
commit a2a3b7b72b
21 changed files with 14258 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/architecture/architecture.md
+350
View File
@@ -0,0 +1,350 @@
# Personal Backlog — Architecture
## 1. Philosophy
> **One deliverable.** The app ships as a single self-contained HTML file (`webapp/index-style-v2.html`). It talks to the filesystem either through a minimal localhost Python server **or** directly via the browser's File System Access API — no cloud, no accounts, no external CDNs at runtime.
>
> The user chooses the mode by how they open the app: run `python3 server/server.py` for universal browser support, or open the HTML file directly in Chrome/Edge for a zero-install experience.
>
> *The source code lives in `web/` as React 18 + JSX files and is compiled to the single-file artifact by `web/bundle.js`. The build step is only needed when changing source; users only ever touch the output.*
## 2. Deployment Topology
### Mode A — API Server (universal browsers)
```
┌─────────────────────────────────────┐
│ Browser (Chrome / Firefox / Safari)│
│ ───────────────────────────────── │
│ Single self-contained HTML file │
│ ├─ inline CSS │
│ ├─ inline React 18 + app JS │
│ └─ inline SVG icons │
└──────────────┬──────────────────────┘
│ HTTP (localhost)
┌──────────────▼──────────────────────┐
│ Python File Server (~420 LoC) │
│ Python 3 stdlib, zero pip deps │
│ Serves HTML + REST API for disk │
└──────────────┬──────────────────────┘
│ read / write / watch
┌──────────┼──────────┐
▼ ▼ ▼
backlog.md backups/ stats.jsonl
(master) (rotating) (append-only)
```
### Mode B — Direct File Access (Chrome/Edge, zero server)
```
┌────────────────────────────────────────┐
│ Browser (Chrome / Edge) │
│ ──────────────────────────────────── │
│ HTML file opened via file:// or │
│ served by any static file server │
│ ├─ inline CSS │
│ ├─ inline React 18 + app JS │
│ └─ inline SVG icons │
└──────────────┬─────────────────────────┘
│ File System Access API
│ (showDirectoryPicker)
┌──────────┼──────────┐
▼ ▼ ▼
backlog.md backups/ stats.jsonl
(master) (rotating) (append-only)
```
> **Note:** Firefox and Safari do not support the File System Access API. Opening the HTML file directly in those browsers starts the app in read-only mode with a warning banner. Use Mode A (Python server) for full functionality in any browser.
## 3. Technology Stack
| Layer | Choice | Rationale |
|-------|--------|-----------|
| **Frontend runtime** | React 18 (production UMD, inlined) | Component model for complex UI; inlined so there are no runtime CDN requests. |
| **Frontend source** | JSX + CSS in `web/` | Multi-file development ergonomics; compiled to single HTML by bundler. |
| **Build tooling** | `bundle.js` + `@babel/core` | Pre-compiles JSX, inlines prod React builds, removes Babel. Dev-only dependency. |
| **Styling** | External `styles.css` (inlined at build) | Edited as plain CSS; inlined into the bundle so the output file is self-contained. |
| **Icons / graphics** | Inline SVG | Scalable, styleable with CSS, no network requests. |
| **Server (API mode)** | Python 3 `http.server` | Ships with macOS/Linux/Windows; zero pip installs. |
| **Server (Direct mode)** | File System Access API | Native browser API for Chrome/Edge; zero install. Not available in Firefox/Safari. |
| **Master storage** | Markdown file (`backlog.md`) | Human-readable, diff-friendly, matches requirements exactly. |
| **Backup storage** | Filesystem directory (`backups/`) | Simple rotation via filename sorting. |
| **Stats storage** | JSONL file (`stats.jsonl`) | Append-only, no locking complexity, human-readable, trivial to parse. |
| **Transport** | HTTP/1.1 + JSON | Universally supported, trivial to debug with curl. |
## 4. Frontend Architecture
The source code is split across files in `web/` and compiled to a single self-contained HTML file in `webapp/`. Logical modules are organized into three layers so that UI components and business logic never know which storage backend is active.
### 4.1 Layered Module Map
```
┌─────────────────────────────────────────────────────────────┐
│ PRESENTATION LAYER │
│ ───────────────── │
│ <App>, <TreeItem>, <AdminPage>, <FilterPanel>, │
│ <TweaksPanel>, <ItemDialog>, <ConfirmDialog>, │
│ <ImportExportDialog> │
│ React components — only read/write data through props │
│ and callbacks passed down from App. │
├─────────────────────────────────────────────────────────────┤
│ DOMAIN / BUSINESS LOGIC LAYER │
│ ──────────────────────────── │
│ Parser, buildDataFromStorage, filter logic in App │
│ Parse markdown → data tree; serialize data tree → markdown │
│ Compute stats, enforce Parent Visibility Rule. │
├─────────────────────────────────────────────────────────────┤
│ INFRASTRUCTURE LAYER │
│ ──────────────────── │
│ Storage, ApiBackend, DirectBackend, SyncPoller │
│ Route reads/writes to disk (API or File System API). │
└─────────────────────────────────────────────────────────────┘
```
### 4.2 Source File Reference
| File | Layer | Responsibility |
|------|-------|----------------|
| `app.jsx` | App | Root `<App>` component. All state lives here via React hooks. Storage init, save/load lifecycle, filter state, debounced autosave. |
| `storage.jsx` | Infrastructure + Domain | `Parser` (parse/serialize markdown), `ApiBackend`, `DirectBackend`, `Storage` (detect + delegate), `SyncPoller`, `buildDataFromStorage`. |
| `tree.jsx` | Presentation | Recursive `<TreeItem>` component. Expand/collapse, drag-and-drop reorder, status glyphs, progress bars. |
| `admin.jsx` | Presentation | `<AdminPage>` — health card, storage stats, backup browser, stats chart, manual actions. |
| `filter-panel.jsx` | Presentation | Left sidebar: status/priority/tag/date filters with multi-select. |
| `tweaks-panel.jsx` | Presentation | Settings slide-out: density, accent hue, status glyph style. |
| `dialogs.jsx` | Presentation | `<ItemDialog>`, `<ConfirmDialog>`, `<ImportExportDialog>`. |
| `helpers.jsx` | Domain | `walkTree`, `findItem`, `countAll`, `countByStatus`, `useTweaks` hook, formatting utilities. |
### 4.3 State Management
All state lives in the `<App>` component via React hooks. No external state library.
```javascript
const [data, setData] = useState(buildEmptyData);
// data = { entries, history, meta, health, stats, backups }
const [storageMode, setStorageMode] = useState('local'); // 'api' | 'direct' | 'local'
const [filters, setFilters] = useState({ statuses, priorities, ... });
const [expandedMap, setExpandedMap] = useState({}); // id → bool, persisted to localStorage
const [saveState, setSaveState] = useState({ status, lastSaved });
```
Only `expandedMap` is persisted to `localStorage`. All backlog data is always sourced from `backlog.md`.
### 4.4 Rendering Strategy
React 18 reconciliation. The full tree re-renders on data or filter changes; React diffs and patches only changed DOM nodes. Filters are applied in the render path — the canonical `data.entries` is never mutated by filtering.
Responsive breakpoints:
- Desktop: sidebar filter panel + main tree.
- Mobile (< 768 px): filter panel becomes a collapsible drawer.
### 4.5 Dual-Mode Storage
`Storage` is a thin router that picks the active backend at boot time.
**Detection order:**
1. Probe `GET http://localhost:8080/api/health`. If it responds within 500 ms → use `ApiBackend`.
2. Else if `window.showDirectoryPicker` is available → use `DirectBackend`.
3. Else show a message: "Please run `python3 server.py` or open this page in Chrome/Edge."
#### ApiBackend
- `load()` → `GET /api/backlog`
- `save(content)` → `POST /api/backlog`
- `listBackups()` → `GET /api/backups`
- `restoreBackup(name)` → `POST /api/backups/restore`
- `getStats()` → `GET /api/stats`
- `appendStats(event)` → `POST /api/stats`
#### DirectBackend
- On first launch the user is prompted to pick a **directory** via `showDirectoryPicker()`.
- The directory handle is stored in IndexedDB (`pb-storage-v2` → `handles` → `root`).
- On reload the handle is retrieved from IndexedDB; `queryPermission()` checks silently — no user gesture needed if permission is still active. If it has lapsed, a connect button is shown.
- `load()` → `dirHandle.getFileHandle('backlog.md').getFile()` → read text.
- `save(content)` → `dirHandle.getFileHandle('backlog.md', { create: true }).createWritable()` → write → close.
- `listBackups()` → iterate `dirHandle.getDirectoryHandle('backups', { create: true })`.
- `restoreBackup(name)` → copy backup file handle content over master.
- `getStats()` / `appendStats()` → read/write `stats.jsonl` via the same directory handle.
- Backup rotation runs in the frontend (same algorithm as server) after every successful save.
## 5. Backend Architecture
### 5.1 Server (`server.py`)
A single Python file (~420 LoC) extending `http.server.BaseHTTPRequestHandler`. Zero external dependencies — stdlib only.
Responsibilities:
1. Serve `index-style-v2.html` for `GET /`.
2. Handle REST API routes.
3. Perform atomic file writes.
4. Manage backup rotation.
5. Append stats events.
6. Compute SHA-256 integrity markers on write; verify on read for warning purposes only (never block loading).
### 5.2 REST API
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/` | Serve `index-style-v2.html` |
| `GET` | `/api/backlog?checksum=<optional>` | Return current markdown content + checksum. If `checksum` matches, return `304 Not Modified`. Checksum mismatch never blocks loading. |
| `POST` | `/api/backlog` | Accept JSON `{ content }`. Write atomically, create backup, verify integrity marker is syntactically valid, return new checksum. |
| `GET` | `/api/backups` | List backups: `[{ name, size, timestamp, valid }]`. |
| `GET` | `/api/backups/<name>` | Download a backup file. |
| `POST` | `/api/backups/restore` | Body `{ name }`. Copy backup over master after checking backup is readable. |
| `POST` | `/api/export` | Body `{ format: "json" }`. Return structured dump. |
| `POST` | `/api/import` | Multipart upload. Validate, then replace master. |
| `POST` | `/api/stats` | Body `{ event, payload }`. Append one line to `stats.jsonl`. |
| `GET` | `/api/stats?from=&to=&aggregate=` | Read stats events and/or roll-ups. |
| `GET` | `/api/health` | `{ status, lastSave, lastBackup, masterSize, backupCount }`. |
### 5.3 Atomic Write Sequence
```
Client POSTs new markdown content
│
▼
Server writes to backlog.md.tmp
│
▼
Server parses tmp file, verifies integrity marker is syntactically valid
│
▼
Server copies tmp → backups/backlog_YYYY-MM-DD_HH-mm-ss.md
│
▼
Server runs backup rotation (prune old files)
│
▼
Server renames tmp → backlog.md
│
▼
Server appends "save_completed" event to stats.jsonl
│
▼
Server returns { ok: true, checksum, saved }
```
### 5.4 Backup Rotation Algorithm
```python
def rotate_backups(backups_dir):
files = sorted(glob("backlog_*.md"), key=extract_timestamp)
# Keep everything for 7 days
for f in files:
if age(f) <= 7 days:
continue
# After 7 days: keep only the newest per calendar day
day = extract_calendar_day(f)
if f != newest_file_for_day(day):
os.remove(f)
# Hard cap: never delete the single most recent backup
```
### 5.5 External Change Detection
- **API mode**: polls `GET /api/backlog?checksum=<local>` every 5 seconds.
- `304 Not Modified` → nothing to do.
- `200 + new content` → file changed externally.
- **Direct mode**: calls `Storage.load()` every 5 seconds and compares checksums.
In both modes:
- If `Store.dirty === false`: auto-reload, show toast notification.
- If `Store.dirty === true`: show conflict modal with options:
- **Overwrite local** (discard unsaved edits, load disk version).
- **Force save** (overwrite disk with local version).
- **Download both** (save local as file, then reload disk version).
## 6. Data Layer Details
### 6.1 Master File (`backlog.md`)
Exactly as specified in [Requirements §3.2](../requirements/requirements.md). The server and client both know how to parse and generate the three sections (`ENTRIES`, `HISTORY`, `INTEGRITY`).
**Checksum algorithm:**
```
sha256( utf8_bytes( entries_section + "\n" + history_section ) )
```
The integrity marker comment is excluded from the hash.
**Checksum policy:**
- On **save**, the writer computes the hash and appends the marker. This proves the file was written completely.
- On **load**, the reader computes the hash and compares it to the stored marker.
- If they match → green status, no banner.
- If they mismatch → **yellow warning banner** ("Checksum mismatch — file was edited outside the app"). The file still loads normally. The next save will overwrite the marker with the correct value.
- If the marker is missing → same yellow warning.
### 6.2 Stats File (`stats.jsonl`)
One JSON object per line, newline-delimited:
```jsonl
{"t":"2025-05-10T14:32:01Z","e":"item_created","d":{"id":"task-1","level":3,"project":"proj-1"}}
{"t":"2025-05-10T14:35:00Z","e":"save_completed","d":{"size":12400,"ms":45}}
```
The server exposes a simple aggregator that reads the last 90 days of lines and computes roll-ups on demand. For large files, a memory-mapped or seek-from-end strategy can be used.
## 7. Safety Considerations
This is a single-user local tool, not a production service. The only real threats are data loss and file corruption.
| Concern | Mitigation |
|---------|------------|
| **Data loss** | Atomic writes (temp file + rename) + automatic backup on every save + rotating backup retention. |
| **File corruption** | Integrity marker proves the file was written completely; parser validates structure before accepting. |
| **Path traversal** | Backup names are generated by the server, never from user input. Restore validates the file exists and is readable. |
| **Network exposure** | Server binds to `0.0.0.0` for convenience (access from phone/tablet on same LAN). No auth — this is intentional for a personal tool. |
## 8. Build & Distribution
The repository contains:
```
personal-backlog/
├── server/
│ └── server.py # Python REST API server (stdlib only)
├── web/ # V2 source (JSX + CSS + bundler)
├── webapp/
│ └── index-style-v2.html # Self-contained SPA (built artifact)
├── backlog.md # master data file (created on first save)
├── backups/ # created automatically
└── stats.jsonl # created automatically
```
### 8.1 Running the App
**Mode A — API Server (all browsers):**
```bash
python3 server/server.py --port 8080 --dir ~/my-backlog
```
Then open `http://localhost:8080`.
**Mode B — Direct Access (Chrome/Edge, zero server):**
```bash
open webapp/index-style-v2.html
```
The app will prompt you to pick a directory. Create a folder, select it, and the app will create `backlog.md`, `backups/`, and `stats.jsonl` inside it.
**Building the bundle from source:**
```bash
cd web && npm install
node bundle.js index.html ../webapp/index-style-v2.html
```
### 8.2 First-Time Setup
- **API mode**: If `backlog.md` does not exist in the server `--dir`, the server creates a blank template with an empty `ENTRIES` section, an empty `HISTORY` table, and a valid `INTEGRITY` marker.
- **Direct mode**: If the selected directory does not contain `backlog.md`, the frontend creates the same blank template via `DirectBackend`.
## 9. Performance Budget
| Target | Limit | How |
|--------|-------|-----|
| Initial load | < 200 KB transferred | Single HTML, no external assets. |
| Startup time | < 1 s for ≤ 1 MB backlog | Parse markdown in one pass; lazy-load admin charts. |
| Save latency | < 300 ms | Atomic rename is instant; SHA-256 of 1 MB is ~5 ms. |
| Render tree | < 50 ms for 500 items | Reuse DOM nodes where possible; virtual scrolling optional future enhancement. |
| Poll overhead | Negligible | 304 responses are empty body; checksum is cached server-side. |
## 10. Future Extensibility (No-Regret Decisions)
1. **Dark mode** — CSS custom properties (`:root` variables) are used for all colors, so a theme switch is a one-line class toggle.
2. **Multi-user** — The server already validates checksums; adding a simple session token or Basic Auth header would be trivial if needed later.
3. **Firefox/Safari direct access** — If those browsers ever gain write access via File System Access API, `DirectBackend` will work without code changes.
doc/architecture/tdd.md
+623
View File
@@ -0,0 +1,623 @@
# Technical Design Document — Personal Backlog
## 1. Overview
This document describes the full technical implementation of the Personal Backlog application — a single-user, locally-hosted task manager backed by a single Markdown file. It covers both the HTML frontend implementations (v1 vanilla JS and v2 React 18), the Python server, the bundling pipeline, and the data contract that ties them together.
## 2. Source Code Structure
```
personal-backlog/
├── server/
│ └── server.py # Python REST API server (~420 LoC)
│
├── web/ # V2 source code (React 18 + JSX) + build tooling
│ ├── index.html # Dev entry point (React + Babel from CDN)
│ ├── styles.css # All CSS
│ ├── storage.jsx # Parser, ApiBackend, DirectBackend, Storage, SyncPoller
│ ├── helpers.jsx # walkTree, findItem, useTweaks hook, event utilities
│ ├── app.jsx # Root <App> component, state, save/load lifecycle
│ ├── tree.jsx # <TreeItem> recursive component
│ ├── dialogs.jsx # Modal dialogs (add/edit/delete/import/export)
│ ├── admin.jsx # Admin dashboard page
│ ├── filter-panel.jsx # Left sidebar: status/priority/tag/date filters
│ ├── tweaks-panel.jsx # Settings panel (density, accent hue, status style)
│ ├── data.jsx # Seed data for testing (not loaded by default)
│ ├── bundle.js # Build tool: multi-file JSX → single-file HTML
│ ├── package.json # Declares @babel/core, @babel/preset-react
│ └── node_modules/
│
├── webapp/ # Built output — ready to open or deploy
│ └── index-style-v2.html # Self-contained single-file SPA (~348 KB)
│
├── design/
│ ├── design-v1/ # V1 source (archived, vanilla JS)
│ └── archive/ # Earlier prototypes
│
└── doc/
├── requirements/requirements.md
└── architecture/
├── architecture.md
└── tdd.md # This file
```
## 3. Data Contract — `backlog.md` Format
The Markdown file is the sole source of truth. Both the Python server and the JavaScript frontend must parse and produce this exact format.
### 3.1 File Structure
```markdown
# Backlog
<!-- SECTION: ENTRIES -->
- [ ] [P0] Task title *(due: 2025-06-01, priority: P0, progress: 50)*
- [x] [P1] Sub-task *(priority: P1, progress: 100)*
- [!] [P1] Blocked task *(priority: P1, reason: waiting for API keys)*
<!-- SECTION: HISTORY -->
| Timestamp | Item ID | Action | Details |
|-----------|---------|--------|---------|
| 2025-05-10T14:32:00Z | i-m1 | status_changed | open → done |
<!-- SECTION: INTEGRITY -->
<!-- saved: 2025-05-10T14:35:12Z | checksum: sha256:abc123... | entries: 42 | history: 128 -->
```
Three mandatory section markers: `<!-- SECTION: ENTRIES -->`, `<!-- SECTION: HISTORY -->`, `<!-- SECTION: INTEGRITY -->`.
### 3.2 Entry Line Format
```
<indent>- [<glyph>] [<priority>] <title> *(<metadata>)*
```
| Part | Format | Example |
|------|--------|---------|
| Indent | 2 spaces per level | ` ` (level 2) |
| Glyph | `[ ]` `[x]` `[!]` `[>]` `[/]` `[-]` | `[x]` |
| Priority prefix | `[P0]` through `[P3]`, followed by space | `[P0] ` |
| Title | Free text | `Ship landing page` |
| Metadata | `*(key: value, ...)*` | `*(due: 2025-06-01, progress: 50)*` |
**Glyph-to-status mapping:**
| Glyph | Status |
|-------|--------|
| `[ ]` | `open` |
| `[/]` | `in-progress` |
| `[!]` | `blocked` |
| `[>]` | `postponed` |
| `[x]` | `done` |
| `[-]` | `cancelled` |
**Metadata keys** (all optional, comma-separated inside `*(...)*`):
| Key | Value format | Default |
|-----|-------------|---------|
| `due` | ISO date `YYYY-MM-DD` | null |
| `priority` | `P0`–`P3` | `P1` |
| `progress` | Integer `0–100` | `0` |
| `reason` | Free text (for blocked items) | null |
| `tags` | Space-separated words | [] |
### 3.3 Checksum Algorithm
```
payload = entries_section_text.trim() + "\n" + history_section_text.trim()
hash = "sha256:" + SHA-256(UTF-8(payload))
```
The `entries_section_text` is everything between `<!-- SECTION: ENTRIES -->` and `<!-- SECTION: HISTORY -->`, excluding the markers themselves. Similarly for history. The integrity marker comment is excluded from the hash.
**Policy:** On save, the writer computes and writes the hash. On load, the reader verifies it. Mismatch = yellow warning banner, never blocks loading. Next save overwrites with the correct hash.
### 3.4 Parser Regex (Critical Implementation Detail)
The parser extracts metadata from entry lines using this regex:
```javascript
const metaM = raw.match(/^(.*?)\s*\*\((.*)\)\*\s*$/);
```
This matches the `*(...)*` wrapper at the end of a line. The trailing `\*\s*$` is essential — the metadata format wraps with `)*` (paren + asterisk), not just `)`. A previous bug where the regex ended with `\)\s*$` caused the entire metadata string to be absorbed into the title on every save/load cycle.
The priority prefix is extracted separately:
```javascript
const pm = title.match(/^\[(P\d)\]\s*/);
if (pm) { priority = pm[1]; title = title.slice(pm[0].length); }
```
### 3.5 Serializer Output Format
The serializer writes entries in this format:
```
<indent>- [<glyph>] [<priority>] <title> *(<metadata>)*
```
**Serialization rules:**
- Priority prefix `[Pn]` is always written before the title
- Metadata `*(...)*` is appended only when at least one metadata field is non-default
- `priority: P1` is omitted from metadata (it's the default, already shown as prefix)
- `progress: 0` is omitted from metadata (default)
- `progress` is only included when > 0
## 4. Frontend Implementation — V2 (React 18)
### 4.1 Component Tree
```
<App>
├── <FilterPanel> (left sidebar)
├── Main area
│ ├── Header + search bar
│ ├── <TreeItem> (recursive, one per entry)
│ │ └── <TreeItem> ... (children)
│ └── Add-item button
├── <AdminPage> (gear icon route)
│ ├── System Health card
│ ├── Storage card (with filesystem paths)
│ ├── Backup Browser
│ ├── Stats Overview
│ └── Manual Actions
├── <TweaksPanel> (settings slide-out)
├── <ItemDialog> (add/edit modal)
├── <ConfirmDialog> (confirmation modal)
└── <ImportExportDialog> (import/export modal)
```
### 4.2 State Management
All state lives in `App` component via React hooks. No external state library.
```javascript
const [data, setData] = useState(buildEmptyData);
const [storageMode, setStorageMode] = useState('local'); // 'api' | 'direct' | 'local'
const [filters, setFilters] = useState({statuses, priorities, tags, dueRange, scope, text});
const [expandedMap, setExpandedMap] = useState({}); // id → bool
const [saveState, setSaveState] = useState({status, lastSaved});
```
**Key invariant:** `data` always reflects the latest saved or loaded state. `isDirtyRef` tracks whether there are unsaved local edits (for conflict detection).
Only UI state (expanded/collapsed map) is persisted to `localStorage`. Backlog data is always sourced from `backlog.md`.
### 4.3 Data Flow
```
User action (edit, status change, reorder)
│
▼
Mutate data object → setData(newData)
│
├─► Immediate re-render (React)
│
└─► Debounced save (300ms)
│
▼
Parser.serialize(data) → markdown string
│
▼
Storage.save(markdown) → backend writes to disk
│
▼
SyncPoller.lastChecksum updated
```
### 4.4 Storage Module (`storage.jsx`)
This is the infrastructure layer — all filesystem access is routed through here.
#### Parser
The `Parser` object provides `parse(text)` and `serialize(data)`:
- **`parse(text)`** — Splits text by section markers, parses entries into a tree, parses history table rows, verifies checksum. Returns `{ entries, history, meta, checksumOk }`.
- **`serialize(data)`** — Rebuilds markdown from the tree, recomputes SHA-256, writes integrity marker. Returns the full markdown string.
Both are async because they use `crypto.subtle.digest()` for SHA-256.
#### Storage Backend Detection
```javascript
Storage.detect() → 'api' | 'direct' | 'local'
```
Detection order:
1. Probe `GET /api/health` with 800ms timeout → `ApiBackend`
2. Check `typeof window.showDirectoryPicker === 'function'` → `DirectBackend`
3. Fall back to `'local'` (localStorage-only, no persistence)
#### ApiBackend
| Method | HTTP | Endpoint |
|--------|------|----------|
| `load()` | `GET` | `/api/backlog` → `{ content, checksum }` |
| `save(content)` | `POST` | `/api/backlog` body `{ content }` → `{ ok, checksum, saved }` |
| `listBackups()` | `GET` | `/api/backups` → `{ backups: [...] }` |
| `restoreBackup(name)` | `POST` | `/api/backups/restore` body `{ name }` → `{ ok }` |
| `getHealthInfo()` | `GET` | `/api/health` → `{ masterSize, backupCount, masterPath, backupsPath, ... }` |
In API mode, `location.protocol` must not be `file:` — detection skips API when opening the HTML file directly.
#### DirectBackend
Uses the File System Access API (Chrome/Edge only):
| Operation | Implementation |
|-----------|---------------|
| Directory handle persistence | IndexedDB (`pb-storage-v2` → `handles` → `root`) |
| Auto-reconnect | `dirHandle.queryPermission({ mode: 'readwrite' })` — silent, no user gesture |
| Manual connect | `dirHandle.requestPermission({ mode: 'readwrite' })` — requires user gesture |
| Read file | `dirHandle.getFileHandle('backlog.md').getFile().text()` |
| Write file | `dirHandle.getFileHandle('backlog.md', {create:true}).createWritable()` |
| List backups | Iterate `dirHandle.getDirectoryHandle('backups')` entries |
| Write backup | Write to `backups/backlog_YYYY-MM-DD-HH-mm-ss.md` |
**Reset:** Delete the `pb-storage-v2` IndexedDB database, then reload the page.
#### SyncPoller
Polls every 5 seconds via `Storage.load()`. Compares the checksum from the loaded content against `lastChecksum`:
- **Checksum unchanged** → no-op
- **Checksum changed, no local edits** → auto-reload, show toast
- **Checksum changed, local edits exist** → show warning toast ("File changed externally — you have unsaved edits")
### 4.5 App Initialization Flow
```
App mounts
│
▼
Storage.detect()
│
├─► 'api' ─► applyStorageData()
│ │
│ ├─ Storage.load()
│ ├─ Storage.listBackups()
│ ├─ Storage.getHealthInfo()
│ ├─ Parser.parse(content)
│ ├─ buildDataFromStorage()
│ ├─ setData(newData)
│ └─ SyncPoller.start()
│
├─► 'direct' ─► tryAutoConnect()
│ │
│ ├─ success ─► applyStorageData() (same as api)
│ └─ fail ─► setNeedsConnect(true), show connect button
│
└─► 'local' ─► setIsLoading(false), use empty data
```
**`isCancelled` pattern:** The `applyStorageData` function accepts an `isCancelled` function (not a boolean) so it can check the cancellation state after each async operation. This prevents setting state on an unmounted component:
```javascript
await applyStorageData(mode, () => cancelled);
```
### 4.6 buildDataFromStorage
This function assembles the complete data object consumed by the UI:
```javascript
async function buildDataFromStorage(parsed, backups, storageMode, sizeInfo)
```
Returns:
```javascript
{
entries, // Hierarchical task tree with levels assigned
history, // Audit log rows
meta, // Integrity marker data
health: {
integrityOk, lastSave, lastBackup, masterSize, backupDirSize,
backupCount, statsSize, historySize, historyOldest,
mode, // 'API server' | 'Direct (File System API)' | 'localStorage only'
masterPath, // Filesystem path (API mode only, null otherwise)
backupsPath, // Backup directory path (API mode only, null otherwise)
},
stats: {
createdThisWeek, completedThisWeek, avgInProgressDays,
mostActiveProject, completionByDay, createdByDay, statusMix,
},
backups, // Backup file list with metadata
}
```
**Progress migration:** When loading, items with missing or non-numeric `progress` get default values based on their status (done→100, in-progress→50, blocked→25, etc.). Items with `status === 'done'` are always forced to `progress: 100`.
## 5. Frontend Implementation — V1 (Vanilla JS, archived)
The V1 implementation is archived at `design/archive/index-v1.html` as a single self-contained file (~41 KB). It uses vanilla JavaScript with direct DOM manipulation — no React, no virtual DOM, no build step.
### 5.1 Architecture
The V1 follows the same three-layer architecture (Presentation → Domain → Infrastructure) but all modules are inlined in a single `<script>` block within the HTML file. Key modules:
| Module | Role |
|--------|------|
| `Parser` | Parse/serialize markdown (identical logic to V2) |
| `Store` | In-memory state, mutations, change events |
| `Renderer` | DOM tree rendering |
| `Storage` | Backend detection and routing |
| `ApiBackend` | HTTP fetch to server API |
| `DirectBackend` | File System Access API |
The V1 Parser uses the same regex for metadata extraction:
```javascript
const metaM = raw.match(/^(.*?)\s*\*\((.*)\)\*\s*$/);
```
### 5.2 Differences from V2
| Aspect | V1 | V2 |
|--------|----|----|
| Framework | Vanilla JS | React 18 |
| DOM updates | Direct manipulation | React reconciliation |
| Source files | Single HTML file | Multi-file JSX + CSS |
| CSS | Inline `<style>` | External `styles.css` (42 KB) |
| Bundle size | ~41 KB | ~346 KB |
| Admin page | Basic | Full dashboard with stats |
| Filter panel | Simple | Advanced with tag autocomplete |
| Settings | None | Tweaks panel (density, accent hue, etc.) |
Both versions use the same `backlog.md` format and are compatible with both storage backends.
## 6. Python Server (`server.py`)
### 6.1 Overview
A single-file HTTP server (~420 LoC) built on Python 3's `http.server` module. Zero external dependencies — only stdlib imports.
### 6.2 Configuration
```python
class Config:
dir # Root data directory (resolved Path)
port # Listen port
master # Path to backlog.md
backups_dir # Path to backups/
stats_file # Path to stats.jsonl
web_dir # Path to web/ (static files to serve)
```
Command-line arguments:
- `--port` (default: 8080) — Listen port
- `--dir` (default: directory of server.py) — Data directory
- `--web-dir` (default: `../webapp/`) — Static file directory
### 6.3 REST API
| Method | Path | Request | Response |
|--------|------|---------|----------|
| `GET` | `/api/health` | — | `{ status, lastSave, lastBackup, masterSize, backupCount, masterPath, backupsPath }` |
| `GET` | `/api/backlog` | — | `{ content, checksum }` |
| `POST` | `/api/backlog` | `{ content }` | `{ ok, checksum, saved }` |
| `GET` | `/api/backups` | — | `{ backups: [{ name, size, timestamp, valid }] }` |
| `GET` | `/api/backups/<name>` | — | Raw markdown file download |
| `POST` | `/api/backups/restore` | `{ name }` | `{ ok }` |
| `POST` | `/api/export` | `{ format: "json" \| "markdown" }` | JSON dump or raw markdown |
| `POST` | `/api/import` | `{ content }` | `{ ok }` |
| `POST` | `/api/stats` | `{ event, payload }` | `{ ok }` |
| `GET` | `/api/stats?from=&to=` | — | `{ events: [...] }` |
Non-API paths serve static files from `web_dir` with MIME type detection. The root path `/` serves `web_dir/index.html`.
### 6.4 Atomic Write Sequence
```
1. Write content to backlog.md.tmp
2. Parse tmp file to verify it's structurally valid
3. Copy tmp → backups/backlog_YYYY-MM-DD_HH-MM-SS-mmm.md
4. Run backup rotation (prune old files)
5. Atomic rename: tmp → backlog.md
6. Append save_completed event to stats.jsonl
7. Return { ok: true, checksum, saved }
```
### 6.5 Backup Rotation
```python
def rotate_backups():
# Keep all backups ≤ 7 days old
# After 7 days: keep only the newest backup per calendar day
# Never delete the single most recent backup
```
### 6.6 CORS & Security
- All responses include `Access-Control-Allow-Origin: *` (single-user local tool, no CSRF protection)
- `OPTIONS` requests return `204 No Content` with CORS headers
- Server binds to `0.0.0.0` for LAN access (phone/tablet)
- Path traversal protection: static file requests are resolved against `web_dir` and checked with `relative_to()`
- No authentication — intentional for a personal local tool
### 6.7 Stats File (`stats.jsonl`)
Append-only JSON Lines file:
```jsonl
{"t":"2025-05-10T14:32:01Z","e":"item_created","d":{"id":"task-1","level":3,"project":"proj-1"}}
{"t":"2025-05-10T14:35:00Z","e":"save_completed","d":{"size":12400,"ms":45}}
```
Fields: `t` (ISO timestamp), `e` (event type), `d` (payload).
## 7. Bundling Pipeline
### 7.1 Purpose
The V2 source code is split across multiple JSX and CSS files for developer ergonomics. The bundler collapses everything into a single HTML file that can be opened offline with zero setup.
### 7.2 Bundle Script (`bundle.js`)
Location: `web/bundle.js` (~234 LoC)
**Prerequisites:**
- Node.js 18+ (uses global `fetch` for CDN downloads)
- `npm install` in `web/` directory (installs `@babel/core` + `@babel/preset-react`)
**Usage:**
```bash
cd web
node bundle.js index.html ../webapp/index-style-v2.html
```
### 7.3 Bundle Process (Step by Step)
```
1. Read index.html
2. Inline stylesheets:
For each <link rel="stylesheet" href="...">:
- If local file: replace with <style>...</style>
- If remote URL: keep as-is
✓ styles.css → <style>
3. Process scripts:
For each <script src="...">:
a) Remote CDN with a swap rule:
- React dev → fetch React production min, inline as <script>
- ReactDOM dev → fetch ReactDOM production min, inline as <script>
- @babel/standalone → remove entirely (no longer needed)
b) Local file with type="text/babel":
- Read file content
- Compile JSX → plain JS via @babel/core + @babel/preset-react
- Strip type="text/babel" attribute
- Inline as <script>...</script>
c) Local file without babel:
- Read and inline as <script>...</script>
d) Remote URL with no swap rule:
- Keep as-is
✓ helpers.jsx (JSX→JS)
✓ storage.jsx (JSX→JS)
✓ tweaks-panel.jsx (JSX→JS)
✓ filter-panel.jsx (JSX→JS)
✓ tree.jsx (JSX→JS)
✓ dialogs.jsx (JSX→JS)
✓ admin.jsx (JSX→JS)
✓ app.jsx (JSX→JS)
4. Compile inline babel blocks:
For each <script type="text/babel">...</script>:
- Compile JSX → JS
- Strip type="text/babel" attribute
✓ ReactDOM.createRoot boot script
5. Write output file
```
### 7.4 CDN Swap Table
| Source CDN URL | Action |
|---------------|--------|
| `unpkg.com/react-dom@*` | Fetch `react-dom.production.min.js`, inline |
| `unpkg.com/react@*` | Fetch `react.production.min.js`, inline |
| `unpkg.com/@babel/standalone@*` | Remove tag entirely |
Production builds are smaller than development builds. The Babel compiler is removed because all JSX has been pre-compiled.
### 7.5 HTML Comment Awareness
The bundler skips all processing for content inside HTML comments (`<!-- ... -->`). This means commented-out `<script>` or `<link>` tags are left untouched, which is important for the data.jsx seed script that is commented out by default in `index.html`.
### 7.6 Script Escaping
Content inlined into `<script>` tags has `</script>` replaced with `<\/script>` to prevent the browser from prematurely closing the script block. Similarly, `</style>` is escaped in inlined CSS.
## 8. Key Data Structures
### 8.1 Entry Item
```javascript
{
id: "i-m1abc", // Unique ID: "i-" + base36 counter
level: 1, // 1–4 (Area→Project→Task→Sub-task)
title: "Ship landing page",
status: "open", // open | in-progress | blocked | postponed | done | cancelled
priority: "P0", // P0 | P1 | P2 | P3
due: "2025-06-01", // ISO date or null
reason: null, // Free text (blocked items)
tags: ["urgent"], // Array of strings
progress: 50, // 0–100 integer
collapsed: false, // UI expand/collapse state
children: [], // Nested entry items
}
```
### 8.2 History Row
```javascript
{
timestamp: "2025-05-10T14:32:00Z",
itemId: "i-m1abc",
action: "status_changed", // status_changed | item_created | item_deleted | item_moved
details: "open → done",
}
```
### 8.3 Integrity Meta
```javascript
{
saved: "2025-05-10T14:35:12Z",
checksum: "sha256:abc123...",
entryCount: 42,
historyCount: 128,
}
```
## 9. Cross-Version Compatibility
### 9.1 V1 ↔ V2
Both versions read and write the same `backlog.md` format. They can be used interchangeably against the same data file. The V2 file (`webapp/index-style-v2.html`) can be served by the Python server or opened directly in Chrome/Edge. The archived V1 file (`design/archive/index-v1.html`) is no longer actively maintained.
### 9.2 API Server ↔ Direct File Access
Both storage backends read/write the same `backlog.md` file on disk. The only difference is *how* they access it:
| Aspect | API Server | Direct File Access |
|--------|-----------|-------------------|
| Read | `GET /api/backlog` → JSON | `dirHandle.getFileHandle().getFile().text()` |
| Write | `POST /api/backlog` ← JSON | `dirHandle.getFileHandle({create:true}).createWritable()` |
| Backup | Server-side copy | Frontend writes to `backups/` dir handle |
| Health | `GET /api/health` | Read file size from `getFile()` |
| Paths shown | Yes (server knows filesystem) | No (browser doesn't expose paths) |
### 9.3 Switching Modes
To switch from API server to direct file access:
1. Stop the server
2. Open `webapp/index-style-v2.html` in Chrome/Edge
3. Select the folder containing your `backlog.md`
To switch from direct file access to API server:
1. Note the folder path where `backlog.md` lives
2. Start `python3 server/server.py --dir /that/folder`
3. Open `http://localhost:8080`
**Important:** Never run both modes simultaneously against the same file — last writer wins.
## 10. Performance Considerations
| Metric | Target | Implementation |
|--------|--------|---------------|
| Initial load | < 1s for ≤1MB file | Single-pass parser, async SHA-256 |
| Save latency | < 300ms | Atomic rename (server), writable stream (direct) |
| Poll overhead | Negligible | 5s interval, checksum comparison, short-circuits on match |
| Render | < 50ms for 500 items | React reconciliation, targeted updates |
| Bundle size | < 400 KB | Production React builds, no source maps |
## 11. Known Limitations
1. **File System Access API** is Chrome/Edge only. Firefox and Safari users must use the Python server.
2. **No concurrent access protection.** If two browser tabs or a browser + external editor write simultaneously, the last writer wins. The SyncPoller detects external changes but cannot prevent race conditions.
3. **No dark mode** yet. CSS custom properties are used throughout, making it a one-line toggle when needed.
5. **DirectBackend backup rotation** does not prune old backups — only the API server does rotation.
doc/requirements/requirements.md
+200
View File
@@ -0,0 +1,200 @@
# Personal Backlog — Requirements
## 1. Purpose
A simplistic, single-user, locally-hosted web UI for tracking personal projects, tasks, and their completion state. All persistent data lives in **one human-readable Markdown file** that contains the full backlog, status history, and audit log. The application loads this file on startup and re-saves it on every change.
## 2. Core Principles
| # | Principle |
|---|-----------|
| 1 | **Markdown as the database** — The `.md` file is the sole source of truth. It must remain readable in any text editor. |
| 2 | **Atomic save with integrity marker** — Every successful write ends with a verifiable marker so the app (and a human) can confirm the file was saved completely. |
| 3 | **Automatic rotating backups** — At least one week of backups must be kept automatically. |
| 4 | **No external runtime dependencies** — The app runs locally in a browser; storage is the filesystem (or a local dev server). |
## 3. Data Model & Hierarchy
### 3.1 Nesting Levels
The backlog supports **up to 4 levels** of nesting:
```
Level 1 — Area / Theme (optional)
Level 2 — Project
Level 3 — Task
Level 4 — Sub-task
```
*One-off tasks* may sit at Level 2 (as a standalone item with no children) or Level 3.
### 3.2 Markdown File Format (Normative)
The master file is divided into **three sections**:
```markdown
# Backlog
<!-- SECTION: ENTRIES -->
## [P1] 🌐 My Project
- [ ] Task A *(due: 2025-06-01, status: open)*
- [x] Sub-task A.1 *(done: 2025-05-10)*
- [!] Task B *(status: blocked, reason: waiting for API keys)*
## [P2] 📦 Another Project
...
<!-- SECTION: HISTORY -->
| Timestamp | Item ID | Action | Details |
|-----------|---------|--------|---------|
| 2025-05-10T14:32:00Z | task-a-1 | status_changed | open → done |
<!-- SECTION: INTEGRITY -->
<!-- saved: 2025-05-10T14:35:12Z | checksum: sha256:abc123... | entries: 42 | history: 128 -->
```
**Rules:**
- `<!-- SECTION: ENTRIES -->` contains the live backlog tree.
- `<!-- SECTION: HISTORY -->` contains an append-only audit table of every mutation.
- `<!-- SECTION: INTEGRITY -->` contains the save marker:
- `saved`: ISO-8601 timestamp of the write.
- `checksum`: SHA-256 over the concatenation of `ENTRIES` + `HISTORY`.
- `entries`: count of backlog items.
- `history`: count of history rows.
- The checksum is a **save-success indicator**, not a hard gate. If the marker is missing or the checksum does not match, the app loads the file anyway but shows a non-blocking warning banner (e.g., "File was edited outside the app — checksum mismatch"). The next save will rewrite a correct marker.
## 4. Functional Requirements
### 4.1 Statuses (REQ-F-001)
Each item has a **status** drawn from the following set:
| Status | Glyph | Meaning |
|--------|-------|---------|
| `open` | `[ ]` | Not started yet |
| `in-progress` | `[/]` | Actively being worked on |
| `blocked` | `[!]` | Cannot proceed; requires external action |
| `postponed` | `[>]` | Intentionally deferred |
| `done` | `[x]` | Completed |
| `cancelled` | `[-]` | No longer relevant |
- Changing status appends a row to the **HISTORY** section.
- `blocked` must allow a short free-text `reason` field.
### 4.1a Progress (REQ-F-001a)
- Every item carries an optional **progress** percentage: `0–100`.
- New tasks default to `0%`.
- Setting status to `done` automatically sets progress to `100%`.
- `blocked` and `cancelled` items may retain any progress value (e.g., partially done before being blocked).
- Progress is stored inline in the Markdown metadata as `progress: N`.
- The UI shows a small progress bar next to the task title.
### 4.2 Priorities & Ordering (REQ-F-002)
- Each item carries a **priority** label: `P0` (burning), `P1`, `P2`, or `P3`.
- Within the **same priority**, items are ordered manually via **drag-and-drop**.
- The UI renders items in priority-descending, then manual-order order.
- `P0` items are visually highlighted (e.g., fire emoji 🔥 or a distinct border) to mark "hot / burning" status.
### 4.3 Due Dates (REQ-F-003)
- Optional `due` date per item (ISO-8601 date: `YYYY-MM-DD`).
- Overdue items are visually flagged.
### 4.4 Tags & Labels (REQ-F-004)
- Items may have free-form **tags** (e.g., `#urgent`, `#research`).
- Tags are stored inline in the Markdown item metadata.
- The UI offers **autocomplete** based on the most recently used tags across the backlog.
- Tags are filterable (multi-select) alongside status and priority.
### 4.5 Filtering & Search (REQ-F-005)
The UI must provide filters for:
- Status (multi-select)
- Priority (multi-select)
- Due date range (`overdue`, `today`, `this week`, `this month`, `custom`)
- Full-text search across titles, descriptions, block reasons, and tags
- Scope: `all`, `top-level only`, or `current project`
**Parent Visibility Rule:** When a filter or quicksearch matches a task at any level, all ancestor projects/items up to the root must remain visible in the tree so the user understands the hierarchical context. Conversely, if a project matches a filter, it may be shown collapsed unless the filter also matches its children.
### 4.6 Mutations & Save (REQ-F-006)
- Every create, update, delete, reorder, or status change triggers a **full re-save** of the master Markdown file.
- The save is **atomic**: write to a temp file, verify the integrity marker, then rename over the original.
- On save failure, the UI shows a non-dismissible banner and keeps the in-memory state intact so the user can retry or export.
### 4.7 External File Monitoring (REQ-F-007)
- The app **watches the master Markdown file** for external changes (e.g., edited via another editor, synced via Dropbox/Git).
- If the file changes on disk and passes the integrity check, the app reloads it automatically and notifies the user.
- If an external change conflicts with unsaved local edits, the UI presents a diff/merge choice rather than silently overwriting.
### 4.8 Import & Export (REQ-F-008)
- **Export**: the user may export the current backlog as a Markdown file or as JSON (full structured dump including history).
- **Import**: the user may import a previously exported Markdown or JSON file.
- Import validates schema (for JSON) and warns on checksum mismatch (for Markdown), but never blocks loading a manually edited file.
## 5. Backup & Recovery
### 5.1 Automatic Rotating Backups (REQ-B-001)
- On every successful save, a timestamped copy is written to a `./backups/` directory.
- Naming convention: `backlog_YYYY-MM-DD_HH-mm-ss.md`
- Rotation policy: keep **at least 7 days** of backups. Older backups may be pruned automatically, but the most recent backup from each calendar day must be preserved for 30 days.
### 5.2 Recovery UI (REQ-B-002)
- The **Admin / Maintenance** page (see §6) lists available backups with:
- Timestamp
- File size
- Entry count
- Integrity check result (valid / corrupt)
- The user may:
- Preview a backup read-only.
- Restore a backup (with a confirmation modal).
- Download any backup as a file.
## 6. Admin & Maintenance Page (REQ-A-001)
A dedicated route/page in the SPA (`/admin` or similar) accessible from a gear icon. It must provide:
| Widget | Content |
|--------|---------|
| **System Health** | Integrity of current file, last save timestamp, last backup timestamp. |
| **Storage Stats** | Master file size, backup directory size, number of backups. |
| **Backup Browser** | List, preview, restore, download backups (§5.2). |
| **Stats Overview** | Key metrics sourced from the stats database (§7): items created/completed this week, average time in `in-progress`, most active project, etc. |
| **Manual Actions** | Force save, force backup, compact history (collapse old `done` items into a summary row). |
## 7. Stats & Metrics Database (REQ-S-001)
### 7.1 Purpose
A separate **append-only** stats store (implementation-agnostic: JSONL file, SQLite, or IndexedDB) that captures usage metrics and enables the admin widgets. It is *not* the source of truth for backlog data; it is purely analytical.
### 7.2 Events to Capture
- `item_created` — id, timestamp, level, project
- `item_status_changed` — id, timestamp, old_status, new_status, duration_in_previous_status
- `item_deleted` — id, timestamp, final_status
- `item_moved` — id, timestamp, old_parent, new_parent
- `item_reordered` — id, timestamp
- `save_completed` — timestamp, file_size, duration_ms
- `backup_created` — timestamp, file_size
- `filter_used` — timestamp, criteria JSON
- `page_view` — timestamp, route
### 7.3 Retention & Privacy
- Retain raw events for 90 days.
- Aggregate older data into daily/weekly roll-ups.
- All data stays local; no telemetry is sent externally.
## 8. Non-Functional Requirements
| ID | Requirement |
|----|-------------|
| REQ-NF-001 | The app must work fully offline after initial load. |
| REQ-NF-002 | Startup time (file load + render) must be < 1 s for a file ≤ 1 MB. |
| REQ-NF-003 | The Markdown file must be valid CommonMark and render legibly in any Markdown viewer. |
| REQ-NF-004 | All dates/times are stored in UTC, displayed in the user's local timezone. |
| REQ-NF-005 | The UI must be keyboard-navigable (expand/collapse, toggle status, quick filter shortcuts). |
| REQ-NF-006 | The UI must be responsive and usable on screens down to 375 px width (mobile). |
## 9. Out of Scope / Future Considerations
1. **Recurring tasks** — Not supported; all items are one-time.
2. **Third-party integrations** — Conversion scripts can be written against the Markdown or JSON export format if needed.
3. **Theme / appearance** — Light theme only for the initial release. CSS custom properties are used throughout so that a dark mode or custom theme can be added later without structural changes.