Initial commitHEADmaster

1 files changed, 236 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..df1f37e
--- /dev/null
+++ b/README.md
@@ -0,0 +1,236 @@
+# MedMan — Self-Hosted Media Manager with Tagging & squashr Integration
+
+MedMan is a web-based media manager designed to work alongside
+[squashr](https://git.kug.is/squashr.git) (SquashFS/OverlayFS/LUKS backup tool).
+
+## Overview
+
+```
+1. Drop files into source directories (or upload via web UI)
+       ↓
+2. MedMan scans/ingests → files are distributed into data directories
+   using a full-hash directory hierarchy (every 2 hex chars = one level)
+       ↓
+3. Browse & tag files through the web interface
+       ↓
+4. Export tags → archival.py packs them into SquashFS containers
+       ↓
+5. Burn containers to CD/DVD/Blu-Ray or archive offline
+       ↓
+6. Offline use: mount container, start MedMan, import index.json →
+   browse & search media with full tag metadata
+```
+
+## Configuration
+
+MedMan is configured primarily through **Docker volume mounts**. Environment
+variables are minimal:
+
+| Variable | Description | Default |
+|---|---|---|
+| `MEDMAN_PORT` | HTTP port | `8080` |
+| `MEDMAN_HOST` | Bind address | `0.0.0.0` |
+| `MEDMAN_W` | Comma-separated weights for data-dir distribution | equal weight |
+
+### Directory Structure
+
+All directories are inside `/app` in the container and wired via
+`docker-compose.yml` mounts:
+
+```
+/app/
+├── srcs/           ← source directories (immediate subdirs)
+│   ├── src0/       ← mount: ./src0:/app/srcs/src0
+│   └── src1/       ← mount: ./src1:/app/srcs/src1
+├── dats/           ← data directories (immediate subdirs)
+│   ├── data0/      ← mount: ./data0:/app/dats/data0
+│   └── data1/      ← mount: ./data1:/app/dats/data1
+├── imports/        ← auto-import directories (immediate subdirs, read-only)
+│   └── my-archive/ ← mount: /mnt/archive:/app/imports/my-archive:ro
+└── db/             ← SQLite database + thumbnails (auto-created)
+```
+
+**Key principle**: Every immediate subdirectory of `srcs/`, `dats/`, and
+`imports/` is automatically discovered. No comma-separated path lists in
+environment variables.
+
+### Data Distribution Weights
+
+`MEDMAN_W` controls how new files are distributed across data directories:
+
+```bash
+# 3 data dirs — dir 0 gets 50%, dirs 1 and 2 get 25% each
+MEDMAN_W=2,1,1
+```
+
+If not set, distribution is equal across all data directories.
+
+## Hash-Based Storage
+
+Every file is SHA-256 hashed. All 32 byte-pairs of the 64-character hex
+hash become directory levels:
+
+```
+Hash:  a1b2c3d4e5f6...a1b2c3d4e5f6 (64 chars)
+Path:  data0/a1/b2/c3/d4/e5/f6/.../a1/b2/c3/d4/e5/f6
+```
+
+- 32 directory levels + filename (last hex pair)
+- Automatic deduplication via content hashing
+- Distribution across data directories according to `MEDMAN_W` weights
+- No `rel` column stored in the database — the path is always deterministically
+  computed from the hash
+
+## Quickstart (Docker)
+
+```bash
+cd medman
+docker compose up -d
+```
+
+MedMan is available at `http://localhost:8080`.
+
+Drop files into `src0/` or `src1/` — they are automatically scanned and
+distributed into the data directories on startup. Upload is also supported
+via the web UI.
+
+## Auto-Import on Startup
+
+Directories mounted under `/app/imports/` are scanned on startup (and on
+each manual scan). Each subdirectory may contain:
+
+- `index.json` — a MedMan export file with tags and file references
+- Files in the canonical hash-prefix hierarchy (same layout as data dirs)
+
+All tags and file registrations from `index.json` are imported automatically
+if the referenced files exist on disk. Import directories receive zero
+distribution weight (new files are never placed there).
+
+Example: Mount a read-only SquashFS container containing an `index.json` and
+canonical file paths:
+
+```yaml
+# docker-compose.yml
+volumes:
+  - /mnt/my-archive:/app/imports/my-archive:ro
+```
+
+## Features
+
+- **Source directories**: files auto-ingested on startup, upload via UI
+- **Manual scan button**: re-scan source and import directories anytime
+- **Hash-based storage**: content-addressable, no duplicates
+- **Multiple data directories**: weighted random distribution
+- **Move**: relocate individual files or selections between data dirs
+- **Tag system**: plain labels, no color coding
+- **Tag search**: combined AND search across multiple tags
+- **Bulk operations**: select-all, batch tag/untag/move/delete
+- **Previous/Next navigation**: arrow keys or buttons in detail view
+- **Checkbox selection**: click the checkbox on any thumbnail to toggle
+- **Export/Import**: tags as JSON (export/import)
+- **archival.py**: create SquashFS + LUKS containers directly from MedMan
+- **Container mount API**: mount SquashFS containers as additional data dirs
+
+## Database Schema
+
+```sql
+tags(id, name, ts)
+files(hash PK, didx, size, mt, w, h, dur, ts)
+ftags(id, hash FK→files, tid FK→tags, UNIQUE(hash, tid))
+```
+
+`didx` is the index into the `dats/` subdirectory list. The on-disk path
+is always `dats/<name>/<h2rel(hash)>`.
+
+## API
+
+| Endpoint | Method | Description |
+|---|---|---|
+| `/api/files` | GET | All files (`?tag=` filter) |
+| `/api/files/<hash>` | DELETE | Delete file |
+| `/api/files/<hash>/move` | POST | `{"didx":N}` move to data dir |
+| `/api/files/<hash>/tags` | GET/POST | Get or add tags |
+| `/api/files/<hash>/tags/<id>` | DELETE | Remove tag from file |
+| `/api/tags` | GET/POST | List or create tags |
+| `/api/tags/<id>` | DELETE | Delete tag |
+| `/api/tags/export` | GET | Full tag export (`?tags=a,b` AND filter) |
+| `/api/tags/import` | POST | Import tag JSON |
+| `/api/search?tags=a,b` | GET | AND tag search |
+| `/api/upload` | POST | Upload file(s) |
+| `/api/scan` | POST | Scan sources + imports |
+| `/api/dat` | GET | Data directory stats |
+| `/api/containers/mount` | POST | Mount container dir as data dir |
+| `/api/containers/mount` | DELETE | Unmount container dir |
+| `/thumb/<hash>` | GET | JPEG thumbnail |
+| `/raw/<hash>` | GET | Original file |
+
+## Export Format (index.json)
+
+```json
+{
+  "ts": "2025-01-15T12:00:00+00:00",
+  "tags": [{"id": 1, "name": "example"}],
+  "files": {
+    "a1b2c3d4...": {"didx": 0, "tags": ["example"]}
+  }
+}
+```
+
+The on-disk path for each file is deterministic: `h2rel(hash)` → 32 hex-pair
+directory levels. No `rel` field is needed.
+
+## archival.py — Creating SquashFS Containers
+
+```bash
+# Build containers via MedMan API
+python3 archival.py --medman-url http://localhost:8080 \
+    --dat /app/data0 /app/data1 \
+    --tags vacation 2024 \
+    --container-size 4700372992 \
+    --output-prefix ./archive
+
+# Build with LUKS encryption
+python3 archival.py --db /app/db/medman.db \
+    --dat /app/data0 /app/data1 \
+    --tags photos \
+    --container-size 25000000000 \
+    --output-prefix ./photos \
+    --cryptsetup --key-file /path/to/keyfile
+
+# Direct DB access (no running MedMan needed)
+python3 archival.py --db /app/db/medman.db \
+    --dat /app/data0 /app/data1 \
+    --container-size 10000000000 \
+    --output-prefix ./offline
+```
+
+The script uses `tar → sqfstar` piping (no staging directories, no temp
+files beyond a single index.json) — exactly like squashr's
+`build_squash_image_tar_sqfstar`.
+
+### Container Size Margin
+
+A margin of 1 MiB is reserved per container for the `index.json` and
+filesystem metadata overhead. Adjust with `--margin` if needed.
+
+### Filter Options
+
+| Option | Description |
+|---|---|
+| `--tags a b` | Only files with ALL given tags (AND) |
+| `--hashes a1b2...` | Specific hashes |
+| `--hashes-from-file` | Read hashes from file |
+| `--from-date 2024-01-01` | Modified on or after |
+| `--to-date 2024-12-31` | Modified on or before |
+| `--mime-type video` | Filter by media type |
+
+## squashr Integration
+
+MedMan containers produced by `archival.py` are standard SquashFS images
+with an `index.json` at the root plus the canonical hash hierarchy. They
+can be mounted and imported into any MedMan instance via the auto-import
+feature or the container mount API.
+
+## License
+
+GNU AGPLv3