aboutsummaryrefslogtreecommitdiffstats

MedMan — Self-Hosted Media Manager with Tagging & squashr Integration

MedMan is a web-based media manager designed to work alongside squashr (SquashFS/OverlayFS/LUKS backup tool).

Overview

1. Drop files into source directories (or upload via web UI)
       ↓
2. MedMan scans/ingests → files are distributed into data directories
   using a full-hash directory hierarchy (every 2 hex chars = one level)
       ↓
3. Browse & tag files through the web interface
       ↓
4. Export tags → archival.py packs them into SquashFS containers
       ↓
5. Burn containers to CD/DVD/Blu-Ray or archive offline
       ↓
6. Offline use: mount container, start MedMan, import index.json →
   browse & search media with full tag metadata

Configuration

MedMan is configured primarily through Docker volume mounts. Environment variables are minimal:

Variable Description Default
MEDMAN_PORT HTTP port 8080
MEDMAN_HOST Bind address 0.0.0.0
MEDMAN_W Comma-separated weights for data-dir distribution equal weight

Directory Structure

All directories are inside /app in the container and wired via docker-compose.yml mounts:

/app/
├── srcs/           ← source directories (immediate subdirs)
│   ├── src0/       ← mount: ./src0:/app/srcs/src0
│   └── src1/       ← mount: ./src1:/app/srcs/src1
├── dats/           ← data directories (immediate subdirs)
│   ├── data0/      ← mount: ./data0:/app/dats/data0
│   └── data1/      ← mount: ./data1:/app/dats/data1
├── imports/        ← auto-import directories (immediate subdirs, read-only)
│   └── my-archive/ ← mount: /mnt/archive:/app/imports/my-archive:ro
└── db/             ← SQLite database + thumbnails (auto-created)

Key principle: Every immediate subdirectory of srcs/, dats/, and imports/ is automatically discovered. No comma-separated path lists in environment variables.

Data Distribution Weights

MEDMAN_W controls how new files are distributed across data directories:

# 3 data dirs — dir 0 gets 50%, dirs 1 and 2 get 25% each
MEDMAN_W=2,1,1

If not set, distribution is equal across all data directories.

Hash-Based Storage

Every file is SHA-256 hashed. All 32 byte-pairs of the 64-character hex hash become directory levels:

Hash:  a1b2c3d4e5f6...a1b2c3d4e5f6 (64 chars)
Path:  data0/a1/b2/c3/d4/e5/f6/.../a1/b2/c3/d4/e5/f6
  • 32 directory levels + filename (last hex pair)
  • Automatic deduplication via content hashing
  • Distribution across data directories according to MEDMAN_W weights
  • No rel column stored in the database — the path is always deterministically computed from the hash

Quickstart (Docker)

cd medman
docker compose up -d

MedMan is available at http://localhost:8080.

Drop files into src0/ or src1/ — they are automatically scanned and distributed into the data directories on startup. Upload is also supported via the web UI.

Auto-Import on Startup

Directories mounted under /app/imports/ are scanned on startup (and on each manual scan). Each subdirectory may contain:

  • index.json — a MedMan export file with tags and file references
  • Files in the canonical hash-prefix hierarchy (same layout as data dirs)

All tags and file registrations from index.json are imported automatically if the referenced files exist on disk. Import directories receive zero distribution weight (new files are never placed there).

Example: Mount a read-only SquashFS container containing an index.json and canonical file paths:

# docker-compose.yml
volumes:
  - /mnt/my-archive:/app/imports/my-archive:ro

Features

  • Source directories: files auto-ingested on startup, upload via UI
  • Manual scan button: re-scan source and import directories anytime
  • Hash-based storage: content-addressable, no duplicates
  • Multiple data directories: weighted random distribution
  • Move: relocate individual files or selections between data dirs
  • Tag system: plain labels, no color coding
  • Tag search: combined AND search across multiple tags
  • Bulk operations: select-all, batch tag/untag/move/delete
  • Previous/Next navigation: arrow keys or buttons in detail view
  • Checkbox selection: click the checkbox on any thumbnail to toggle
  • Export/Import: tags as JSON (export/import)
  • archival.py: create SquashFS + LUKS containers directly from MedMan
  • Container mount API: mount SquashFS containers as additional data dirs

Database Schema

tags(id, name, ts)
files(hash PK, didx, size, mt, w, h, dur, ts)
ftags(id, hash FK→files, tid FK→tags, UNIQUE(hash, tid))

didx is the index into the dats/ subdirectory list. The on-disk path is always dats/<name>/<h2rel(hash)>.

API

Endpoint Method Description
/api/files GET All files (?tag= filter)
/api/files/<hash> DELETE Delete file
/api/files/<hash>/move POST {"didx":N} move to data dir
/api/files/<hash>/tags GET/POST Get or add tags
/api/files/<hash>/tags/<id> DELETE Remove tag from file
/api/tags GET/POST List or create tags
/api/tags/<id> DELETE Delete tag
/api/tags/export GET Full tag export (?tags=a,b AND filter)
/api/tags/import POST Import tag JSON
/api/search?tags=a,b GET AND tag search
/api/upload POST Upload file(s)
/api/scan POST Scan sources + imports
/api/dat GET Data directory stats
/api/containers/mount POST Mount container dir as data dir
/api/containers/mount DELETE Unmount container dir
/thumb/<hash> GET JPEG thumbnail
/raw/<hash> GET Original file

Export Format (index.json)

{
  "ts": "2025-01-15T12:00:00+00:00",
  "tags": [{"id": 1, "name": "example"}],
  "files": {
    "a1b2c3d4...": {"didx": 0, "tags": ["example"]}
  }
}

The on-disk path for each file is deterministic: h2rel(hash) → 32 hex-pair directory levels. No rel field is needed.

archival.py — Creating SquashFS Containers

# Build containers via MedMan API
python3 archival.py --medman-url http://localhost:8080 \
    --dat /app/data0 /app/data1 \
    --tags vacation 2024 \
    --container-size 4700372992 \
    --output-prefix ./archive

# Build with LUKS encryption
python3 archival.py --db /app/db/medman.db \
    --dat /app/data0 /app/data1 \
    --tags photos \
    --container-size 25000000000 \
    --output-prefix ./photos \
    --cryptsetup --key-file /path/to/keyfile

# Direct DB access (no running MedMan needed)
python3 archival.py --db /app/db/medman.db \
    --dat /app/data0 /app/data1 \
    --container-size 10000000000 \
    --output-prefix ./offline

The script uses tar → sqfstar piping (no staging directories, no temp files beyond a single index.json) — exactly like squashr's build_squash_image_tar_sqfstar.

Container Size Margin

A margin of 1 MiB is reserved per container for the index.json and filesystem metadata overhead. Adjust with --margin if needed.

Filter Options

Option Description
--tags a b Only files with ALL given tags (AND)
--hashes a1b2... Specific hashes
--hashes-from-file Read hashes from file
--from-date 2024-01-01 Modified on or after
--to-date 2024-12-31 Modified on or before
--mime-type video Filter by media type

squashr Integration

MedMan containers produced by archival.py are standard SquashFS images with an index.json at the root plus the canonical hash hierarchy. They can be mounted and imported into any MedMan instance via the auto-import feature or the container mount API.

License

GNU AGPLv3

generated by cgit v1.2.3 (git 2.39.1) at 2026-09-04 05:43:40 +0000