path: root/README.md

# MedMan — Self-Hosted Media Manager with Tagging & squashr Integration

MedMan is a web-based media manager designed to work alongside
[squashr](https://git.kug.is/squashr.git) (SquashFS/OverlayFS/LUKS backup tool).

## Overview

```
1. Drop files into source directories (or upload via web UI)
       ↓
2. MedMan scans/ingests → files are distributed into data directories
   using a full-hash directory hierarchy (every 2 hex chars = one level)
       ↓
3. Browse & tag files through the web interface
       ↓
4. Export tags → archival.py packs them into SquashFS containers
       ↓
5. Burn containers to CD/DVD/Blu-Ray or archive offline
       ↓
6. Offline use: mount container, start MedMan, import index.json →
   browse & search media with full tag metadata
```

## Configuration

MedMan is configured primarily through **Docker volume mounts**. Environment
variables are minimal:

| Variable | Description | Default |
|---|---|---|
| `MEDMAN_PORT` | HTTP port | `8080` |
| `MEDMAN_HOST` | Bind address | `0.0.0.0` |
| `MEDMAN_W` | Comma-separated weights for data-dir distribution | equal weight |

### Directory Structure

All directories are inside `/app` in the container and wired via
`docker-compose.yml` mounts:

```
/app/
├── srcs/           ← source directories (immediate subdirs)
│   ├── src0/       ← mount: ./src0:/app/srcs/src0
│   └── src1/       ← mount: ./src1:/app/srcs/src1
├── dats/           ← data directories (immediate subdirs)
│   ├── data0/      ← mount: ./data0:/app/dats/data0
│   └── data1/      ← mount: ./data1:/app/dats/data1
├── imports/        ← auto-import directories (immediate subdirs, read-only)
│   └── my-archive/ ← mount: /mnt/archive:/app/imports/my-archive:ro
└── db/             ← SQLite database + thumbnails (auto-created)
```

**Key principle**: Every immediate subdirectory of `srcs/`, `dats/`, and
`imports/` is automatically discovered. No comma-separated path lists in
environment variables.

### Data Distribution Weights

`MEDMAN_W` controls how new files are distributed across data directories:

```bash
# 3 data dirs — dir 0 gets 50%, dirs 1 and 2 get 25% each
MEDMAN_W=2,1,1
```

If not set, distribution is equal across all data directories.

## Hash-Based Storage

Every file is SHA-256 hashed. All 32 byte-pairs of the 64-character hex
hash become directory levels:

```
Hash:  a1b2c3d4e5f6...a1b2c3d4e5f6 (64 chars)
Path:  data0/a1/b2/c3/d4/e5/f6/.../a1/b2/c3/d4/e5/f6
```

- 32 directory levels + filename (last hex pair)
- Automatic deduplication via content hashing
- Distribution across data directories according to `MEDMAN_W` weights
- No `rel` column stored in the database — the path is always deterministically
  computed from the hash

## Quickstart (Docker)

```bash
cd medman
docker compose up -d
```

MedMan is available at `http://localhost:8080`.

Drop files into `src0/` or `src1/` — they are automatically scanned and
distributed into the data directories on startup. Upload is also supported
via the web UI.

## Auto-Import on Startup

Directories mounted under `/app/imports/` are scanned on startup (and on
each manual scan). Each subdirectory may contain:

- `index.json` — a MedMan export file with tags and file references
- Files in the canonical hash-prefix hierarchy (same layout as data dirs)

All tags and file registrations from `index.json` are imported automatically
if the referenced files exist on disk. Import directories receive zero
distribution weight (new files are never placed there).

Example: Mount a read-only SquashFS container containing an `index.json` and
canonical file paths:

```yaml
# docker-compose.yml
volumes:
  - /mnt/my-archive:/app/imports/my-archive:ro
```

## Features

- **Source directories**: files auto-ingested on startup, upload via UI
- **Manual scan button**: re-scan source and import directories anytime
- **Hash-based storage**: content-addressable, no duplicates
- **Multiple data directories**: weighted random distribution
- **Move**: relocate individual files or selections between data dirs
- **Tag system**: plain labels, no color coding
- **Tag search**: combined AND search across multiple tags
- **Bulk operations**: select-all, batch tag/untag/move/delete
- **Previous/Next navigation**: arrow keys or buttons in detail view
- **Checkbox selection**: click the checkbox on any thumbnail to toggle
- **Export/Import**: tags as JSON (export/import)
- **archival.py**: create SquashFS + LUKS containers directly from MedMan
- **Container mount API**: mount SquashFS containers as additional data dirs

## Database Schema

```sql
tags(id, name, ts)
files(hash PK, didx, size, mt, w, h, dur, ts)
ftags(id, hash FK→files, tid FK→tags, UNIQUE(hash, tid))
```

`didx` is the index into the `dats/` subdirectory list. The on-disk path
is always `dats/<name>/<h2rel(hash)>`.

## API

| Endpoint | Method | Description |
|---|---|---|
| `/api/files` | GET | All files (`?tag=` filter) |
| `/api/files/<hash>` | DELETE | Delete file |
| `/api/files/<hash>/move` | POST | `{"didx":N}` move to data dir |
| `/api/files/<hash>/tags` | GET/POST | Get or add tags |
| `/api/files/<hash>/tags/<id>` | DELETE | Remove tag from file |
| `/api/tags` | GET/POST | List or create tags |
| `/api/tags/<id>` | DELETE | Delete tag |
| `/api/tags/export` | GET | Full tag export (`?tags=a,b` AND filter) |
| `/api/tags/import` | POST | Import tag JSON |
| `/api/search?tags=a,b` | GET | AND tag search |
| `/api/upload` | POST | Upload file(s) |
| `/api/scan` | POST | Scan sources + imports |
| `/api/dat` | GET | Data directory stats |
| `/api/containers/mount` | POST | Mount container dir as data dir |
| `/api/containers/mount` | DELETE | Unmount container dir |
| `/thumb/<hash>` | GET | JPEG thumbnail |
| `/raw/<hash>` | GET | Original file |

## Export Format (index.json)

```json
{
  "ts": "2025-01-15T12:00:00+00:00",
  "tags": [{"id": 1, "name": "example"}],
  "files": {
    "a1b2c3d4...": {"didx": 0, "tags": ["example"]}
  }
}
```

The on-disk path for each file is deterministic: `h2rel(hash)` → 32 hex-pair
directory levels. No `rel` field is needed.

## archival.py — Creating SquashFS Containers

```bash
# Build containers via MedMan API
python3 archival.py --medman-url http://localhost:8080 \
    --dat /app/data0 /app/data1 \
    --tags vacation 2024 \
    --container-size 4700372992 \
    --output-prefix ./archive

# Build with LUKS encryption
python3 archival.py --db /app/db/medman.db \
    --dat /app/data0 /app/data1 \
    --tags photos \
    --container-size 25000000000 \
    --output-prefix ./photos \
    --cryptsetup --key-file /path/to/keyfile

# Direct DB access (no running MedMan needed)
python3 archival.py --db /app/db/medman.db \
    --dat /app/data0 /app/data1 \
    --container-size 10000000000 \
    --output-prefix ./offline
```

The script uses `tar → sqfstar` piping (no staging directories, no temp
files beyond a single index.json) — exactly like squashr's
`build_squash_image_tar_sqfstar`.

### Container Size Margin

A margin of 1 MiB is reserved per container for the `index.json` and
filesystem metadata overhead. Adjust with `--margin` if needed.

### Filter Options

| Option | Description |
|---|---|
| `--tags a b` | Only files with ALL given tags (AND) |
| `--hashes a1b2...` | Specific hashes |
| `--hashes-from-file` | Read hashes from file |
| `--from-date 2024-01-01` | Modified on or after |
| `--to-date 2024-12-31` | Modified on or before |
| `--mime-type video` | Filter by media type |

## squashr Integration

MedMan containers produced by `archival.py` are standard SquashFS images
with an `index.json` at the root plus the canonical hash hierarchy. They
can be mounted and imported into any MedMan instance via the auto-import
feature or the container mount API.

## License

GNU AGPLv3