personal-data consolidation · CLI + local web UI · Python, SQLite

TidyDisk: one organized library out of a decade of scattered personal files

Old phone backups, dead-drive dumps, and “I’ll sort this later” folders spread across disks that don’t agree on names or languages. TidyDisk pulls the whole pile into a single deduplicated library. It fingerprints everything, drops exact duplicates and junk, and files what’s left where it belongs. For the media in the pile it goes further, laying things out the way Jellyfin or Plex expect them: movies as Title (Year), shows as Series/Season NN, photos into year folders. When the rules can’t tell what something is, a model (local or cloud) proposes an answer and you approve it.

The problem

Every family has a version of this: a decade of phone backups, old-drive dumps, and half-finished “I’ll sort this later” folders, scattered across drives that don’t agree on a naming convention or a language. I pointed it at 2 TB and 388,609 real files.

TidyDisk is the release version, built through multiple rounds of logic distillation. Here it is running the whole job (scan, sort, organize, dedup, verify) against a small synthetic library:

tidydisk: a real session (synthetic demo library)

$ tidydisk scan librarylibrary indexed: 8 files (8 hashed this run, 0 unreadable)$ tidydisk scan old-laptopsource 'old-laptop' scanned: 12 files (0 unreadable)$ tidydisk verdicts old-laptopverdicts for 'old-laptop' (12 files): JUNK=2  ORGANIZED=2  REVIEW=0  UNIQUE=8 # 8 files exist nowhere else: they get copied in FIRST (order is enforced)$ tidydisk plan organize old-laptopplan 819a184e0d02 (organize, 8 ops): T:\.tidydisk\plans\plan-organize-old-laptop-819a184e.jsonl  note: coverage: 0.0% unmatched (threshold 5.0%)  note: 5 content-routed (Jellyfin layout), 3 provenance-flat$ tidydisk apply "T:\.tidydisk\plans\plan-organize-old-laptop-819a184e.jsonl"REHEARSAL 819a184e0d02: would_do=8$ tidydisk apply "T:\.tidydisk\plans\plan-organize-old-laptop-819a184e.jsonl" --executeEXECUTE 819a184e0d02: done=8 # only now may duplicates leave: staged to a same-drive folder, never deleted$ tidydisk plan dedup old-laptopplan f31c4c366eff (dedup, 4 ops): T:\.tidydisk\plans\plan-dedup-old-laptop-f31c4c36.jsonl  note: confirmed ORGANIZED twins (quick match, full SHA-256 above 256KiB, P21/A3); 0 failed confirm (kept in place, not staged)$ tidydisk apply "T:\.tidydisk\plans\plan-dedup-old-laptop-f31c4c36.jsonl" --executeEXECUTE f31c4c366eff: done=4$ tidydisk verify libraryunindexed: 0missing_on_disk: 0drifted: 0protected_in_staging: 0unjournaled_staging: 0

What that run is doing

Every file gets sorted into one of four verdicts: JUNK, ORGANIZED (already a copy of something in the library), UNIQUE (exists nowhere else), or REVIEW (needs a human or a model). Uniques get copied in first, filed by content. Only then do confirmed duplicates get cleared out, and a closing verifyre-checks every path against the index.

Files the rules can’t place go to a model you choose: local on your own machine, or Claude or Gemini with your own key. It proposes; you approve.

The same two passes, in the browser

tidydisk serve is the product surface for people who don't live in a terminal. Pass 1 seals one proposal; Pass 2 executes exactly the sections you approve. Everything starts rejected. Real screenshots of that review, against the same synthetic demo world above.

tidydisk review screen: a sealed proposal with per-cluster approve/reject controls, rehearsal counts, and a gated dedup section — Pass 2, the review. Each cluster shows its destination, row count, and the rule that produced it; dedup stays gated until organize has executed.

tidydisk overview screen: library index stats, sources, staging roots, and the latest sealed proposal — The control room: library stats, sources, and staging roots, which are same-drive and reversible.

tidydisk settings screen: three critic-provider cards for local Ollama, Anthropic API, and Gemini API with key status chips — The agent layer's front door: local model, Claude, or Gemini. Pasted keys go to a workspace `.env` and are never shown again.

No delete

It's human in the loop, and the code makes permanent deletes impossible: files get staged or trashed, never deleted outright.

How this got built

The road taken to the 0.1.0 release: every round left behind a mechanism and the regression test that pins it.

Every change went through the same steps: a written plan, a coding pass that implements one scoped piece at a time, and a separate adversarial pass that tries to break what the first pass built. Nothing merges on “tests pass on my machine”: every change runs the full suite locally, then twice more in a fresh Linux container, to catch the class of bug that only shows up crossing a platform boundary.

966TESTS, INCL. HYPOTHESIS PROPERTY TESTS

109DOCUMENTED MECHANISMS + INVARIANTS

69MODULES, ONE OF WHICH CAN TOUCH DISK

2×LINUX VERIFICATION RUNS PER WINDOWS CHANGE

the discipline behind every commit

$ python -m pytest -q966 passed in 149.11s (0:02:29) # same change, a fresh Linux container, twice, to catch anything the# Windows-only local run would never see$ docker run --rm -v "$(pwd)":/repo -w /repo python:3.11-slim bash -c "pip install -q -e .[dev]; python -m pytest -q; python -m pytest -q"=== run 1 ===962 passed, 4 skipped in 194.54s (0:03:14)=== run 2 ===962 passed, 4 skipped in 187.06s (0:03:07) $ git commit -m "tidydisk 0.1.0: initial release"[public-master (root-commit) b176417] tidydisk 0.1.0: initial release 165 files changed, 34188 insertions(+)

Why I built it this way

AI-assisted engineering can write the code. The harder question is whether the process around it is good enough to trust the result with real files: plans that get gated instead of assumed, invariants enforced by a test instead of a comment, and the same verification bar a human contributor would have to clear. That’s what I was testing with TidyDisk.

← all projects