Local Player: atomic progress that travels with the drive

Losing where I stopped in a video was the bug that annoyed me most. So I treated progress as critical data.

Context

In part 1 I told you about the Local Player’s overall architecture. Here I’ll tell you about the subsystem that cost me the most sleep: saving the lesson position in server/progress/store.js.

I decided progress would live next to the library, in <lib>/.courseplayer/progress.json. That way it travels with the HDD or USB stick — I unplug it, take it to another machine, and pick up where I left off. Each entry is {position, duration, completed, updatedAt} under a libId\0rel key.

The problem I wanted to solve

It was always the same story: I’d shut down the server mid-save, the power would go out, or progress.json would get corrupted — and I’d lose everything. Or worse: I’d clear the transcode cache and the progress went with it.

My rule became: a normal save never deletes a key, never removes a completed, never loses a valid position. The frontend had to cooperate too, with throttling and a few guards against regression.

The most important decision

Three simple layers on the backend:

1. atomic writes (writeFileAtomic: writes to a tmp file, fsyncs, renames, fsyncs the directory);
2. serial queue (updateProgress) — one save at a time, never two together;
3. double backup (.bak and .bak.1; if the main file corrupts, it’s preserved as .corrupt-<ts> and restored from backup).

save → tmp exclusivo → fsync arquivo → rename → fsync dir
corrompido → preserva .corrupt-<ts> → tenta .bak → tenta .bak.1

I considered using SQLite with WAL, but it would pull in a native dependency and complicate things on FAT/exFAT. Atomic JSON with backup covers my real volume, and anyone can open and audit it.

flowchart TD
  Player["timeupdate / pause / ended"] --> Fila["updateProgress (serial)"]
  Fila --> Atomica["writeFileAtomic"]
  Atomica --> Disco["progress.json"]
  Disco --> Backup[".bak / .bak.1"]

How I used AI in development

It was great for generating the test matrix (progress-invariance, progress-persistence, progress-forensic, library-progress) — position regression, preserved completed, per-library isolation.

But it tried to talk me into parallel writes with aggressive debouncing on the backend to “optimize”. I had to lock the rule in AGENTS.md: serial queue, no exceptions.

And I did validation the hard way: I wrote progress, killed the server mid-save, restarted, and checked it came back. I corrupted the JSON on purpose to watch .corrupt-<ts> be born and the restore kick in.

What I discovered implementing it

• Clearing is surgical: by <course>/ prefix or global. And POST /api/transcode/clear clears transcode and cancels jobs, but never touches progress.json — there’s a test just for that.
• Rescan never deletes progress.
• On the frontend (public/js/player.js): timeupdate with 5s throttle, pause/ended/beforeunload with sendBeacon; above 95% marks as completed; rewatching doesn’t unmark; position 0 doesn’t erase; nothing is written without metadata; resume only if 3 < position < duration - 2.
• Removing a library is config-only (with an active job it returns 409); the default one can’t be removed (403).

What worked

• LP_DATA_DIR to run tests with a real server in a temporary directory, without touching my data.
• LP_PROGRESS_FORENSIC=1 when I need verbose logging.
• Per-library keys ended progress leaking between the internal HDD and the USB stick.

What didn’t work

• The first direct fs.writeFile: a kill mid-write produced truncated JSON and total loss. The backup only became useful after atomicity.
• I tried persisting the sidebar’s open folders alongside and gave up — the tree re-renders a lot and fought the saved state.

What I’d do differently today

I would have written the guards (“normal save never…”) as tests before the code, not after. Many AI bugs would have fallen at the first node --test.

Result

Passing today: kill mid-save preserves, corrupted becomes .corrupt-<ts> and restores, transcode clear doesn’t affect progress, rescan preserves everything.

Missing: compacting progress.json for huge libraries — today it rewrites the whole file on every save.

What I learned

1. Atomicity is cheap

Tmp + fsync + rename solves 90% with no new dependency.

2. “Don’t regress” is a specification

It has to be a test, not a comment.

3. Format follows portability

JSON in .courseplayer travels with the disk; a local database would chain my data to one machine.

Next steps

• Part 1: vibecoding with invariants and zero build step
• Part 3: transcode fallback and Whisper P0-P3

Conclusion

Reliable progress doesn’t need a new stack. It needs total ordering of writes, fsync in the right place, and backup tested with real kills.

References

• server/progress/store.js, server/core/fs-atomic.js, server/progress/routes.js
• test/progress-invariance.test.js, test/progress-persistence.test.js, test/progress-forensic.test.js
• docs/VALIDATION.md section 4