voice-transcriptor/docs/superpowers/specs/2026-08-30-audio-preprocessing-chunking-design.md

Audio Preprocessing and Chunking Design

Goal and Scope

Add a cancellable, duration-based FFmpeg pipeline for m4a, mp3, wav, mp4, mov, webm, and mkv. It converts speech into compact overlapping chunks, records exact source timestamps, and exposes settings and progress through the desktop GUI. Transcription API calls remain out of scope.

The repository currently lacks its planned main window and bootstrap, so this change also adds the minimal host UI needed to select a recording, start preprocessing outside the GUI thread, show progress, and cancel a job.

Architecture

• models.py defines preprocessing settings and immutable chunk, manifest, and progress values.
• services/chunking.py contains pure validation, boundary, overlap, duration, and offset calculations.
• services/preprocessing.py owns job directories, FFprobe validation, incremental FFmpeg execution, progress parsing, cancellation, manifests, and cleanup.
• services/settings.py and ui/settings_dialog.py persist and edit the advanced values.
• ui/main_window.py adapts service callbacks to Qt signals and runs preprocessing on a worker thread.
• app.py and __main__.py wire the runnable application.

Services never import PySide6.

Inspection and Validation

Every job uses the existing shell-free FFprobe service. Its query is extended to identify audio and video streams. Preprocessing rejects unreadable paths, unsupported extensions, missing tools, sources without audio, unavailable or non-positive durations, non-positive chunk duration, negative overlap, and overlap greater than or equal to chunk duration. Typed service errors expose sanitized GUI messages.

Exact Chunk Math

Defaults are 900 seconds per chunk and 15 seconds overlap. Chunking is duration-based and assumes no universal upload-size limit.

For source duration D, chunk duration C, and overlap O, the stride is C - O. The first chunk is [0, min(C, D)]. Each next chunk begins one stride after the previous start and ends at min(start + C, D). Generation stops when a chunk reaches D. Thus adjacent full chunks share exactly O, and the last end equals D.

Calculations use Decimal values derived from FFprobe strings to prevent accumulated binary-float drift. JSON timestamps are decimal seconds serialized as strings. Each chunk records its zero-based index, relative path, exact source start/end, and duration. A future local transcription timestamp maps to the source timeline by adding the chunk's source start.

Incremental FFmpeg Processing

One FFmpeg process is launched per boundary. It selects 0:a:0, disables video/subtitle/data output, seeks to the exact start, limits output to the exact duration, downmixes to mono, resamples to 24 kHz, and encodes AAC-LC at 64 kbps in .m4a. Machine-readable -progress pipe:1 -nostats output drives progress.

Mono AAC-LC at 24 kHz and 64 kbps retains strong speech-recognition quality while keeping a 15-minute chunk near 7.2 MB. Video audio goes directly into each chunk without a large intermediate file. FFmpeg streams input and output; Python retains only progress lines and metadata, so multi-hour sources do not enter RAM.

Manifest and Temporary Lifecycle

Each run creates a unique voice-transcriptor-<job-id>-* system-temporary directory containing manifest.json and chunks/. The manifest records schema/job identifiers, source metadata, effective settings, output codec settings, state (running, completed, cancelled, or failed), every planned chunk and timestamp, completed count, and an optional sanitized failure message.

The manifest is atomically written before chunking and after every chunk or terminal transition. It never contains credentials.

The service returns a context-managed job result. The GUI closes it after success, cancellation, or failure. Cleanup removes only the exact verified job directory unless retain_temporary_files is enabled. Retained paths are logged for debugging; cleanup failures are reported because they may leave material data behind.

Cancellation and Progress

A thread-safe token is checked before probing, before every chunk, and while consuming FFmpeg progress. Cancellation terminates the active process, waits briefly, and kills it only if necessary. It prevents later chunks, marks the manifest cancelled, and removes partial current output when possible.

Progress carries percentage, phase, and message. FFmpeg out_time is combined with completed planned work, remains monotonic, and reaches 100 only after the completed manifest is written. The Qt worker emits progress, completion, cancellation, and sanitized failure signals. The window disables conflicting controls during a job, enables Cancel, restores controls on all terminal paths, and requests cancellation when closing.

Settings and GUI

AppSettings gains chunk_duration_seconds=900, chunk_overlap_seconds=15, and retain_temporary_files=false. Older settings files load these defaults; invalid persisted values use the existing warning/fallback behavior. Saving stays atomic and excludes API keys.

An initially collapsed Advanced group provides integer-second duration/overlap controls and a retain-files checkbox. Validation requires overlap smaller than duration.

The main window provides supported-format selection, metadata, a preprocessing action, progress bar, Cancel button, and ordered status messages. Completion reports prepared chunks without invoking the transcription stub or any API.

Testing and Verification

Pure tests cover sub-chunk sources, exact/fractional final boundaries, multi-hour duration, default and zero overlap, total planned duration including overlaps, source timestamp offsets, and invalid values.

Service tests use controlled process fakes to verify commands, incremental progress, manifest transitions, cancellation, and cleanup without requiring FFmpeg. Probe tests cover audio/video metadata and missing audio. Settings and Qt tests cover defaults, round trips, advanced validation, progress, controls, and cancellation.

Final verification runs the entire pytest suite, Python compilation, and whitespace checks. If FFmpeg and FFprobe are installed, a generated fixture is processed end-to-end; missing binaries are reported separately rather than failing portable unit tests.

Success Criteria

Supported inputs are inspected and converted incrementally into overlapping speech-oriented chunks; each chunk has exact source timestamps in an atomic manifest; multi-hour operation is memory-bounded; cleanup honors configuration; GUI progress and cancellation do not block; advanced settings persist; no transcription API is called; and all tests pass.