Turn a paper into a narrated video.
PaperCast is an end-to-end research-communication pipeline. It finds the papers worth covering, writes an approvable script from one, designs and renders the slides, speaks them in Mandarin, and composes a subtitled MP4 — with a human gate at the one place a human decision actually matters.
Python 3.11+ · FastAPI · SQLAlchemy · PyMuPDF · headless Chrome · ffmpeg
Discovery in front, a gate in the middle, video at the end.
Most "paper to video" tools are one prompt and a template. PaperCast is a job runner: every stage writes a manifest, every artifact is content-addressed, and the stage that costs money never runs before the stage that checks the work.
Sources are polled, papers scored on a dual-axis rubric, duplicates collapsed by identity key.
You pick the paper and upload the PDF. Nothing enters the library on the model's say-so.
One call writes the whole script; a draft→review→revise loop owns its quality.
The script and its primary points are approved by a person before any slide exists.
The approved script is cut into spans; one HTML/CSS/SVG slide is designed per span.
Rules, then a narration judge, then an optional per-slide vision pass — before TTS spends.
Per-slide Mandarin audio, SRT/VTT subtitles, provenance JSON, a synchronized MP4.
Four screens, one job.
The papers find you.
Radar sits in front of the pipeline. Configurable sources — RSS, Crossref, PubMed — are polled on their own schedule by a standalone collector process, filtered against a keyword library, deduplicated, and scored on two independent axes: academic weight out of 25, and how well the work will actually carry as a video out of 25.
- Priority, not a feed. Candidates group into P0 / P1 / P2 with S/A/B/C tiers, so triage is a decision about the top band rather than a scroll.
- Deterministic scoring. The rubric is pure code — the same paper scores the same every time, and re-discovery re-scores in place.
- Identity, not bytes. A canonical key (
doi:→pubmed:→arxiv:→ title hash) dedups a paper across re-discovery and manual upload. - Approval is an upload. Papers are paywalled; approving a candidate means supplying its PDF. A rejected candidate is never resurrected.
Approved papers, and nothing else.
The library is the set of papers a person has signed off on. Radar approvals and direct uploads land in the same place, merged when the bytes already match something on the shelf. A job can only ever be created from an approved paper.
- Files are truth, the database is an index. Catalog rows point at files on disk; a repair endpoint reconverges them. No content is ever stored in the DB.
- Tags and full-text search over title, authors and abstract, so a back catalogue stays navigable.
- One paper, many jobs. Re-running a paper in a different template, language or content style never duplicates the source.
Long jobs you can leave alone.
With real models a single stage runs twenty minutes or more, so the job list is built for work that outlives your attention: every job shows its stage, every job is resumable, and cancellation is honoured inside a stage — not merely at its boundaries, which is the difference between stopping a job and watching it finish anyway.
- Content-hash caching per slide. Render, storyboard, TTS and compose each keep a per-slide manifest; a resume recomputes only the slides whose inputs changed.
- Two ways past the cache, both deliberate.
resynthesizefor speech-only changes, and a single-stagererunthat takes free-text instructions. - A re-run is a revision. The stage's model gets the artifacts it already produced plus your request — and because that request is not in the cache key, a later ordinary run still hits.
- Outputs are never overwritten. Video generations are content-addressed (
papercast.g2.mp4); history is prunable, not clobbered.
From approved script to a finished cut.
A job is five inspectable steps — Settings, Script, Slides, Narration, Video — and the pane shows each rendered slide beside the exact words it speaks. Once a stage is done its output is read-only: the deck you are looking at is the deck that was rendered, narrated and cut, not a draft that can drift underneath the video.
- The storyboard is the contract. A versioned JSON pairs every rendered slide with its span of the approved script. It is extraction, not authoring — the stage never writes narration, and every stage after it reads that one file.
- Slides are drawn, not generated. All HTML, CSS and SVG, screenshotted by a headless browser at a fixed 1920×1080 — no image-generation model, so charts, comparison tables and big-number callouts say exactly what the data says.
- Timing comes from the audio, not an estimate. One speech task per slide; the WAV that comes back is measured, and that measured duration is what drives the cut and the subtitle timings.
- Compose is one ffmpeg pass per slide, concatenated by stream copy into the final MP4 — alongside SRT and VTT subtitles and a provenance record embedding the full storyboard and the ffprobe output.
- QA is a gate, not a report. Every slide has segments, every segment has at least one citation, every image is exactly 1920×1080, every referenced audio file exists — and the shipped narration is compared back against the script that was approved at the start.
One call writes the script. A loop owns its quality.
A single long-context call writes the entire approvable artifact — the primary points, the cover title and one continuous script. A loop outside the writer then decides whether it is good enough. Every iteration sees the whole paper and the whole artifact, so nothing is ever built on a summary of a summary, and the script is cut into slides only after a person has approved the prose.
# every iteration sees the whole paper # AND the whole artifact goal_reached, draft, findings = False, None, [] while not goal_reached: # ONE call writes the WHOLE artifact draft = write(draft, findings) # rules first, then a reviewer goal_reached, findings = review(draft)
The harness decides when it is done, never the model.
Two layers judge every draft: arithmetic first — schema, length, banned wording, restatement, and every Arabic number checked against the source text — then a separate reviewer model whose findings are only kept if the quote is verbatim in the draft. The loop keeps the best draft, not the last, ranked on severity-weighted badness. Minor findings never buy a revision; they go to the human at the gate.
Primary points are the checkable artifact
Three to five co-primary points for a review, one to three for a narrow empirical paper, each with a compact basis. It is how a reviewer checks the model's reading before approving its writing — and it forbids collapsing a multi-axis source into one thesis.
Location alone scores near a bare retry
A model repairs an error readily once told where it is, and cannot reliably find it itself. So every signal comes from outside the writer, and every finding names the admissible alternative. Rejected drafts are never accumulated — each revision sees the current draft only.
The cut can never rewrite the script
After approval, one call returns a cut marker and a picture per span. The cutting itself is deterministic, and reconstruction is verified character for character. Unusable markers degrade to an even split and say so.
Detection wired to action.
A finished video is the worst place to find a defect — and by the time one exists, the pipeline has already produced everything needed to catch it: rendered slides, a storyboard contract and the full narration. The review stage reads that evidence and turns it into a decision somebody can act on, while acting on it is still cheap.
Judged before the paid step
Review runs between storyboard and synthesize: the slides are rendered, the contract is written, and text-to-speech — one cloud task per slide — has not yet been billed.
The family routes the repair
A closed vocabulary maps each finding id to a severity and the stage that owns the fix. That routing key is exactly the shape the existing re-run endpoint already takes, so repair reuses the whole path instead of inventing a second one.
One substring test kills the dominant failure
A finding whose quote is not verbatim in the artifact is dropped. The characteristic judge failure — a plausible invented problem — does not survive it.
Rules, then one whole-narration call
Deterministic rules are free and catch most of it. The narration judge then reads the entire script in one call, because the defects that matter are cross-slide — a repeated point, a jump between two spans — and a per-slide judge structurally cannot see them.
A noisy family gets demoted, not learned from
Dismissals are recorded, and a finding family dismissed more than 30% of the time is demoted before any proposal is drawn from it. A review panel nobody trusts is one nobody reads.
Shadow mode is what ships
Findings are recorded; nothing is rewritten unless you switch repair on. Switched on, it queues at most three passes and compares a severity-weighted score rather than a count, so trading one serious defect for four trivial ones does not read as progress.
Built for calls that cost minutes and money.
Three kinds of call failure, three different answers
Transient — a 429, a 5xx, a reset connection: ask the same model again after a capped
backoff honouring Retry-After. Permanent — a retired id or an exhausted
quota: bench the model for the process so later calls skip it instead of paying its failure.
Timeout: move to the next model in the fallback chain without retrying.
A stalled stream is a failed call and a retryable one — gateways report a lost upstream in-band, so a half-written reply is never returned as the model's answer.
A failed call must never cost work already done
The narrative loop keeps its best draft when a later call fails. The reviewer is advisory and swallows everything it can raise. A failed slide costs one slide, not the deck. The first draft is the expensive one; losing it because the next call broke is the worst trade available in the stage.
A re-run that names slides revises only those
"Slides 3 and 5", "page three", "the one about trial results" — working out which slides a person meant is a reading task, so one small call resolves the request against the deck's own slide titles, and every number it returns is range-checked. Those slides are then edited in place, one call each, and the rest of the deck stays byte-identical. Naming all of them, or none, falls back to the whole-deck path.
Per-workspace overrides that cannot leak
The deployment's .env is the default, not the last word: a workspace overrides
an allowlisted subset — voice, pacing, template, content style, budgets. It is a context
variable rather than process env on purpose, because the runner executes jobs from different
workspaces concurrently on a thread pool. Credentials are never overridable.
Session per job, trace per stage, generation per call
Optional Langfuse tracing answers what the job logs cannot: exactly what was sent to the model, and which prompt version produced which answer. Synthesis is traced too — the one paid per-slide stage was the one with a hole in its session. No audio leaves the box, and the traced parameter list is an allowlist.
Real models by default, hermetic in CI
Every provider selector defaults to the real model, so the product works out of the box. The deterministic offline stand-ins still exist and are pinned in the test config, so the suite needs no network, no GPU and no browser — and never sleeps on a retry.
What it runs on.
| Input | A research paper PDF — uploaded directly, or approved out of Research Radar. |
|---|---|
| Output | A synchronized MP4 with per-slide Mandarin narration, SRT and VTT subtitles, and a provenance.json embedding the full storyboard and ffprobe output. |
| Frames | 1920×1080 landscape (a hard contract, asserted by QA) and a 1080×1920 vertical design system for short-video output. |
| Slides | All HTML/CSS/SVG, screenshotted by headless Chrome. Typed slide kinds: title, points, process, diagram, stats, compare, bars, figure, template, closing. |
| Speech | Alibaba Model Studio Qwen-Audio-TTS over a DashScope WebSocket by default; any OpenAI-compatible /v1/audio/speech endpoint, including a local Qwen3-TTS server, as an alternative. |
| Platform | Python ≥ 3.11 · FastAPI REST API under /api/v1 · SQLAlchemy catalog with Alembic migrations · PyMuPDF · ffmpeg + ffprobe · a no-build web UI. |
| Deployment | Single-node Docker Compose — app, PostgreSQL and MinIO — with a single-process job runner gated by a file lock, and a standalone radar collector beside it. |
| Interfaces | The web workspace, a full REST API, and a CLI that runs the whole pipeline or any single stage standalone. |
Research explained, at the pace research arrives.
PaperCast is built and run at 67AI Lab as a working system, not a demo — the engineering notes behind it, and the rest of the lab's writing on agentic AI, are on the blog.