Inside the Studio: How We Turn Content Into a Two-Voice Podcast
It's easier than it looks.
There’s a particular kind of magic the first time you press “Generate Audio Overview,” wait about a minute, and get back a genuinely listenable conversation between two people discussing the exact content you were reading. No robotic monotone. No “the article states that.” Just two named hosts talking, complete with the occasional “umm” and “you know,” in whichever language you happen to be using the app in.
This post is the first in a series pulling back the curtain on the Studio panel — the content-generation half of our educational assistant. We’ll start with the feature people ask about most: the audio overview. Specifically, three things that make it work well:
How the audio actually gets generated — from raw content to a stitched MP3 with captions.
How personas make every language feel native — why the French version isn’t just “the English one, translated.”
How we make it fast — fan-out threading and ordered aggregation, so a ten-minute podcast doesn’t take ten minutes to synthesize.
Let’s get into it.
The 10,000-foot view
At its core, an audio overview is a two-stage pipeline. The first stage is a writer; the second is a voice cast.
The LLM doesn’t produce audio, and the text-to-speech engine doesn’t produce prose. Each does one job well. The interesting engineering lives in how we structure the handoff between them — and how we parallelize the slow part.
Here’s the whole thing end to end, with a little more honesty about where the wall-clock time goes:
Two details worth flagging early, because they save an enormous amount of time and money:
Everything is content-hashed and cached. The cache key is built from stable identifiers — the course/content/document IDs, the chosen voices, the target length, the language, the model — not the raw bytes of the source. So two users asking for the same content share the same audio even if their text chunks arrive in a slightly different order. A cache hit skips both the LLM call and all the TTS work.
The slow part is the voices, not the writing. One LLM call produces the whole script in a single request. But a script has dozens of turns, and each turn is an independent network round-trip to the speech service. That’s the part we parallelize.
Stage one: writing a script worth listening to
First, a word on where that “content” comes from — because it isn’t a raw file we hand the model. When you select content in the app, the backend doesn’t reach for the original source file. Instead it pulls the relevant chunks out of the vector store — and those chunks are the content the LLM summarizes into the script.
Every artifact in the app has already been split into passages and embedded into ChromaDB during ingestion. A selection is really a query against that store: fetch the chunks tagged with this course and section, or with this document’s ID, and stitch them back together in order. Those retrieved chunks are exactly what the LLM reads and condenses into the two-voice script.
The upshot: the “wall of content” the writer sees is an assembled set of retrieved chunks, not the source document itself. That keeps the pipeline uniform — a piece of course content and a user upload look identical by the time they reach the LLM — and it means the same retrieval layer that powers chat also feeds the Studio.
With the content in hand, the script generator’s job is to convert that wall of text into a natural back-and-forth between an interviewer and a subject-matter expert (SME). We ask the LLM for a single JSON object with a strict shape:
{
"discussion": [
{ "name": "Steve", "statement": "So today we're digging into..." },
{ "name": "Hannah", "statement": "Right, and the key idea here is..." }
],
"simple_summary": "A one-sentence description of the topic."
}
That JSON contract matters more than it looks. A few of the prompt rules that earn their keep:
“Respond with ONE valid JSON object and nothing else.” LLMs love to be helpful — offering to split a long task into parts, asking clarifying questions, apologizing about token limits. For an automated pipeline, a “meta-conversation” is a failure, not a courtesy. The prompt bans it explicitly.
Make it sound human. We ask for conversational pauses (“umm,” “you know”), an intro where the interviewer previews the topics, and a natural flow rather than a robotic Q&A. Crucially, we tell it not to say “the article” or “the text” — the source is the hosts’ knowledge, not a thing they’re reviewing.
Length steering. If you want a ten-minute podcast, you have to fight the model’s instinct to wrap up early. We convert target minutes to a word budget (Azure Neural TTS averages ~165 words per minute), then bracket it with directives at the top and bottom of the prompt: aim for N words, plan for at least M conversational turns, keep going if your draft feels short.
And because models still under-deliver sometimes, we measure adherence after the fact:
word_count = sum(len(t["statement"].split()) for t in result["discussion"])
pct = (word_count / target_words) * 100
if pct < 60 and attempt < 2:
# Retry with a stronger, "you already failed once" directive.
...
If the script comes back under 60% of the target word count, we retry once with a blunter prompt (“PREVIOUS ATTEMPT FAILED… produce the full JSON now — no questions, no apologies”). There’s also a JSON-repair fallback: if validation fails, we hand the broken JSON back to the model and ask it to fix it. Belt and suspenders.
The output is validated against a Pydantic model before it’s allowed anywhere near the voice stage. If it isn’t valid, it isn’t a script.
Stage two: personas make each language native
Here’s where the app does something more thoughtful than “translate the English version.”
When you generate an overview in French, you don’t get Steve and Hannah speaking French. You get Lucas and Camille — French names, mapped to French neural voices. Every supported locale has its own hand-picked cast: a host and an SME, each with a culturally appropriate name and a matching Azure voice.
These pairings live in a single source-of-truth table in the backend, keyed by locale:
VOICES_BY_LANGUAGE = {
"fr-FR": {
"host_name": "Lucas",
"host_voice": "fr-FR-HenriNeural",
"sme_name": "Camille",
"sme_voice": "fr-FR-VivienneMultilingualNeural",
},
# ...one entry per supported locale
}
def voices_for(language):
"""Host/SME voice pair for `language`, falling back to en-US."""
return VOICES_BY_LANGUAGE.get(language) or VOICES_BY_LANGUAGE["en-US"]
Why bother? Because names carry a lot of the “this was made for me” feeling. A Spanish speaker hearing Mateo interview Sofía has a fundamentally different experience than hearing a translated “Steve.” It’s a small dictionary of engineering choices, but it’s the difference between localization and mere translation.
The persona names flow into the script prompt too — the LLM is told “the interviewer’s name is Lucas, the SME is Camille” — so the names the model writes into the dialogue are the same names the voices are assigned to at synthesis time. The written world and the spoken world agree.
A design note worth calling out: these voice choices live in code, not in user settings. They’re engineering decisions tied to the Azure Speech voice catalog — which voices sound good, which are multilingual, which pair well together — not preferences a user should have to configure. Keeping them in one table also makes adding a language a checklist item rather than an archaeology dig.
Stage three: fan-out, or why a 10-minute podcast doesn’t take 10 minutes
Now the fun part. A finished script might have 40 turns. Each turn is a separate call to Azure Speech — connect, send text, receive audio, disconnect. Done sequentially, those round-trips stack up: even at half a second of overhead each, 40 turns is real time spent waiting on the network.
So we don’t do them sequentially. We fan out.
The shape of this matters, so let’s walk the important decisions.
Authenticate once, reuse everywhere. We fetch a single Azure AD token before spawning any workers and pass it into every thread. No worker does its own auth handshake.
Pre-resolve everything a worker needs. Before fanning out, we walk the script and turn each entry into a tidy (index, voice_name, statement) tuple — resolving the speaker’s name to their voice, and dropping any malformed turns. Workers never touch the speaker-config dictionaries; they get exactly what they need and nothing else. This keeps the concurrent code dead simple and free of shared mutable state.
turns = [] # (index, voice_name, statement)
for turn in script["discussion"]:
name, statement = turn.get("name"), turn.get("statement")
if not name or not statement:
continue
voice = interviewer["voice_name"] if name == interviewer["name"] else sme["voice_name"]
turns.append((len(turns), voice, statement))
Bounded concurrency. We don’t spawn 40 threads for 40 turns. The pool is capped (default six workers, tunable via an environment variable) to stay comfortably under the speech resource’s per-account concurrency limits. Fan-out without a bound is just a different way to get rate-limited.
max_workers = int(os.getenv("TTS_MAX_WORKERS", "6"))
results = []
with ThreadPoolExecutor(max_workers=max_workers) as pool:
for outcome in pool.map(_synthesize_turn, turns):
results.append(outcome)
Each worker is self-contained and failure-tolerant. A worker builds its own speech config, synthesizes one statement, and returns (index, statement, audio_segment) — or (index, statement, None) if that one turn failed. One bad turn doesn’t sink the whole podcast; it just leaves a gap we handle gracefully at aggregation time.
Aggregation: order out of chaos
Here’s the subtle bit. Threads finish in whatever order they finish — turn 12 might come back before turn 3. If we stitched audio in completion order, the conversation would be scrambled nonsense.
That’s why every worker carries its original index all the way through. Aggregation is then a three-step dance:
results.sort(key=lambda r: r[0]) # 1. back into script order
valid = [(i, s, seg) for i, s, seg in results if seg] # 2. drop failed turns
# 3. concatenate, inserting a pause between (not after the last) turn
full_audio = AudioSegment.empty()
current_ms = 0
last = len(valid) - 1
for i, (_idx, statement, seg) in enumerate(valid):
timestamps.append({"statement": statement, "start": current_ms, "end": current_ms + len(seg)})
full_audio += seg
current_ms += len(seg)
if i != last:
full_audio += AudioSegment.silent(duration=700)
current_ms += 700
Three things fall out of this loop for free:
Correct order. Sorting by the original index un-scrambles the parallel results.
Natural pacing. A 700 ms pause between turns keeps the conversation from sounding rushed — but we deliberately don’t trail a pause after the final turn. That trailing silence used to inflate the reported duration by hundreds of milliseconds per turn (about 26 seconds on a ten-minute podcast), and players ignore it anyway.
Free captions. Because we track
start/endin milliseconds as we stitch, we get a precise timestamp for every statement — which we turn straight into a WebVTT caption file. Same data, two outputs.
The concurrency model here is the classic scatter-gather: scatter N independent units of work across a bounded pool, gather them back, and restore order using a key you attached before scattering. It’s a pattern worth internalizing because it shows up everywhere — parallel HTTP fetches, map-reduce, batch inference. The audio overview is just an especially audible example of it.
Putting it together
Zoom back out and the whole feature is a study in doing the slow things once and the parallel things in bulk:
One LLM request writes the entire script, with length steering and self-correcting retries.
Per-locale personas — real names, real voices — make every language feel authored rather than translated.
Bounded fan-out turns dozens of sequential network calls into a handful of parallel batches, and index-preserving aggregation reassembles them in the right order with the right pacing and caption timing.
Content-hash caching means the second person to want a piece of content waits milliseconds, not a minute.
None of these pieces is exotic on its own. The LLM writes; the TTS speaks; a thread pool parallelizes; a hash caches. The value is in the orchestration — the same lesson that keeps showing up as we build out the Studio.
Next in the series, we’ll look at the other Studio tools — quizzes, study guides, and mind maps — and the shared generation machinery underneath them. Different outputs, surprisingly similar plumbing.
Thanks for reading. If you’re building something similar and want the gnarly details on any one stage, reply and let me know which — I’m happy to go deeper.