OpenAI Whisper-1
A much simpler integration, but a cleanup-oriented transcription path was misaligned with the product’s job: reveal disfluencies exactly enough to coach them.
model: "whisper-1"
AI Speech Coach · PM + Developer · 2026
Speak Is Easy
A mirror for the words you do not hear yourself say.
I designed and built a bilingual speech coach that turns a one-minute recording into an honest view of filler words, pace, speaking patterns, structure, and vocabulary—without polishing away the evidence first.
Role — Product, design + development
Status — Live web product
Built — April 2026
Try the live product ↗
Problem & opportunity
The invisible speaking habit.
Filler words, vague openings, repeated phrases, and weak structure happen in real time. The speaker is busy thinking about the next idea, so the behavior is nearly invisible until someone else points it out.
Most practice tools either require another person or produce a polished transcript. I saw an opportunity for a private, repeatable loop: give people something specific to talk about, preserve what they actually said, and turn the recording into concrete coaching.
“The transcript is not the deliverable. It is the evidence.”
— Product framingThe product loop
A repeatable deliberate-practice loop.
Instead of asking users to invent both a topic and a practice routine, Speak Is Easy supplies the prompt, constrains the session, and turns the result into a next attempt.
01Pick a promptChoose a category or ask to be surprised.
02Speak for 1–3 minutesA timer and waveform keep the task focused.
03See the patternReview fillers, pace, structure, and vocabulary.
04Try againUse a better phrase or structure on the next round.
The critical model decision
Model fidelity over transcript polish.
I briefly replaced the first transcription pipeline with OpenAI Whisper-1 because its REST API was simpler than maintaining a binary WebSocket integration. I reverted that change sixteen minutes later.
For most transcription products, corrections and cleaned-up language are features. For a speech coach, they can destroy the signal. If the model removes an “um,” repairs a broken sentence, or guesses the intended phrase, the coach is evaluating an edited performance instead of the one the user gave.
Volcengine SeedASR streaming
The streaming ASR path preserves the raw phrasing needed for contextual filler detection, while supporting both English and Chinese in the same practice product.
volc.seedasr.sauc.duration
Decision principle
Optimize the model for the product’s truth—not for the prettiest output.
Latency & architecture
Streaming architecture for near-instant results.
The original flow uploaded audio only after the recording ended. That made the processing screen absorb the entire transcription delay before coaching could even begin. I moved ASR into the recording window instead.
ClientBrowser audio
TransportRailway relay
TranscriptionSeedASR
AnalysisClaude coach
OutputResults
Parallel work: PCM audio streams through a persistent WebSocket while the session is in progress. When the user stops, the transcript is already available and only the coaching pass remains.
From detection to coaching
Coaching beyond the filler count.
The first version focused on filler detection: what the speaker said, how often, and what to try instead. I expanded the analysis into a fuller coaching report that can identify thinking and speaking patterns, recommend a structure, and surface vocabulary worth reaching for.
Speak Is Easy · Coach notes1:00 session
Your point is there. The path to it can be cleaner.
What I noticedHedged opening
You softened the claim before the listener knew what it was.
Try this structurePoint → Reason → Example
Lead with the answer, then earn it with one concrete example.
VocabularyReplace “basically”
Pause, or use “in essence” only when you are truly summarizing.
Product guardrails
Explicit guardrails for contextual AI.
Keep the original transcript intact
The coaching layer tags evidence without rewriting the words it is meant to evaluate.
Detect fillers contextually
“Like” is not always a filler. The model judges usage in the sentence instead of applying a blind keyword match.
Do not invent insight from thin evidence
Sessions under roughly twenty spoken words skip the deeper critique and ask the user to speak longer next time.
Static interface, dynamic coaching
Navigation stays predictable while the analysis follows the language of the recording.
Outcome & reflection
A live product and a clearer model-selection standard.
Speak Is Easy is live as a mobile-responsive English and Chinese web product, with a bilingual prompt library, real-time streaming transcription, contextual coaching, and PostHog instrumentation across the core funnel.
The central lesson was not to choose the least capable model. It was to define capability in product terms. A transcription model can score higher, sound cleaner, and still be the wrong component if it removes the behavior the user came to understand.
- Model quality is task-specific. Fidelity to disfluency mattered more than editorial polish.
- Latency is a workflow problem. Streaming moved waiting time into a moment the user was already spending.
- Feedback must end in action. Structure and replacement language make the next practice round more useful than a score alone.