Product Guide 20
Fintech AI

AI Video Interaction: How to Build It

A step-by-step guide to building a system where a person talks to an AI on video and it answers out loud, with a face. Six steps, the real alternatives at each one with published prices, the latency arithmetic nobody does honestly, and where the regulator still requires a human.

Verified September 2026Free · No signupOfficial sources only
BeginnerStart here. No prior knowledge assumed.

How to use this page

This page walks you through building one product: a system where a person talks to an AI on a live video call and the AI answers, out loud, with a face, fast enough to feel like a conversation.

How to read it. Each step below is written to be read straight through, top to bottom, without clicking anything. That gives you the whole workflow in about five minutes.

Inside each step, the ways of doing it are folded away behind a +. Open one only when you are actually building that step. Everything collapsed is a choice; everything already open is the workflow.

Prices are only shown where the company publishes them on its own site. Where a vendor does not publish a price, this page says so rather than repeating a figure from somewhere else.

Watch out

Read this before you build anything. If your plan is to replace the human officer in an Indian video KYC call, stop. The RBI's V-CIP rules require an authorised official of the regulated entity to conduct the interaction, and require that “the activity log along with the credentials of the official performing the V-CIP shall be preserved” [1]. That sentence was written assuming a human. This site found no published RBI guidance permitting an AI to conduct a V-CIP session. Everything on this page is buildable; pointing it at regulated identity verification as a replacement for the officer is not something any official source currently supports.

What this actually is

An AI video interaction is a loop that runs about three times a minute for as long as the call lasts:

hear them → work out what to say → say it → show a face saying it — and get all four done before the silence becomes uncomfortable.

That last clause is the entire engineering problem. Every individual piece below is a solved, purchasable product. Chaining them inside a human conversational rhythm is not.

Where it is genuinely useful in a fintech today: explaining a rejected application, walking someone through a form they have abandoned twice, collections conversations at a scale no call centre can staff, first-line support in a language your team does not speak, and internal interview or training simulations.

Where it is not: anywhere a regulator has named a human as the control. See the warning above.

The whole build, in one table

Six steps. Read this table once and the rest of the page is detail.

#StepIn plain words
1Move the audio and videoGet their camera and microphone to your servers and your answer back, live.
2Hear themTurn speech into text as they talk — and work out when they have finished talking.
3Decide what to sayThe model turn. The one architectural fork on this page.
4Say itTurn the answer back into audio, in under a fifth of a second.
5Put a face on itA streaming avatar whose mouth matches the audio.
6Record and keep itWhere the video lives afterwards, for how long, and in which country.

Steps 2, 3 and 4 collapse into one purchase if you choose a speech-to-speech model. Step 3 explains why you might not want them to.

IntermediateBuild it. Pipelines, tools and working code.

Step 1 — move the audio and video

Step 1

Move the audio and video — both ways, live

What it does
carries their microphone and camera to your servers, and your reply back, with low enough delay to hold a conversation.
Why it is not trivial
a phone on Indian mobile data, behind a corporate firewall, on a bad day. Reconnection, packet loss and echo are the actual work.
The standard underneath
WebRTC, a joint W3C and IETF standard, first published as a W3C Recommendation in January 2021 and re-issued as a Recommendation dated 13 March 2025 [2]. It gives you the browser media path free — not the servers, recording or scale.
Pick by
whether you need to self-host, and whether you are billed per participant or per agent.
Option A — LiveKit (open source, or their cloud)

What it is: an open-source media server you can run yourself, plus a managed cloud. The only genuinely self-hostable option here.

Published price: Build $0/month with 1,000 agent minutes; Ship $50/month with 5,000 included then $0.01/min; Scale $500/month with 50,000 included. WebRTC minutes on Ship are $0.0005/min after 150,000 [3].

What you give up by self-hosting: LiveKit publishes the tradeoff plainly — self-hosted is a single-home server capped around 3,000 users per room with no uptime SLA, against the cloud’s 99.99% target [4].

Watch the billing unit: LiveKit prices agent minutes, everyone else prices participant minutes. The numbers are not directly comparable.

Pick it if data residency or on-premise is a hard requirement — this is the only row on this list you can run inside your own datacentre.

Option B — Daily

Published price: $0.0015–$0.004 per participant-minute for video and audio, volume-tiered; audio only $0.00036–$0.00099. Cloud recording $0.01349 per recorded minute. Free tier 10,000 minutes a month [5].

Also relevant: Daily prices its agent-hosting product separately — $0.01/min for a half-vCPU worker up to $0.03/min, with voice-only 1:1 transport free and voice-plus-video at $0.004 per participant-minute [6].

Pick it if you want the cheapest published per-minute transport and do not need to self-host.

Option C — Agora

Published price: audio $0.99 per 1,000 minutes; video HD $3.99; Full HD $8.99; 2K $15.99 [7].

Two things to know before you commit. The free 10,000 minutes a month ends in service suspension rather than overage billing. And Cloud Proxy — which you will need if your customers sit behind restrictive corporate firewalls — carries a $500/month minimum base fee on top of per-minute rates [7].

They also sell the whole loop: a Conversational AI Engine at $0.10/minute covering speech-in, model and speech-out, whether you use their keys or your own [8]. That is several times what assembling the parts costs — a convenience premium, priced openly.

Option D — Twilio

Published price: $0.004 per participant per minute for group rooms up to 50 participants; participant recordings another $0.004 per participant-minute; compositions $0.01 per composed minute [9].

Status note worth knowing: Twilio announced an end-of-life for Programmable Video and then reversed that decision in October 2024. The product continues [10].

Do not leave recordings there. Twilio storage is $0.00167 per GB per day — roughly $0.05 per GB-month, several times any object store in Step 6. Move finished recordings out.

Feeds into: Step 2 gets the audio stream from here. If it breaks: nothing else on this page runs — and reconnection behaviour, not raw latency, is what customers will complain about.

Step 2 — hear them

Step 2

Hear them — and know when they have stopped

What it does
converts speech to text as it arrives, and decides the moment the person has finished their turn.
The underrated half
end-of-turn detection. Answer too early and you talk over them; too late and you feel slow. This is a bigger source of felt lag than transcription speed.
Pick by
language coverage first, price second. Indian-language support varies enormously here.
Option A — Deepgram

Published price: Nova-3 monolingual streaming $0.0048/min promotional, $0.0077/min regular; multilingual $0.0058 / $0.0092; Flux $0.0065 / $0.0077 [11].

Indian languages: the strongest on this list — Hindi, Indian English, Assamese, Bengali, Gujarati, Kannada, Marathi, Punjabi, Tamil, Telugu and Urdu are all listed [12]. The cited page does not break that list down by streaming versus batch, so confirm your specific language streams before committing to it.

And it is the only vendor here that documents latency honestly. Deepgram publishes a methodology page separating network transit (20–200ms) from server-side processing (150–300ms), gives total observed transcript latency as 200–500ms, states plainly that processing time “cannot be measured directly from the client,” and tells you to track p50/p95/p99 rather than one number [13]. Use this page as your yardstick for judging everyone else’s claims.

Its Flux model folds turn-detection into the transcription model rather than bolting silence-detection on top. Published end-of-turn figures: p90 of 1 second, p95 of 1.5 seconds, with an eager mode firing 150–250ms earlier [14]. Note those percentiles — the headline is not the number you will live with.

Option B — AssemblyAI

Published price: Universal-Streaming $0.15/hour, which is $0.0025/min — the cheapest streaming rate on this page [15].

But check the language tier before you celebrate. The $0.15/hour multilingual tier covers European languages only. Hindi streaming requires Universal-3.5 Pro at $0.45/hour, three times the base rate, and no other Indian regional languages are listed for streaming [16].

On its latency claim: the product page headlines “300ms latency” without defining what is timed or what is excluded [17]. Their pricing page separately publishes a percentile-labelled figure — around 134ms at p50 — but for the synchronous API, not the streaming one [15]. Two numbers measuring two different things, and neither documents the streaming path end to end.

Option C — OpenAI

Published price: gpt-live-transcribe and gpt-realtime-whisper both $0.017/min; gpt-realtime-translate $0.034/min; the batch gpt-transcribe $0.0045/min [18].

Genuine streaming is supported, returning transcript deltas as speech arrives [19].

Two gaps, stated plainly: OpenAI publishes no latency figure at all for any transcription model, and this site could not find published Indian-language support for them. At roughly 3.5× Deepgram’s rate with less documentation, this is the option to pick for reasons other than price.

Option D — Google, or Azure

Google Cloud Speech-to-Text, published: $0.016/min up to 500,000 minutes a month, falling to $0.01, then $0.008, then $0.004/min above two million [25]. At entry volumes that is over three times Deepgram and six times AssemblyAI; it only becomes competitive in the millions of minutes. No latency figure published.

Azure Speech: this site could not verify a single rate. Microsoft’s pricing page renders every figure as a placeholder and requires sign-in [27]. The free tier (5 audio hours a month) and the billing units are published; the prices are not. That procurement friction is itself worth factoring in.

Feeds into: Step 3 — unless you choose speech-to-speech there, in which case this step disappears into it. If it breaks: wrong words reach the model and it answers the wrong question, confidently.

Step 3 — decide what to say

Step 3

Decide what to say — the one real architectural fork

What it does
takes what was said and produces what to say back.
The fork
one model that hears audio and speaks audio directly, or a chain of speech-to-text, then a text model, then text-to-speech.
Why it matters more than speed
in a regulated business the question is not only how fast it answers, but whether you can prove afterwards exactly what it said.
Pick by
whether anyone will ever audit this conversation.
Option A — Speech-to-speech (OpenAI Realtime, Gemini Live)

What it is: one model interprets audio, decides, and responds in speech. OpenAI’s framing: it “processes and generates audio directly through a single model and API,” which “reduces latency, preserves nuance in speech, and produces more natural, expressive responses” [22].

Published price — OpenAI: gpt-realtime-2.1 at $32.00 per million audio input tokens and $64.00 per million audio output; the mini at $10.00 / $20.00. Cached audio input is $0.40 per million — eighty times cheaper than uncached, which tells you how much of a long conversation is re-sent context [18].

Published price — Google Gemini Live: Google publishes both token and per-minute rates, which almost nobody does: $0.005/min audio in and $0.018/min out for Gemini 3.8 Live [23]. Roughly $0.023/min all-in makes this the single biggest cost lever on the page.

Gemini’s hard session limits are a real design constraint: audio-only sessions cap at 15 minutes and audio-plus-video at two minutes [24]. A video interview is longer than two minutes. You will be writing session-resumption code.

Neither vendor publishes a latency figure. The architectural argument that removing two hand-offs is faster is plausible; the evidence is simply absent.

Option B — Chained (speech-to-text, then a text model, then text-to-speech)

What it is: the three separate steps on this page, wired together. More moving parts, more serialisation points.

What you get for that: OpenAI’s own documentation makes the case better than this page could. Chained gives “control over each speech and text stage” and the ability to “inspect or transform intermediate text and replace each component independently” — recommended where “each stage needs to be visible or replaceable,” with their own examples being storing transcripts, running policy checks, or calling internal systems before generating speech [20].

Pick it if you need to run a compliance check on an answer before it is spoken, swap one component without re-architecting, or hand a regulator a transcript you can stand behind.

The fact that should decide this for a regulated build

Speech-to-speech models do emit transcripts of both sides. But OpenAI states in its own documentation:

“The realtime model doesn’t have enough information to precisely align transcript and audio.” [21]

Read that carefully. In a speech-to-speech architecture the transcript is a parallel artefact generated alongside the audio — not a record of it — and the vendor says the alignment between the two is imprecise.

In a chained architecture the text you sent to the speech engine is, by construction, exactly what was said.

If this conversation is retained for five years and may be read by an auditor or a court, that distinction is not cosmetic. It is the strongest argument on this page for the slower architecture — and it comes from the vendor selling the faster one.

Feeds into: Step 4, or straight to Step 5 if the model speaks. If it breaks: you find out at the audit, not at the demo.

Step 4 — say it out loud

Step 4

Say it out loud — before the sentence is finished

What it does
turns the answer into audio, streaming the first sound before the sentence is finished.
The metric that matters
time to first audio, not total generation time. The rest arrives while the first part is already playing.
Pick by
published latency, Indian-language coverage, and concurrency limits.
Option A — ElevenLabs

Published latency, with the honesty footnote that should be standard: Flash v2.5 at ~75ms†, where the dagger reads “excluding application & network latency” — model inference only. v3 Conversational at ~280ms† [28]. Add Step 1’s network transit back on before you plan around it.

Published price: $0.05 per 1,000 characters for Flash and v3 Conversational; $0.10 per 1,000 for v3 and v2 Multilingual [29].

A contractual requirement, not a nicety: ElevenLabs’ use policy requires that organisations “clearly and prominently disclose to their users they are interacting with AI rather than a human,” and prohibits cloning a voice “without consent or legal right” or in a way “intended to deceive others about whether the voice was generated by artificial intelligence” [30]. If you build on ElevenLabs, disclosure is in your contract.

Option B — Cartesia

Published latency: “sub-90ms” for Sonic 3.5 [31]. Note what is missing next to ElevenLabs’ dagger: no statement of what is being timed and no exclusions. Weaker claim, same ballpark.

Hindi is named explicitly among its 42 languages.

Published price: credit-based — Pro $5/month for roughly 133 minutes, Startup $49 for ~1,667, Scale $299 for ~10,667. Voice agents $0.06/min [32].

Check which concurrency ceiling applies to you. Cartesia publishes two sets: 2, 3, 5 and 15 simultaneous requests by tier for standalone text-to-speech, and a higher 8, 12, 20 and 60 for Voice Agents [32]. If you are building the agent product the second set binds — but check it against your expected simultaneous calls before you compare price.

Option C — PlayAI

Published latency, and the clearest metric definition on the page: Play 3.0 Mini at 190ms time to first audio, with “consistent latency of <200ms with streaming”; PlayDialog at 350ms [33]. “Time to first audio” is unambiguous — credit where due.

Price: not published. This site could find no per-character rate, plan cost or credit allocation on their pricing page — only a direction to contact sales. Budget for a procurement conversation.

Option D — Google

Published price, per million characters: Standard and WaveNet $4; Neural2 $16; Chirp 3 HD $30; Studio $160; Instant Custom Voice $60 [26].

A cost trap worth knowing: character counting includes spaces, newlines and all SSML tags. Heavily marked-up speech costs materially more than the words suggest.

No latency figure published, and no separate streaming price.

Feeds into: Step 5, which needs this audio to drive the mouth. If it breaks: the avatar sits there, mouth closed, while the caller waits.

Step 5 — put a face on it

Step 5

Put a face on it — and only now, not first

What it does
renders a talking face, live, in sync with the audio from Step 4.
The distinction that eliminates most vendors
batch video generation and real-time streaming avatars are different products. Most well-known avatar companies do the first. You need the second.
Pick by
session length caps and concurrency, which bite before price does.
Option A — HeyGen LiveAvatar

Published latency, metric named: under 300ms median time to first frame [34]. Note that measures the first frame, not a complete response.

Published price: one credit is 10 US cents. Full Mode consumes a credit per 30 seconds ($0.20/min); Lite Mode a credit per minute ($0.10/min). Plans: Starter $19/month, Essential $99, Business $475 [35].

Session length is capped by tier — 5, 20 and 60 minutes. Check yours against how long your calls actually run.

One inconsistency to resolve in writing before signing: HeyGen’s product site claims unlimited concurrency on Essential and above, while its own help centre publishes caps of 5, 20 and 40. Those cannot both be true.

The “$0.01/min at scale” figure on their marketing page is attached to the Enterprise tier — which itself carries no published price. Treat it as an opening position in a sales conversation, not a rate you can plan against.

Option B — Tavus

The most meaningful latency claim here, because the metric is genuinely end-to-end: “less than one second of latency between utterances,” defined in their docs as the full round-trip from a participant speaking to the response [36][37].

But note where it lives: the number appears on a launch blog post, not in the documentation, which publishes no benchmarked figure. The accompanying “world’s fastest” claim is unfalsifiable.

Published price: Starter $22/month for 60 minutes then $0.35/min; Growth $397 for 1,300 minutes at $0.31/min with 10 concurrent; recordings $0.03/min. Every conversation carries a 30-second minimum charge — which matters if your calls are short [38].

Their six-stage streaming architecture is published in full, which makes it the easiest vendor here to reason about.

Option C — Synthesia Interactive Avatars

The surprise on this list. Synthesia is known for batch video generation, but its Interactive Avatars product is genuinely real-time — the avatar “publishes it to your LiveKit room as a regular participant” [39].

Published price: $0.12/minute pay-as-you-go, no commitment or minimum, plus 500 free minutes on a new API key [39].

Latency: not published — and Synthesia says so itself, noting it is still improving “expressiveness, listening behaviors, interruption handling, and latency.” That candour is worth something, but it also tells you this is the newest product here.

Option D — Simli, and why D-ID is a different thing

Simli publishes under 300ms for the speech-to-video stage, and is the only vendor on this page that publishes a whole-stack latency breakdown — see the latency section below, where it does most of the work. Rates: not published. The cited page advertises a free monthly allocation and refers to volume discounts and pay-as-you-go billing, but publishes no per-minute figure this site could read [40].

D-ID Agents are message-based, not streaming. Their own help documentation bills per “message (generated video) of up to 15 seconds” at half a credit, with each further 15-second interval costing another half credit [41]. That is discrete video generations answering prompts, not a continuously streamed face. Dollar pricing is not published. If you want D-ID here, the Talks Streams API is the endpoint to look at — this site could not verify its latency or price.

Feeds into: Step 1 carries these frames back to the caller. If it breaks: you still have a working voice product — which is worth knowing, because it means the face should be the last thing you add, not the first.

The latency budget

This is the part that decides whether you have a product or a demo.

What the research actually says

The gap between two people’s turns in ordinary conversation, measured across languages, has a mode of about 200 milliseconds. Between 51% and 55% of turn transitions happen inside 200ms; 70–82% inside 500ms. Cross-language averages run from −78ms in Dutch — speakers beginning before the other finishes — to +468ms in Danish [42].

But the number you should design against is not 200ms. It is this one:

The number that matters

Past roughly 600–700 milliseconds, a pause stops being heard as slowness and starts being heard as meaning. Gaps that long are associated with “dispreferred” responses — reluctance, difficulty, disagreement — and experimental work finds that “gaps of 600 ms or longer generate inferences of this unwelcome kind” [42].

For a fintech that is a product problem, not a performance problem. A delay before your AI answers a question about someone’s rejected loan does not read as lag. It reads as hesitation.

One consolation, and it explains why this is hard: humans do not hit 200ms by thinking fast. Producing even a single word takes around 600ms from stimulus to speech, and a simple sentence around 1,500ms [42]. People achieve 200ms gaps by predicting how the incoming turn will end and preparing during it. Deepgram’s eager end-of-turn mode and Tavus’s turn-taking model are engineering attempts at exactly that trick.

The arithmetic, honestly

Simli publishes a full-stack breakdown, which is unusual and unflattering to the whole industry [40]:

StagePublished range
Speech to text~100–500ms
The model turn~250–450ms
Text to speech~250–1,200ms
Speech to video<300ms
Total — this page’s sum, not Simli’s~900–2,450ms

Now do it the optimistic way, taking every vendor’s best published figure from the steps above: 20–200ms network each way, 100ms end-of-turn — the floor of Deepgram’s published 100–500ms range [13] — 150ms transcription, 75ms to first audio, 300ms to first avatar frame. That is about 665ms before the model has thought at all, taking the best end of every published range at once — already past the threshold where people start reading meaning into the pause.

Watch out

Three things go wrong when people add these numbers up. One: nearly every figure above excludes something. ElevenLabs’ 75ms excludes network and application time by explicit footnote; Deepgram’s 150ms is server-side only; HeyGen’s 300ms is first frame, not full response. Summing numbers that each exclude the network and calling the result end-to-end understates reality by the entire round trip. Two: use p90, not best case. Deepgram is the only vendor publishing both, and the gap is instructive — a published end-of-turn range starting at 100ms [13] against a p90 of 1 second for that same step [14]. Over a ten-turn conversation, a p90 event is not an edge case, it is near-certain. Three: no speech-to-speech vendor publishes a latency number at all, so the claim that the single-model architecture is faster cannot currently be checked.

The honest answer: expect 1.5 to 2.5 seconds end to end today against the ~200ms of natural conversation — roughly an order of magnitude off. The most aggressive credible claim in the market, Tavus’s sub-one-second utterance-to-utterance, is still five times the natural gap and still above the threshold where the pause means something.

Design for that instead of pretending otherwise: fill the gap with a listening animation, let the avatar start a filler word while the answer generates, and never let the face sit frozen.

Step 6 — record it and keep it

Step 6

Record it, keep it, and keep it in the right country

What it does
stores the session so it can be produced later, intact, for as long as the rules require.
The cost nobody models
egress. Storage is cheap; reading it back out repeatedly for audits is not.
The constraint that overrides cost
for Indian regulated use, residency. See the note below this step.
Pick by
where your regulator says the data must physically sit.
Option A — Cloudflare R2

Published price: $0.015 per GB-month standard; Infrequent Access $0.01 with a 30-day minimum and $0.01/GB retrieval; Class A operations $4.50 per million, Class B $0.36 per million [43].

Egress is free. That is the whole argument for R2 and it is a large one — for an archive auditors pull from repeatedly, retrieval cost can exceed storage cost everywhere else.

Check region pinning before assuming this satisfies an Indian residency requirement.

Option B — Google Cloud Storage

Published price: $0.020 per GB-month for Standard storage; Class A operations $0.005 per 1,000, Class B $0.0004 per 1,000 [44].

On the India region: Google lists asia-south1 (Mumbai) as a Standard location, but publishes per-region rates behind an interactive selector rather than a static table, and this site could not extract a Mumbai figure differing from us-central1. Check it in the console for your own region — assume neither parity nor a premium.

Egress to the internet is $0.12/GB for the first 10TB, falling to $0.11 and $0.08. Inbound is free.

Do the sum before choosing: storage sits about a third above R2, but the real gap is egress — total cost can differ by more than double once retrieval is counted. A region that satisfies your residency rule may still be worth paying for.

Option C — AWS S3, and the only WORM story with a published assessment

Published price — with a caveat about AWS rather than about S3. The S3 pricing page renders its storage, request and data-transfer rates client-side, so none of them can be read as published text [45]. What AWS does state in a static page is a worked tiering example putting the first 50TB per month of S3 Standard at $0.023 per GB [46].

This site is not repeating the widely-quoted per-GB egress figure, because it could not be traced to any AWS-published page — only to third-party blogs. Budget for the pricing calculator rather than a price list.

Where S3 wins outright: S3 Object Lock. It is the only object-store immutability feature with a published third-party regulatory assessment — assessed by Cohasset Associates for SEC 17a-4, CFTC and FINRA environments [47].

Compliance mode is genuinely immutable, and AWS states the consequence starkly: “The only way to delete an object under the compliance mode before its retention date expires is to delete the associated AWS account” [47]. Test in governance mode first. That is not a drill you want to discover in production.

Option D — leave it where it was recorded (do not)

Transport vendors will happily store recordings for you. Twilio’s rate is $0.00167 per GB per day, about $0.05 per GB-month [9] — between two and three times any object store above, compounding every day of a five-year retention period.

Record there, move immediately, keep the lifecycle rule boring.

Feeds into: your audit trail, and whatever your regulator asks for in three years. If it breaks: you discover the gap at exactly the moment you cannot afford one.

India

If this touches Indian regulated identity work, residency is not optional. The RBI requires that “the entire data and recordings of V-CIP shall be stored in a system / systems located in India,” with recordings carrying a date and time stamp and the live GPS co-ordinates of the customer [1]. The Direction separately requires records to be kept for at least five years — a general record-management rule rather than a V-CIP-specific one, but it governs these recordings all the same [1]. That requirement outranks every cost comparison in Step 6 — pick the region first, then optimise inside it.

What it costs

A worked example, using only rates each company publishes, for one thousand calls of five minutes — 5,000 call-minutes a month. Note the transport row: a call has two participants, so per-participant billing counts 10,000 minutes, not 5,000. That is exactly the trap Step 1 warns about.

LayerChoicePublished rateMonthly
TransportDaily, video$0.004/participant-min~$40
Speech inDeepgram Nova-3$0.0077/min regular~$39
Model turnGemini 3.8 Live~$0.023/min in+out~$115
Speech outElevenLabs Flash$0.05/1,000 charssee note
The faceSynthesia Interactive$0.12/min~$600
StorageGCS Mumbai$0.024/GB-monthsee note
Note

Two of those cells are deliberately not filled in, and that is the honest answer. Text-to-speech is billed per character, not per minute, so it depends entirely on how much your agent talks — and if you chose a speech-to-speech model in Step 3, this line disappears into the model cost instead. Storage depends on bitrate and retention: five-year retention makes the monthly figure climb every month for five years, then plateau. Model both before committing.

The shape to take away: the face is the most expensive layer by a wide margin, usually several times everything else combined. Voice-only is not a lesser product — it is a different price bracket. Build and ship voice-only first; add the face when you can point to what it earns.

The second-biggest lever is Step 3. Gemini Live’s published per-minute rate is dramatically below OpenAI Realtime’s token pricing for conversational use — but read the two-minute audio-plus-video session cap before you build on it.

AdvancedShip it. Failure modes, thresholds and evidence.

Three versions you could build

The two-week version

Voice only, no face. LiveKit or Daily for transport, Gemini Live for the whole middle, nothing recorded beyond a transcript. You will learn more from two weeks of real calls than two months of architecture diagrams, and you will find out whether anyone actually wants to talk to it.

What you are deliberately not building: the avatar, the audit trail, and the retention policy. Do not put this in front of a customer whose identity you are verifying.

The proper version — start here if you are serious

Chained, not speech-to-speech — for the transcript-alignment reason in Step 3. Deepgram for speech-in with Indian language coverage, your own text model so you can run a policy check before anything is spoken, ElevenLabs Flash for speech-out, one streaming avatar vendor, recordings written straight to object storage in-region with a lifecycle rule and immutability switched on.

Budget the latency work properly. The engineering is not in connecting the six steps — a week gets you a working loop. It is in the difference between p50 and p90, which is where the product either feels alive or feels broken.

The enterprise version

Everything above, plus self-hosted transport for residency, immutable storage with a documented retention schedule, per-turn logging of what the model was sent and what it said, a human escalation path that triggers on low confidence or on the customer asking for one, and a written answer to the question of what happens when the model says something wrong on a recorded call that you must keep for five years.

That last one is a policy document, not a feature. Write it before launch, not after the incident.

What goes wrong

The regulatory hole, first, because it is the big one

India’s V-CIP rules require an authorised official of the regulated entity to conduct the session, and require that the official’s credentials be preserved in the activity log [1]. An AI has no credentials to preserve. This site found no published RBI guidance on AI-conducted V-CIP in either direction — it is not expressly forbidden, and it is plainly not contemplated.

Treat that as an open regulatory question, not a loophole. Build the AI to assist the officer — pre-filling, translating, prompting, checking — rather than to be the officer, until somebody in authority says otherwise in writing.

Disclosure is not optional, and may be contractual

ElevenLabs’ own use policy already requires organisations to “clearly and prominently disclose to their users they are interacting with AI rather than a human” [30]. If you are serving European users, the EU AI Act adds its own transparency duty on top. Assume you must say so, plainly, at the start of the call.

The p90 problem

Every vendor markets its best case. Deepgram publishes both, and the distance between the 100ms floor of its published end-of-turn range and a p90 of one second is the distance between the demo and the product. Instrument percentiles from day one; a mean will hide the calls that lost you the customer.

Session caps and concurrency, discovered late

Gemini Live caps audio-plus-video sessions at two minutes [24]. HeyGen caps session length by tier and contradicts itself on concurrency between its own two pages. Cartesia publishes concurrency limits as low as two simultaneous requests. None of these appear in a latency benchmark or a price comparison, and all of them will stop your product working on the day it gets busy.

The cost surprise

Conversational context is re-sent every turn, so a long call costs more than its length suggests. OpenAI’s eighty-fold discount for cached audio input [18] exists precisely because of this. Anyone modelling cost from the headline rate will be wrong in both directions.

And what you cannot read from the call

Once you have a video stream of a person’s face, vendors will offer to tell you how they feel, whether they are stressed, and whether they are lying. That is a separate question with a much worse evidence base and live legal exposure — it has its own page.

Where to go next

Read next:

Sources

Every figure on this page traces to one of these. Prices are shown only where the company publishes them on its own site; where a vendor does not, this page says so rather than repeating a number from elsewhere.

  1. officialRBI Master Direction on KYC (V-CIP provisions) — the authorised-official requirement, liveness and face-match components, India data-residency, geo-tagging, activity-log credentials and five-year retention. https://www.rbi.org.in/Scripts/BS_ViewMasDirections.aspx?id=11566
  2. officialW3C / IETF — WebRTC — WebRTC as a joint W3C and IETF standard, a W3C Recommendation since 26 January 2021. https://www.w3.org/TR/webrtc/
  3. vendorLiveKit pricing — plan tiers, agent-minute and WebRTC-minute rates. https://livekit.com/pricing
  4. vendorLiveKit self-hosting documentation — the single-home ~3,000-user cap, absence of an SLA when self-hosted, and the 99.99% cloud target. https://docs.livekit.io/transport/self-hosting/
  5. vendorDaily WebRTC infrastructure pricing — per-participant-minute video and audio rates, recording rates, free tier. https://www.daily.co/pricing/webrtc-infrastructure/
  6. vendorDaily Pipecat Cloud pricing — agent-hosting rates and transport rates. https://www.daily.co/pricing/pipecat-cloud/
  7. vendorAgora video calling pricing — per-1,000-minute rates by resolution, free-tier suspension behaviour and the Cloud Proxy base fee. https://www.agora.io/en/pricing/video-calling/
  8. vendorAgora Conversational AI Engine pricing — the $0.10/minute bundled rate. https://docs.agora.io/en/conversational-ai/overview/pricing
  9. vendorTwilio Video pricing — participant-minute, recording, composition and per-day storage rates. https://www.twilio.com/en-us/video/pricing
  10. vendorTwilio changelog — the reversal of the Programmable Video end-of-life decision. https://www.twilio.com/en-us/changelog/-twilio-video-will-remain-a-standalone-product
  11. vendorDeepgram pricing — streaming and batch per-minute rates for Nova-3 and Flux. https://deepgram.com/pricing
  12. vendorDeepgram models and languages — Hindi, Indian English and nine further Indian languages in streaming. https://developers.deepgram.com/docs/models-languages-overview
  13. vendorDeepgram — measuring streaming latency — the separation of network transit from server-side processing, total observed transcript latency, and the instruction to track percentiles. https://developers.deepgram.com/docs/measuring-streaming-latency
  14. vendorDeepgram — introducing Flux — fused turn detection, p90 and p95 end-of-turn figures, eager end-of-turn. https://deepgram.com/learn/introducing-flux-conversational-speech-recognition
  15. vendorAssemblyAI pricing — Universal-Streaming and Universal-3.5 Pro hourly rates. https://www.assemblyai.com/pricing
  16. vendorAssemblyAI streaming language support — Hindi available only on the Universal-3.5 Pro streaming tier. https://www.assemblyai.com/docs/faq/language-support-for-real-time-transcription
  17. vendorAssemblyAI Universal-Streaming product page — the 300ms latency headline, stated without a defined metric or statement of exclusions. https://www.assemblyai.com/universal-streaming
  18. vendorOpenAI API pricing — transcription and Realtime API rates, including the cached-audio-input discount. https://developers.openai.com/api/docs/pricing
  19. vendorOpenAI realtime transcription guide — streaming transcript deltas and the commit model. https://developers.openai.com/api/docs/guides/realtime-transcription
  20. vendorOpenAI voice agents guide — the documented comparison of speech-to-speech and chained architectures, and when each is recommended. https://developers.openai.com/api/docs/guides/voice-agents
  21. vendorOpenAI realtime conversations guide — the statement that the realtime model cannot precisely align transcript and audio. https://developers.openai.com/api/docs/guides/realtime-conversations
  22. vendorOpenAI — introducing gpt-realtime — the single-model architecture claim. https://openai.com/index/introducing-gpt-realtime/
  23. vendorGoogle Gemini API pricing — Live API token and per-minute audio rates. https://ai.google.dev/gemini-api/docs/pricing
  24. vendorGoogle Gemini Live API capabilities — the 15-minute audio and two-minute audio-plus-video session limits. https://ai.google.dev/gemini-api/docs/live-api/capabilities
  25. vendorGoogle Cloud Speech-to-Text pricing — tiered per-minute rates. https://cloud.google.com/speech-to-text/pricing
  26. vendorGoogle Cloud Text-to-Speech pricing — per-million-character rates by voice type and the SSML character-counting rule. https://cloud.google.com/text-to-speech/pricing
  27. vendorMicrosoft Azure Speech pricing — free-tier allowances and billing units; per-unit rates are not rendered without sign-in. https://azure.microsoft.com/en-us/pricing/details/speech/
  28. vendorElevenLabs models — Flash v2.5 and v3 Conversational latency figures with the stated exclusion of application and network latency. https://elevenlabs.io/docs/overview/models
  29. vendorElevenLabs API pricing — per-1,000-character rates. https://elevenlabs.io/pricing/api
  30. vendorElevenLabs use policy — the AI-disclosure requirement and the voice-cloning consent terms. https://elevenlabs.io/use-policy
  31. vendorCartesia Sonic 3.5 documentation — the sub-90ms latency claim and language coverage. https://docs.cartesia.ai/build-with-cartesia/tts-models/sonic-3-5
  32. vendorCartesia pricing — credit tiers, published minute equivalents and concurrency limits. https://www.cartesia.ai/pricing
  33. vendorPlayAI model reference — time-to-first-audio figures for Play 3.0 Mini and PlayDialog. https://docs.play.ht/reference/models
  34. vendorHeyGen LiveAvatar — the median-time-to-first-frame claim. https://www.liveavatar.com/
  35. vendorHeyGen LiveAvatar pricing help article — credit mechanics, plan tiers, session-length and concurrency figures. https://help.heygen.com/en/articles/12758516-introducing-liveavatar
  36. vendorTavus — conversational video interface launch post — the sub-one-second utterance-to-utterance claim. https://www.tavus.io/post/introducing-the-worlds-fastest-conversational-video-interface-for-developers
  37. vendorTavus CVI documentation — the six-stage streaming architecture and the definition of utterance-to-utterance latency. https://docs.tavus.io/sections/conversational-video-interface/overview-cvi
  38. vendorTavus pricing — plan tiers, per-minute overage, recording rate and the 30-second minimum charge. https://www.tavus.io/pricing
  39. vendorSynthesia Interactive Avatars — the LiveKit-participant architecture and the published per-minute rate. https://www.synthesia.io/features/avatars/interactive-avatars
  40. vendorSimli — the sub-300ms speech-to-video figure and the published whole-stack latency breakdown. https://www.simli.com/
  41. vendorD-ID agent pricing help article — per-message credit consumption for 15-second video generations. https://help.d-id.com/hc/en-us/articles/31262931985169-How-does-Agent-pricing-work
  42. researchLevinson & Torreira (2015), Frontiers in Psychology 6:731 — the ~200ms modal turn gap, the cross-language distribution, the 600ms threshold for dispreferred-response inference, and human language-production times. https://www.frontiersin.org/articles/10.3389/fpsyg.2015.00731/full
  43. vendorCloudflare R2 pricing — per-GB-month storage, operation rates and free egress. https://developers.cloudflare.com/r2/pricing/
  44. vendorGoogle Cloud Storage pricing — regional storage rates including asia-south1, operation rates and egress tiers. https://cloud.google.com/storage/pricing
  45. vendorAmazon S3 pricing — request rates and the published starting storage rate; per-GB egress tiers do not render on the page. https://aws.amazon.com/s3/pricing/
  46. vendorAWS pricing calculator assumptions — worked volume-tier storage rates. https://aws.amazon.com/calculator/calculator-assumptions/
  47. officialAWS S3 Object Lock documentation — the Cohasset Associates assessment for SEC 17a-4, CFTC and FINRA, and compliance-mode immutability. https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html

Checked September 2026. Prices and model availability in this market change monthly; the date is part of the claim.