A step-by-step guide to turning a recorded call into a written record you can search, prove and produce to an auditor years later. Ten steps, each opening in plain words, with the real alternatives at each one — licences, published accuracy, and what the Indian rules require.
This page walks you through one product: taking a recorded call — a video KYC session, a collections call, a support call — and turning it into a written record you can search, prove and hand to an auditor three years later.
How to read it. Every step opens with a short story in plain words: what is actually happening, and why anyone needs this step at all. Read those ten stories top to bottom and you will understand the whole product in about ten minutes, without knowing a single tool name.
Underneath each story are the technical details, and below those, the ways of doing it folded away behind a +. Open one only when you are building that step. Everything collapsed is a choice; everything already open is the workflow.
Prices only where the company publishes them. Licences stated for every open-source tool, because “free” and “usable by a regulated business” are not the same thing and this page names three popular tools where they come apart.
This guide is the second half of a pair. Guide 20 ends by telling you to record the call and keep it for five years. This guide is what you do with that recording. If you have not read that one, you do not need to — but its Step 6 is where this page begins.
You have an audio file. A person is talking in it. You need six things from it that the audio alone cannot give you:
Ten steps. Each one produces something the next one needs.
Read this table once and the rest of the page is detail.
| # | Step | In plain words |
|---|---|---|
| 1 | Ingest | Open the file, check what it actually is, convert it to one standard shape, fingerprint it. |
| 2 | Find the speech | Work out which parts are talking and which are silence. Not an optimisation — a safety control. |
| 3 | Transcribe | Turn speech into words. The step everyone thinks is the whole product. |
| 4 | Align | Pin each word to the exact millisecond it was spoken. |
| 5 | Separate the speakers | Work out who said what — or avoid needing to, which is usually the right answer. |
| 6 | Punctuate | Add the full stops and capitals, if your transcriber didn't. |
| 7 | Rewrite the numbers | "two lakh fifty thousand" becomes 2,50,000. This is what makes search work. |
| 8 | Flag the sensitive bits | Find names, account numbers, Aadhaar. Know why this will partly fail. |
| 9 | Assemble the record | Put it all in one structured document with everything an auditor will ask for. |
| 10 | Store and index it | Somewhere searchable, in India, tamper-evident, for as long as the rules say. |
Steps 4, 5 and 6 may not apply to you. If you use one of OpenAI’s own Whisper checkpoints, step 6 is already done — it writes punctuation itself, though check the output if you are running an Indian-language fine-tune. If you record each person on their own audio channel, step 5 disappears entirely, and that is the single best engineering decision on this page. Each step says plainly when you can skip it. A shorter pipeline is a better pipeline.
A file lands in your system. You did not make it, and you do not actually know what is inside it.
It might be a video call from a browser, or a phone call recorded by a telecom system, or something an agent uploaded from a laptop. The sound inside could be in any of a dozen formats, at different qualities, with the customer and the agent mixed together or kept apart. Some files have video attached that you do not need.
Every later step expects one exact shape of sound. So the first job is to look inside the file, convert what you find into that one shape, and take a fingerprint — a short code that proves, years later, that the file you are holding is the file you were given and nobody has quietly changed it.
What it is: the universal tool for reading, converting and inspecting audio and video. Its inspection command, ffprobe, “gathers information from multimedia streams and prints it in human- and machine-readable fashion” [1].
The licence has two modes and the difference is decided when the binary is built, not when you use it. FFmpeg is “licensed under the GNU Lesser General Public License (LGPL) version 2.1 or later”, but “incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 or later. If those parts get used the GPL applies to all of FFmpeg” [2].
Building with --enable-gpl — required by libx264, libx265 and others — makes the whole thing GPL. Building with --enable-nonfree produces a binary that is, in the project’s own words, unredistributable [3].
What this means for you: an audio-only pipeline needs none of the GPL-only libraries, so a stock LGPL build is enough. But most Linux distributions ship a GPL build. If you ship a container image to anyone, run ffmpeg -version and read what your base image actually compiled in. This site did not verify how the common Debian, Ubuntu or Alpine packages are built — check yours.
Look inside first: ffprobe -v error -show_format -show_streams -of json input.webm [1].
Convert to the standard shape: ffmpeg -i input.webm -ac 1 -ar 16000 -c:a pcm_s16le output.wav — -ar sets the sampling frequency, -ac the number of channels [4].
Strip the video: add -vn, which “disables video recording” [4].
Now the trap. With no -map, FFmpeg picks “the stream with the most channels” [4]. On a call recorded with the agent on one track and the customer on another, that default silently selects one of them and discards the other — with no error. Always name the track: -map 0:a:0 and -map 0:a:1.
Browser calls: WebRTC requires Opus and G.711 as mandatory codecs [5], and browser recording typically produces Opus in a WebM container at 48kHz [6].
Phone calls: usually G.711 at 8kHz. These are two genuinely different quality regimes and you should not treat them as one input.
Google’s own guidance, which contradicts the usual advice: “If possible, set the sampling rate of the audio source to 16000 Hz. Otherwise, set the sample_rate_hertz to match the original sample rate of the audio source (instead of re-sampling)” [7].
Converting 8kHz phone audio up to 16kHz adds no information — everything above 4kHz was gone at capture. You do it only because a model demands that input, never to improve accuracy. Expect worse results on phone calls and do not be surprised by them.
And convert only once. A published study of Opus compression found relative word-error degradation of 202% at 8kbps, 12.6% at 16kbps and 2.4% at 128kbps [8] — on far-field English with a proprietary dataset, so treat the shape of that curve as the lesson, not the numbers. Google’s own advice is to “use a lossless codec to record and transmit audio” [7].
Hash the original file bytes. This is what an auditor wants: proof the artefact you hold is the artefact you received.
Also hash the decoded audio. FFmpeg has a built-in hash output that “computes and prints a cryptographic hash of all the input audio and video frames”, supporting MD5, SHA160, SHA256, SHA512, CRC32 and adler32 [9].
Why both? A file hash changes if anyone remuxes the container, even when the sound is bit-identical — metadata, timestamps and internal ordering all differ. So a file hash answers “is this the same upload” and cannot answer “is this the same call”. The decoded hash answers the second.
Two practical warnings. Pass -hash sha256 explicitly; this site could not verify what the default is. And store the exact command and FFmpeg version alongside the hash, because changing the sample rate or the resampler changes the hash for the same audio.
Feeds into: everything. If it breaks: quietly, and you find out much later — a dropped second channel or a double re-encode produces a working pipeline with a worse answer, not an error.
Most of a real call is not talking.
It is ringing, hold music, the customer walking to another room, the long pause while somebody finds their card. If you hand the whole recording to a transcription model, it will dutifully try to transcribe the silence too.
And here is the part that should worry you: it does not return nothing. It makes something up. Fed silence and background noise, these models have a strong documented tendency to write plausible-sounding sentences that nobody said — most often "thank you" or "thanks for watching", learned from the internet videos they were trained on.
So this step finds the parts where someone is actually speaking and passes on only those. It makes the pipeline faster and cheaper, but that is not why it is here. It is here to stop invented sentences entering a financial record you must keep for five years.
On 301,317 non-speech audio files fed to Whisper large-v3, “hallucinations appeared 121,378 times (40.3%)”. They are strikingly repetitive — the top 30 account for 67% of all output, led by “thank you” at 24.76% and “thanks for watching” at 10.32%. The authors’ recommended mitigation is a filter combined with voice activity detection [10].
In a peer-reviewed study of real transcripts, roughly 1% contained “entire hallucinated phrases or sentences which did not exist in any form in the underlying audio”, and 38% of those contained explicit harms — violence, fabricated personal associations, invented claims of authority. Critically, speakers with longer non-vocal pauses were hallucinated against at disproportionately higher rates [11].
That last finding is the mechanism. Silence induces hallucination. A collections call with long dead air is exactly the worst case.
And it improves accuracy, not just safety: WhisperX reports word error falling from 10.52% to 9.70% with voice-detection preprocessing, with a 2.7× speed-up at batch size 1 rising to 11.8× batched [12].
Licence: MIT, not gated [13]. The cleanest licence position of the three — no account, no terms to accept, no contact details surrendered.
Size and speed: the model is around two megabytes; the project states one 30ms chunk takes under 1ms on a single CPU thread, with published measurements from 189µs to 830µs per chunk depending on version and runtime [14].
Handles 8kHz natively as well as 16kHz — which matters, because that is what phone calls are.
Published accuracy on a 17-hour multi-domain set: ROC-AUC 0.97 and accuracy 0.92, against WebRTC’s 0.73 and 0.74 [15]. Caveat stated honestly: this is Silero benchmarking its own model against a competitor. The datasets are named and public, which is more than most vendors offer, but it is not independent, and none of them is Indian-language telephony.
Licence: BSD 3-Clause for the underlying code, MIT for the common Python wrapper [16]. Permissive, ungated.
How it works: not a neural network. It is a Gaussian Mixture Model comparing log energy across six frequency bands and running a likelihood-ratio test between “noise” and “speech” [17]. Fast, tiny, and old.
The failure that matters: in the comparison above it scored 0.00 accuracy on environmental-noise-only audio [15] — it classifies most background noise as speech. On a noisy Indian collections call with traffic, a television and background conversation, it will not remove any of it.
Note also that Google publishes no accuracy claim for it at all. Everything you will read about its quality is third-party.
Licence: MIT for the toolkit; the segmentation model is MIT but gated — you must share contact information and accept conditions before downloading [18].
The gate is not a licence restriction, it is a contact-information wall, and MIT permits commercial use. But it has an operational consequence: your build pipeline now depends on a third party’s consent flow and a Hugging Face token. Mirror the weights into your own artifact store once accepted, and record the acceptance.
This site found no published standalone accuracy figure for pyannote as a speech detector — the project publishes diarization error rates, which is a different measurement. If you are not already using pyannote for Step 5, there is no evidence-backed reason to choose it here.
Feeds into: Step 3, which should only ever see speech. If it breaks: invented sentences enter an auditable record, and nothing downstream can tell they were invented.
This is the step everyone thinks is the whole product: sound goes in, words come out.
It is also where most of your accuracy is won or lost, and where the advice you will find online is most likely to be wrong for you. Almost every recommendation written in English assumes the call is in English, recorded on a decent microphone, with one person speaking clearly at a time.
Your calls are not that. They are in Hindi, or Hindi mixed with English, over a phone line, from someone in a market or on a bus, with an accent from one of twenty-odd language backgrounds. The gap between those two situations is not small — on published benchmarks it is the difference between getting about 3 words wrong in 100 and getting 32 wrong in 100.
So this step is mostly about choosing a model that has actually been measured on speech like yours.
Licence: MIT for the fine-tuned models, stated in the project repository [19]. Built by AI4Bharat and IIT Madras, published at Interspeech 2023, covering 12 Indian languages. Hindi fine-tuning used 2,150 hours of Indian speech.
Published Hindi word error rate, averaged across seven benchmarks: 13.6%, against Google’s 23.9% and Azure’s 20.0% [20]. Across all 12 languages: 24.6 against Google’s 44.3.
Now look at the telephone column, because it is the honest one. On Gramvaani — spontaneous, telephone-quality, rural Hindi — every system roughly doubles or triples its error. Google goes from 19.4 to 59.9. IndicWhisper goes from 11.4 to 26.8 [20]. It is still the best of the group by a wide margin, and it is still getting one word in four wrong. Plan for that number, not the headline.
Also from AI4Bharat: a 600-million-parameter Conformer covering all 22 scheduled Indian languages, MIT licensed but gated on Hugging Face, with published Hindi accuracy of 13.2% word error on an independent benchmark [21].
Licence: permissive, with a discrepancy worth knowing. The repository says “Whisper’s code and model weights are released under the MIT License” [22], while the model card for large-v3 states Apache 2.0 [23]. Both permit commercial use; your compliance team will still ask, so have the answer ready.
What hardware you need, from the project’s own table: large needs about 10GB of video memory, turbo about 6GB, medium about 5GB, small about 2GB [22].
But use faster-whisper instead of the original. Also MIT [24], it is “a reimplementation of OpenAI’s Whisper model… up to 4 times faster… while using less memory”, and its published benchmark runs large-v2 in 2,926MB at int8 precision — which puts the largest model on an 8GB consumer card. It ships Silero VAD integration, covering Step 2 for you.
On Hindi: OpenAI publishes per-language accuracy only as an image, not as numbers. Independent measurement puts Whisper V3 at 32.4% word error on accented Hindi [25] — against the 2.7% its own paper reports on clean read English [55]. That order-of-magnitude gap is the single most useful fact on this page.
All rates below are pre-recorded/batch, read from the vendors’ own pricing pages in September 2026.
Google: dynamic batch $0.003/min; standard on-demand tiers from $0.016 down to $0.004 [26]. Supports Hindi plus nine other Indian languages. Each channel is billed separately.
AssemblyAI: Universal-2 at $0.15/hour ($0.0025/min), Universal-3.5 Pro at $0.21/hour; speaker separation is a separately priced add-on at +$0.02/hour [27]. Hindi plus nine Indian languages.
Deepgram: Nova-3 monolingual $0.0043/min, multilingual $0.0052 [28]. Hindi plus nine Indian languages, and speaker separation is included for pre-recorded audio.
AWS Transcribe: $0.006/min batch [29]. Supports Hindi — but note that content redaction is not supported for Hindi [30], which matters for Step 8.
Azure: this site could not verify a single rate. The pricing page renders every figure as a placeholder and requires sign-in [31]. The billing units are published; the prices are not.
NVIDIA’s open speech models, Parakeet and Canary, are excellent, cleanly licensed under CC-BY-4.0, ungated, and fast. They are also the models most confidently recommended in every technical blog post about self-hosted transcription.
They support zero Indian languages. Parakeet v3 and Canary-1b-v2 each cover 25 languages, all European [32][33]. Parakeet v2 is English only.
If your calls are in Hindi or any Indian language, the most-recommended open option on the internet is simply not available to you, and no amount of tuning changes that. Cross it off and move on.
Feeds into: Steps 4 through 8, all of which operate on this text. If it breaks: every downstream step confidently processes the wrong words — and Step 4 will pin them to precise timestamps, which makes them look more credible, not less.
Your transcription already came with rough timings. They are usually good enough to follow along, and usually wrong by a few tenths of a second.
That matters more than it sounds. If a dispute turns on whether the customer said "yes" before or after the agent explained the charges, a third of a second is the whole argument. And if you want a reviewer to click a word and hear it, the timings have to actually land on the word.
Alignment fixes this by going back over the audio with the words already known. It is a much easier problem than transcription — the model no longer has to work out what was said, only when.
But there is a trap here that you should understand before you build it, and it is the reverse of what you would expect. Alignment makes a transcript look more trustworthy without making it more correct. If Step 3 heard the wrong word, Step 4 will pin that wrong word to a precise millisecond, and it will look authoritative.
Licence: Apache 2.0 [34] — the cleanest of the group, and notably the toolkit licence genuinely applies here, unlike NVIDIA’s punctuation models in Step 6.
No pronunciation dictionary required, works with CTC models, handles hour-long files, supports 14+ languages, outputs word and segment-level timings [35].
Published accuracy, and it contains the number that should shape your expectations: on the AMI test set at a 200ms threshold, 98.35% precision and recall using a correct reference transcript — but 85.65% precision and 71.09% recall when fed a machine transcript [36].
Read that gap. Alignment cannot repair Step 3. It can only place whatever words it is given.
Licence: BSD 2-Clause [37].
What it does: replaces Whisper’s generated timings with measured ones, by running a phoneme-level model over the audio and applying dynamic time warping. Published word-segmentation precision and recall of 84.1/60.3 on meeting audio and 93.2/65.4 on telephone speech [12].
Note that recall of 65% figure. Even at its best, on telephone audio, roughly a third of words are not placed within 200ms of where they belong.
On languages: five languages use built-in alignment models — English, French, German, Spanish and Italian — and a further set, including Hindi, have default models pulled from Hugging Face automatically [37]. So Hindi word-level timings work without configuration, which is worth knowing given how much else on this page does not cover Indian languages.
Its own documented limits are worth quoting: words containing characters outside the alignment dictionary cannot be timed, and “overlapping speech is not handled particularly well” [37].
Licence: MIT [38]. Well-established, built on Kaldi, and measured as “comparable to human annotators on average” with mean word boundary error of 24ms on conversational speech [39].
But it needs a pronunciation dictionary as well as an acoustic model, which is the practical barrier — the other options here need neither.
And for Indian languages the models mostly do not exist. Among South Asian languages only Tamil has a published pretrained acoustic model; there is no Hindi model [40].
It also fails hard rather than degrading. In an independent evaluation it scored respectably on the files it processed — but only 5 of 16 files aligned at all [36].
One serious warning if you go looking for an Indian-language alternative. AI4Bharat publishes IndicMFA with models for Indian languages, but no licence is stated on the repository [41]. Under standard terms, code published without a licence is all-rights-reserved by default. Do not deploy it in a regulated business without written clarification.
There is one peer-reviewed study of using this technology for forensic transcription, and it is a warning rather than an endorsement.
On poor-quality audio, the researchers found the forced aligner incorrectly matched transcripts to non-speech sounds including drumming and laughter. Their conclusion is blunt: “computational methods are not suitable for solving the issue of transcription of indistinct forensic audio for a range of reasons.” They specifically flag that transcripts can be matched to audio through alignment, appearing correct despite being entirely inaccurate, and note that “injustices can, and have, occurred” where questionable transcripts were treated as evidence [42].
This site found no published source establishing forced alignment as a method for producing evidentiary transcripts. The one relevant paper argues the opposite.
So: use alignment to make a transcript navigable and reviewable by a human. Do not present the precision of the timestamps as evidence of the accuracy of the words.
Feeds into: Step 5, which needs word timings to attach speakers, and Step 9. If it breaks: the record still reads correctly but nobody can jump to the moment — annoying, not dangerous. Unlike most steps here, this one fails safely.
A transcript that says "yes, I agree to the charges" is worth very little if you cannot prove who said it.
When both people are recorded onto one mixed track, working out who spoke when is a genuinely hard computing problem with a name — diarization — and it is wrong more often than you would guess. On standard telephone benchmarks across all speaker counts the published error rate runs around 28%. For a two-party call like yours the best published figure is far better, around 6–7%, because two speakers is genuinely the easy end of the problem — but that is still one word in fifteen attributed to the wrong person.
But there is a way to make the entire problem disappear, and it costs nothing. If you record each person onto their own separate audio track at the time of the call — which your calling system can almost certainly already do — then you never have to guess. Track one is the agent. Track two is the customer. It is correct by construction, permanently, with no model involved.
This is the single best decision available on this page, and it is made in Step 1, not here.
Every major vendor treats per-channel audio as the superior path, and several say so outright.
AssemblyAI: “Multichannel is more accurate since each speaker’s audio is processed independently” [43]. AssemblyAI states separately that the two features cannot be enabled together and that doing so returns an error [44] — so this is a choice between them, not a combination.
Azure: “Diarization is only supported on single-channel (mono) audio” [45]. Stereo and diarization are alternatives by design.
Deepgram returns a separate transcript per channel, noting that diarization only becomes necessary when multiple speakers share a channel [46].
AWS transcribes each channel separately with channel labels and handles the case where both talk at once, to a maximum of two channels [47]. AWS states elsewhere that channel identification is available at no additional cost [29], but this site could not find a billing sentence on the channel-identification page itself — confirm the charging model for your region before assuming stereo is free.
Google supports up to eight channels with a channel tag per result — but bills the sum of all channels, so stereo costs double [48].
Two honest caveats: it only works if each channel genuinely holds one party, and AWS caps at two channels. For a KYC or support call where you control the recording stack, neither is a problem.
Licence: MIT for the toolkit; the pipeline models are MIT or CC-BY-4.0, all gated — contact details required [49][50]. Commercial use is permitted by both licences.
Published accuracy — and note which benchmark you should care about. The flattering numbers are broadcast and meeting audio: 7.9% on REPERE, 11.2% on VoxConverse. On CALLHOME, which is actual telephone conversation, it is 28.5% for the open 3.1 model and 26.7% for the community model [50].
The paid pipeline is materially better on telephone audio (16.6%), which tells you something about where the open model sits.
No published accuracy exists for Hindi or any Indian language, from this or any other diarization system this site could find. If your calls are in Indian languages, you are operating without a benchmark.
Read this before copying any tutorial. There are three Sortformer models with three different licences:
diar_sortformer_4spk-v1 — CC-BY-NC-4.0. Non-commercial. Unusable by a business. [51]diar_streaming_sortformer_4spk-v2 — CC-BY-4.0, commercial use permitted with attribution [52]diar_streaming_sortformer_4spk-v2.1 — NVIDIA Open Model License [53]Version 1 is the one most tutorials and blog posts reference, and it is the one you may not use. That is the trap.
Published accuracy shows how much speaker count dominates: on CALLHOME, 6.57% error with 2 speakers rising to 28.74% with 6 [52]. Maximum 4 speakers by design, and NVIDIA states performance “may degrade on non-English speech”.
Telephone audio is the hard case, and it is your case: 28.5% error on telephone benchmarks against 7.9% on broadcast [50].
Speaker count is the dominant variable. 2 speakers 6.57%, 6 speakers 28.74% [52]. Your two-party call is genuinely the easy end.
Boundaries are systematically clipped. An independent benchmark across 196 hours and five systems found the dominant failure is missed speech at segment edges, with roughly 350ms average missed duration across all models [54] — every speaker turn trimmed by about a third of a second.
Overlapping speech breaks everything. Every source says so; WhisperX states plainly that “overlapping speech is not handled particularly well” [37]. People interrupt each other constantly on collections calls.
And none of the cloud vendors publishes an error rate at all — their accuracy claims cannot be checked against any stated benchmark.
Feeds into: Step 9, where speaker attribution becomes a field in the record. If it breaks: you attribute a commitment to the wrong person — the single most damaging error this pipeline can make, and the reason separate channels are worth the plumbing.
Some transcription systems hand you a bare stream of lowercase words with no full stops. It is exhausting to read and hard to search.
Restoring the capitals and punctuation is its own small prediction problem: the model has to work out, from the words alone, where the sentences end and which words are names.
The good news is that you may have nothing to do here. Whisper was deliberately trained to produce written-form text, punctuation and capitals included, so a Whisper-based pipeline gets this free. Most cloud services include it too — though two of them have it switched off by default, which is the most common reason a transcript comes back looking unreadable.
Read this step only if your transcriber does not punctuate. Otherwise skip to Step 7.
OpenAI’s Whisper checkpoints do this natively. From the paper: models are trained “to predict the raw text of transcripts without any significant standardization”, and the training data was deliberately filtered to remove transcripts that had normalised away “complex punctuation… or stylistic aspects such as capitalization” [55].
The cloud services, and their defaults:
punctuate=true [58]None of them charges extra for it.
One caveat that applies to this page’s own recommendation. The native-punctuation property belongs to OpenAI’s released checkpoints. This site found nothing establishing that the Indian-language fine-tunes recommended in Step 3 preserve it — fine-tuning on unpunctuated Indian speech corpora is exactly what would remove it. Transcribe one call and look at the output before deciding you can skip this step.
Licence: MIT [60]. Published at scale, covering 11 Indic languages, restoring sentence ends, commas and question marks.
Published accuracy: Hindi F1 of 0.81, with Marathi and Punjabi at 0.86, Gujarati 0.85, Tamil 0.77 [61].
This is the clean choice for a regulated business — genuinely MIT, no derivative obligations, no gate.
AI4Bharat’s Cadence covers English plus all 22 scheduled Indian languages with around 30 punctuation labels including the Devanagari danda, and outperforms the alternatives: Hindi 0.82 on written text [62].
Its model card says MIT. Treat that with care. Cadence is built on Google’s Gemma, which makes it a model derivative under the Gemma Terms of Use — and a downstream publisher cannot licence away Google’s terms. Gemma does permit commercial use, but it attaches obligations MIT does not: you must pass the terms through to recipients, include a specific notice, mark modified files, and enforce Google’s prohibited-use policy contractually [63].
So it is usable, but it is not MIT in substance, and a compliance review will find that.
And if you choose indic-punct for licence cleanliness, know what you are giving up. The two Hindi figures above come from different papers and different test sets, so they are not comparable. The one direct head-to-head, in Cadence’s own paper, puts Cadence at 0.76 against indic-punct at 0.54 on a shared benchmark [62] — a gap of roughly twenty points, not one.
One number worth carrying: Cadence scores 0.79 on written text but 0.62 on actual speech transcripts [62]. Punctuation restoration is measurably harder on the thing you actually want to use it for.
deepmultilingualpunctuation: MIT licensed, English/German/French/Italian only, no Indian language support [64].
Look at the shape of its published results rather than the average. English F1 by mark: full stops 0.948, question marks 0.890, commas 0.819, colons 0.575, hyphens 0.425 [64]. The macro average of 0.775 hides a five-fold spread — full stops are close to solved, while hyphens land below a coin flip and colons are not far above one.
It was also evaluated on European Parliament speeches — prepared political oratory. An Indian collections call is a different distribution and those numbers will not transfer.
NVIDIA NeMo’s punctuation model predicts only commas, periods and question marks [65], with a published 77% F1 on an internal, unpublished dataset and no per-mark breakdown [66]. And its licence is a trap: the NeMo framework is Apache 2.0, but the pretrained weights are governed by NVIDIA’s NGC Terms of Use, not the framework licence [66]. Read the model page, not the repository.
Feeds into: Step 7, which needs sentence structure to spot numbers and dates. If it breaks: the transcript is ugly and harder to search, but nothing is factually wrong. Low stakes.
Somebody says, out loud: "two lakh fifty thousand rupees".
Your transcript now contains those words. Six months later a compliance officer searches your archive for 2,50,000 and gets nothing. The call is there. The search cannot see it, because a computer matching text has no idea those two things are the same.
This step rewrites spoken numbers into the written forms people actually search for: amounts, dates, phone numbers, account numbers. It is unglamorous and it is the difference between an archive you can use and an archive you merely have.
For India there is a specific wrinkle worth knowing. We group digits differently — 2,50,000 rather than 250,000 — and we count in lakhs and crores rather than millions. Most tools built elsewhere get this wrong. One published Indian tool gets it right.
Licence: MIT [60]. Covers 12 languages including Hindi, English, Tamil, Bengali, Telugu, Marathi and Gujarati.
Worked examples, taken verbatim from the project’s own documentation [60]:
Look carefully at those commas. 10,01,420 and 4,04,00,000 use Indian grouping — two, two, three from the right. Western grouping would give 1,001,420 and 40,400,000. So this handles both the lakh/crore vocabulary and the correct comma placement, which is exactly what you need and what most tools miss.
One honest caution: the paper dates from 2022 and this site could not confirm recent maintenance activity. For a regulated business that is a supply-chain question worth checking before you depend on it.
Licence: Apache 2.0 [67] — and unlike the NeMo punctuation model in Step 6, this one genuinely is permissive. The distinction matters; do not blur them.
Hindi is supported, for both directions of normalisation, along with Marathi [68]. This contradicts the common assumption that the toolkit is European-only.
Published accuracy: 97.43% sentence accuracy for English on a public benchmark [69]. No Hindi accuracy figure is published.
And a gap you should verify yourself before choosing this: Hindi support exists in the matrix, but this site found no published example, test case or documentation demonstrating that NeMo’s Hindi grammar handles lakh, crore or Indian digit grouping. Do not assume it does. indic-punct is the one with published evidence.
Whisper does this natively. From OpenAI’s paper, training on raw written-form text “removes the need for a separate inverse text normalization step in order to produce naturalistic transcriptions” [55].
Deepgram offers it via a smart_format flag covering numerals, dates, times, currency, phone numbers and email addresses — but the documentation states it has “the broadest support for English-language models” and is primarily for English [70]. No Indian number format support is documented.
AWS Transcribe does number and currency conversion only for languages with that support, and Indian numbering is not mentioned in its documentation [57].
So the free option works for English and leaves Indian formats unhandled.
This site searched for published grammars, test sets or documented examples handling Indian date conventions in any inverse-normalisation toolkit, in either NeMo or indic-punct, and found none. The indic-punct documentation describes its component as focusing on number conversion.
So if your calls contain spoken dates that must be searchable — EMI due dates, appointment dates, dates of birth in a KYC call — expect to write and test that rule yourself, and budget for it rather than discovering it late.
A note on evidence. The argument that unnormalised numbers break search is mechanically obvious, but this site could not find published research measuring it. Treat it as a consequence of how string matching works, not as a cited finding — the distinction matters on a site that cites everything else.
Feeds into: Step 8, which looks for identifier patterns, and Step 10, where search runs. If it breaks: the archive quietly fails to answer the questions you built it to answer.
Somewhere in the call, the customer read out their Aadhaar number. Twelve digits, aloud, to an agent. Those digits are now sitting in your transcript in plain text, in a database, and they will be there for five years.
So you run a detector over the transcript to find identifiers and personal details, and mask them. Reasonable. There are good tools for it, including ones that know about Aadhaar, PAN and voter ID specifically.
Here is the problem, and it is genuinely uncomfortable. These detectors work partly by checking the maths: a real Aadhaar number has a built-in check digit that only fits if every other digit is right. That check is what makes the detector confident rather than flagging every twelve-digit string in your database.
But speech recognition misheard a digit. It usually does. And a number with one digit wrong fails the check — so the detector concludes it is not an Aadhaar number at all, and leaves it exactly where it is. Unmasked. With no error shown to anyone.
The very thing that makes the detector precise on clean text is what makes it fail silently on yours.
Licence: MIT [71]. Works by combining regular-expression patterns, surrounding context words, checksum validation and name recognition.
Six Indian entity types are supported [72]: Aadhaar (pattern, context and checksum), PAN, passport, voter ID, vehicle registration (with checksum) and GSTIN. That covers every identifier you are likely to hear on an Indian fintech call.
Quote Microsoft’s own disclaimer to anyone who thinks this solves the problem: “because it is using automated detection mechanisms, there is no guarantee that Presidio will find all sensitive information. Consequently, additional systems and protections should be employed” [73].
AWS Comprehend: $0.0001 per 100 characters for PII detection, roughly $1 per million characters, with 5 million characters free monthly for the first 12 months [74]. Four Indian types: Aadhaar, PAN, NREGA and voter number [75].
But there is a constraint that probably rules it out for you: Amazon Comprehend detects PII in English or Spanish only [75]. For a Hindi or Hinglish transcript — the normal case in Indian fintech — it does not apply at all.
Google Sensitive Data Protection: free up to 1 GiB monthly, then $3.00 per GiB, falling to $2.00 above 1 TiB [76]. Four Indian types: Aadhaar, PAN, GST and passport [77]. Note the unit — billing per gigabyte makes the free tier very generous for text, since a million-character transcript corpus is about a megabyte.
Azure AI Language: only two Indian types, PAN and Aadhaar [78] — the weakest coverage. And this site could not verify its prices; the page renders placeholders.
Name and entity detection measured on machine transcripts versus clean text: F1 dropped from 0.631 to 0.237 on one dataset and 0.964 to 0.564 on another [79].
And the point that should change how you read vendor claims: the same paper, summarising the prior literature it builds on, notes that transcription performance does not correlate with downstream performance on language tasks [79]. A good transcription accuracy number is not evidence that your PII detection will work.
It is worse than just misheard words. A second study found that entity recognition models “fail spectacularly even if no word errors are introduced by the ASR” [80] — the loss of sentence structure in spontaneous speech is by itself enough to break detection.
On the specific checksum argument in the story above: this site found no published study of checksum-validated identifier detection over machine transcripts. It follows from two things that are documented — Presidio validates the Aadhaar checksum, and a check digit exists precisely to reject single-digit corruption — but it is deduction, not a citation, and this page says so rather than dressing it up.
The Aadhaar (Sharing of Information) Regulations, 2016, Regulation 6(3): no entity holding an Aadhaar number “shall make public any database or record containing the Aadhaar numbers of individuals, unless the Aadhaar numbers have been redacted or blacked out through appropriate means, both in print and electronic form” [81].
Read that precisely, because it is commonly overstated: it is a redaction-before-publication duty, not a blanket ban on holding Aadhaar numbers internally. But Regulation 6(5) does bind retention — entities shall not retain Aadhaar numbers “for longer than is necessary for the purpose specified” [81]. An indefinitely-retained transcript archive is hard to square with that.
The RBI adds a redaction duty in the KYC context: where a customer submits proof of possession of Aadhaar, the regulated entity shall ensure the customer “redacts or blacks out his Aadhaar number… where the authentication of Aadhaar number is not required” [82], repeated specifically for video KYC.
On the checksum itself: Aadhaar is universally implemented as using the Verhoeff algorithm and Presidio validates it as such, but this site found no UIDAI publication stating it. Treat it as de facto rather than officially specified. Note also that a failed check means “not valid or mistranscribed” — your pipeline cannot tell which.
Feeds into: Step 9, where flags become fields, and Step 10, where they drive access control. If it breaks: an unmasked Aadhaar number sits in your archive and nothing in the system knows.
You now have words, times, speakers, tidy numbers and flags on the sensitive parts. They are in several different shapes from several different tools.
This step puts them into one structured document. That sounds like tidying-up. It is actually the step where you decide whether this thing is a record or just a convenient text file.
Here is the question that should design your format. If this transcript is ever produced in a proceeding in India, somebody from your company has to sign a certificate saying what device produced it, how it was produced, and that the system was working properly at the time. That certificate is a legal requirement, and it is only signable if your system wrote those facts down when it made the record.
So do not design the format around what is convenient to display. Design it around what that certificate has to say.
This site checked the four formats people reach for. None is a record format:
And the vendors do not agree either. All four of Deepgram, AssemblyAI, AWS and Google express the same atom — token, start, end, confidence, speaker — and disagree on the name of every field, the type of every field, the unit of time, and whether the speaker lives on the word or in a side table that you must join on timestamps [86][87][88][89].
So writing your own canonical schema plus a per-vendor adapter is not a workaround. It is a design requirement. State it as one.
Under Section 63 of the Bharatiya Sakshya Adhiniyam, 2023, an electronic record is admissible on conditions including that the computer was operating properly, or that any malfunction did not affect the accuracy of the contents — and sub-section (4) requires a certificate identifying the record, describing the manner of its production, and giving particulars of the device [90].
That is your schema requirement. Working backwards from what a truthful certificate needs:
Design the record so the certificate can be signed truthfully. That single sentence is the most useful thing on this page for an Indian builder.
ISO 15489-1:2016 covers records management concepts and metadata “regardless of structure or form” [91] — it tells you a record needs metadata and controls, not what a transcript must contain. It is a paid standard (CHF 155).
ISO/IEC 27037:2012 covers handling of digital evidence [92], not transcript structure.
NIST’s forensic standards registry carries forensic audio standards, but this site found no registry standard governing transcription output [93].
There is even an open issue on the W3C timed-text list about exactly this gap for public-sector meeting transcripts [94]. The gap is acknowledged, not closed.
Feeds into: Step 10. If it breaks: you have a transcript you cannot defend — readable, searchable, and missing the one thing that would let you produce it.
Last step, and the one with the longest tail: where does this live, and for how long?
Three forces pull in different directions. The regulator says keep KYC records for five years, stored in India, retrievable quickly. The privacy law says delete personal data once the purpose is served. And your compliance team needs to search years of calls and be confident they found every one that matters.
The good news is that the law resolves the first two itself — the privacy law has a carve-out for anything you are legally required to keep, and it was written with exactly this banking case in mind.
The hard part is the technology. If you lock everything into storage that nobody can delete, you have solved the regulator and created a privacy problem for every recording that was never legally required. And the strongest immutable storage available has, in the vendor's own words, one way out: delete the entire cloud account.
PostgreSQL full-text search gives you indexed matching, boolean and phrase operators, and ranking [95]. And it ships a Hindi stemmer — inherited from Snowball, registered in the source alongside Nepali and Tamil, UTF-8 only [96]. That contradicts the usual assumption. Bengali is absent.
But read Postgres’s own warning on ranking: “the concept of relevancy is vague and very application-specific… The built-in ranking functions are only examples”, and ranking “can be expensive since it requires consulting the tsvector of each matching document” [97].
OpenSearch is Apache 2.0 under a Linux Foundation project [98] — the least legally complicated option if you must self-host in India. It ships Hindi and Bengali analyzers with Indic Unicode normalisation, which genuinely helps when transcripts come from different vendors. Elasticsearch is now tri-licensed, and its default distribution remains under the Elastic License. Read what that actually restricts, because it is commonly stated backwards: Elastic’s own FAQ says you “may freely use Elasticsearch inside your SaaS or self-managed application, and redistribute it with your application” subject to three limitations [99]. The restriction bites when your customers get direct access to substantial portions of the Elasticsearch APIs or the Kibana interface — that is, when you are effectively reselling Elasticsearch itself rather than building on it.
The catch nobody mentions: none of this handles Hinglish. A customer saying “mera chargeback abhi tak nahi aaya” in Roman script is not Hindi to a Devanagari stemmer and not English to an English one. Plan for trigram and literal-phrase matching alongside language analysis, not instead of thinking about it.
Vector search finds calls where someone said “the money came back” when you searched “chargeback”. Genuinely useful. But not as your system of record, for a reason its own documentation states plainly.
From pgvector: “By default, pgvector performs exact nearest neighbor search, which provides perfect recall”; adding an index “trades some recall for speed”; and — the sentence to remember — “Unlike typical indexes, you will see different results for queries after adding an approximate index” [100].
An index is normally a pure speed optimisation. Here it changes the answers. A system that returned 14 calls last month and 12 this month, with identical data, because someone tuned a search parameter, is not one you want to explain to a regulator. OpenSearch documents the same trade-off [101].
And a sobering benchmark: exhaustive retrieval is a named research problem with a NIST evaluation track behind it, and even the best automated systems reached recall “on the order of 0.95” — not 1.0 [102]. So “prove you found everything” is not achievable by any method. What is achievable is a deterministic, reproducible, documented process — which is why lexical search should be your production path and semantic search a discovery aid on top.
The RBI requires transaction records “for at least five years from the date of transaction”, and identification records “for at least five years after the business relationship is ended” [82].
Those are two different clocks. A call recording that evidences both a transaction and an identity check is governed by whichever expires later. Most implementations apply one policy and get this wrong.
Records may be kept “in hard or soft format” [82] — so the RBI does not mandate immutable storage. That is your evidentiary choice, not a regulatory requirement.
For video KYC specifically, the data and recordings “shall be stored in a system / systems located in India”, with a date and time stamp that “affords easy historical data search”, and the activity log with the official’s credentials preserved [82].
SEBI advisers must keep records five years, and where a dispute is raised, “till resolution of the dispute” or “till further intimation from SEBI” [103]. That open-ended clause is why legal hold must be a separate mechanism from your retention clock — it cannot be expressed as a duration.
AWS S3 Object Lock in compliance mode is assessed by an independent firm as meeting SEC 17a-4(f) and FINRA 4511(c) requirements [104]. It is also absolute: “a protected object version can’t be overwritten or deleted by any user, including the root user”, and “the only way to delete an object under the compliance mode before its retention date expires is to delete the associated AWS account” [105].
Azure immutable blob storage is equivalent in shape and has the same independent assessment; a policy “must be locked” to be compliant, and a locked policy cannot be deleted or shortened [106]. Google Bucket Lock is likewise irreversible once locked [107].
Test in governance mode first. A classification mistake under compliance mode has no remedy short of destroying the account.
Legally, there is no conflict. The DPDP Act’s erasure duty opens with a carve-out: a Data Fiduciary shall erase personal data on withdrawal of consent “unless retention is necessary for compliance with any law for the time being in force” [108], and the same words qualify the erasure right. The Act is “in addition to and not in derogation of” other law.
The legislature saw this case coming — the Act’s own illustration to that section is a banking record-keeping example. A regulated lender declining an erasure request for KYC records inside the five-year window is acting under the statute, not against it.
Operationally the problem is real, and it is at the edges:
The design that resolves it: encrypt each record with its own key held outside the immutable store. Retain the ciphertext to satisfy the regulator; destroy the key when erasure is genuinely due. But state the caveat honestly: this site found no publication from MeitY, the Data Protection Board or the RBI confirming that key destruction constitutes “erasure” under the Act. It is the standard engineering answer to an open legal question, not a settled one.
Feeds into: whatever an auditor asks you for in 2031. If it breaks: you find out at the worst possible moment, which is the entire reason this pipeline exists.
Record each party on a separate channel. Run faster-whisper with its built-in voice detection. Store the JSON in Postgres with a full-text index. That is Steps 1, 2, 3, 9 and 10 — and by recording separate channels you have skipped Step 5, the least reliable step on this page.
What you are deliberately not building: alignment, identifier flagging, and any retention policy. Do not point it at regulated KYC recordings yet.
Separate channels throughout. IndicWhisper or IndicConformer if calls are in Indian languages, faster-whisper otherwise. Voice detection before transcription, always. NeMo Forced Aligner for word timings. indic-punct for Indian number formats. Presidio for identifier flagging — with access control and encryption doing the real work, because Step 8 explains why the detector alone is not a control. Records written with hashes, model versions and settings from the start, so the certificate in Step 9 can be signed. Storage in India, classified per record at ingestion, governance-mode immutability, legal hold as a separate axis.
Budget your time for Steps 8, 9 and 10. Steps 1 through 7 are a week of plumbing that mostly works first time. The last three are where the judgement is, and where a mistake is expensive and slow to surface.
Everything above, plus per-record encryption keys held outside the immutable store, a documented retention basis on every record, reproducible lexical search as the production path with semantic search clearly marked as a discovery aid, and a written answer to the question of what you do when a customer requests erasure of a record you are legally required to keep.
That last one is a policy document, not a feature. Step 10 gives you the legal answer; you still need to write down who decides and how it is recorded.
The single most dangerous failure here, and the least obvious. 40.3% hallucination on non-speech audio; “thank you” and “thanks for watching” accounting for a third of it [10]. Voice detection before transcription is not optional in a regulated pipeline.
Whisper: 2.7% word error on clean read English [55], 32.4% on accented Hindi [25]. Google speech: 19.4% on read Hindi, 59.9% on telephone Hindi [20]. Never accept a vendor accuracy figure without asking which dataset. Measure on a sample of your own calls before committing.
Three live traps on this page. NVIDIA’s most-referenced diarization model is non-commercial [51]. NeMo’s punctuation weights follow NGC terms, not the framework’s Apache licence [66]. Cadence is labelled MIT but is a Gemma derivative carrying Google’s terms [63]. And IndicMFA states no licence at all [41] — which defaults to all rights reserved.
These two come apart. Entity detection fell from 0.631 to 0.237 F1 on machine transcripts, and the same paper notes that transcription quality does not correlate with downstream task performance [79]. Do not let a good accuracy number reassure you about Step 8.
“You will see different results for queries after adding an approximate index” [100]. If compliance search runs on approximate vector search, the same question asked twice can return different sets. Keep lexical search as the system of record.
Two RBI clocks run from different dates [82], and locking non-mandated recordings into compliance-mode immutability creates a privacy exposure you cannot undo without deleting the cloud account [105]. Classify at ingestion.
No published accuracy for speaker separation in any Indian language. No published Indian date handling in any normalisation toolkit. No confirmation that key destruction satisfies the DPDP erasure duty. No published study of checksum-based PII detection over machine transcripts. These are real holes, and a guide that pretended otherwise would be less useful.
Read next:
Every figure, licence and quote on this page traces to one of these. Prices only where the company publishes them; licences read from the repository or model card, not assumed from the project they belong to.
Checked September 2026. Model availability, licences and prices in this field change monthly; the date is part of the claim.
Opens your assistant with this page as the source, and a question rather than a summary. It will ask what you are building before it answers.
Nothing is sent from here. The link carries only this page’s title and address.