A company picks a speech recognition provider for its voice agent roughly like this: it opens a leaderboard, looks at the top row, and checks whether Ukrainian is on the list of supported languages. It almost always is. That is usually where the check ends.
The problem is that the number in the top row was not measured on what the agent will be working with. And we could not find one that was: not on vendor pages, not on independent leaderboards, not in arXiv papers.
Three public numbers on Ukrainian — and none of them is about your call
These are the three numbers quoted most often when Ukrainian speech recognition comes up.
Number | What was measured | Who measured it | Recording conditions |
|---|---|---|---|
3.5% WER | Scribe v1, Ukrainian | ElevenLabs, its own page | FLEURS — read speech, studio quality |
2.3% WER | Scribe v2, overall ranking | Artificial Analysis, AA-WER v2.0 | a set with no Ukrainian |
8.5% WER | Soniox model, Ukrainian | Soniox, comparison with Google | YouTube audio, 2025; the page no longer opens |
Each of these numbers is correct. The problem starts the moment they are put side by side and compared.
The first comes from ElevenLabs' own page on Ukrainian. It gives 3.5% WER — word error rate, the share of words in a transcript that were recognized incorrectly — on the FLEURS benchmark, and the table states outright that this is Scribe v1. FLEURS is speech read aloud and recorded in good conditions: a narrator, silence, the full frequency band.
The second number comes from the AA-WER v2.0 index by Artificial Analysis, an independent party. In August 2026 Scribe v2 topped that index at 2.3%, followed by Gemini 3 Pro at 2.9%; as of October 2026 it sits fourth at 2.2%. But the index is built from three datasets: AA-AgentTalk (50% weight), VoxPopuli-Cleaned-AA (25%) and Earnings22-Cleaned-AA (25%), and no Ukrainian result has been published there — nor any per-language breakdown at all. Calling it the best model in the world is an accurate statement about an index your language was never in.
The third number has the most interesting story. Soniox's page comparing itself with Google gave 8.5% WER for Ukrainian against 18.3% — vendor material, but with the conditions stated honestly: a 2025 study on real-world YouTube audio. As of 25 August 2026 that page no longer opens. Search engines still remember it, but the server returns a 404.
What is left in its place, on Soniox's live page about Ukrainian, as of October 2026, is the wording "World’s most accurate Ukrainian speech-to-text" and "industry-leading error rates" — claims to top accuracy and the lowest error rates without a single number. The same vendor does have a benchmarks section, but it measures only English.
The gap between 3.5% and 8.5% read as a twofold advantage for the first model. In reality it is the gap between a narrator recorded in a studio and clips from YouTube: what was being compared was conditions, not models. In different years, too, and on versions the vendors chose themselves. And now one of the two numbers has simply vanished — it can no longer be checked against the primary source.
Why the phone channel has to be measured separately
All three measurements have one thing in common: none was made on telephony.
A phone call is a narrowband signal: an 8 kHz sampling rate, a usable band of roughly 3.4 kHz, G.711 codec compression. Compared with a studio recording sampled at 16 kHz or higher, the upper part of the spectrum has been physically cut out. That is exactly where the cues that distinguish voiceless consonants sit — and Ukrainian sibilants, the hissing and hushing sounds the language is full of, are the first to suffer. Add background noise on the caller's side: the street, a car, a shop, a child nearby.
A model that scores 3.5% on a narrator's recording will score something else on this material. How different depends on the specific model — and that is precisely the test we could not find for Ukrainian in open sources.
There is one more trap, a purely technical one. Many teams plan to pull call audio through the telephony provider's regular HTTP API, then discover that all it holds is an after-the-fact recording, available once the conversation is over. That is enough for call analytics, but not for a dialogue. A live stream comes from a SIP trunk, and that means your own media layer in the call path: an SBC that receives RTP and passes it on to recognition. A side benefit of this setup is that the same layer can strip out noise before the audio reaches the model.
Where exactly Ukrainian breaks on a call
Accent is not the main difficulty. The real failure points are elsewhere, and they are specific to the Ukrainian market.
Switching languages within a single utterance. A person starts in Ukrainian, drops in a Russian word, switches back. Surzhyk, this everyday mix of the two languages, is the norm rather than the exception, and for a model it is the hardest class: it has to hold two phonetic systems at once and decide which one a given word belongs to.
Proper names. Names of cities, villages and districts, surnames, company and brand names. Here the model gets no hint from the context of the sentence, and the word it gets wrong is the very one the call was made for.
Fast, emotional speech, interruptions, cut-offs. A person calling with a problem does not talk like a narrator: they speak faster, swallow word endings and cut the other person off mid-sentence.
The first two classes are not a hunch from practice but a measured fact. It just happens to have been measured in other languages.
The lesson about vendor numbers: version, date, conditions
When Grow2.ai was working out the architecture of a voice agent for lead qualification, the assessment began with an alarming number: one candidate's Ukrainian recognition supposedly had a 13.5% error rate. For a scenario where the agent has to hear a city name correctly, that looked like a death sentence — and it nearly ruled the option out before any testing. This is an episode from our internal assessment, not a published measurement: we are not naming the provider or the date, because the number itself is an illustration here, not evidence.
When we tried to trace the number to its source, it fell apart. The figure referred to the previous generation of the model and had been measured on dirty YouTube audio — material known to be harder than average. It had nothing to do with the current version, which was already out by then.
The conclusion from this episode is not about a particular provider. It is that any number on recognition accuracy without three coordinates — model version, measurement date, recording conditions — is not information. It is a marketing line that can be used just as easily to sell something as to wrongly reject something.
Both traps, by the way, sit on the very same vendor page: the Ukrainian number there is given for the first version of the model, while the headline about first place in the world rests on an index that has no Ukrainian in it.
You don't need verbatim accuracy
Most of the anxiety around WER goes away with one observation about what exactly the agent needs to hear.
WER measures the share of incorrectly recognized words across the whole transcript. But a qualification agent is not producing a verbatim record. It has to pull a few fields out of the conversation that determine where the lead goes next: for example, the caller's city and the reason for the call. The rest of the conversation can be recognized as roughly as you like — it has no effect on routing.
So the metric that matters is not WER but accuracy on the decision fields. A transcript with 12% errors, none of which fell on a city name, is better for qualification than a transcript with 6% errors, half of which landed precisely on the place names. A model that looks weaker overall may turn out to be the better fit — if it handles proper names better.
That also points to how to reduce the risk. Instead of chasing the last percent of overall accuracy, you narrow the task: a short script in which the agent has to find out two or three things in 20–30 seconds; asking again when confidence is low; prompting the model with a list of expected vocabulary — the cities that actually come up in your calls. Models support that last technique directly: in Scribe v2, for example, keyterm prompting lets you pass words and phrases the model should pay special attention to: as of October 2026, up to 1,000 in batch mode and up to 50 in real time.
A test plan for before launch, not after
Since there is no public number, you have to get one yourself. Below is what Grow2.ai builds into the test plan before a pilot. It is a plan, not a report on a measured result: valid numbers come only from the audio of a specific project, and there is nowhere else to get them.
- Build your own test set. Real call recordings from your telephony, not studio phrases. A few dozen conversations are enough, but they must reflect the mix you actually get: cities, Surzhyk, noise, cut-offs.
- Label the ground truth by hand. Without a correct answer there is nothing to compare against. You don't label the whole text word for word, only the decision fields — what the person actually said about the city and the reason for the call.
- Run several candidates on the same set. A comparison only makes sense when conditions are the same for everyone — exactly what the public numbers lack.
- Measure accuracy by field, not WER. Plus, separately: the share of cases where the model was unsure and the agent should have asked again.
- Test recognition and dialogue logic separately. Otherwise you can't tell what broke: the agent didn't hear the city, or it heard it and made the wrong call on where to send the lead.
- Test denoising as a separate variant. The same set, run with and without audio cleanup before recognition, shows whether this layer is worth the extra architectural complexity.
A run like this costs a few days of work and yields a number that is on no leaderboard but is about your calls specifically. The logic is the same as in the checklist for launching an agent in two weeks: testing on real data comes before scaling, not after.
What to measure after launch
The pilot gives you a starting point; after that you need continuous oversight. From day one, your monitoring should show accuracy on the decision fields — the share of calls where the city and the reason for the call were identified correctly — and, next to it, routing errors split by cause: the lead went to the wrong stage because the agent misheard, or because it decided wrongly.
The re-ask rate is tracked separately, and it has no desirable direction. If the agent asks again too often, the conversation becomes exhausting and people hang up. If it never asks again, it is guessing. The normal range has to be found by feel on live traffic, and it differs from scenario to scenario.
The fourth metric is the share of conversations the agent judged atypical and handed to an operator. This is not a sign of a weak system. An agent that hands a difficult case to a human in time is behaving correctly; the problem is when it hands off nothing. That boundary of competence is set the same way as the other safeguards — more on those in the breakdown of the three protection levels of an AI agent.
What this means for the decision to deploy
The absence of a public measurement is no reason to give up on a voice agent in Ukrainian. It is a reason not to buy a solution off a leaderboard.
Choosing a recognition provider stops being part of procurement and becomes part of the pilot. That also changes the shape of the agreement with your contractor: testing on your recordings has to be in the plan before the final architecture is fixed, not after the work has been signed off. The general approach to which tasks are worth handing to an agent in sales at all is covered in the guide for executives, and the arithmetic of the comparison with a sales rep's work is covered separately.
For a business owner, this section comes down to one question for the contractor: what do you do with an utterance the model recognized incorrectly — rewrite it entirely or ask the person again? The answer "we rewrite it" means the agent will be correcting your customers' city names and surnames by guesswork. What follows is the mechanics of this layer for those who will be building it.
For the technical team: a correction layer on top of recognition
On top of a single recognition pass you can put a language model that refines the result over several steps: it classifies the caller's utterance as a confirmation, new information or a correction, finds the erroneous fragment in the transcript and edits just that fragment instead of rewriting the whole utterance. That is the design in the Agentic ASR paper (arXiv:2605.29430, preprint submitted on 28 May 2026), where the largest share of semantic errors is removed on the very first refinement — including on a benchmark with proper names, the same class that breaks Ukrainian telephony.
We covered the correction mechanics, the S²ER metric, the full tables and the limits of this result separately: correcting speech recognition errors in three steps. The same piece covers which of those numbers carry over to Ukrainian and which don't, and what that means for the price and timeline of a pilot.
For deployment, the mechanics matter more than the percentages. Correction works through dialogue: the model clarifies whatever it is unsure of. In a voice agent that translates into simple behavior — ask again rather than guess. An agent that asks for the city name again when recognition confidence is low is doing the same thing the correcting model does in this paper, only it uses the caller in place of a second model.
The next step is to discuss a voice agent pilot: in one meeting we work out which decision fields are critical in your calls and how to set up a test on your recordings. Get in touch.
And one detail that usually gets noticed only after launch: the test set has to live on. Calls change with ad campaigns and the season, and within a few months a set collected once before the pilot no longer describes what actually comes in on the line.
Sources:
- The Batch, Issue 367 — "Agents Come to Speech Recognition" (21 August 2026)
- Agentic ASR — arXiv:2605.29430
- Artificial Analysis — Speech to Text Leaderboard (AA-WER v2.0)
- ElevenLabs — Ukrainian Speech to Text
- Soniox — Ukrainian Speech to Text (the comparison page against Google that gave 8.5% WER returned a 404 as of 25 August 2026)