A 3.54 average WER is a good number for an open model. It is also just a number. The more useful question is what happens when you hand the model a noisy phone recording of someone who speaks fast and changes their mind halfway through a sentence.
That is what I want to find out about HojoAI/Hojo-ASR-Multi-V1, an open multilingual speech recognition model from @hojoHQ. This post keeps three things separate: what Hojo reports, what I tested, and what I think.
The model uses an Encoder-Adapter-LLM design with a customized multi-frame acoustic fusion structure.
The decoder is built on Qwen3. The model tree lists Qwen/Qwen3-4B-Instruct-2507 as the base model. That describes the decoder, not the size of the whole model.
It is released under Apache 2.0 and installs with pip install -U hojo-asr.
Inference accepts WAV file paths, raw audio bytes and SCP files.
The card says it targets noisy conditions, informal pronunciation and spoken self-correction.
The card lists support for German, French, Spanish, Portuguese, Japanese, Italian, Arabic, Korean and Russian.
Hojo also shared a multilingual Open ASR Leaderboard snapshot in which the model has an average WER of 3.54 and sits 7th, behind Microsoft Azure, ElevenLabs, AssemblyAI and Reson8 entries. That snapshot covers five languages only: German, French, Italian, Spanish and Portuguese, across 13 benchmark slices. Leaderboard positions change as new submissions arrive, so treat the rank as a snapshot, not a permanent standing.
A few things stand out when I do the arithmetic myself.
The headline average hides a wide spread. The best slice is Italian FLEURS at 2.30. The worst is Italian MLS at 5.35. The same language sits at both ends, so the dataset matters as much as the language. FLEURS is read speech, MLS comes from audiobooks, and CoVoST is crowdsourced. None of them is a noisy call recorded on a phone.
The 3.54 is a two-step average. According to Hojo's technical team, scores are averaged within each language first, and those five language-level averages are then averaged to get the final score. The benchmark organizers provide an official script for this, and that script is the reference, not my own arithmetic. For comparison, averaging the model card's per-dataset table the same way gives about 3.57, and a plain mean of all 13 slices gives about 3.52. I did not reproduce 3.54 exactly from the card's table, so I treat the official script's output as the number that counts. The practical point is that every language carries equal weight, no matter how many datasets or samples it has.
The gaps near the top are small. In the snapshot, the leader sits at 2.61 and Hojo at 3.54, a difference of under one WER point. Whether that matters depends on the job. For meeting transcripts used for search, probably not. For transcripts that feed an agent that takes actions, one wrong entity name can be expensive.
The 3.54 is also an average over five languages and specific datasets. It says nothing about every language or every real-world recording, and I would not read it that way.
What an LLM decoder changes (my interpretation)
Traditional ASR stacks map audio to text with a decoder that knows little about language. Putting an LLM decoder behind an acoustic encoder gives the system a stronger language prior. That can help with ambiguous sounds resolved by context, informal or disfluent speech, and punctuation and casing.
It can also hurt. An LLM decoder can produce fluent text that was never said. Hallucinated insertions and silently “corrected” speech are the failure modes I look for first, especially on silence, background music and heavily accented audio. WER does not capture these well, because a fluent wrong sentence can score only slightly worse than a hesitant right one.
That is why I am testing with my own audio instead of repeating the leaderboard.
How I am testing it
I am running the model locally through the hojo-asr package and recording the setup for every clip: audio language and duration, recording device and environment, inference hardware, package and model version, batch size, ground-truth transcript, raw output and WER.
The test set:
Clean read speech in one of the five benchmarked languages, as a sanity check against the model card.
A phone recording in a cafe or on a street.
A non-native speaker.
Fast, informal speech with hesitations and self-corrections.
Technical vocabulary from my own field: APIs, product names, acronyms.
Code-switching, for example English terms inside a German or Spanish sentence.
Long-form audio of 10 minutes or more.
A language the card lists but the leaderboard does not cover, such as Japanese or Korean, labeled clearly as outside the benchmarked set.
5. Could this be an input layer for hardware?
This section is discussion, not testing. Local inference is supported, which matters for privacy-sensitive products such as earbuds, recorders and in-car systems. Several questions decide whether the model fits a real device:
Memory and compute. A model with a 4B-class decoder is a different proposition from a small on-device recognizer. VRAM and throughput need to be measured on the actual hardware.
Latency vs throughput. Leaderboard RTFx measures throughput. It is not end-to-end latency on a device.
Streaming. The model card does not claim real-time streaming, and I have not verified it.
On-device use.“Runs locally” and “runs on a wearable” are separate claims, and only the first has been tested.
Downstream cost of errors. If transcripts feed an agent, a wrong entity name can trigger a wrong action. Error types matter as much as WER.
Try it yourself
Take your hardest multilingual clip and run it through HojoAI/Hojo-ASR-Multi-V1. Post the transcript next to the ground truth and tag @hojoHQ. If you find where the 3.54 breaks, that is the useful result.
