Transcribing recordings in other languages: what models do well and where they stumble

Anyone who studies, researches or works internationally ends up with recordings in more than one language: an interview in Turkish, a family recording in Arabic, a guest lecture in English, a meeting where Polish and German run together. Modern speech recognition handles this surprisingly well — if you know its weak spots.

How many languages do the models recognize?

OpenAI's Whisper was trained on 99 languages; Sprechverlauf uses the fast Whisper Large v3 Turbo in the free trial. Microsoft MAI-Transcribe-2, the model in the paid plans, supports 60 languages according to its model card and separates speakers too. Microsoft reports an average word error rate of 5.2 percent on the FLEURS benchmark. That is an average: large languages with lots of training data do better, rarer ones worse.

Pitfall 1: closely related languages

Croatian, Serbian and Bosnian; Hindi and Urdu; Malay and Indonesian: these sound so alike that automatic language detection can miss on short clips. A Serbian interview may come out in Croatian spelling, or Urdu in Devanagari instead of Arabic script. The fix is simple: select the language explicitly before you start.

Pitfall 2: dialects and slang

Arabic is the clearest example: Egyptian, Levantine and Moroccan Arabic differ a lot, yet most writing uses Modern Standard Arabic. Models tend to smooth spoken dialect toward the standard language or mishear regional words. The same applies, to a lesser extent, to Swiss German, Brazilian Portuguese or Cantonese.

Pitfall 3: tonal languages and scripts without spaces

In Chinese, Vietnamese or Thai, pitch changes the meaning of a syllable. Reverb, music and heavy compression blur exactly that — a clean recording counts double here. Chinese, Japanese and Thai also do not separate words with spaces; the model has to set word boundaries itself. Check names and technical terms carefully.

Pitfall 4: switching languages mid-conversation

In many families and teams, people switch languages mid-sentence. If you force one language, the model has to write everything in it. For mixed recordings, “automatic” is the better choice; Microsoft says MAI-Transcribe-2 is built for this kind of code-switching.

In short

• One language: select it before starting, especially for short clips and related languages.
• Several languages in one conversation: leave it on “automatic”.
• Dialect: expect deviations and check quotes at the timestamp.
• Tonal languages: record cleanly, no background music.