Speech1 min readDraft: no results reported
Listening in Darija and Kiswahili
How we are building speech evaluation for languages that most speech models were not trained to hear.
Most farmers we work with do not speak to their phone in a standard written language. They speak Darija, Tunisian Derja, Kiswahili, Hausa, Yoruba, Igbo, Ewe, Setswana, often mixed with French or English in the same sentence. This draft explains how we are building an evaluation for that reality. It reports no results yet.
Why standard benchmarks miss it
Public speech benchmarks are dominated by read speech in standard languages, recorded indoors. Field speech is spontaneous, code-switched, and recorded outdoors with wind, engines and animals.
What we are collecting
- Short spontaneous questions about crops, recorded with consent in the field.
- Native-speaker transcripts, including code-switching.
- Agronomic terms in each dialect: local crop names, pest names, product slang.
How we will score it
Word error rate alone is not enough. We also score whether the agronomic meaning survived: the crop, the symptom, the field and the action requested.
Results will be published here, per language, with the number of speakers and hours behind each figure.