HarkHark

Small on-device speech models make far more mistakes on accented English than on the voices they were trained on, and they keep getting the same names and terms wrong. Sending audio to a server fixes accuracy at the cost of privacy.

People need captions that improve for their own voice without their audio leaving the device.

I built Hark: live captions that run entirely in the browser. I wrote the speech engine myself, in Python and again in JavaScript, from the audio features to the search over a streaming model, and tested the two against each other piece by piece.

When you correct a caption, Hark learns the rare words you say that it keeps missing and listens harder for them next time, from plain counts stored in your browser.

I tested it on eight accented speakers held out from all tuning.

Misses on the names speakers had corrected fell from 18 of 22 to 13 of 22, while overall accuracy never got worse: 16.4% of words wrong before, 16.3% after 200 fixes.

My first version made captions worse overall, so I kept that run, fixed the cause and reported both. The evidence is honest but thin, and the write-up says so.