TinyML: AI on tiny chipsMiddleLesson 2795 min read

Wake words and gestures

An always-on detector checks again and again, all day. Learn the trade-off between false alarms and misses, and how to choose a threshold.

✦ Ask the tutor 🧪 Lab · 20 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Wake words and gestures

An always-on detector checks again and again, all day. Learn the trade-off between false alarms and misses, and how to choose a threshold.

1/5

In simple words

A dog that barks at every sound keeps the whole house awake. A dog that never barks misses the real thief. A good guard dog is somewhere in between. An always-on detector has the same problem: you choose how "easily" it barks.

A wake word detector listens all the time for one word or phrase. When it hears it, it wakes a bigger system. Gesture detectors do the same with motion: a flick of the wrist, a double tap. This is called always-on detection, and it is one of the most common TinyML jobs, because the tiny model can run cheaply while everything else sleeps.

How always-on detection works

1
Keep a rolling buffer
Store the last second or so of audio or motion in RAM.
2
Check often
Several times a second, turn the latest window into features and run the model.
3
Get a score
The model gives a score between 0 and 1: how likely the wake word is.
4
Compare with a threshold
If the score is at or above the threshold, it fires.
5
Smooth and wait
Often require a few high scores in a row, then ignore new triggers for a short time.

Two kinds of mistakes

Mistake	What happens	How it feels
False alarm (false positive)	It fires when nobody said the word	The device wakes up for the TV or a pressure cooker whistle. Annoying, uses battery, and may feel like spying.
Miss (false negative)	Someone said the word and it did not fire	The user repeats themselves, gets frustrated, and gives up.

The threshold moves the balance. A low threshold fires easily: fewer misses, more false alarms. A high threshold fires rarely: fewer false alarms, more misses. You cannot make both zero; you choose where to sit.

Why false alarms need very small rates

An always-on detector makes a decision many times every second, all day. Even a rate that sounds tiny adds up.

Python: how a small false-alarm rate adds up over a daypython

# A detector that checks for the wake word twice every second, all day.
checks_per_day = 2 * 60 * 60 * 24
for false_alarm_rate in [0.01, 0.001, 0.0001, 0.00001]:
    wakes = checks_per_day * false_alarm_rate
    print(f"false alarm on {false_alarm_rate:.3%} of checks -> about {wakes:,.1f} false wakes a day")

So teams often talk about false alarms per hour or per day of background sound, and misses as a share of real wake words. Measure false alarms on long recordings of normal life without the wake word: talking, TV, kitchen noise, traffic.

Weak testing

Test only on clean recordings in a quiet room
Report one accuracy number
Pick the threshold that looks best on the training data
Test with only the developer's voice

Strong testing

Hours of real background: TV, family talk, traffic, festivals
Report false alarms per hour and miss rate separately
Choose the threshold on held-out data, for a false-alarm target
Many voices, ages, accents and languages, near and far from the device

Real example

A team builds a voice switch for a shop's shutter light. In testing it works well. In the shop, a Hindi film song with a similar-sounding word sets it off several times each evening. They add recordings of songs and TV to the background class, and raise the threshold a little. Misses go up slightly, but false alarms become rare.

Note

Gestures have the same trade-off. A "shake to call for help" gesture must not fire when someone jogs, but must not miss a real shake. Choose the threshold based on which mistake is worse for your users, and test with real daily movement.

Watch out

For safety uses, such as fall detection or alarms, do not rely on a TinyML detector alone. Add a way to cancel false alarms and a way to call for help manually, and test with the people who will really use it.

Watch and read more

Lab

Working functions that compute false-alarm and miss rates, and choose a threshold for a false-alarm target.

~20 min

The problem

The starter makes made-up detector scores: 200 clips where someone said the wake word, and 5000 clips of other sounds.

Write rates(threshold). The detector fires when score ≥ threshold. Return (false_alarm_rate, miss_rate): the share of other clips that fire, and the share of word clips that do not.
Write pick_threshold(max_false_alarm): among the candidate thresholds, return the lowest one whose false-alarm rate is at most the limit. (Lowest, because it misses the fewest real words.)
Run it. How many misses do you pay to go from a 1% false-alarm limit to 0.1%?
Write two sentences: which mistake is worse for a kitchen voice switch, and which is worse for a "call for help" gesture?

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Ctrl + Enter to run

import numpy as np

# Made-up scores from a wake-word detector, between 0 and 1.
# 200 clips where someone said the wake word, 5000 clips of other sounds
# (talking, TV, pressure cooker whistles, traffic).
rng = np.random.default_rng(42)
word_scores = np.clip(rng.normal(0.80, 0.12, 200), 0, 1)
other_scores = np.clip(rng.beta(2, 9, 5000), 0, 1)

def rates(threshold):
    """The detector fires when score >= threshold.
    false_alarm_rate = share of OTHER clips that fire.
    miss_rate        = share of WORD clips that do not fire.
    Return (false_alarm_rate, miss_rate), each between 0 and 1."""
    raise NotImplementedError("Use np.mean on a comparison, for example np.mean(other_scores >= threshold)")

def pick_threshold(max_false_alarm, candidates=np.arange(0.05, 1.0, 0.05)):
    """Among the candidates, return the LOWEST threshold whose false-alarm rate
    is at most max_false_alarm (lowest threshold = fewest misses)."""
    raise NotImplementedError

print("threshold  false alarms  misses")
for t in [0.3, 0.5, 0.7, 0.9]:
    fa, miss = rates(t)
    print(f"  {t:.1f}       {fa:6.2%}      {miss:6.2%}")

for limit in [0.01, 0.001]:
    t = pick_threshold(limit)
    print(f"limit {limit:.1%}: threshold {t:.2f}, rates {rates(t)}")

Output

No output. Add print(...) to see values.

You are done when

rates returns the false-alarm share of other clips and the miss share of word clips
pick_threshold returns the lowest candidate that meets the false-alarm limit
The learner sees that a stricter false-alarm limit costs more misses
The written answer ties the choice of threshold to which mistake hurts users more

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1With a 1% false-alarm limit you miss very few wake words. Why might that still be a bad product?Reveal

1% of checks is a lot for an always-on device. If it checks twice a second, that is well over a thousand false wakes a day. The limit should be set per hour or per day of real background sound, which means a much smaller rate per check.

Q2Your detector's false alarms are fine at home but very high at a wedding. What would you do?Reveal

Collect recordings from loud events with music, crowds and speeches, and add them to the background training data and to the false-alarm test. Then re-train and choose the threshold again. You might also require several high scores in a row. Check the miss rate again afterwards, because it may rise.

Please sign in to continue.

Questions people ask

Why not just make the model better, so no threshold is needed?

A better model helps: its scores separate the two groups more clearly, so any threshold gives fewer mistakes. But scores for real words and background will still overlap a bit, so you always need to choose a threshold.

What does "requiring several high scores in a row" do?

A single noisy spike above the threshold no longer fires. Real words usually give high scores over several checks in a row. This cuts false alarms, with a small delay and sometimes a few more misses.

Should every device use the same threshold?

Often one default is chosen, but some products adjust it by setting or environment. A noisy kitchen and a quiet bedroom are different. Test the default in the noisiest place you expect.

Can I pick any wake word?

Short or common words are easy to trigger by accident. Words with a few clear syllables that do not appear in normal talk usually work better. Test your choice against real speech in the languages your users speak.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Wake words and gestures

Wake words and gestures

How always-on detection works

Keep a rolling buffer

Check often

Get a score

Compare with a threshold

Smooth and wait

Two kinds of mistakes

Why false alarms need very small rates

Watch and read more

The problem

Practice lab

You are done when

Hard questions

Questions people ask