TinyML: AI on tiny chipsMiddleLesson 2766 min read

Collecting sensor data

A TinyML model is only as good as its data. Learn how accelerometers and microphones produce data, how to label it, and how to cut it into windows.

✦ Ask the tutor 🧪 Lab · 25 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Collecting sensor data

A TinyML model is only as good as its data. Learn how accelerometers and microphones produce data, how to label it, and how to cut it into windows.

1/5

In simple words

Imagine teaching a friend to recognise cricket shots just by feeling the bat's movement. You would record many shots, and write down which was a cover drive and which was a pull. Without those notes, your friend could feel the bat but never know the names. Collecting and labelling data is exactly this.

Two common sensors

An accelerometer measures acceleration along three axes: x, y and z. When the device is still, it still reads about 1 g, because it feels gravity. So the numbers tell you both how the device is tilted and how it moves. Phones, fitness bands and many small boards have one.

A microphone measures air pressure changes many thousands of times a second. For speech, a common rate is 16,000 samples per second. Audio models usually do not use raw samples directly; they first turn short pieces of audio into a spectrogram, a picture of which frequencies are loud over time.

Sensor	Typical data	Good for
Accelerometer	3 numbers (x, y, z), tens to hundreds of times a second	Gestures, activity (still, walking, running), falls, machine vibration
Gyroscope	3 numbers for rotation speed	Turning and twisting gestures, often used with an accelerometer
Microphone	Thousands of samples a second	Wake words, sounds such as coughs, alarms or machine noise
Small camera	Low-resolution images	Person present or not, simple objects

Sampling rate and windows

The sampling rate is how many readings the sensor gives each second. A model does not look at one reading; it looks at a window, for example one second of data. Windows usually overlap: a new window starts every half second, so a gesture that falls across the edge of one window is still fully inside another.

Python: two seconds of made-up walking data, with labelspython

import numpy as np
import pandas as pd

RATE = 50          # samples per second
SECONDS = 2
t = np.arange(RATE * SECONDS) / RATE
rng = np.random.default_rng(0)

# 2 seconds of made-up "walking" on the z axis: gravity + a step rhythm + noise
z = 1.0 + 0.3 * np.sin(2 * np.pi * 1.8 * t) + rng.normal(0, 0.02, len(t))
df = pd.DataFrame({"time_s": t, "z_g": z.round(3), "label": "walking"})
print(len(df), "samples in", SECONDS, "seconds")
print(df.head(6))

Labelling

A label is the right answer for a piece of data: "walking", "still", "wake word", "background". The model learns the link between data and labels, so bad labels teach bad lessons.

1
Plan the classes
Decide the few classes you need. Always include "none of these" or "background".
2
Record in real conditions
Use the same board, the same sensor placement and the places where it will really be used.
3
Label while recording
Press a button or say the label, so you do not have to guess later.
4
Vary people and places
Different people, speeds, rooms, noise levels and ways of holding the device.
5
Keep some data aside
Hold back recordings from some people or days for testing. Never train on them.

Data that fools you

All "walking" recorded by one person on one day
Only quiet room recordings for a wake word used in a busy kitchen
Test windows cut from the same recording as training windows
No "background" class, so every sound must be a command

Data that holds up

Many people, several days, different shoes and floors
Recordings with fans, TV, traffic and pressure cooker whistles
Test data from different people or sessions than training
A large, varied "background" or "unknown" class

Watch out

Recording people's voices or movements is personal data. Ask for clear permission, explain what you will record and why, store it safely, and delete it when you are done. Do not record people who have not agreed.

Note

Many TinyML platforms can collect data straight from a supported board or a phone into a labelled dataset. That is handy, but the same rules apply: variety, clear labels and a separate test set.

Try this. In the code above, change the rate to 100 samples per second. How many samples are in 2 seconds now? What happens to the memory needed for one window?

Watch and read more

Lab

A working windowing function that turns a labelled accelerometer recording into training examples.

~25 min

The problem

The starter makes a made-up 23-second accelerometer recording (still, walking, shaking, still) at 50 samples per second, with one label per sample. It also draws the signal.

Write make_windows(signal, labels, size, step). Start a window at 0, step, 2 × step and so on, while a full window still fits. Each window is signal[start:start+size].
Keep a window only if all its labels are the same, and give it that label. Drop windows that mix two activities.
Return X with shape (windows, size, 3) and y with one label per window.
Run it with size 50 and step 25. Then try step 50 (no overlap). How many windows do you get now?

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Ctrl + Enter to run

import numpy as np
import matplotlib.pyplot as plt

# A made-up 3-axis accelerometer recording (in g), 50 samples per second.
# Someone holds a phone still, walks, then shakes it. Each sample has a label.
RATE = 50
rng = np.random.default_rng(7)

def segment(kind, seconds):
    t = np.arange(seconds * RATE) / RATE
    s = rng.normal(0, 0.02, (len(t), 3))
    s[:, 2] += 1.0                                   # gravity on the z axis
    if kind == "walking":
        s[:, 2] += 0.3 * np.sin(2 * np.pi * 1.8 * t)  # about 2 steps per second
    if kind == "shaking":
        s += 0.8 * np.sin(2 * np.pi * 6 * t)[:, None]
    return s, [kind] * len(t)

parts = [segment("still", 6), segment("walking", 8), segment("shaking", 5), segment("still", 4)]
signal = np.concatenate([p[0] for p in parts])
labels = np.array(sum((p[1] for p in parts), []))
print("recording:", signal.shape, "samples x axes;", len(signal) / RATE, "seconds")

def make_windows(signal, labels, size, step):
    """Cut the recording into windows of "size" samples, starting every "step" samples.
    Keep a window only if all its samples have the same label.
    Return X with shape (n_windows, size, 3) and y with one label per window."""
    raise NotImplementedError("Loop over start positions 0, step, 2*step, ...")

X, y = make_windows(signal, labels, size=50, step=25)
print("windows:", X.shape)
for kind in np.unique(y):
    print(f"  {kind:8} {np.sum(y == kind)} windows")

plt.figure(figsize=(8, 3))
plt.plot(np.arange(len(signal)) / RATE, signal)
plt.xlabel("seconds"); plt.ylabel("g"); plt.legend(["x", "y", "z"])
plt.show()

Output

No output. Add print(...) to see values.

You are done when

make_windows returns X with shape (windows, size, 3) and a matching y
Windows that mix two labels are dropped
Every activity ends up with some windows
The learner compares the number of windows with and without overlap

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Why drop windows that mix two labels, instead of giving them the majority label?Reveal

A window that is half "still" and half "walking" is not a clean example of either. Training on it with one label teaches the model a blurry boundary. Dropping it is simple and safe. Some projects keep such windows with the majority label, or add a "transition" class, but you should decide that on purpose.

Q2Your gesture model works for you but fails for your friend. Name two likely data problems.Reveal

First, all the training data came from one person, so the model learned your speed and style, not the gesture in general. Second, the device may have been held or worn in a different position, which changes which axis sees gravity and movement. Collect from more people and placements, and test on people who are not in the training data.

Please sign in to continue.

Questions people ask

How much data do I need?

There is no fixed number. Start with enough to cover the variety you expect: several people, places and ways of doing each action. Train, test on held-out data, and look at the mistakes. If a class keeps failing, collect more of exactly those cases.

Can I use data from the internet?

Public datasets are useful for practice and for some sounds. But your sensor, board and placement matter a lot. A model trained on phone data may do poorly on a band worn on the wrist. Always add data from your own device, and check each dataset's licence.

Why do windows overlap?

A gesture or a word does not know where your windows start. With overlap, an event that is cut in half by one window is likely to be whole in the next. It also gives more training examples from the same recording.

Can I make fake data?

Synthetic data is great for learning and testing code, as in the labs here. It can also help add variety, such as mixing background noise into clean recordings. But a model must be tested on real recordings before you trust it.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Collecting sensor data

Collecting sensor data

Two common sensors

Sampling rate and windows

Labelling

Plan the classes

Record in real conditions

Label while recording

Vary people and places

Keep some data aside

Watch and read more

The problem

Practice lab

You are done when

Hard questions

Questions people ask