TinyML: AI on tiny chipsMiddleLesson 2776 min read

Training a tiny model

Turn each window into a few useful numbers, such as the mean and the energy, and train a small classifier. Then measure it honestly on held-out data.

✦ Ask the tutor 🧪 Lab · 25 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Training a tiny model

Turn each window into a few useful numbers, such as the mean and the energy, and train a small classifier. Then measure it honestly on held-out data.

1/4

In simple words

A doctor does not need a whole video of you to check if you have a fever. One number, your temperature, says a lot. Features are like that: a few well-chosen numbers that sum up a whole second of sensor data, so a small model can decide quickly.

A window of 50 readings on 3 axes is 150 numbers. A tiny chip could use all of them, but a smaller input means a smaller, faster model. So we often compute a few features per window first.

Two simple, powerful features

Feature	How to compute it	What it tells you
Mean (per axis)	The average of the readings in the window	Which way gravity points, so how the device is tilted or lying
Energy (per axis)	The average of (reading − mean)², also called the variance	How much the device is moving or shaking
Others you may add	Minimum, maximum, peaks per second, frequency features	Speed of a rhythm, sharp hits, steps

Python: mean and energy for a still window and a walking windowpython

import numpy as np

rng = np.random.default_rng(1)
t = np.arange(50) / 50
still = np.column_stack([rng.normal(0, 0.02, 50), rng.normal(0, 0.02, 50), 1 + rng.normal(0, 0.02, 50)])
walk = still.copy()
walk[:, 2] += 0.3 * np.sin(2 * np.pi * 1.8 * t)

def features(window):
    mean = window.mean(axis=0)                     # where gravity points
    energy = ((window - mean) ** 2).mean(axis=0)   # how much it moves
    return np.concatenate([mean, energy])

np.set_printoptions(precision=4, suppress=True)
print("still:  ", features(still))
print("walking:", features(walk))

The means are almost the same, because both windows have gravity on z. The z energy is very different, because walking moves up and down. Neither number alone separates every class, but together they often do.

Choosing a small classifier

Model	Why it suits a tiny chip	Watch out for
Decision tree (small depth)	Just a few comparisons; easy to read and to write in C	Deep trees overfit
Logistic regression	One weight per feature per class; very small	Only draws straight boundaries
Small neural network	Learns more complex patterns; well supported by TinyML runtimes	Needs more data and care

In this track we use scikit-learn in Python for the ideas. Many TinyML tools train small neural networks with other libraries and then convert them for the chip. The steps are the same: features in, labels out, test on held-out data.

Honest testing

1
Split before training
Keep a test set from other people or sessions, and do not look at it while you tune.
2
Train on the training set
Fit the model and choose settings using only training data (or a separate validation set).
3
Test once
Measure accuracy on the held-out test set.
4
Read the confusion matrix
See which classes get mixed up. That tells you what data or features to add.

Note

The labs use clean, made-up data, so scores come out very high. Real data from real people is messier. A model that scores near perfect on your first real test is a reason to check for leaks between training and test data, not to celebrate.

Watch out

Accuracy alone can hide problems. If 95% of windows are "still", a model that always says "still" gets 95% and is useless. Look at each class, and at the kinds of mistakes that matter for your use.

Real example

A college team in Coimbatore builds a band that tells a weaver's hand movements from rest, to count work cycles. The first model reaches high accuracy on their own recordings. On new weavers it fails often. The fix was not a bigger model: they recorded more people and added a feature for the rhythm of the movement.

Try this. In the lab, first use only the mean as the feature. Which classes get mixed up in the confusion matrix? Then add energy and compare.

Watch and read more

scikit-learn: decision treesscikit-learn · reading

Lab

A feature function that lets a small decision tree reach 90% or more on held-out data.

~25 min

The problem

The starter makes made-up accelerometer windows for four classes: flat, tilted, walking and shaking. Training and test data come from different random seeds, so the test windows are new.

Write features(window). It gets one window of shape (50, 3) and returns a short list of numbers (between 2 and 12).
Start with only the mean of each axis. Run it and read the confusion matrix. Which classes get confused?
Add the energy of each axis: the mean of (reading − mean)². Run again.
Aim for 90% or more held-out accuracy. Write one sentence on what each feature adds.

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Ctrl + Enter to run

import numpy as np

CLASSES = ["flat", "tilted", "walking", "shaking"]
RATE = 50      # samples per second
SIZE = 50      # one window = 1 second

def make_window(label, rng):
    t = np.arange(SIZE) / RATE
    w = rng.normal(0, 0.02, (SIZE, 3))          # small sensor noise
    if label == "flat":
        w[:, 2] += 1.0                           # gravity on z
    elif label == "tilted":
        w[:, 0] += 0.7; w[:, 2] += 0.7           # gravity split between x and z
    elif label == "walking":
        f = rng.uniform(1.6, 2.2)                # steps per second
        p = rng.uniform(0, 2 * np.pi)
        w[:, 2] += 1.0 + 0.3 * np.sin(2 * np.pi * f * t + p)
        w[:, 0] += 0.1 * np.sin(np.pi * f * t + p)
    else:  # shaking
        f = rng.uniform(5, 7)
        p = rng.uniform(0, 2 * np.pi)
        w[:, 2] += 1.0
        w[:, :] += rng.uniform(0.6, 1.0) * np.sin(2 * np.pi * f * t + p)[:, None] * rng.uniform(0.5, 1, 3)
    return w

def make_dataset(n_per_class, seed):
    rng = np.random.default_rng(seed)
    X, y = [], []
    for label in CLASSES:
        for _ in range(n_per_class):
            X.append(make_window(label, rng)); y.append(label)
    return np.array(X), np.array(y)

def features(window):
    """Turn one window (50 x 3) into a short list of numbers.
    Ideas: the mean of each axis (which way gravity points),
    and the energy of each axis (how much it moves around its mean)."""
    raise NotImplementedError("Return, for example, 3 means + 3 energies")

from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score, confusion_matrix

X_train, y_train = make_dataset(40, seed=1)   # data to learn from
X_test, y_test = make_dataset(40, seed=2)     # held-out data, never used for training

F_train = np.array([features(w) for w in X_train])
F_test = np.array([features(w) for w in X_test])
print("features per window:", F_train.shape[1])

model = DecisionTreeClassifier(max_depth=4, random_state=0)
model.fit(F_train, y_train)
pred = model.predict(F_test)
print("held-out accuracy:", round(accuracy_score(y_test, pred), 3))
print(confusion_matrix(y_test, pred, labels=CLASSES))

Output

No output. Add print(...) to see values.

You are done when

features returns a short list of numbers for one window
The mean-only version mixes up classes that share the same tilt
Adding energy separates still classes from moving ones
Held-out accuracy is 90% or more on fresh data

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1The mean-only model confuses "flat" with "walking" and "shaking". Why, and why does energy fix it?Reveal

All three have gravity mainly on the z axis, so their average readings are similar. What differs is movement: flat hardly changes, walking goes up and down, shaking moves a lot on all axes. Energy measures how much the readings move around their mean, so it separates them. The mean still matters for telling flat from tilted.

Q2Your features work in Python, but on the board the model predicts "shaking" for everything. What would you check?Reveal

Compare the feature numbers on the board and in Python for the same recorded window. Common causes: a different unit (raw sensor counts instead of g), a different axis order, a different window length or sampling rate, or a missing scaling step. The model must get exactly the same kind of numbers it was trained on.

Please sign in to continue.

Questions people ask

Why not feed the raw window straight into a neural network?

You can, and many TinyML projects do, especially with small convolutional networks. It needs more data and a bigger model. Features are a good first step: small, fast, and easy to understand. Try features first, then compare.

Should I compute features on the chip?

Yes, the same features must be computed on the board, in exactly the same way as in training. If your Python code and your C code compute "energy" differently, the model gets inputs it never saw. Test both on the same window and compare the numbers.

What is a confusion matrix?

A table with the true class in each row and the predicted class in each column. Numbers on the diagonal are correct; everything else shows which class was mistaken for which.

How do I know if my model is overfitting?

If training accuracy is much higher than held-out accuracy, the model learned details of the training data that do not carry over. Use a smaller model (for example a shallower tree), more varied data or fewer features.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Training a tiny model

Training a tiny model

Two simple, powerful features

Choosing a small classifier

Honest testing

Split before training

Train on the training set

Test once

Read the confusion matrix

Watch and read more

The problem

Practice lab

You are done when

Hard questions

Questions people ask