Machine learning fundamentalsMiddleLesson 1165 min read

Clustering: finding groups without labels

k-means finds groups of similar rows when nobody has labelled them. Learn how it works, how to pick the number of groups, and why the groups need a human to interpret.

✦ Ask the tutor🧪 Lab · 20 min📝 Lesson test
Lesson in motion

In 60 seconds

Clustering: finding groups without labels

k-means finds groups of similar rows when nobody has labelled them. Learn how it works, how to pick the number of groups, and why the groups need a human to interpret.

1/4
In simple words
You pour a big box of mixed buttons onto the floor and sort them into piles of similar ones, without anyone telling you the right piles. Maybe you sort by size, maybe by colour. That is clustering. The computer makes piles of similar things, and then you decide what each pile means.
In clustering there are no labels. We only have features, and we ask the computer to find groups of rows that are similar to each other. It is a kind of unsupervised learning.

How k-means works

  1. 1

    Choose k

    Decide how many groups you want, for example 3.
  2. 2

    Start

    Place k centre points, often at randomly chosen rows.
  3. 3

    Assign

    Give each row to its nearest centre.
  4. 4

    Move

    Move each centre to the average of the rows assigned to it.
  5. 5

    Repeat

    Assign and move again until the centres stop moving much.
Because the start is random, different starts can give different groups. scikit-learn's n_init setting runs several starts and keeps the best one. Setting random_state makes the result repeat.
Python: grouping made-up shop customerspython
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans

# Made-up customers of a small shop: visits per month, average spend (arbitrary units).
rng = np.random.default_rng(5)
a = rng.normal([2, 80], [1, 15], (40, 2))     # rare visits, bigger baskets
b = rng.normal([12, 25], [2, 8], (40, 2))     # frequent visits, small baskets
c = rng.normal([6, 50], [1.5, 10], (40, 2))   # in between
X = np.vstack([a, b, c])

km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
print("Centres (visits, spend):")
print(km.cluster_centers_.round(1))
print("Customers per group:", np.bincount(km.labels_))

plt.scatter(X[:, 0], X[:, 1], c=km.labels_)
plt.scatter(km.cluster_centers_[:, 0], km.cluster_centers_[:, 1], marker="x", s=200, color="red")
plt.xlabel("visits per month"); plt.ylabel("average spend")
The model finds three groups and gives each one a number: 0, 1 or 2. The numbers mean nothing by themselves. You look at the centres and decide: "group 1 visits often with small baskets, perhaps nearby families."

How many groups? The elbow method

k-means reports inertia_: the total squared distance from each row to its centre. More groups always lower it, so the lowest is not the goal. Plot inertia for k = 1, 2, 3 and so on. Look for the "elbow", where adding another group stops helping much. It is a rough guide, not a rule; the right k also depends on what is useful for your purpose.

Things to watch

  • Scale first. k-means uses distances, just like nearest neighbours. A feature with big numbers will dominate.
  • Round, similar-sized groups. k-means works best when groups are blob-shaped and of similar spread. Long, curved or very uneven groups need other methods, such as DBSCAN.
  • It always finds k groups. Even in data with no real groups at all, k-means will still split it into k piles. Check that the groups make sense.
  • Groups are not labels of truth. Be careful with groups of people. Naming a cluster of customers with a label like "poor" or "risky" can be unfair and wrong. Describe what the numbers show.
Note
If you do have true labels for checking (as with the iris dataset), you can compare the clusters with them. adjusted_rand_score gives 1 for a perfect match and about 0 for random grouping. In real clustering tasks you usually do not have labels, which is why interpretation matters so much.
Try this. In the code, change n_clusters=3 to 5. What does k-means do with the extra groups?

Watch and read more

scikit-learn: Clusteringscikit-learn · reading

Lab

An elbow table for k-means on scaled iris measurements, a 3-group model, and a check of how well its groups match the real species.

~20 min

The problem

Pretend you do not know the iris species. You only have the four measurements.

  1. Run the starter. It scales the features into X_scaled.
  2. For k from 1 to 8, fit KMeans(n_clusters=k, n_init=10, random_state=0) on X_scaled and store its inertia_ in inertias[k].
  3. Fit the final kmeans with 3 groups (same settings).
  4. Now use the hidden species labels only to check: set ari to adjusted_rand_score(true_species, kmeans.labels_).
  5. Print the table and the score. Where is the elbow? Which species do the groups mix up?

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Output

No output. Add print(...) to see values.

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1The elbow looks like it could be at 2 or at 3. Why is that, and how would you decide without labels?Reveal

Two of the three species have similar measurements, so the data looks like one clear group plus one larger, mixed group. The elbow is often unclear in real data. Without labels, decide by usefulness: look at each group's centre, ask whether the split is meaningful for your purpose, and check that the groups are stable across different random starts.

Q2Why is it not fair to call the adjusted Rand score a measure of "accuracy" for clustering in general?Reveal

In most real clustering there are no true labels, and the groups found may be meaningful even if they do not match any existing labels. Here we only have species labels because it is a teaching dataset. The score tells you how well the clusters match one particular grouping, not whether the clusters are "right".

Please sign in to continue.

Questions people ask

What is the difference between clustering and classification?

Classification learns from labelled examples and predicts known groups. Clustering has no labels and invents groups from similarity. The groups may or may not match anything meaningful.

Can k-means use text columns?

Not directly. k-means needs numbers where distance makes sense. Turn categories into numbers carefully, or use a method designed for mixed data.

Why do I get a different result each run?

The random start changes. Set random_state for repeatable results and use n_init of 10 or more so the best of several starts is kept.

What else is clustering used for?

Grouping similar documents, finding unusual rows that fit no group well, compressing colours in an image, and exploring a new dataset before building a supervised model.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress