Machine learning fundamentalsMiddleLesson 1085 min read

Data, features and labels

Every supervised problem is a table: rows are examples, feature columns are what you know, and the label column is what you want to predict.

✦ Ask the tutor 🧪 Lab · 15 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Data, features and labels

Every supervised problem is a table: rows are examples, feature columns are what you know, and the label column is what you want to predict.

1/5

In simple words

Think of a school register. Each row is one student. The columns say height, age, and class. If you want to guess a student's class from their height and age, then height and age are the clues, and class is the answer. In machine learning, clues are called features and the answer is called the label.

Almost every problem in this track starts as a table.

Row (example, sample): one thing we know about, such as one flower, one loan application or one field.
Feature: a column the model is allowed to use as input, such as petal length or rainfall.
Label (target): the column we want to predict, such as the flower's species or the crop yield.

By habit, programmers call the features X (a table, capital letter) and the labels y (one column, small letter).

A real toy dataset: iris flowers

scikit-learn comes with small datasets for learning. The iris dataset has 150 flowers. Each has four measurements, and a label for which of three species it is.

Python: look at features and labelspython

from sklearn.datasets import load_iris

iris = load_iris(as_frame=True)
X = iris.data          # features: a pandas DataFrame
y = iris.target        # labels: 0, 1 or 2

print("Rows and feature columns:", X.shape)
print("Feature names:", list(X.columns))
print("Species names:", iris.target_names.tolist())
print()
print(X.head())
print()
print("How many of each label:")
print(y.value_counts().sort_index())

Two kinds of label, two kinds of problem

Label looks like	Problem type	Examples
A number on a scale	Regression	Crop yield, house rent, travel time
One of a few groups	Classification	Spam or not, species, loan approve or reject

Good features and bad features

A feature is useful if it carries information about the label and will really be available when you make the prediction.

Bad features

Row numbers or ids, which only carry order, not meaning
Anything you only learn after the answer is known
A copy of the label under a different name
Information that is personal and not needed

Good features

Measurements known at prediction time
Things that plausibly affect the label
Recorded the same way for every row
Collected with consent and a clear purpose

Danger

Label leakage is when the label, or something that gives it away, sneaks into the features. Example: predicting if a patient was admitted, using a column "ward number" that is only filled in after admission. The model looks brilliant in testing and is useless in real life. Always ask of every feature: "Will I know this at the moment I need the prediction?"

Numbers only, please

scikit-learn models need numbers. Text columns like "district" must be turned into numbers first, for example with one column per district holding 0 or 1 (called one-hot encoding). Missing values usually must be filled or removed first. The cleaning lesson in this track shows how.

Note

Data quality matters more than the choice of model. A simple model on clean, honest data usually beats a clever model on messy data. The lesson Step 7 · Data is the product makes the same point for large language models.

Try this. In the code, print X.describe(). Which feature has the biggest spread of values?

Watch and read more

scikit-learn: Toy datasetsscikit-learn · reading

Lab

The wine dataset split cleanly into a feature table X and a label column y, with no leakage.

~15 min

The problem

The wine dataset has 178 wines, 13 chemical measurements each, and a label for which of three growers made it.

Run the starter. It prints the full table, which includes a target column: the label.
The starter sets X = df. That leaks the label into the features. Fix it so X has only the 13 feature columns.
Set y to the label column.
Set n_classes to the number of different labels, computed from y (do not type the number by hand).
Print which feature has the largest average value.

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Ctrl + Enter to run

Output

No output. Add print(...) to see values.

You are done when

X has 178 rows and 13 columns, and no target column
y is the target column
n_classes is computed from y, for example with y.nunique()
The printed feature with the largest mean comes from X.mean()

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1The largest-average feature is not necessarily the most useful for prediction. Why?Reveal

The average depends on the unit. A column measured in large units has a big average, but that says nothing about whether it separates the three growers. Usefulness depends on how differently the feature behaves across the labels, which later lessons measure with models.

Q2Suppose the wine table also had a column "grower_name" in text. What would happen if you kept it in X?Reveal

It is the label written in words, so it is leakage: a model would learn to read the name and look perfect. In real use you would not have the grower's name, since that is exactly what you want to predict. Remove any column that is the answer in disguise.

Please sign in to continue.

Questions people ask

Is the label always one column?

In this track, yes. Some problems predict several things at once, but the ideas are the same.

What if a feature is text, like a comment?

Text must be turned into numbers. Simple ways count words; modern ways use embeddings. For this track we stick to tables of numbers.

Should I use every column I have?

No. Remove ids, leaked columns and personal data you do not need. More columns can also make overfitting easier. Start with features that make sense and add more if testing shows they help.

What does as_frame=True do?

It returns the data as a pandas DataFrame with column names, instead of a bare numpy array. That makes it easier to read and check.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.