Machine learning fundamentalsMiddleLesson 1095 min read

Your first prediction: linear regression

Fit a straight line to data with scikit-learn, use it to predict numbers, and compare it with a simple baseline so you know if it really helps.

✦ Ask the tutor🧪 Lab · 20 min📝 Lesson test
Lesson in motion

In 60 seconds

Your first prediction: linear regression

Fit a straight line to data with scikit-learn, use it to predict numbers, and compare it with a simple baseline so you know if it really helps.

1/4
In simple words
A caterer notices that bigger weddings need more rice. With 100 guests they cooked about this much, with 300 guests about three times more. If you draw these on a graph, the points roughly follow a straight line. Next time, they can read the line to guess how much rice to cook. That line is a linear regression model.
Linear regression predicts a number with a straight-line rule:
prediction = slope × feature + intercept
With several features, each feature gets its own slope (called a coefficient), and the parts are added up. Training finds the slopes and intercept that make the squared errors on the training examples as small as possible.

The scikit-learn pattern

Every scikit-learn model follows the same three steps. Learn them once and you can use dozens of models.
  1. 1

    Create

    model = LinearRegression()
  2. 2

    Fit (train)

    model.fit(X_train, y_train) finds the best slopes.
  3. 3

    Predict

    model.predict(X_new) gives numbers for new rows.
Python: made-up guests and ricepython
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

# Made-up data: wedding guests and kg of rice cooked. Not real figures.
rng = np.random.default_rng(1)
guests = rng.integers(50, 500, 25)
rice_kg = 0.15 * guests + 5 + rng.normal(0, 4, 25)

X = guests.reshape(-1, 1)          # scikit-learn wants a 2-D table
model = LinearRegression().fit(X, rice_kg)

print("Slope (kg per guest):", round(model.coef_[0], 3))
print("Intercept (kg):", round(model.intercept_, 2))
print("Prediction for 250 guests:", round(model.predict([[250]])[0], 1), "kg")

xs = np.array([[50], [500]])
plt.scatter(guests, rice_kg, label="made-up examples")
plt.plot(xs, model.predict(xs), color="red", label="learned line")
plt.xlabel("guests"); plt.ylabel("rice (kg)"); plt.legend()
The learned slope comes out close to 0.15, the number we used to make the data. That is a good sign: the method recovered the pattern we hid in it, despite the noise.

Measuring error

MeasureWhat it meansNotes
MAE (mean absolute error)Average size of the mistakeSame unit as the label; easy to explain
RMSE (root mean squared error)Like MAE but punishes big mistakes moreSame unit as the label
R² (R-squared)How much of the variation the model explains1 is perfect, 0 is no better than predicting the average; can be negative

Always compare with a baseline

A baseline is the simplest possible prediction. For regression, predict the average of the training labels for every row. If your model cannot beat that, it has learned nothing useful. A number like "MAE = 20" means nothing on its own; "MAE 20 against a baseline of 22" tells you the model helps, but only a little.
Watch out
Test on data the model did not train on. Error on the training rows is too optimistic. We use train_test_split to keep some rows aside. The "Doing it honestly" unit goes deeper.

When a straight line is the wrong shape

Some relationships curve. Crops need rain, but too much rain can also hurt. A straight line cannot go up and then down. Later, in the project lesson, you will see how to let a linear model bend by adding a squared feature.
Try this. In the code, change the noise from 4 to 20. Does the learned slope move further from 0.15?

Watch and read more

scikit-learn: Linear modelsscikit-learn · reading

Lab

A linear regression on the diabetes dataset using one feature, scored on a held-out test set and compared with a baseline.

~20 min

The problem

The diabetes dataset has 442 patients, 10 measurements each (already scaled, so values are small numbers around 0), and a number measuring how the disease progressed one year later. This is a teaching dataset, not medical advice.

  1. Run the starter. It uses only the bmi column as the feature and splits the data.
  2. Create a LinearRegression named model and fit it on the training rows only.
  3. Compute test_mae: the mean absolute error of the model on the test rows.
  4. Compute baseline_mae: the test error if you predict the training average for every test row.
  5. Print both. Does the model beat the baseline? By how much?

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Output

No output. Add print(...) to see values.

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Why must the baseline use the training average and not the test average?Reveal

In real use you do not know the answers for new rows, so you cannot know their average. Using the test average would secretly give the baseline information from the test set. Both the model and the baseline must only use what was known at training time.

Q2Add a second feature, <code>s5</code>, so X has two columns. What do you expect, and how would you check it is not luck?Reveal

Likely the test error drops, since another measurement adds information. To check it is not luck of one split, repeat with several different random_state values, or use cross-validation (in the lesson on train, validation and test sets), and see if the gain holds most of the time.

Please sign in to continue.

Questions people ask

Why does scikit-learn want X as a 2-D table even for one feature?

Because the same code must work for one feature or a hundred. X is always rows by columns. reshape(-1, 1) turns a flat list into one column, or you can pass a DataFrame with one column.

Does a big slope mean a feature is important?

Not by itself. The slope depends on the unit. A slope per gram looks tiny next to a slope per kilogram. To compare features fairly, scale them first, and even then be careful: related features can share credit in odd ways.

Does linear regression show cause and effect?

No. It finds a pattern in the data. More ice-cream sales and more drownings rise together in summer, but one does not cause the other. Causes need careful study design, not just a fitted line.

What is random_state?

A number that fixes the random choices, such as which rows go to the test set. The same number gives the same split every time, so your results can be repeated.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress