Small and on-device modelsMiddleLesson 2296 min read

Quantisation in practice

Quantisation stores each weight with fewer bits so a model needs less memory. See exactly what 8-bit and 4-bit mean with a tiny numpy example.

✦ Ask the tutor🧪 Lab · 30 min📝 Lesson test
Lesson in motion

In 60 seconds

Quantisation in practice

Quantisation stores each weight with fewer bits so a model needs less memory. See exactly what 8-bit and 4-bit mean with a tiny numpy example.

1/5
In simple words
Imagine measuring everyone's height in your class to the exact millimetre, then rounding each one to the nearest 5 cm. The list becomes much shorter to write, and you still know who is tall and who is short. That rounding is quantisation.
A model's weights are usually stored as 16-bit numbers. Quantisation stores them with fewer bits, such as 8 or 4. Fewer bits means a smaller file, less memory and often faster generation, because moving data from memory is a big part of the work. The price is a little rounding error in every weight.

What 8-bit and 4-bit really mean

FormatHow many different values it can storeMemory vs 16-bit
16-bit65,5361×
8-bit integer (int8)256about ½
4-bit16about ¼
Only 16 possible values sounds too few. The trick is a scale: you store small whole numbers plus one scale number that says how big each step is. To use the weight, you multiply the whole number by the scale. That is called dequantising.
  1. 1

    Find the range

    Look at a group of weights and find the biggest absolute value.
  2. 2

    Pick a scale

    For int8, scale = biggest value ÷ 127, so the biggest weight maps to 127.
  3. 3

    Round

    Divide each weight by the scale, round to the nearest whole number, and store it as int8.
  4. 4

    Dequantise when needed

    Multiply back by the scale to get a value close to the original.

A tiny example you can run

Python: quantise a small array to int8 and back (numpy)python
import numpy as np

x = np.array([0.82, -1.37, 0.05, 2.41, -0.66, 0.0, 1.10, -2.02], dtype=np.float32)

# 1. One scale for the whole array: the largest value maps to 127
scale = np.abs(x).max() / 127

# 2. Quantise: divide, round, clip to the int8 range, store as int8
q = np.clip(np.round(x / scale), -127, 127).astype(np.int8)

# 3. Dequantise: multiply back by the scale
x_back = q.astype(np.float32) * scale

print("scale     :", round(float(scale), 5))
print("int8      :", q)
print("restored  :", np.round(x_back, 3))
print("max error :", float(np.abs(x - x_back).max()))
print("bytes     : float32 =", x.nbytes, "| int8 =", q.nbytes, "+ one scale")
Run it and you will see the restored numbers are very close to the originals, while each value now takes 1 byte instead of 4. The largest error is at most half a step (half of the scale).

Why outliers hurt, and why blocks help

If one weight is much bigger than the rest, the scale becomes large, and all the small weights get rounded to the same few values. Real formats deal with this in several ways. A common one is to quantise in small blocks, each with its own scale, so one big value only affects its own block. Many GGUF formats and other 4-bit methods work this way. Some methods also look at real example data to decide which weights matter most.

Quality vs size

  • 8-bit is often very close to the original for many models and tasks.
  • 4-bit saves much more memory and is popular for laptops and phones, with a larger but often acceptable drop in quality.
  • Below 4 bits, quality often falls faster. Smaller models can be more sensitive than big ones.
  • The only reliable answer for your task is your own test set, run on each version.
Risky
  • Choosing the smallest file because it downloads faster
  • Checking quality with two or three chat questions
  • Comparing a 4-bit model against a different model at 16-bit
Sound
  • Picking the lowest bits that still pass your test set
  • Testing on your real task, in your users' languages
  • Comparing the same model at different precisions on the same test
Note
The quantisation formats table and methods like GPTQ and AWQ are covered in "Step 17 · Evaluate, shrink, serve". This lesson is about understanding what happens to the numbers.
Try this. In the code, change 2.41 to 50.0 and run it again. What happens to the small values like 0.05 and -0.66?

Watch and read more

llama.cpp (includes quantisation tools)ggml-org · reading

Lab

Your own int8 and 4-bit quantiser in numpy, with measured errors and an outlier experiment.

~30 min

The problem

  1. Run the starter code. Note the int8 maximum error.
  2. Add a 4-bit version: use the range -7 to 7, so the scale is the largest value ÷ 7. Compare its maximum error with int8.
  3. Add one outlier (for example 50.0) to the array. Record both errors again. What happened to the small values?
  4. Now quantise in blocks of 4 values, each block with its own scale. Record the errors again. Write three sentences on what you learned.

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Output

No output. Add print(...) to see values.

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Blocks with their own scales reduce error. Why not make every block just one value long?Reveal

Each block needs its own scale, and the scale takes memory too, often more bits than the value itself. With one value per block you would store a full scale for every weight and save nothing. Real formats pick a block size that balances accuracy against the extra memory for scales.

Q2Quantising a model to 4-bit makes it about four times smaller than 16-bit. Why is generation often faster too, not only smaller?Reveal

When a model generates text, it must read its weights from memory for every new token. Often the chip can do the maths faster than memory can deliver the numbers. Smaller weights mean less data to move, so each token can be produced sooner. How much faster depends on the hardware and runtime.

Please sign in to continue.

Questions people ask

Does quantisation change what the model knows?

It does not remove facts on purpose, but the rounding adds small errors to every weight. Most of the time the answers stay similar. Sometimes, especially at very low bits, answers get worse in ways that are hard to predict.

Can I quantise a model myself?

Yes. llama.cpp includes tools to convert and quantise models into GGUF, and there are Python libraries for other formats. Many publishers and community members also share ready-made quantised files, but check that the source is trustworthy.

Is 4-bit always faster than 8-bit?

Often, because less data has to move from memory. But it depends on the hardware and the runtime. Some chips have fast support for certain formats and not others. Measure on your own machine.

Why clip to -127 and not -128?

int8 can store -128, but using -127 to 127 keeps the range the same on both sides, so positive and negative values are treated equally. It is a common simple choice, not the only one.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress