What is quantization in the context of AI models?

Reducing Precision, Not Reducing the Model Itself

Quantization is one of the most widely used techniques for making AI models smaller and faster. Rather than removing parts of a model, it changes how each parameter is numerically represented. AI models are typically trained using high-precision numbers, often 32-bit floating point values, which allow for very fine-grained distinctions between values. Quantization converts these numbers into lower-precision formats, like 16-bit, 8-bit, or even smaller representations, which take up less memory and can often be processed faster.

Think of it like rounding measurements. A model trained with 32-bit precision might store a parameter as a very specific decimal number. Quantization rounds that number to a coarser scale, similar to switching from measuring in millimeters to measuring in centimeters. You lose some fine detail, but for many purposes the loss is small enough not to matter, while the savings in storage and computation are substantial.

Why Lower Precision Still Produces Useful Models

It might seem surprising that reducing numerical precision doesn’t ruin a model’s performance, but neural networks tend to be somewhat tolerant of small numerical imprecision, since their outputs emerge from combining huge numbers of parameters together. Small rounding effects in individual parameters often average out or have limited impact on the model’s overall behavior, especially at moderate quantization levels like 8-bit.

There are two broad ways quantization gets applied. Post-training quantization takes a model that has already finished training and reduces its precision afterward, which is simpler but can sometimes cost more in accuracy. Quantization-aware training instead incorporates the lower precision into the training process itself, allowing the model to adjust and compensate for the reduced precision as it learns, generally preserving more of the original accuracy at the cost of a more involved training process.

The Practical Payoff: Smaller, Faster, Cheaper

The benefits of quantization show up directly in deployment. A quantized model takes up less disk space and memory, which matters both for data center efficiency and for running AI on devices with limited storage, like phones or laptops. It can also run faster, particularly on hardware specifically designed to accelerate lower-precision arithmetic, since processing an 8-bit number generally requires less computational work than a 32-bit one. This combination of smaller size and faster computation is a major reason quantization is one of the first techniques applied when adapting a large model for more efficient or wider deployment.

Bottom Line

Quantization shrinks AI models by reducing the numerical precision used to represent their parameters rather than changing the model’s structure. It’s one of the most common and effective compression techniques because it typically delivers meaningful savings in memory and speed with only a modest, often acceptable, cost to accuracy.

Go deeper

• Why do some ai models require significantly more memory to run than others of similar size?
• What does training a model actually mean at a basic level?
• What Hardware Do You Need to Run an Open-Source AI Model Yourself?