Quantization is a method that reduces the precision of a model’s weights and activations, leading to more efficient use of disk storage, less memory usage, and fewer compute requirements. This approach holds great promise for large language models (LLMs) looking to optimize performance on smaller hardware.