A Record of Learning About Quantization and Deployment Techniques for LLMs in Edge AI

I am compiling this memorandum based on related materials to document what I have learned about quantization and deployment ...
Nature

Quantization Techniques in Neural Network Inference

Quantization in neural network inference refers to the process of mapping high-precision parameters and activations to lower-precision representations, typically using integer or even binary values.