Emerging techniques knowledge distillation
Guides knowledge distillation of large language models into smaller student models, covering temperature scaling, logit and response distillation, reverse KLD, strategy selection, and paper references. Use when the user asks how to compress or distill an LLM,