ToDi, Per Token KL Divergence Control for LLM Distillation.

ToDi: Per Token KL Divergence Control for LLM Distillation

A seven billion parameter model writes a clean, helpful answer, then you try to ship it. It will not fit on the phone, the latency blows past your budget, and the energy draw is not something a small product can…

ToDi: Per Token KL Divergence Control for LLM Distillation Read More »