EasyDistill Packages LLM Distillation, But Its Top-k Shortcut Has A Cost
Most teams that want a smaller, cheaper version of a large language model end up stitching together the same handful of techniques by hand, supervised fine-tuning on a bigger model’s outputs, maybe a logit matching loss if they have access…
EasyDistill Packages LLM Distillation, But Its Top-k Shortcut Has A Cost Read More »