Embarrassingly Simple Self-Distillation Improves Code Generation

Can a large language model (LLM) improve at code generation using only its own raw outputs, without a verifier, a teacher model, or reinforcement learning? We answer in the affirmative with simple self-distillation (SSD): sample solutions from the model with certain temperature and truncation configurations, then fine-tune on those samples with standard supervised fine-tuning. SSD improves Qwen3-3

[misshiki]

自己蒸留（SSD）でコード生成性能向上。モデル自身の出力で再学習し、Qwen3-30Bのpass@1が42.4→55.3%。検証器・RL不要。精度と探索のトレードオフ再調整が効果。