On-Policy Self-Distillation

Papers on training models to distill their own knowledge during on-policy learning, where a model learns from its own generated reasoning or behaviors without requiring a separate teacher network.