DOPD: Dual On-policy Distillation
Xinlei Yu, Gen Li, Qingyi Si, Guibin Zhang, Yuqi Xu, Congcong Wang, Shuai Dong, Kaiwen Tuo, Xiangyu Zeng, Kaituo Feng, Qunzhong Wang, Yang Shi, Xiaobin Hu, Xiangyu Yue, Jiaqi Wang, Shuicheng Yan
Read on arXiv →
Key claim
DOPD improves distillation by managing privileged information effectively.
In plain English
Imagine you're trying to teach a model to make decisions based on examples it sees, like a student learning from a teacher. In machine learning, we often use a method called distillation, where a 'teacher' model guides a 'student' model to learn from its outputs. However, this can lead to problems when the student gets confused by extra information that it can't fully understand — this is known as privilege illusion. It happens because the student might mix up what it can actually learn from the teacher with what it can't replicate on its own. This confusion can hinder the learning process, especially when only a few pieces of information are truly valuable for making decisions.
To tackle this, the authors propose a new method called DOPD, which stands for advantage-aware dual distillation. The idea is to smartly manage how the student learns from the teacher by adjusting the strength and type of guidance it receives based on its current understanding. This way, the student can focus on the most important signals and avoid getting lost in the noise of irrelevant information. The results show that DOPD consistently outperforms traditional distillation methods, making it a promising approach for improving model training in both language and vision tasks.
For anyone building models, this means you can achieve better performance by using this new method of distillation, especially in complex scenarios where information can be misleading.
The approach introduces a new paradigm for distillation that leverages privileged information to enhance performance.
The claims are supported by extensive experiments across multiple model settings, demonstrating consistent improvements.
Deep reliability assessment
The methodology supports the claim that token-level routing between privileged teacher and privileged self-supervision can outperform vanilla OPD in the authors' Qwen3/Qwen3-VL distillation setups. The broader claim that DOPD generally alleviates "privilege illusion" is plausible but somewhat overclaimed unless the paper directly isolates information-asymmetry effects across more privilege types, model families, and production-like data regimes.
Reproducibility
No code repository is mentioned in the provided abstract, introduction, results, or conclusion excerpts. The paper uses named public benchmarks and includes prompts for privileged input generation, but the provided text is insufficient to confirm full reproducibility details such as training data, exact hyperparameters, checkpoints, or scripts.
Key figure
Figure 1 compares DOPD with vanilla OPD, teacher/student policies, and other OPD variants across LLM and VLM benchmark suites, showing DOPD with the highest reported average performance in both settings.