Progressive$^2$：渐进式师生协同进化的知识蒸馏方法
原标题：Progressive$^2$: A Teacher-Student Progressive Co-Evolving Knowledge Distillation Method for Substantial Model Compression
AI 摘要
arXiv 上发表了一篇名为 Progressive$^2$ 的论文，提出了一种新的知识蒸馏方法，通过渐进增强教师模型和渐进缩小学生模型来协同进化，以解决服务器与客户端能力差距大时蒸馏性能下降的问题。该方法包含教师侧的多特征融合适配器和学生侧的渐进式网络缩减，可灵活部署，旨在实现模型压缩与性能的平衡。
正文节选
Computer Science > Machine Learning Title:Progressive$^2$: A Teacher-Student Progressive Co-Evolving Knowledge Distillation Method for Substantial Model Compression View PDF HTML (experimental) Abstract:Knowledge distillation (KD) is a widely utilized technique for transferring knowledge from a large model (the teacher) to a smaller model (the student). Owing to its flexibility and broad applicability, KD has been extensively applied in the compression of server-side models to meet t