Cover Image

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Yuhang, Zhu, Jing, Xu, Paiheng, Liu, Xiaoyu, Wang, Xiyao, Koutra, Danai, Ai, Wei, Huang, Furong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Explore Spurious Correlations at the Concept Level in Language Models for Text Classification
by: Zhou, Yuhang, et al.
Published: (2023)

DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Data
by: Zhou, Yuhang, et al.
Published: (2025)

GFairHint: Improving Individual Fairness for Graph Neural Networks via Fairness Hint
by: Xu, Paiheng, et al.
Published: (2023)

Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications
by: Zhou, Yuhang, et al.
Published: (2024)

Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios
by: Zhou, Yuhang, et al.
Published: (2024)

Unveiling the Impact of Local Homophily on GNN Fairness: In-Depth Analysis and New Benchmarks
by: Loveland, Donald, et al.
Published: (2024)

Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning
by: Zhu, Jing, et al.
Published: (2024)

Tackling Size Generalization of Graph Neural Networks on Biological Data from a Spectral Perspective
by: Li, Gaotang, et al.
Published: (2023)

Random Search Neural Networks for Efficient and Expressive Graph Learning
by: Ito, Michael, et al.
Published: (2025)

GRAPHTEXTACK: A Realistic Black-Box Node Injection Attack on LLM-Enhanced GNNs
by: Ma, Jiaji, et al.
Published: (2025)

Understanding GNNs and Homophily in Dynamic Node Classification
by: Ito, Michael, et al.
Published: (2025)

Glance for Context: Learning When to Leverage LLMs for Node-Aware GNN-LLM Fusion
by: Loveland, Donald, et al.
Published: (2025)

LinkGPT: Teaching Large Language Models To Predict Missing Links
by: He, Zhongmou, et al.
Published: (2024)

BiFedKD: Bidirectional Federated Knowledge Distillation Framework for Non-IID and Long-Tailed ECG Monitoring
by: Shu, Zixuan, et al.
Published: (2026)

Knowledge Distillation in Federated Learning: A Survey on Long Lasting Challenges and New Solutions
by: Laiqiao Qin, et al.
Published: (2025)

Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions
by: Qin, Laiqiao, et al.
Published: (2024)

TouchUp-G: Improving Feature Representation through Graph-Centric Finetuning
by: Zhu, Jing, et al.
Published: (2023)

CSRec: Rethinking Sequential Recommendation from A Causal Perspective
by: Liu, Xiaoyu, et al.
Published: (2024)

On the Impact of Feature Heterophily on Link Prediction with Graph Neural Networks
by: Zhu, Jiong, et al.
Published: (2024)

VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition
by: Zhang, Zaiwei, et al.
Published: (2024)

Balance Divergence for Knowledge Distillation
by: Qi, Yafei, et al.
Published: (2025)

Integrating Knowledge Distillation Methods: A Sequential Multi-Stage Framework
by: Tian, Yinxi, et al.
Published: (2026)

Integrated Multi-Level Knowledge Distillation for Enhanced Speaker Verification
by: Yang, Wenhao, et al.
Published: (2024)

The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education
by: Xu, Paiheng, et al.
Published: (2024)

Memorization Inheritance in Sequence-Level Knowledge Distillation for Neural Machine Translation
by: Dankers, Verna, et al.
Published: (2025)

Beyond Unimodal Boundaries: Generative Recommendation with Multimodal Semantics
by: Zhu, Jing, et al.
Published: (2025)

Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences
by: Wang, Xiyao, et al.
Published: (2024)

Learn from Balance: Rectifying Knowledge Transfer for Long-Tailed Scenarios
by: Huang, Xinlei, et al.
Published: (2024)

Learning Laplacian Positional Encodings for Heterophilous Graphs
by: Ito, Michael, et al.
Published: (2025)

Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models
by: Cui, Xiao, et al.
Published: (2024)

Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation
by: Wei, Jingxuan, et al.
Published: (2024)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
by: Fang, Luyang, et al.
Published: (2025)

Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networks
by: Zhang, Tianqing, et al.
Published: (2025)

Large Language Models and Causal Inference in Collaboration: A Survey
by: Liu, Xiaoyu, et al.
Published: (2024)

Parameter Efficient Diverse Paraphrase Generation Using Sequence-Level Knowledge Distillation
by: Jayawardena, Lasal, et al.
Published: (2024)

Target-Balanced Score Distillation
by: Xu, Zhou, et al.
Published: (2025)

GATES: Self-Distillation under Privileged Context with Consensus Gating
by: Stein, Alex, et al.
Published: (2026)

Multi-Label Knowledge Distillation
by: Yang, Penghui, et al.
Published: (2023)

Early Exit and Multi Stage Knowledge Distillation in VLMs for Video Summarization
by: Khan, Anas Anwarul Haq, et al.
Published: (2025)

Multimodal Distillation-Driven Ensemble Learning for Long-Tailed Histopathology Whole Slide Images Analysis
by: Ling, Xitong, et al.
Published: (2025)