Image de couverture de livre

Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chen, Xuxi, Wang, Zhendong, Sow, Daouda, Yang, Junjie, Chen, Tianlong, Liang, Yingbin, Zhou, Mingyuan, Wang, Zhangyang
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Rethinking PGD Attack: Is Sign Function Necessary?
par: Yang, Junjie, et autres
Publié: (2023)

Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining
par: Sow, Daouda, et autres
Publié: (2025)

Algorithm Design for Online Meta-Learning with Task Boundary Detection
par: Sow, Daouda, et autres
Publié: (2023)

Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
par: Yang, Hongru, et autres
Publié: (2024)

Meta ControlNet: Enhancing Task Adaptation via Meta Learning
par: Yang, Junjie, et autres
Publié: (2023)

LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning
par: Perin, Gabriel J., et autres
Publié: (2025)

Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK
par: Yang, Hongru, et autres
Publié: (2023)

Diffusion Policies creating a Trust Region for Offline Reinforcement Learning
par: Chen, Tianyu, et autres
Publié: (2024)

Enhancing and Accelerating Diffusion-Based Inverse Problem Solving through Measurements Optimization
par: Chen, Tianyu, et autres
Publié: (2024)

Make Optimization Once and for All with Fine-grained Guidance
par: Shi, Mingjia, et autres
Publié: (2025)

Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay
par: Sun, Yifan, et autres
Publié: (2025)

Enhancing Adversarial Training via Reweighting Optimization Trajectory
par: Huang, Tianjin, et autres
Publié: (2023)

Sparse Mixture-of-Experts for Compositional Generalization: Empirical Evidence and Theoretical Foundations of Optimal Sparsity
par: Zhao, Jinze, et autres
Publié: (2024)

Improving Sample Efficiency of Model-Free Algorithms for Zero-Sum Markov Games
par: Feng, Songtao, et autres
Publié: (2023)

Few-Step Diffusion via Score identity Distillation
par: Zhou, Mingyuan, et autres
Publié: (2025)

Neon: Negative Extrapolation From Self-Training Improves Image Generation
par: Alemohammad, Sina, et autres
Publié: (2025)

Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generation
par: Zhou, Mingyuan, et autres
Publié: (2024)

Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts
par: Yin, Yueqin, et autres
Publié: (2024)

In-Context Learning with Representations: Contextual Generalization of Trained Transformers
par: Yang, Tong, et autres
Publié: (2024)

DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
par: Li, Chaojian, et autres
Publié: (2021)

On the Continuity of Schur-Horn Mapping
par: Chen, Hengzhun, et autres
Publié: (2024)

Distilled Protein Backbone Generation
par: Xie, Liyang, et autres
Publié: (2025)

Non-asymptotic Convergence of Training Transformers for Next-token Prediction
par: Huang, Ruiquan, et autres
Publié: (2024)

You Only Debias Once: Towards Flexible Accuracy-Fairness Trade-offs at Inference Time
par: Han, Xiaotian, et autres
Publié: (2025)

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
par: Yin, Yueqin, et autres
Publié: (2024)

Score Distillation Beyond Acceleration: Generative Modeling from Corrupted Data
par: Zhang, Yasi, et autres
Publié: (2025)

Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation
par: Zhou, Mingyuan, et autres
Publié: (2024)

Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation
par: Chen, Tianyu, et autres
Publié: (2025)

How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias
par: Huang, Ruiquan, et autres
Publié: (2025)

Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training
par: Zhang, Jiacheng, et autres
Publié: (2024)

Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods
par: Zhao, Wanru, et autres
Publié: (2026)

LLM-AutoDiff: Auto-Differentiate Any LLM Workflow
par: Yin, Li, et autres
Publié: (2025)

Improved Focus on Hard Samples for Lung Nodule Detection
par: Chen, Yujiang, et autres
Publié: (2024)

CPL: Critical Plan Step Learning Boosts LLM Generalization in Reasoning Tasks
par: Wang, Tianlong, et autres
Publié: (2024)

IFFair: Influence Function-driven Sample Reweighting for Fair Classification
par: Yang, Jingran, et autres
Publié: (2025)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
par: Zhao, Jiawei, et autres
Publié: (2024)

Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning
par: Yang, Puning, et autres
Publié: (2025)

Hard Constraints Meet Soft Generation: Guaranteed Feasibility for LLM-based Combinatorial Optimization
par: Liu, Yang, et autres
Publié: (2026)

Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity
par: Liu, Shiwei, et autres
Publié: (2021)

On the Effectiveness of Training Data Optimization for LLM-based Code Generation: An Empirical Study
par: Kuang, Shiqi, et autres
Publié: (2025)