:: Library Catalog

Imagen de Portada

Guardado en:

Detalles Bibliográficos
Autores principales:	Fan, Simin, Grangier, David, Ablin, Pierre
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language
Acceso en línea:	https://arxiv.org/abs/2410.02498
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
por: Grangier, David, et al.
Publicado: (2024)

Need a Small Specialized Language Model? Plan Early!
por: Grangier, David, et al.
Publicado: (2024)

Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
por: Ablin, Pierre, et al.
Publicado: (2025)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
por: Bethune, Louis, et al.
Publicado: (2025)

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
por: Saada, Thiziri Nait, et al.
Publicado: (2025)

Optimal Splitting of Language Models from Mixtures to Specialized Domains
por: Seto, Skyler, et al.
Publicado: (2026)

The AdEMAMix Optimizer: Better, Faster, Older
por: Pagliardini, Matteo, et al.
Publicado: (2024)

Scaling Laws for Mixture Pretraining Under Data Constraints
por: Sedova, Anastasiia, et al.
Publicado: (2026)

Nectar: Neural Estimation of Cached-Token Attention via Regression
por: Monteiro, João, et al.
Publicado: (2026)

Training Bilingual LMs with Data Constraints in the Targeted Language
por: Seto, Skyler, et al.
Publicado: (2024)

No Need to Talk: Asynchronous Mixture of Language Models
por: Filippova, Anastasiia, et al.
Publicado: (2024)

Scaling Laws for Optimal Data Mixtures
por: Shukor, Mustafa, et al.
Publicado: (2025)

Pretraining with hierarchical memories: separating long-tail and common knowledge
por: Pouransari, Hadi, et al.
Publicado: (2025)

Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment
por: Lu, Keming, et al.
Publicado: (2024)

The Geometries of Truth Are Orthogonal Across Tasks
por: Azizian, Waiss, et al.
Publicado: (2025)

DoGE: Domain Reweighting with Generalization Estimation
por: Fan, Simin, et al.
Publicado: (2023)

Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees
por: Wu, Yongtao, et al.
Publicado: (2025)

Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection
por: Sun, Guanglong, et al.
Publicado: (2026)

Why Is RLHF Alignment Shallow? A Gradient Analysis
por: Young, Robin
Publicado: (2026)

Robust Multi-Objective Preference Alignment with Online DPO
por: Gupta, Raghav, et al.
Publicado: (2025)

Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learning
por: Fan, Ziqing, et al.
Publicado: (2025)

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models
por: Wang, Fei, et al.
Publicado: (2024)

DSPA: Dynamic SAE Steering for Data-Efficient Preference Alignment
por: Wedgwood, James, et al.
Publicado: (2026)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
por: Imai, Saki, et al.
Publicado: (2026)

MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
por: Wang, Tianze, et al.
Publicado: (2025)

Value Alignment from Unstructured Text
por: Padhi, Inkit, et al.
Publicado: (2024)

Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimization
por: Yang, Junming, et al.
Publicado: (2025)

Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation
por: Chen, Simin, et al.
Publicado: (2025)

A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability
por: Cao, Chengtai, et al.
Publicado: (2022)

A Common Pitfall of Margin-based Language Model Alignment: Gradient Entanglement
por: Yuan, Hui, et al.
Publicado: (2024)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
por: Abbes, Istabrak, et al.
Publicado: (2025)

Efficient Alignment of Large Language Models via Data Sampling
por: Khera, Amrit, et al.
Publicado: (2024)

SAIL: Self-Improving Efficient Online Alignment of Large Language Models
por: Ding, Mucong, et al.
Publicado: (2024)

Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
por: Lu, Yining, et al.
Publicado: (2025)

Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
por: Rannen-Triki, Amal, et al.
Publicado: (2024)

Progressive Mixed-Precision Decoding for Efficient LLM Inference
por: Chen, Hao Mark, et al.
Publicado: (2024)

When Inverse Data Outperforms: Exploring the Pitfalls of Mixed Data in Multi-Stage Fine-Tuning
por: Deng, Mengyi, et al.
Publicado: (2025)

Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs
por: Gallego, Víctor
Publicado: (2024)

Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models
por: Li, Chengao, et al.
Publicado: (2025)

Alignment-Constrained Dynamic Pruning for LLMs: Identifying and Preserving Alignment-Critical Circuits
por: Patel, Dev, et al.
Publicado: (2025)