Cover Image

Aioli: A Unified Optimization Framework for Language Model Data Mixing

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Mayee F., Hu, Michael Y., Lourie, Nicholas, Cho, Kyunghyun, Ré, Christopher
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Olmix: A Framework for Data Mixing Throughout LM Development
by: Chen, Mayee F., et al.
Published: (2026)

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
by: Hu, Michael Y., et al.
Published: (2026)

Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check
by: Lourie, Nicholas, et al.
Published: (2025)

Smoothie: Label Free Language Model Routing
by: Guha, Neel, et al.
Published: (2024)

Show Your Work with Confidence: Confidence Bands for Tuning Curves
by: Lourie, Nicholas, et al.
Published: (2023)

Hyperparameter Loss Surfaces Are Simple Near their Optima
by: Lourie, Nicholas, et al.
Published: (2025)

Language Models as Causal Effect Generators
by: Bynum, Lucius E. J., et al.
Published: (2024)

Neural Neural Scaling Laws
by: Hu, Michael Y., et al.
Published: (2026)

Archon: An Architecture Search Framework for Inference-Time Techniques
by: Saad-Falcon, Jon, et al.
Published: (2024)

Efficient semantic uncertainty quantification in language models via diversity-steered sampling
by: Park, Ji Won, et al.
Published: (2025)

Training Language Models with Language Feedback at Scale
by: Scheurer, Jérémy, et al.
Published: (2023)

Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
by: Ye, Jiasheng, et al.
Published: (2024)

Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates
by: Narayan, Avanika, et al.
Published: (2024)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
by: Ge, Ce, et al.
Published: (2024)

An Overview of Large Language Models for Statisticians
by: Ji, Wenlong, et al.
Published: (2025)

Rethinking Data Mixing from the Perspective of Large Language Models
by: Xu, Yuanjian, et al.
Published: (2026)

LoLCATs: On Low-Rank Linearizing of Large Language Models
by: Zhang, Michael, et al.
Published: (2024)

Unveiling Imitation Learning: Exploring the Impact of Data Falsity to Large Language Model
by: Cho, Hyunsoo
Published: (2024)

xVal: A Continuous Numerical Tokenization for Scientific Language Models
by: Golkar, Siavash, et al.
Published: (2023)

Preference Learning Algorithms Do Not Learn Preference Rankings
by: Chen, Angelica, et al.
Published: (2024)

Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models
by: Ghandeharioun, Asma, et al.
Published: (2024)

Language Models Optimized to Fool Detectors Still Have a Distinct Style (And How to Change It)
by: Soto, Rafael Rivera, et al.
Published: (2025)

Is Child-Directed Speech Effective Training Data for Language Models?
by: Feng, Steven Y., et al.
Published: (2024)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Translating Hanja Historical Documents to Contemporary Korean and English
by: Son, Juhee, et al.
Published: (2022)

A Unified Framework for Model Editing
by: Gupta, Akshat, et al.
Published: (2024)

Non-convolutional Graph Neural Networks
by: Wang, Yuanqing, et al.
Published: (2024)

Meta-Statistical Learning: Supervised Learning of Statistical Estimators
by: Peyrard, Maxime, et al.
Published: (2025)

Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
by: Opsahl-Ong, Krista, et al.
Published: (2024)

UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
by: Jin, Yiqiao, et al.
Published: (2026)

Training Greedy Policy for Proposal Batch Selection in Expensive Multi-Objective Combinatorial Optimization
by: Lee, Deokjae, et al.
Published: (2024)

AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models
by: Cheng, Jiale, et al.
Published: (2024)

SLOT: Sample-specific Language Model Optimization at Test-time
by: Hu, Yang, et al.
Published: (2025)

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models
by: Weng, Zixuan, et al.
Published: (2026)

Improving Code Generation by Training with Natural Language Feedback
by: Chen, Angelica, et al.
Published: (2023)

MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining
by: Wen, Bingbing, et al.
Published: (2026)

PromptBench: A Unified Library for Evaluation of Large Language Models
by: Zhu, Kaijie, et al.
Published: (2023)

Large Language Models as Optimizers
by: Yang, Chengrun, et al.
Published: (2023)

MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
by: Zhang, Shiyue, et al.
Published: (2023)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)