Image de couverture de livre

Establishing Task Scaling Laws via Compute-Efficient Model Ladders

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bhagia, Akshita, Liu, Jiacheng, Wettig, Alexander, Heineman, David, Tafjord, Oyvind, Jha, Ananya Harsh, Soldaini, Luca, Smith, Noah A., Groeneveld, Dirk, Koh, Pang Wei, Dodge, Jesse, Hajishirzi, Hannaneh
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DataDecide: How to Predict Best Pretraining Data with Small Experiments
par: Magnusson, Ian, et autres
Publié: (2025)

Paloma: A Benchmark for Evaluating Language Model Fit
par: Magnusson, Ian, et autres
Publié: (2023)

OLMES: A Standard for Language Model Evaluations
par: Gu, Yuling, et autres
Publié: (2024)

What's In My Big Data?
par: Elazar, Yanai, et autres
Publié: (2023)

Organize the Web: Constructing Domains Enhances Pre-Training Data Curation
par: Wettig, Alexander, et autres
Publié: (2025)

Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging
par: Morrison, Jacob, et autres
Publié: (2024)

Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions
par: Wiegreffe, Sarah, et autres
Publié: (2024)

OLMoE: Open Mixture-of-Experts Language Models
par: Muennighoff, Niklas, et autres
Publié: (2024)

Fluid Language Model Benchmarking
par: Hofmann, Valentin, et autres
Publié: (2025)

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
par: Heineman, David, et autres
Publié: (2025)

Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training
par: Merrill, William, et autres
Publié: (2025)

Olmix: A Framework for Data Mixing Throughout LM Development
par: Chen, Mayee F., et autres
Publié: (2026)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
par: Soldaini, Luca, et autres
Publié: (2024)

EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Trees
par: Zeng, Zhiyuan, et autres
Publié: (2025)

FlexOlmo: Open Language Models for Flexible Data Use
par: Shi, Weijia, et autres
Publié: (2025)

Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index
par: Xu, Hao, et autres
Publié: (2025)

EMO: Pretraining Mixture of Experts for Emergent Modularity
par: Wang, Ryan, et autres
Publié: (2026)

OLMo: Accelerating the Science of Language Models
par: Groeneveld, Dirk, et autres
Publié: (2024)

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
par: Na, Clara, et autres
Publié: (2024)

ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
par: Chen, Tong, et autres
Publié: (2025)

Because we have LLMs, we Can and Should Pursue Agentic Interpretability
par: Kim, Been, et autres
Publié: (2025)

Digital Socrates: Evaluating LLMs through Explanation Critiques
par: Gu, Yuling, et autres
Publié: (2023)

Just CHOP: Embarrassingly Simple LLM Compression
par: Jha, Ananya Harsh, et autres
Publié: (2023)

2 OLMo 2 Furious
par: OLMo, Team, et autres
Publié: (2024)

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens
par: Liu, Jiacheng, et autres
Publié: (2024)

Train Separately, Merge Together: Modular Post-Training with Mixture-of-Experts
par: Morrison, Jacob, et autres
Publié: (2026)

BaRDa: A Belief and Reasoning Dataset that Separates Factual Accuracy and Reasoning Ability
par: Clark, Peter, et autres
Publié: (2023)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
par: Zhao, Bowen, et autres
Publié: (2024)

Olmo Hybrid: From Theory to Practice and Back
par: Merrill, William, et autres
Publié: (2026)

Set the Clock: Temporal Alignment of Pretrained Language Models
par: Zhao, Bowen, et autres
Publié: (2024)

Neologism Learning for Controllability and Self-Verbalization
par: Hewitt, John, et autres
Publié: (2025)

Reliable, Adaptable, and Attributable Language Models with Retrieval
par: Asai, Akari, et autres
Publié: (2024)

OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens
par: Liu, Jiacheng, et autres
Publié: (2025)

Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
par: Liu, Jiacheng, et autres
Publié: (2023)

ComPO: Community Preferences for Language Model Personalization
par: Kumar, Sachin, et autres
Publié: (2024)

Biased or Flawed? Mitigating Stereotypes in Generative Language Models by Addressing Task-Specific Flaws
par: Jha, Akshita, et autres
Publié: (2024)

AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters
par: Lucy, Li, et autres
Publié: (2024)

Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
par: Kim, Joongwon, et autres
Publié: (2024)

ScienceMeter: Tracking Scientific Knowledge Updates in Language Models
par: Wang, Yike, et autres
Publié: (2025)

HREF: Human Response-Guided Evaluation of Instruction Following in Language Models
par: Lyu, Xinxi, et autres
Publié: (2024)