Image de couverture de livre

What Should Baby Models Read? Exploring Sample-Efficient Data Composition on Model Performance

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yam, Hong Meng, Paek, Nathan J
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Mini Minds: Exploring Bebeshka and Zlata Baby Models
par: Proskurina, Irina, et autres
Publié: (2023)

Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective
par: Gonzalez, Emmanuel Anaya, et autres
Publié: (2025)

Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models
par: Zhou, Ke, et autres
Publié: (2025)

BabyVLM: Data-Efficient Pretraining of VLMs Inspired by Infant Learning
par: Wang, Shengao, et autres
Publié: (2025)

Should You Use Your Large Language Model to Explore or Exploit?
par: Harris, Keegan, et autres
Publié: (2025)

When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets?
par: Iyer, Srikrishna
Publié: (2024)

Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning
par: Zhao, Jun, et autres
Publié: (2024)

Repetition over Diversity: High-Signal Data Filtering for Sample-Efficient German Language Modeling
par: Aynetdinov, Ansar, et autres
Publié: (2026)

Thinking in Many Modes: How Composite Reasoning Elevates Large Language Model Performance with Limited Data
par: Ahmad, Zishan, et autres
Publié: (2025)

Bias Dynamics in BabyLMs: Towards a Compute-Efficient Sandbox for Democratising Pre-Training Debiasing
par: Trhlik, Filip, et autres
Publié: (2026)

Baby Scale: Investigating Models Trained on Individual Children's Language Input
par: Feng, Steven Y., et autres
Publié: (2026)

Learning to Plan for Language Modeling from Unlabeled Data
par: Cornille, Nathan, et autres
Publié: (2024)

Balanced Data Sampling for Language Model Training with Clustering
par: Shao, Yunfan, et autres
Publié: (2024)

Models Can and Should Embrace the Communicative Nature of Human-Generated Math
par: Boguraev, Sasha, et autres
Publié: (2024)

Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements
par: Haller, Patrick, et autres
Publié: (2025)

Exploring Data-Efficient Adaptation of Large Language Models for Code Generation
par: Jiang, Xue, et autres
Publié: (2024)

Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs
par: Smit, Andries, et autres
Publié: (2023)

What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis
par: Wang, Peiran, et autres
Publié: (2025)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
par: Zhang, Xuemiao, et autres
Publié: (2025)

Target-Aware Language Modeling via Granular Data Sampling
par: Chang, Ernie, et autres
Publié: (2024)

Exploring the Performance of Large Language Models on Subjective Span Identification Tasks
par: Dmonte, Alphaeus, et autres
Publié: (2026)

Express Your Doubts -- Probabilistic World Modeling Should not be Based on Token logprobs
par: Wagner, Eitan, et autres
Publié: (2025)

You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models
par: Mąka, Paweł, et autres
Publié: (2025)

PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Sampling
par: Xie, Xudong, et autres
Publié: (2024)

Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?
par: Hase, Peter, et autres
Publié: (2024)

Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning
par: Zhang, Yang, et autres
Publié: (2025)

Exploring Data and Parameter Efficient Strategies for Arabic Dialect Identifications
par: Kanjirangat, Vani, et autres
Publié: (2025)

Dense X Retrieval: What Retrieval Granularity Should We Use?
par: Chen, Tong, et autres
Publié: (2023)

BabyHGRN: Exploring RNNs for Sample-Efficient Training of Language Models
par: Haller, Patrick, et autres
Publié: (2024)

Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement
par: Kong, Injin, et autres
Publié: (2026)

PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
par: Ranaldi, Leonardo, et autres
Publié: (2023)

Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs
par: von Recum, Alexander, et autres
Publié: (2024)

Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models
par: Karov, Bar, et autres
Publié: (2025)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
par: Guo, Ping, et autres
Publié: (2025)

WebDS: An End-to-End Benchmark for Web-based Data Science
par: Hsu, Ethan, et autres
Publié: (2025)

Exploring the Knowledge Mismatch Hypothesis: Hallucination Propensity in Small Models Fine-tuned on Data from Larger Models
par: Wee, Phil, et autres
Publié: (2024)

Where Should I Study? Biased Language Models Decide! Evaluating Fairness in LMs for Academic Recommendations
par: Shailya, Krithi, et autres
Publié: (2025)

S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models
par: He, Tao, et autres
Publié: (2025)

FBS: Modeling Native Parallel Reading inside a Transformer
par: Wang, Tongxi
Publié: (2026)

What Should Embeddings Embed? Autoregressive Models Represent Latent Generating Distributions
par: Zhang, Liyi, et autres
Publié: (2024)