Cover Image

Bias Dynamics in BabyLMs: Towards a Compute-Efficient Sandbox for Democratising Pre-Training Debiasing

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Trhlik, Filip, Caines, Andrew, Buttery, Paula
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Quantifying Generative Media Bias with a Corpus of Real-world and Generated News Articles
by: Trhlik, Filip, et al.
Published: (2024)

BabyLM's First Words: Word Segmentation as a Phonological Probing Task
by: Goriely, Zébulon, et al.
Published: (2025)

Are BabyLMs Second Language Learners?
by: Edman, Lukas, et al.
Published: (2024)

Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
by: Ganescu, Bianca-Mihaela, et al.
Published: (2025)

Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction
by: Salhan, Suchir, et al.
Published: (2025)

Mitigating Frequency Bias and Anisotropy in Language Model Pre-Training with Syntactic Smoothing
by: Martinez, Richard Diehl, et al.
Published: (2024)

Child-directed speech facilitates production, not comprehension, in BabyLMs
by: Bunzeck, Bastian, et al.
Published: (2026)

BabyLMs for isiXhosa: Data-Efficient Language Modelling in a Low-Resource Context
by: Matzopoulos, Alexis, et al.
Published: (2025)

Less is More: Pre-Training Cross-Lingual Small-Scale Language Models with Cognitively-Plausible Curriculum Learning Strategies
by: Salhan, Suchir, et al.
Published: (2024)

CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs
by: Capone, Luca, et al.
Published: (2025)

Do Construction Distributions Shape Formal Language Learning In German BabyLMs?
by: Bunzeck, Bastian, et al.
Published: (2025)

Mask and You Shall Receive: Optimizing Masked Language Modeling For Pretraining BabyLMs
by: Edman, Lukas, et al.
Published: (2025)

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes
by: Goriely, Zébulon, et al.
Published: (2024)

Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models
by: Zeng, Linda, et al.
Published: (2026)

Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
by: Askari, Raha, et al.
Published: (2025)

BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop
by: Charpentier, Lucas, et al.
Published: (2025)

Learning Dynamics of Meta-Learning in Small Model Pretraining
by: Africa, David Demitri, et al.
Published: (2025)

Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models
by: Weiss, Yuval, et al.
Published: (2025)

BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop
by: Choshen, Leshem, et al.
Published: (2026)

Rethinking AI Cultural Alignment
by: Bravansky, Michal, et al.
Published: (2025)

Experiments in News Bias Detection with Pre-Trained Neural Transformers
by: Menzner, Tim, et al.
Published: (2024)

Characterizing Stereotypical Bias from Privacy-preserving Pre-Training
by: Arnold, Stefan, et al.
Published: (2024)

Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
by: Damani, Mehul, et al.
Published: (2025)

BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction
by: Li, Jiangmeng, et al.
Published: (2024)

Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
by: Prashanth, USVSN Sai, et al.
Published: (2024)

Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts
by: Lin, Yujie, et al.
Published: (2026)

Word Sense Linking: Disambiguating Outside the Sandbox
by: Bejgu, Andrei Stefan, et al.
Published: (2024)

Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages
by: Africa, David Demitri, et al.
Published: (2025)

BLiSS 1.0: Evaluating Bilingual Learner Competence in Second Language Small Language Models
by: Gao, Yuan, et al.
Published: (2025)

Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
by: Warstadt, Alex, et al.
Published: (2025)

Great Memory, Shallow Reasoning: Limits of $k$NN-LMs
by: Geng, Shangyi, et al.
Published: (2024)

BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency
by: Haga, Akari, et al.
Published: (2024)

Identifying the Risks of LM Agents with an LM-Emulated Sandbox
by: Ruan, Yangjun, et al.
Published: (2023)

Rethinking Reflection in Pre-Training
by: AI, Essential, et al.
Published: (2025)

Pico: A Modular Framework for Hypothesis-Driven Small Language Model Research
by: Martinez, Richard Diehl, et al.
Published: (2025)

Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs
by: Chatterjee, Sagnik, et al.
Published: (2026)

General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level
by: Shi, Bingkang, et al.
Published: (2023)

InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuning
by: Zhu, Guanghao, et al.
Published: (2025)

NAIPv2: Debiased Pairwise Learning for Efficient Paper Quality Estimation
by: Zhao, Penghai, et al.
Published: (2025)

When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets?
by: Iyer, Srikrishna
Published: (2024)