Cover Image

Less is More: Pre-Training Cross-Lingual Small-Scale Language Models with Cognitively-Plausible Curriculum Learning Strategies

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Salhan, Suchir, Martinez, Richard Diehl, Goriely, Zébulon, Buttery, Paula
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

What is the Best Sequence Length for BABYLM?
by: Salhan, Suchir, et al.
Published: (2025)

IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modeling
by: Goriely, Zébulon, et al.
Published: (2025)

ByteSpan: Information-Driven Subword Tokenisation
by: Goriely, Zébulon, et al.
Published: (2025)

Mitigating Frequency Bias and Anisotropy in Language Model Pre-Training with Syntactic Smoothing
by: Martinez, Richard Diehl, et al.
Published: (2024)

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes
by: Goriely, Zébulon, et al.
Published: (2024)

Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages
by: Africa, David Demitri, et al.
Published: (2025)

BabyLM's First Words: Word Segmentation as a Phonological Probing Task
by: Goriely, Zébulon, et al.
Published: (2025)

Pico: A Modular Framework for Hypothesis-Driven Small Language Model Research
by: Martinez, Richard Diehl, et al.
Published: (2025)

BLiSS 1.0: Evaluating Bilingual Learner Competence in Second Language Small Language Models
by: Gao, Yuan, et al.
Published: (2025)

Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
by: Ganescu, Bianca-Mihaela, et al.
Published: (2025)

Tending Towards Stability: Convergence Challenges in Small Language Models
by: Martinez, Richard Diehl, et al.
Published: (2024)

Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models
by: Weiss, Yuval, et al.
Published: (2025)

Learning Dynamics of Meta-Learning in Small Model Pretraining
by: Africa, David Demitri, et al.
Published: (2025)

The Distribution of Phoneme Frequencies across the World's Languages: Macroscopic and Microscopic Information-Theoretic Models
by: Martín, Fermín Moscoso del Prado, et al.
Published: (2026)

Modelling the Diachronic Emergence of Phoneme Frequency Distributions
by: Martín, Fermín Moscoso del Prado, et al.
Published: (2026)

A Computational Operationalisation of Competing Maturational Theories of Syntactic Development via Statistical Grammar Induction
by: Marcheva, Mila, et al.
Published: (2026)

Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction
by: Salhan, Suchir, et al.
Published: (2025)

Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale
by: Zheng, Wenzhen, et al.
Published: (2024)

Soft Prompt Tuning for Cross-Lingual Transfer: When Less is More
by: Philippy, Fred, et al.
Published: (2024)

Bias Dynamics in BabyLMs: Towards a Compute-Efficient Sandbox for Democratising Pre-Training Debiasing
by: Trhlik, Filip, et al.
Published: (2026)

Lost in Interpretation: The Plausibility-Faithfulness Trade-off in Cross-Lingual Explanations
by: Banerjee, Somnath, et al.
Published: (2026)

BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
by: Jumelet, Jaap, et al.
Published: (2025)

Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
by: Fujii, Kazuki, et al.
Published: (2024)

Curriculum Learning for Cross-Lingual Data-to-Text Generation With Noisy Data
by: Hari, Kancharla Aditya, et al.
Published: (2024)

Less is More: Adapting Text Embeddings for Low-Resource Languages with Small Scale Noisy Synthetic Data
by: Navasardyan, Zaruhi, et al.
Published: (2026)

Analysis of Multi-Source Language Training in Cross-Lingual Transfer
by: Lim, Seong Hoon, et al.
Published: (2024)

Luth: Efficient French Specialization for Small Language Models and Cross-Lingual Transfer
by: Lasbordes, Maxence, et al.
Published: (2025)

Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models
by: Xu, Zixiang, et al.
Published: (2025)

Pre-Training Curriculum for Multi-Token Prediction in Language Models
by: Aynetdinov, Ansar, et al.
Published: (2025)

Language Anisotropic Cross-Lingual Model Editing
by: Xu, Yang, et al.
Published: (2022)

Winning with Less for Low Resource Languages: Advantage of Cross-Lingual English_Persian Argument Mining Model over LLM Augmentation
by: Jahan, Ali, et al.
Published: (2025)

Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias
by: Wan, Zhongwei, et al.
Published: (2023)

Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
by: Akhlaghi, Amir Mohammad, et al.
Published: (2025)

LIMR: Less is More for RL Scaling
by: Li, Xuefeng, et al.
Published: (2025)

Cross-Lingual Optimization for Language Transfer in Large Language Models
by: Lee, Jungseob, et al.
Published: (2025)

Cross-Lingual Interleaving for Speech Language Models
by: Moumen, Adel, et al.
Published: (2025)

Less Is More: Engineering Challenges of On-Device Small Language Model Integration in a Mobile Application
by: Oliveira, William
Published: (2026)

Evaluating Cross-Lingual Unlearning in Multilingual Language Models
by: Lizzo, Tyler, et al.
Published: (2026)

IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning
by: Gupta, Navya, et al.
Published: (2026)

Curriculum Learning for Small Code Language Models
by: Naïr, Marwa, et al.
Published: (2024)