Copertina

BabyLlama-2: Ensemble-Distilled Models Consistently Outperform Teachers With Limited Data

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tastet, Jean-Loup, Timiryasov, Inar
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Machine Learning I.2.7
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Ensemble Language Models for Multilingual Sentiment Analysis
di: Hasan, Md Arid
Pubblicazione: (2024)

Influence-driven Curriculum Learning for Pre-training on Limited Data
di: Schoenegger, Loris, et al.
Pubblicazione: (2025)

Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning
di: Xu, Shuyao, et al.
Pubblicazione: (2025)

Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation
di: Stacey, Joe, et al.
Pubblicazione: (2023)

Overcoming Long-Context Limitations of State-Space Models via Context-Dependent Sparse Attention
di: Zhan, Zhihao, et al.
Pubblicazione: (2025)

Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
di: Saparkhan, Raman, et al.
Pubblicazione: (2026)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
di: Oketunji, Abiodun Finbarrs
Pubblicazione: (2023)

AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
di: Wu, Zhengxuan, et al.
Pubblicazione: (2025)

Stratified Hazard Sampling: Minimal-Variance Event Scheduling for CTMC/DTMC Discrete Diffusion and Flow Models
di: Jang, Seunghwan, et al.
Pubblicazione: (2026)

A Llama walks into the 'Bar': Efficient Supervised Fine-Tuning for Legal Reasoning in the Multi-state Bar Exam
di: Fernandes, Rean, et al.
Pubblicazione: (2025)

Large Language Model (LLM) Bias Index -- LLMBI
di: Oketunji, Abiodun Finbarrs, et al.
Pubblicazione: (2023)

Engineering A Large Language Model From Scratch
di: Oketunji, Abiodun Finbarrs
Pubblicazione: (2024)

PSK at SemEval-2026 Task 9: Multilingual Polarization Detection Using Ensemble Gemma Models with Synthetic Data Augmentation
di: Pulipaka, Srikar Kashyap
Pubblicazione: (2026)

The Data Efficiency Frontier of Financial Foundation Models: Scaling Laws from Continued Pretraining
di: Ponnock, Jesse
Pubblicazione: (2025)

Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment
di: Zhang, Yizhuo, et al.
Pubblicazione: (2025)

DynaSemble: Dynamic Ensembling of Textual and Structure-Based Models for Knowledge Graph Completion
di: Nandi, Ananjan, et al.
Pubblicazione: (2023)

RBCorr: Response Bias Correction in Language Models
di: Bhatt, Om, et al.
Pubblicazione: (2026)

Compact Example-Based Explanations for Language Models
di: Schoenegger, Loris, et al.
Pubblicazione: (2026)

On the Effect of (Near) Duplicate Subwords in Language Modelling
di: Schäfer, Anton, et al.
Pubblicazione: (2024)

Consistency Evaluation of News Article Summaries Generated by Large (and Small) Language Models
di: Gilhuly, Colleen, et al.
Pubblicazione: (2025)

The Open Source Advantage in Large Language Models (LLMs)
di: Manchanda, Jiya, et al.
Pubblicazione: (2024)

Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models
di: Zhang, Gongbo, et al.
Pubblicazione: (2026)

Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation
di: Shafique, Muhammad Ali, et al.
Pubblicazione: (2025)

MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems
di: Ye, Xinwu, et al.
Pubblicazione: (2025)

Scaling Laws for Forgetting When Fine-Tuning Large Language Models
di: Kalajdzievski, Damjan
Pubblicazione: (2024)

Sparsing Law: Towards Large Language Models with Greater Activation Sparsity
di: Luo, Yuqi, et al.
Pubblicazione: (2024)

Better and Worse with Scale: How Contextual Entrainment Diverges with Model Size
di: Kukreja, Dikshant, et al.
Pubblicazione: (2026)

BLP-2023 Task 2: Sentiment Analysis
di: Hasan, Md. Arid, et al.
Pubblicazione: (2023)

MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering
di: Ness, Robert Osazuwa, et al.
Pubblicazione: (2024)

Hardware Co-Design Scaling Laws via Roofline Modelling for On-Device LLMs
di: Sun, Luoyang, et al.
Pubblicazione: (2026)

From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge
di: Chowdhury, Nafis, et al.
Pubblicazione: (2025)

Distilling Knowledge from Large Language Models: A Concept Bottleneck Model for Hate and Counter Speech Recognition
di: Labadie-Tamayo, Roberto, et al.
Pubblicazione: (2025)

Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms
di: Hanna, Michael, et al.
Pubblicazione: (2024)

Where Should LoRA Go? Component-Type Placement in Hybrid Language Models
di: Borobia, Hector, et al.
Pubblicazione: (2026)

TCNN: Triple Convolutional Neural Network Models for Retrieval-based Question Answering System in E-commerce
di: Song, Shuangyong, et al.
Pubblicazione: (2020)

Decoding-Time Debiasing via Process Reward Models: From Controlled Fill-in to Open-Ended Generation
di: Khan, Muneeb Ur Raheem
Pubblicazione: (2026)

Constraint-Driven Small Language Models Based on Agent and OpenAlex Knowledge Graph: Mining Conceptual Pathways and Discovering Innovation Points in Academic Papers
di: Xia, Ziye, et al.
Pubblicazione: (2025)

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment
di: Jaber, Jaber, et al.
Pubblicazione: (2026)

Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models
di: Bhandari, Pranav, et al.
Pubblicazione: (2026)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
di: Fadli, Samih
Pubblicazione: (2025)