Image de couverture de livre

Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bick, Aviv, Katsch, Tobias, Sohoni, Nimit, Desai, Arjun, Gu, Albert
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
par: Bick, Aviv, et autres
Publié: (2025)

Retrieval-Aware Distillation for Transformer-SSM Hybrids
par: Bick, Aviv, et autres
Publié: (2026)

GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling
par: Katsch, Tobias
Publié: (2023)

Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models
par: Bick, Aviv, et autres
Publié: (2024)

Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
par: Wu, Xiaojun, et autres
Publié: (2025)

Rethinking Large Language Model Distillation: A Constrained Markov Decision Process Perspective
par: Zimmer, Matthieu, et autres
Publié: (2025)

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
par: Park, Seonghyeon, et autres
Publié: (2026)

Mamba: Linear-Time Sequence Modeling with Selective State Spaces
par: Gu, Albert, et autres
Publié: (2023)

FTA generation using GenAI with an Autonomy sensor Usecase
par: Shetiya, Sneha Sudhir, et autres
Publié: (2024)

Distributed Interpretability and Control for Large Language Models
par: Desai, Dev Arpan, et autres
Publié: (2026)

Solving Robust Markov Decision Processes: Generic, Reliable, Efficient
par: Meggendorfer, Tobias, et autres
Publié: (2024)

Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners
par: Paliotta, Daniele, et autres
Publié: (2025)

Dead Weights, Live Signals: Feedforward Graphs of Frozen Language Models
par: Armstrong, Marcus, et autres
Publié: (2026)

A Comparison of Methods for Neural Network Aggregation
par: Pomerat, John, et autres
Publié: (2023)

Privileged Information Distillation for Language Models
par: Penaloza, Emiliano, et autres
Publié: (2026)

RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
par: Botev, Aleksandar, et autres
Publié: (2024)

Interpreting Affine Recurrence Learning in GPT-style Transformers
par: Bhargav, Samarth, et autres
Publié: (2024)

Leveraging Zero-Shot Prompting for Efficient Language Model Distillation
par: Vöge, Lukas, et autres
Publié: (2024)

Recurrent Diffusion for Large-Scale Parameter Generation
par: Wang, Kai, et autres
Publié: (2025)

Distilling Time Series Foundation Models for Efficient Forecasting
par: Li, Yuqi, et autres
Publié: (2026)

Spatio-Temporal Forecasting of PM2.5 via Spatial-Diffusion guided Encoder-Decoder Architecture
par: Pandey, Malay, et autres
Publié: (2024)

Scaling Inference-Efficient Language Models
par: Bian, Song, et autres
Publié: (2025)

Evaluating Computational Accuracy of Large Language Models in Numerical Reasoning Tasks for Healthcare Applications
par: Malghan, Arjun R.
Publié: (2025)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
par: Zhu, Hourun, et autres
Publié: (2025)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
par: Yang, Runming, et autres
Publié: (2024)

Data-Efficient Symbolic Regression via Foundation Model Distillation
par: Ying, Wangyang, et autres
Publié: (2025)

Random Conditioning with Distillation for Data-Efficient Diffusion Model Compression
par: Kim, Dohyun, et autres
Publié: (2025)

Spectra 1.1: Scaling Laws and Efficient Inference for Ternary Language Models
par: Vaidhya, Tejas, et autres
Publié: (2025)

Rethinking the Role of Temperature in Large Language Model Distillation
par: Luong, Hoang-Chau, et autres
Publié: (2026)

Using Large Language Models for Hyperparameter Optimization
par: Zhang, Michael R., et autres
Publié: (2023)

Efficient Prediction of Pass@k Scaling in Large Language Models
par: Kazdan, Joshua, et autres
Publié: (2025)

Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR
par: Segal-Feldman, Yael, et autres
Publié: (2024)

GradMetaNet: An Equivariant Architecture for Learning on Gradients
par: Gelberg, Yoav, et autres
Publié: (2025)

Distill-then-Replace: Efficient Task-Specific Hybrid Attention Model Construction
par: Xia, Xiaojie, et autres
Publié: (2026)

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
par: Shing, Makoto, et autres
Publié: (2025)

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
par: Wu, Xiaoyou, et autres
Publié: (2026)

MoGU: Mixture-of-Gaussians with Uncertainty-based Gating for Time Series Forecasting
par: Aviv, Gilad, et autres
Publié: (2025)

Meta Reinforcement Learning with Finite Training Tasks -- a Density Estimation Approach
par: Rimon, Zohar, et autres
Publié: (2022)

Distillation of Large Language Models via Concrete Score Matching
par: Kim, Yeongmin, et autres
Publié: (2025)

Curriculum Learning-Guided Progressive Distillation in Large Language Models
par: Cao, Jincheng, et autres
Publié: (2026)