Copertina

M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Bhendawade, Nikhil, Najibi, Mahyar, Naik, Devang, Belousova, Irina
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Speculative Streaming: Fast LLM Inference without Auxiliary Models
di: Bhendawade, Nikhil, et al.
Pubblicazione: (2024)

FS-DFM: Fast and Accurate Long Text Generation with Few-Step Diffusion Language Models
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2025)

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2026)

DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2026)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
di: Fu, Qichen, et al.
Pubblicazione: (2024)

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
di: Merth, Thomas, et al.
Pubblicazione: (2024)

OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
di: Mehta, Sachin, et al.
Pubblicazione: (2024)

CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data
di: Mehta, Sachin, et al.
Pubblicazione: (2024)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
di: Samragh, Mohammad, et al.
Pubblicazione: (2024)

Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
di: Bhendawade, Nikhil, et al.
Pubblicazione: (2025)

Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference
di: Liu, Baihui, et al.
Pubblicazione: (2026)

R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training
di: Ge, Albert, et al.
Pubblicazione: (2025)

Residual Stream Duality in Modern Transformer Architectures
di: Zhang, Yifan
Pubblicazione: (2026)

On the Spatial Structure of Mixture-of-Experts in Transformers
di: Bershatsky, Daniel, et al.
Pubblicazione: (2025)

LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference
di: Kapadia, Shashank, et al.
Pubblicazione: (2026)

Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
di: NVIDIA, et al.
Pubblicazione: (2025)

Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
di: NVIDIA, et al.
Pubblicazione: (2026)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
di: Cho, Minsik, et al.
Pubblicazione: (2024)

Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts
di: He, Shwai, et al.
Pubblicazione: (2025)

Mixture of Chapters: Scaling Learnt Memory in Transformers
di: Tibrewal, Tasmay Pankaj, et al.
Pubblicazione: (2026)

Multi-Head Mixture-of-Experts
di: Wu, Xun, et al.
Pubblicazione: (2024)

MoSLD: An Extremely Parameter-Efficient Mixture-of-Shared LoRAs for Multi-Task Learning
di: Zhao, Lulu, et al.
Pubblicazione: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
di: Gu, Naibin, et al.
Pubblicazione: (2025)

Probabilistic Consensus through Ensemble Validation: A Framework for LLM Reliability
di: Naik, Ninad
Pubblicazione: (2024)

R$^2$PO: Decoupling Training Trajectories from Inference Responses for LLM Reasoning
di: Wang, Jingchu, et al.
Pubblicazione: (2026)

Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
di: Liu, Yilun, et al.
Pubblicazione: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
di: Hayou, Soufiane, et al.
Pubblicazione: (2024)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
di: Pan, Bowen, et al.
Pubblicazione: (2024)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
di: Tang, Chuanyu, et al.
Pubblicazione: (2024)

Do We Need Frontier Models to Verify Mathematical Proofs?
di: Naik, Aaditya, et al.
Pubblicazione: (2026)

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
di: Xiao, Da, et al.
Pubblicazione: (2025)

HMI: Hierarchical Knowledge Management for Efficient Multi-Tenant Inference in Pretrained Language Models
di: Zhang, Jun, et al.
Pubblicazione: (2025)

MoM: Linear Sequence Modeling with Mixture-of-Memories
di: Du, Jusen, et al.
Pubblicazione: (2025)

Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes
di: Nath, Abhijnan, et al.
Pubblicazione: (2025)

Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Models
di: Sakarvadia, Mansi, et al.
Pubblicazione: (2023)

Nexus: Specialization meets Adaptability for Efficiently Training Mixture of Experts
di: Gritsch, Nikolas, et al.
Pubblicazione: (2024)

LEAP: Learnable End-to-End Adaptive Pruning of Large Language Models
di: Mozaffari, Mohammad, et al.
Pubblicazione: (2026)

Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
di: Chen, Yilong, et al.
Pubblicazione: (2026)

NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
di: NVIDIA, et al.
Pubblicazione: (2025)

Scaling Inference-Efficient Language Models
di: Bian, Song, et al.
Pubblicazione: (2025)