Cover Image

Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Garcia, Roberto, Liu, Jerry, Sorvisto, Daniel, Eyuboglu, Sabri
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Constructing Efficient Fact-Storing MLPs for Transformers
by: Dugan, Owen, et al.
Published: (2025)

Decorrelation Speeds Up Vision Transformers
by: Carrigg, Kieran, et al.
Published: (2025)

Adaptive Budget Allocation for Orthogonal-Subspace Adapter Tuning in LLMs Continual Learning
by: Wan, Zhiyi, et al.
Published: (2025)

AdapterTune: Zero-Initialized Low-Rank Adapters for Frozen Vision Transformers
by: Khazem, Salim
Published: (2026)

ARA: Adaptive Rank Allocation for Efficient Large Language Model SVD Compression
by: Xv, Lin, et al.
Published: (2025)

Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters
by: Bhardwaj, Kartikeya, et al.
Published: (2024)

SpeedUpNet: A Plug-and-Play Adapter Network for Accelerating Text-to-Image Diffusion Models
by: Chai, Weilong, et al.
Published: (2023)

Sparse High Rank Adapters
by: Bhardwaj, Kartikeya, et al.
Published: (2024)

The Quest for Winning Tickets in Low-Rank Adapters
by: Damirchi, Hamed, et al.
Published: (2025)

Asymmetry in Low-Rank Adapters of Foundation Models
by: Zhu, Jiacheng, et al.
Published: (2024)

Towards Symmetric Low-Rank Adapters
by: Panoutsos, Tales, et al.
Published: (2025)

LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization
by: Bouquet, Yann, et al.
Published: (2026)

Unlocking the Global Synergies in Low-Rank Adapters
by: Zhang, Zixi, et al.
Published: (2024)

PreLoRA: Hybrid Pre-training of Vision Transformers with Full Training and Low-Rank Adapters
by: Thapa, Krishu K, et al.
Published: (2025)

Ensembles of Low-Rank Expert Adapters
by: Li, Yinghao, et al.
Published: (2025)

Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression
by: Crawshaw, Michael, et al.
Published: (2025)

Aggregating Low Rank Adapters in Federated Fine-tuning
by: Trautmann, Evelyn, et al.
Published: (2025)

Flora: Low-Rank Adapters Are Secretly Gradient Compressors
by: Hao, Yongchang, et al.
Published: (2024)

LoQT: Low-Rank Adapters for Quantized Pretraining
by: Loeschcke, Sebastian, et al.
Published: (2024)

Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences
by: Dimitriadis, Nikolaos, et al.
Published: (2024)

MGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMs
by: Li, Guangyan, et al.
Published: (2025)

IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring
by: Cui, Xuan, et al.
Published: (2026)

Continual Low-Rank Adapters for LLM-based Generative Recommender Systems
by: Yoo, Hyunsik, et al.
Published: (2025)

SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection
by: Ataiefard, Foozhan, et al.
Published: (2024)

Learning Adapter Rank via Symmetry Breaking
by: Doyle, Cooper, et al.
Published: (2025)

Mitigating Subject Dependency in EEG Decoding with Subject-Specific Low-Rank Adapters
by: Klein, Timon, et al.
Published: (2025)

Simple linear attention language models balance the recall-throughput tradeoff
by: Arora, Simran, et al.
Published: (2024)

Just read twice: closing the recall gap for recurrent language models
by: Arora, Simran, et al.
Published: (2024)

BoostLoRA: Growing Effective Rank by Boosting Adapters
by: Anantha, Raviteja, et al.
Published: (2026)

On the Duality between Gradient Transformations and Adapters
by: Torroba-Hennigen, Lucas, et al.
Published: (2025)

Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models
by: Narayan, Avanika, et al.
Published: (2025)

Allocation of Parameters in Transformers
by: Yu, Ruoxi, et al.
Published: (2025)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
by: Cong, Peizhuang, et al.
Published: (2025)

Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapters
by: Xiang, Haotian, et al.
Published: (2026)

Speed Up the Cold-Start Learning in Two-Sided Bandits with Many Arms
by: Bayati, Mohsen, et al.
Published: (2022)

CR-BLEA: Contrastive Ranking for Adaptive Resource Allocation in Bilevel Evolutionary Algorithms
by: Xu, Dejun, et al.
Published: (2025)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

QWO: Speeding Up Permutation-Based Causal Discovery in LiGAMs
by: Shahverdikondori, Mohammad, et al.
Published: (2024)

Multiple Choice Learning of Low-Rank Adapters for Language Modeling
by: Letzelter, Victor, et al.
Published: (2025)