Cover Image

LoLCATs: On Low-Rank Linearizing of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Michael, Arora, Simran, Chalamala, Rahul, Wu, Alan, Spector, Benjamin, Singhal, Aaryan, Ramesh, Krithik, Ré, Christopher
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ThunderKittens: Simple, Fast, and Adorable AI Kernels
by: Spector, Benjamin F., et al.
Published: (2024)

Just read twice: closing the recall gap for recurrent language models
by: Arora, Simran, et al.
Published: (2024)

ParallelKittens: Systematic and Practical Simplification of Multi-GPU AI Kernels
by: Sul, Stuart H., et al.
Published: (2025)

LoLA: Low-Rank Linear Attention With Sparse Caching
by: McDermott, Luke, et al.
Published: (2025)

Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes
by: Arora, Simran, et al.
Published: (2023)

DenseLoRA: Dense Low-Rank Adaptation of Large Language Models
by: Mu, Lin, et al.
Published: (2025)

LoRTA: Low Rank Tensor Adaptation of Large Language Models
by: Hounie, Ignacio, et al.
Published: (2024)

DiffLoRA: Differential Low-Rank Adapters for Large Language Models
by: Misrahi, Alexandre, et al.
Published: (2025)

LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging
by: Liu, Zehua, et al.
Published: (2025)

LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model
by: Hu, Yuxuan, et al.
Published: (2025)

DS@GT at Touché: Large Language Models for Retrieval-Augmented Debate
by: Miyaguchi, Anthony, et al.
Published: (2025)

BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models
by: Chang, Yupeng, et al.
Published: (2024)

Linearizing Large Language Models
by: Mercat, Jean, et al.
Published: (2024)

Temporally Consistent Factuality Probing for Large Language Models
by: Bajpai, Ashutosh, et al.
Published: (2024)

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning
by: Zhang, Jingfan, et al.
Published: (2024)

LoRA$^2$ : Multi-Scale Low-Rank Approximations for Fine-Tuning Large Language Models
by: Zhang, Jia-Chen, et al.
Published: (2024)

RedPajama: an Open Dataset for Training Large Language Models
by: Weber, Maurice, et al.
Published: (2024)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

RELIC: Investigating Large Language Model Responses using Self-Consistency
by: Cheng, Furui, et al.
Published: (2023)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
by: Rajabzadeh, Hossein, et al.
Published: (2024)

Generalist Large Language Models Outperform Clinical Tools on Medical Benchmarks
by: Vishwanath, Krithik, et al.
Published: (2025)

Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
by: Saad-Falcon, Jon, et al.
Published: (2024)

NLoRA: Nyström-Initiated Low-Rank Adaptation for Large Language Models
by: Guo, Chenlu, et al.
Published: (2025)

Simple linear attention language models balance the recall-throughput tradeoff
by: Arora, Simran, et al.
Published: (2024)

HaLoRA: Hardware-aware Low-Rank Adaptation for Large Language Models Based on Hybrid Compute-in-Memory Architecture
by: Wu, Taiqiang, et al.
Published: (2025)

Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models
by: Liang, Yan-Shuo, et al.
Published: (2025)

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
by: Meng, Xiangdi, et al.
Published: (2024)

HealthBench: Evaluating Large Language Models Towards Improved Human Health
by: Arora, Rahul K., et al.
Published: (2025)

LCQ: Low-Rank Codebook based Quantization for Large Language Models
by: Cai, Wen-Pu, et al.
Published: (2024)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry
by: Zhang, Michael, et al.
Published: (2024)

LoQT: Low-Rank Adapters for Quantized Pretraining
by: Loeschcke, Sebastian, et al.
Published: (2024)

BeamLoRA: Beam-Constraint Low-Rank Adaptation
by: Gu, Naibin, et al.
Published: (2025)

Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Models
by: Luong, Hoang-Chau, et al.
Published: (2026)

LoKI: Low-damage Knowledge Implanting of Large Language Models
by: Wang, Runyu, et al.
Published: (2025)

Ranking Large Language Models without Ground Truth
by: Dhurandhar, Amit, et al.
Published: (2024)

ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation
by: Song, Yurun, et al.
Published: (2024)

OLoRA: Orthonormal Low-Rank Adaptation of Large Language Models
by: Büyükakyüz, Kerim
Published: (2024)

Derivative-Free Optimization for Low-Rank Adaptation in Large Language Models
by: Jin, Feihu, et al.
Published: (2024)

Flexora: Flexible Low Rank Adaptation for Large Language Models
by: Wei, Chenxing, et al.
Published: (2024)