Image de couverture de livre

SoftLMs: Efficient Adaptive Low-Rank Approximation of Language Models using Soft-Thresholding Mechanism

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Bhatnagar, Priyansh, Wen, Linfeng, Kang, Mingu
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
par: Qin, Guanghui, et autres
Publié: (2021)

Enabling Approximate Joint Sampling in Diffusion LMs
par: Bansal, Parikshit, et autres
Publié: (2025)

Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
par: Liu, Hengchang, et autres
Publié: (2026)

Soft Adaptive Policy Optimization
par: Gao, Chang, et autres
Publié: (2025)

Soft-Masked Diffusion Language Models
par: Hersche, Michael, et autres
Publié: (2025)

Dynamic Rank Reinforcement Learning for Adaptive Low-Rank Multi-Head Self Attention in Large Language Models
par: Erden, Caner
Publié: (2025)

LCQ: Low-Rank Codebook based Quantization for Large Language Models
par: Cai, Wen-Pu, et autres
Publié: (2024)

Training Bilingual LMs with Data Constraints in the Targeted Language
par: Seto, Skyler, et autres
Publié: (2024)

Accelerating the Low-Rank Decomposed Models
par: Hajimolahoseini, Habib, et autres
Publié: (2024)

AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability
par: Agrawal, Sudhanshu, et autres
Publié: (2024)

Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention
par: Liao, Huanxuan, et autres
Publié: (2025)

Soft Prompting for Unlearning in Large Language Models
par: Bhaila, Karuna, et autres
Publié: (2024)

Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
par: Ji, Yixin, et autres
Publié: (2024)

Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices
par: Zhao, Pengxiang, et autres
Publié: (2024)

Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
par: Ponkshe, Kaustubh, et autres
Publié: (2024)

Soft Self-Consistency Improves Language Model Agents
par: Wang, Han, et autres
Publié: (2024)

LoRA-GA: Low-Rank Adaptation with Gradient Approximation
par: Wang, Shaowen, et autres
Publié: (2024)

Compositional preference models for aligning LMs
par: Go, Dongyoung, et autres
Publié: (2023)

Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models
par: Siddiqui, S M Tahmid, et autres
Publié: (2026)

SoftMatcha 2: A Fast and Soft Pattern Matcher for Trillion-Scale Corpora
par: Yoneda, Masataka, et autres
Publié: (2026)

SERA: Soft-Verified Efficient Repository Agents
par: Shen, Ethan, et autres
Publié: (2026)

Soft-prompt Tuning for Large Language Models to Evaluate Bias
par: Tian, Jacob-Junqi, et autres
Publié: (2023)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
par: Rajabzadeh, Hossein, et autres
Publié: (2024)

A Bayesian Interpretation of Adaptive Low-Rank Adaptation
par: Chen, Haolin, et autres
Publié: (2024)

Batched Low-Rank Adaptation of Foundation Models
par: Wen, Yeming, et autres
Publié: (2023)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
par: Barazandeh, Babak, et autres
Publié: (2025)

Efficient Minimum Bayes Risk Decoding using Low-Rank Matrix Completion Algorithms
par: Trabelsi, Firas, et autres
Publié: (2024)

SARA: Singular-Value Based Adaptive Low-Rank Adaption
par: Gu, Jihao, et autres
Publié: (2024)

Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
par: Ahmed, Kareem, et autres
Publié: (2026)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
par: Liu, Zeyu, et autres
Publié: (2024)

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
par: Rawat, Ankit Singh, et autres
Publié: (2024)

Adaptive Soft Rolling KV Freeze with Entropy-Guided Recovery: Sublinear Memory Growth for Efficient LLM Inference
par: Metinov, Adilet, et autres
Publié: (2025)

SOS! Soft Prompt Attack Against Open-Source Large Language Models
par: Yang, Ziqing, et autres
Publié: (2024)

When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
par: Jeong, Soyeong, et autres
Publié: (2025)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
par: Li, Shiwei, et autres
Publié: (2025)

A3 : an Analytical Low-Rank Approximation Framework for Attention
par: Wong, Jeffrey T. H., et autres
Publié: (2025)

Unlocking Memorization in Large Language Models with Dynamic Soft Prompting
par: Wang, Zhepeng, et autres
Publié: (2024)

ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
par: Saxena, Utkarsh, et autres
Publié: (2024)

FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing
par: Smith, James Seale, et autres
Publié: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
par: Hayou, Soufiane, et autres
Publié: (2024)