Cover Image

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Meng, Fanxu, Wang, Zhaohui, Zhang, Muhan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

HD-PiSSA: High-Rank Distributed Orthogonal Adaptation
by: Wang, Yiding, et al.
Published: (2025)

CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning
by: Meng, Fanxu, et al.
Published: (2024)

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
by: Li, Yanan, et al.
Published: (2025)

Singular Value Decomposition on Kronecker Adaptation for Large Language Model
by: Chong, Yee Hin, et al.
Published: (2025)

GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding
by: Meng, Fanxu
Published: (2026)

Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation
by: Tang, Pingzhi, et al.
Published: (2026)

KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models
by: Wang, Fan, et al.
Published: (2024)

SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models
by: Cao, Yang, et al.
Published: (2024)

TransMLA: Multi-Head Latent Attention Is All You Need
by: Meng, Fanxu, et al.
Published: (2025)

Singular Vectors of Attention Heads Align with Features
by: Franco, Gabriel, et al.
Published: (2026)

TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference
by: Tang, Xiaojuan, et al.
Published: (2025)

VectorFit : Adaptive Singular & Bias Vector Fine-Tuning of Pre-trained Foundation Models
by: Hegde, Suhas G, et al.
Published: (2025)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
by: Zhou, Ruijie, et al.
Published: (2026)

A Generalized Singular Value Theory for Neural Networks
by: Brown, Brian Charles, et al.
Published: (2026)

Internal Value Alignment in Large Language Models through Controlled Value Vector Activation
by: Jin, Haoran, et al.
Published: (2025)

SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors
by: Lingam, Vijay, et al.
Published: (2024)

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition
by: Nasiri, Hamid, et al.
Published: (2025)

FANoise: Singular Value-Adaptive Noise Modulation for Robust Multimodal Representation Learning
by: Li, Jiaoyang, et al.
Published: (2025)

Beyond the Singular: Revealing the Value of Multiple Generations in Benchmark Evaluation
by: Zhang, Wenbo, et al.
Published: (2025)

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain
by: Wi, Hyowon, et al.
Published: (2025)

FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Models
by: Ramesh, Hariharan, et al.
Published: (2025)

The Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models
by: Zhao, Zhiyu, et al.
Published: (2026)

Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
by: Ahmad, Areeb, et al.
Published: (2025)

Why Deep Jacobian Spectra Separate: Depth-Induced Scaling and Singular-Vector Alignment
by: Haas, Nathanaël, et al.
Published: (2026)

Invisible Backdoor Attack Through Singular Value Decomposition
by: Chen, Wenmin, et al.
Published: (2024)

SPA-Cache: Singular Proxies for Adaptive Caching in Diffusion Language Models
by: Sun, Wenhao, et al.
Published: (2026)

Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure
by: Yang, Haotong, et al.
Published: (2023)

GADPN: Graph Adaptive Denoising and Perturbation Networks via Singular Value Decomposition
by: Deng, Hao, et al.
Published: (2026)

VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models
by: Xu, Hefei, et al.
Published: (2026)

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
by: Xu, Yufei, et al.
Published: (2026)

On the Limits of Self-Improving in Large Language Models: The Singularity Is Not Near Without Symbolic Model Synthesis
by: Zenil, Hector
Published: (2026)

There is a Singularity in the Loss Landscape
by: Lowell, Mark
Published: (2022)

Knowledge is Not Enough: Injecting RL Skills for Continual Adaptation
by: Tang, Pingzhi, et al.
Published: (2026)

LFA applied to CNNs: Efficient Singular Value Decomposition of Convolutional Mappings by Local Fourier Analysis
by: van Betteray, Antonia, et al.
Published: (2025)

IoMT-based Automated Leukemia Classification using CNN and Higher Order Singular Value
by: Marzijarani, Shabnam Bagheri, et al.
Published: (2025)

NANOZK: Layerwise Zero-Knowledge Proofs for Verifiable Large Language Model Inference
by: Wang, Zhaohui Geoffrey
Published: (2026)

SELF: A Robust Singular Value and Eigenvalue Approach for LLM Fingerprinting
by: Zhang, Hanxiu, et al.
Published: (2025)

Dispelling the Curse of Singularities in Neural Network Optimizations
by: Cao, Hengjie, et al.
Published: (2026)

Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting
by: Tian, Zailong, et al.
Published: (2026)

An Empirical Study of Realized GNN Expressiveness
by: Wang, Yanbo, et al.
Published: (2023)