Imagen de Portada

Generalized Fisher-Weighted SVD: Scalable Kronecker-Factored Fisher Approximation for Compressing Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chekalina, Viktoriia, Moskovskiy, Daniil, Cherniuk, Daria, Kurkin, Maxim, Kuznetsov, Andrey, Frolov, Evgeny
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
por: Chekalina, Viktoriia, et al.
Publicado: (2024)

DyKAF: Dynamical Kronecker Approximation of the Fisher Information Matrix for Gradient Preconditioning
por: Yudin, Nikolay, et al.
Publicado: (2025)

ERC-SVD: Error-Controlled SVD for Large Language Model Compression
por: Bai, Haolei, et al.
Publicado: (2025)

LoTR: Low Tensor Rank Weight Adaptation
por: Bershatsky, Daniel, et al.
Publicado: (2024)

Knowledge Graph Completion with Mixed Geometry Tensor Factorization
por: Yusupov, Viacheslav, et al.
Publicado: (2025)

CoRoVA: Compressed Representations for Vector-Augmented Code Completion
por: Cherniuk, Daria, et al.
Publicado: (2025)

<think> So let's replace this phrase with insult... </think> Lessons learned from generation of toxic texts with LLMs
por: Pletenev, Sergey, et al.
Publicado: (2025)

NoReGeo: Non-Reasoning Geometry Benchmark
por: Abdullaeva, Irina, et al.
Publicado: (2026)

Dynamic Low-rank Approximation of Full-Matrix Preconditioner for Training Generalized Linear Models
por: Matveeva, Tatyana, et al.
Publicado: (2025)

Dataless Weight Disentanglement in Task Arithmetic via Kronecker-Factored Approximate Curvature
por: Porrello, Angelo, et al.
Publicado: (2026)

Quantization of Large Language Models with an Overdetermined Basis
por: Merkulov, Daniil, et al.
Publicado: (2024)

SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers
por: Chekalina, Viktoriia, et al.
Publicado: (2024)

Fisher Mask Nodes for Language Model Merging
por: K, Thennal D, et al.
Publicado: (2024)

The Approximate Fisher Influence Function: Faster Estimation of Data Influence in Statistical Models
por: Lev, Omri, et al.
Publicado: (2024)

Fisher Flow Matching for Generative Modeling over Discrete Data
por: Davis, Oscar, et al.
Publicado: (2024)

Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models
por: Liu, Ji, et al.
Publicado: (2024)

AlignMerge - Alignment-Preserving Large Language Model Merging via Fisher-Guided Geometric Constraints
por: Roy, Aniruddha, et al.
Publicado: (2025)

RTD-Lite: Scalable Topological Analysis for Comparing Weighted Graphs in Learning Tasks
por: Tulchinskii, Eduard, et al.
Publicado: (2025)

Weighted Stochastic Differential Equation to Implement Wasserstein-Fisher-Rao Gradient Flow
por: Rahimi, Herlock
Publicado: (2025)

DipSVD: Dual-importance Protected SVD for Efficient LLM Compression
por: Ding, Xuan, et al.
Publicado: (2025)

Beyond Variance: Knowledge-Aware LLM Compression via Fisher-Aligned Subspace Diagnostics
por: Shihab, Ibne Farabi, et al.
Publicado: (2026)

Rank-1 Approximation of Inverse Fisher for Natural Policy Gradients in Deep Reinforcement Learning
por: Huo, Yingxiao, et al.
Publicado: (2026)

KFCPO: Kronecker-Factored Approximated Constrained Policy Optimization
por: Lim, Joonyoung, et al.
Publicado: (2025)

AA-SVD : Anchored and Adaptive SVD for Large Language Model Compression
por: Sinha, Atul Kumar, et al.
Publicado: (2026)

Intrinsic Structure as a Proxy for Saliency: SVD-Based Weight Preservation for Mixed-Precision Quantization in Large Language Models
por: Landge, Shashank, et al.
Publicado: (2025)

FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation
por: Wu, Zhuguanyu, et al.
Publicado: (2025)

Singular Value Decomposition on Kronecker Adaptation for Large Language Model
por: Chong, Yee Hin, et al.
Publicado: (2025)

Run LoRA Run: Faster and Lighter LoRA Implementations
por: Cherniuk, Daria, et al.
Publicado: (2023)

Compressing Quantum Fisher Information
por: Tang, Rui Jie, et al.
Publicado: (2026)

Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
por: Gong, Wenbo, et al.
Publicado: (2025)

On the Computation of the Fisher Information in Continual Learning
por: van de Ven, Gido M.
Publicado: (2025)

Scalable Whole Slide Image Representation Using K-Mean Clustering and Fisher Vector Aggregation
por: Gupta, Ravi Kant, et al.
Publicado: (2025)

End-to-End Graph-Sequential Representation Learning for Accurate Recommendations
por: Baikalov, Vladimir, et al.
Publicado: (2024)

Position-Aware Sequential Attention for Accurate Next Item Recommendations
por: Nabiev, Timur, et al.
Publicado: (2026)

OmniFusion Technical Report
por: Goncharova, Elizaveta, et al.
Publicado: (2024)

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator
por: Li, YuXin, et al.
Publicado: (2025)

Beyond the Mean: Fisher-Orthogonal Projection for Natural Gradient Descent in Large Batch Training
por: Lu, Yishun, et al.
Publicado: (2025)

Adaptive Variance-Penalized Continual Learning with Fisher Regularization
por: Sarkar, Krisanu
Publicado: (2025)

Fisher-Bingham-like normalizing flows on the sphere
por: Glüsenkamp, Thorsten
Publicado: (2025)

GWT: Scalable Optimizer State Compression for Large Language Model Training
por: Wen, Ziqing, et al.
Publicado: (2025)