Cover Image

Towards Higher Effective Rank in Parameter-efficient Fine-tuning using Khatri--Rao Product

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Albert, Paul, Zhang, Frederic Z., Saratchandran, Hemanth, Hengel, Anton van den, Abbasnejad, Ehsan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RandLoRA: Full-rank parameter-efficient fine-tuning of large models
by: Albert, Paul, et al.
Published: (2025)

Knowledge Composition using Task Vectors with Learned Anisotropic Scaling
by: Zhang, Frederic Z., et al.
Published: (2024)

Procedural Pretraining: Warming Up Language Models with Abstract Data
by: Jiang, Liangze, et al.
Published: (2026)

Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers
by: Shinnick, Zachary, et al.
Published: (2025)

Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learning
by: McDonnell, Mark D., et al.
Published: (2024)

RanPAC: Random Projections and Pre-trained Models for Continual Learning
by: McDonnell, Mark D., et al.
Published: (2023)

Learning to Reason and Navigate: Parameter Efficient Action Planning with Large Language Models
by: Mohammadi, Bahram, et al.
Published: (2025)

InvariantStock: Learning Invariant Features for Mastering the Shifting Market
by: Cao, Haiyao, et al.
Published: (2024)

Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling
by: Rodriguez-Opazo, Cristian, et al.
Published: (2024)

Enhancing Transformers Through Conditioned Embedded Tokens
by: Saratchandran, Hemanth, et al.
Published: (2025)

Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoning
by: Shinnick, Zachary, et al.
Published: (2025)

SineProject: Machine Unlearning for Stable Vision Language Alignment
by: Garg, Arpit, et al.
Published: (2025)

From Activation to Initialization: Scaling Insights for Optimizing Neural Fields
by: Saratchandran, Hemanth, et al.
Published: (2024)

Leaner Transformers: More Heads, Less Depth
by: Saratchandran, Hemanth, et al.
Published: (2025)

Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting
by: Xu, Runze, et al.
Published: (2026)

Weight Conditioning for Smooth Optimization of Neural Networks
by: Saratchandran, Hemanth, et al.
Published: (2024)

Frame-wise Conditioning Adaptation for Fine-Tuning Diffusion Models in Text-to-Video Prediction
by: Liu, Zheyuan, et al.
Published: (2025)

Preconditioners for the Stochastic Training of Neural Fields
by: Chng, Shin-Fang, et al.
Published: (2024)

Adaptive Randomized Tensor Train Rounding using Khatri-Rao Products
by: Daas, Hussam Al, et al.
Published: (2025)

Structured Initialization for Vision Transformers
by: Zheng, Jianqiao, et al.
Published: (2025)

Khatri-Rao Clustering for Data Summarization
by: Ciaperoni, Martino, et al.
Published: (2026)

Preconditioned Attention: Enhancing Efficiency in Transformers
by: Saratchandran, Hemanth
Published: (2026)

ViewFusion: Towards Multi-View Consistency via Interpolated Denoising
by: Yang, Xianghui, et al.
Published: (2024)

Data Denoising and Derivative Estimation for Data-Driven Modeling of Nonlinear Dynamical Systems
by: Yao, Jiaqi, et al.
Published: (2025)

Always Skip Attention
by: Ji, Yiping, et al.
Published: (2025)

Invertible Neural Warp for NeRF
by: Chng, Shin-Fang, et al.
Published: (2024)

Efficient Learning With Sine-Activated Low-rank Matrices
by: Ji, Yiping, et al.
Published: (2024)

Points-to-3D: Structure-Aware 3D Generation with Point Cloud Priors
by: Xia, Jiatong, et al.
Published: (2026)

LoRA-FA: Efficient and Effective Low Rank Representation Fine-tuning
by: Zhang, Longteng, et al.
Published: (2023)

Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning
by: Sehanobish, Arijit, et al.
Published: (2024)

Improving the Inclusivity of Dutch Speech Recognition by Fine-tuning Whisper on the JASMIN-CGN Corpus
by: Shekoufandeh, Golshid, et al.
Published: (2025)

Parameter-efficient Fine-tuning in Hyperspherical Space for Open-vocabulary Semantic Segmentation
by: Peng, Zelin, et al.
Published: (2024)

ClickSAM: Fine-tuning Segment Anything Model using click prompts for ultrasound image segmentation
by: Guo, Aimee, et al.
Published: (2024)

Scaling up Multi-domain Semantic Segmentation with Sentence Embeddings
by: Yin, Wei, et al.
Published: (2022)

The Inlet Rank Collapse in Implicit Neural Representations: Diagnosis and Unified Remedy
by: Zheng, Jianqiao, et al.
Published: (2026)

External Prompt Features Enhanced Parameter-efficient Fine-tuning for Salient Object Detection
by: Liang, Wen, et al.
Published: (2024)

Let Your Video Listen to Your Music!
by: Zhang, Xinyu, et al.
Published: (2025)

From Tables to Signals: Revealing Spectral Adaptivity in TabPFN
by: Zheng, Jianqiao, et al.
Published: (2025)

Rethinking Attention: Polynomial Alternatives to Softmax in Transformers
by: Saratchandran, Hemanth, et al.
Published: (2024)

D'OH: Decoder-Only Random Hypernetworks for Implicit Neural Representations
by: Gordon, Cameron, et al.
Published: (2024)