Cover Image

On the Duality between Gradient Transformations and Adapters

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Torroba-Hennigen, Lucas, Lang, Hunter, Guo, Han, Kim, Yoon
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Hyperloop Transformers
by: Zeitoun, Abbas, et al.
Published: (2026)

Towards Verifiable Text Generation with Symbolic References
by: Hennigen, Lucas Torroba, et al.
Published: (2023)

Principled Gradient-based Markov Chain Monte Carlo for Text Generation
by: Du, Li, et al.
Published: (2023)

Learning to Decode Collaboratively with Multiple Language Models
by: Shen, Shannon Zejiang, et al.
Published: (2024)

FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients
by: Yu, Hongyeon, et al.
Published: (2026)

Theoretical Analysis of Weak-to-Strong Generalization
by: Lang, Hunter, et al.
Published: (2024)

A Latent-Variable Model for Intrinsic Probing
by: Stańczak, Karolina, et al.
Published: (2022)

Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapters
by: Sanchez, Bryan
Published: (2026)

Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
by: Chen, Yuyan, et al.
Published: (2024)

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
by: Guo, Han, et al.
Published: (2023)

Not All Adapters Matter: Selective Adapter Freezing for Memory-Efficient Fine-Tuning of Language Models
by: Son, Hyegang, et al.
Published: (2024)

Residual Stream Duality in Modern Transformer Architectures
by: Zhang, Yifan
Published: (2026)

Gated Linear Attention Transformers with Hardware-Efficient Training
by: Yang, Songlin, et al.
Published: (2023)

Parallelizing Linear Transformers with the Delta Rule over Sequence Length
by: Yang, Songlin, et al.
Published: (2024)

Block Circulant Adapter for Large Language Models
by: Ding, Xinyu, et al.
Published: (2025)

Unlocking the Global Synergies in Low-Rank Adapters
by: Zhang, Zixi, et al.
Published: (2024)

On Structured State-Space Duality
by: Hu, Jerry Yao-Chieh, et al.
Published: (2025)

LoQT: Low-Rank Adapters for Quantized Pretraining
by: Loeschcke, Sebastian, et al.
Published: (2024)

MAAT: Multi-phase Adapter-Aware Targeted Unlearning
by: Yagnik, Suryash, et al.
Published: (2026)

The Diffusion Duality
by: Sahoo, Subham Sekhar, et al.
Published: (2025)

PaTH Attention: Position Encoding via Accumulating Householder Transformations
by: Yang, Songlin, et al.
Published: (2025)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
by: Shi, Haizhou, et al.
Published: (2024)

SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
by: Liu, Yongkang, et al.
Published: (2026)

AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model
by: Yan, Yongyu, et al.
Published: (2022)

Approximate Attributions for Off-the-Shelf Siamese Transformers
by: Möller, Lucas, et al.
Published: (2024)

Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis
by: Yang, Hongru, et al.
Published: (2024)

Ensembles of Low-Rank Expert Adapters
by: Li, Yinghao, et al.
Published: (2025)

MoKA: Mixture of Kronecker Adapters
by: Sadeghi, Mohammadreza, et al.
Published: (2025)

Random Initialization of Gated Sparse Adapters
by: Retault, Vi, et al.
Published: (2025)

How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
by: Im, Shawn, et al.
Published: (2026)

Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding
by: Jeon, Jaehyun, et al.
Published: (2025)

Transformer-VQ: Linear-Time Transformers via Vector Quantization
by: Lingle, Lucas D.
Published: (2023)

SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks
by: Song, Jiwon, et al.
Published: (2024)

Enhancing Learned Knowledge in LoRA Adapters Through Efficient Contrastive Decoding on Ascend NPUs
by: Heisler, Morgan Lindsay, et al.
Published: (2025)

FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
by: Williams, Joshua Nathaniel, et al.
Published: (2024)

Bias in LLMs as Annotators: The Effect of Party Cues on Labelling Decision by Large Language Models
by: Vera, Sebastian Vallejo, et al.
Published: (2024)

Diversity Measurement and Subset Selection for Instruction Tuning Datasets
by: Wang, Peiqi, et al.
Published: (2024)

Learning Adapter Rank via Symmetry Breaking
by: Doyle, Cooper, et al.
Published: (2025)

Dual-Personalizing Adapter for Federated Foundation Models
by: Yang, Yiyuan, et al.
Published: (2024)

Neutral Residues: Revisiting Adapters for Model Extension
by: Talla, Franck Signe, et al.
Published: (2024)