Cover Image

Reparameterized LLM Training via Orthogonal Equivalence Transformation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qiu, Zeju, Buchholz, Simon, Xiao, Tim Z., Dax, Maximilian, Schölkopf, Bernhard, Liu, Weiyang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Orthogonal Finetuning Made Scalable
by: Qiu, Zeju, et al.
Published: (2025)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
by: Qiu, Zeju, et al.
Published: (2026)

Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
by: Shi, Kexuan, et al.
Published: (2026)

Identifying Intervenable and Interpretable Features via Orthogonality Regularization
by: Miller, Moritz, et al.
Published: (2026)

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
by: Liu, Weiyang, et al.
Published: (2023)

Rigidity-Aware Geometric Pretraining for Protein Design and Conformational Ensembles
by: Ni, Zhanghan, et al.
Published: (2026)

Can Large Language Models Understand Symbolic Graphics Programs?
by: Qiu, Zeju, et al.
Published: (2024)

Controlling Text-to-Image Diffusion by Orthogonal Finetuning
by: Qiu, Zeju, et al.
Published: (2023)

Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
by: Xiao, Tim Z., et al.
Published: (2025)

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
by: You, Haoran, et al.
Published: (2024)

Algorithmic causal structure emerging through compression
by: Wendong, Liang, et al.
Published: (2025)

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector
by: Zhang, Andi, et al.
Published: (2024)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
by: Huang, Yangyi, et al.
Published: (2026)

Verbalized Machine Learning: Revisiting Machine Learning with Language Models
by: Xiao, Tim Z., et al.
Published: (2024)

Limits of Transformer Language Models on Learning to Compose Algorithms
by: Thomm, Jonathan, et al.
Published: (2024)

Learning to Reason Efficiently with A* Post-Training
by: Opedal, Andreas, et al.
Published: (2026)

A Compact Representation for Bayesian Neural Networks By Removing Permutation Symmetry
by: Xiao, Tim Z., et al.
Published: (2023)

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
by: Rajendran, Goutham, et al.
Published: (2024)

Improving Large Language Model Safety with Contrastive Representation Learning
by: Simko, Samuel, et al.
Published: (2025)

Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
by: Goddard, Charles, et al.
Published: (2025)

Counterfactual reasoning: an analysis of in-context emergence
by: Miller, Moritz, et al.
Published: (2025)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
by: Li, Zeju, et al.
Published: (2026)

MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs
by: Opedal, Andreas, et al.
Published: (2024)

ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms
by: Xu, Bingxin, et al.
Published: (2025)

Causal Component Analysis
by: Wendong, Liang, et al.
Published: (2023)

OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
by: Shin, Seungjun, et al.
Published: (2025)

Improving LLM Code Reasoning via Semantic Equivalence Self-Play with Formal Verification
by: Barone, Antonio Valerio Miceli, et al.
Published: (2026)

Intrinsically Interpretable Attention via Sparse Post-Training
by: Draye, Florent, et al.
Published: (2025)

Robustness of Nonlinear Representation Learning
by: Buchholz, Simon, et al.
Published: (2025)

Generalized Interpolating Discrete Diffusion
by: von Rütte, Dimitri, et al.
Published: (2025)

Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
by: Guo, Siyuan, et al.
Published: (2024)

Selection of LLM Fine-Tuning Data based on Orthogonal Rules
by: Li, Xiaomin, et al.
Published: (2024)

MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
by: Lin, Bokai, et al.
Published: (2024)

Can Large Language Models Infer Causation from Correlation?
by: Jin, Zhijing, et al.
Published: (2023)

Analyzing the Role of Semantic Representations in the Era of Large Language Models
by: Jin, Zhijing, et al.
Published: (2024)

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
by: Xi, Zhiheng, et al.
Published: (2024)

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
by: Arbel, Iftach, et al.
Published: (2024)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by: Xiao, Chaojun, et al.
Published: (2024)

Rethinking GSPO: The Perplexity-Entropy Equivalence
by: Liu, Chi
Published: (2025)

Muon is Scalable for LLM Training
by: Liu, Jingyuan, et al.
Published: (2025)