Image de couverture de livre

Reparameterized LLM Training via Orthogonal Equivalence Transformation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Qiu, Zeju, Buchholz, Simon, Xiao, Tim Z., Dax, Maximilian, Schölkopf, Bernhard, Liu, Weiyang
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Orthogonal Finetuning Made Scalable
par: Qiu, Zeju, et autres
Publié: (2025)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
par: Qiu, Zeju, et autres
Publié: (2026)

Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
par: Shi, Kexuan, et autres
Publié: (2026)

Identifying Intervenable and Interpretable Features via Orthogonality Regularization
par: Miller, Moritz, et autres
Publié: (2026)

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
par: Liu, Weiyang, et autres
Publié: (2023)

Rigidity-Aware Geometric Pretraining for Protein Design and Conformational Ensembles
par: Ni, Zhanghan, et autres
Publié: (2026)

Can Large Language Models Understand Symbolic Graphics Programs?
par: Qiu, Zeju, et autres
Publié: (2024)

Controlling Text-to-Image Diffusion by Orthogonal Finetuning
par: Qiu, Zeju, et autres
Publié: (2023)

Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling
par: Xiao, Tim Z., et autres
Publié: (2025)

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
par: You, Haoran, et autres
Publié: (2024)

Algorithmic causal structure emerging through compression
par: Wendong, Liang, et autres
Publié: (2025)

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector
par: Zhang, Andi, et autres
Publié: (2024)

PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective
par: Huang, Yangyi, et autres
Publié: (2026)

Verbalized Machine Learning: Revisiting Machine Learning with Language Models
par: Xiao, Tim Z., et autres
Publié: (2024)

Limits of Transformer Language Models on Learning to Compose Algorithms
par: Thomm, Jonathan, et autres
Publié: (2024)

Learning to Reason Efficiently with A* Post-Training
par: Opedal, Andreas, et autres
Publié: (2026)

A Compact Representation for Bayesian Neural Networks By Removing Permutation Symmetry
par: Xiao, Tim Z., et autres
Publié: (2023)

Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
par: Rajendran, Goutham, et autres
Publié: (2024)

Improving Large Language Model Safety with Contrastive Representation Learning
par: Simko, Samuel, et autres
Publié: (2025)

Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
par: Goddard, Charles, et autres
Publié: (2025)

Counterfactual reasoning: an analysis of in-context emergence
par: Miller, Moritz, et autres
Publié: (2025)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
par: Li, Zeju, et autres
Publié: (2026)

MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs
par: Opedal, Andreas, et autres
Publié: (2024)

ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms
par: Xu, Bingxin, et autres
Publié: (2025)

Causal Component Analysis
par: Wendong, Liang, et autres
Publié: (2023)

OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference
par: Shin, Seungjun, et autres
Publié: (2025)

Improving LLM Code Reasoning via Semantic Equivalence Self-Play with Formal Verification
par: Barone, Antonio Valerio Miceli, et autres
Publié: (2026)

Intrinsically Interpretable Attention via Sparse Post-Training
par: Draye, Florent, et autres
Publié: (2025)

Robustness of Nonlinear Representation Learning
par: Buchholz, Simon, et autres
Publié: (2025)

Generalized Interpolating Discrete Diffusion
par: von Rütte, Dimitri, et autres
Publié: (2025)

Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
par: Guo, Siyuan, et autres
Publié: (2024)

Selection of LLM Fine-Tuning Data based on Orthogonal Rules
par: Li, Xiaomin, et autres
Publié: (2024)

MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
par: Lin, Bokai, et autres
Publié: (2024)

Can Large Language Models Infer Causation from Correlation?
par: Jin, Zhijing, et autres
Publié: (2023)

Analyzing the Role of Semantic Representations in the Era of Large Language Models
par: Jin, Zhijing, et autres
Publié: (2024)

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
par: Xi, Zhiheng, et autres
Publié: (2024)

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
par: Arbel, Iftach, et autres
Publié: (2024)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
par: Xiao, Chaojun, et autres
Publié: (2024)

Rethinking GSPO: The Perplexity-Entropy Equivalence
par: Liu, Chi
Publié: (2025)

Muon is Scalable for LLM Training
par: Liu, Jingyuan, et autres
Publié: (2025)