Image de couverture de livre

Harnessing Orthogonality to Train Low-Rank Neural Networks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Coquelin, Daniel, Flügel, Katharina, Weiel, Marie, Kiefer, Nicholas, Debus, Charlotte, Streit, Achim, Götz, Markus
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Feed-Forward Optimization With Delayed Feedback for Neural Network Training
par: Flügel, Katharina, et autres
Publié: (2023)

AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
par: Coquelin, Daniel, et autres
Publié: (2024)

Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients
par: Flügel, Katharina, et autres
Publié: (2024)

Model Fusion via Neuron Transplantation
par: Öz, Muhammed, et autres
Publié: (2025)

A Comparative Study of Pruning Methods in Transformer-based Time Series Forecasting
par: Kiefer, Nicholas, et autres
Publié: (2024)

Energy Consumption in Parallel Neural Network Training
par: Huber, Philipp, et autres
Publié: (2025)

Differentiable Power-Flow Optimization
par: Öz, Muhammed, et autres
Publié: (2026)

ReCycle: Fast and Efficient Long Time Series Forecasting with Residual Cyclic Transformers
par: Weyrauch, Arvid, et autres
Publié: (2024)

Sampling Parallelism for Fast and Efficient Bayesian Learning
par: Özdemir, Asena Karolin, et autres
Publié: (2026)

Jigsaw: Training Multi-Billion-Parameter AI Weather Models with Optimized Model Parallelism
par: Kieckhefen, Deifilia, et autres
Publié: (2025)

Bayesian Lottery Ticket Hypothesis
par: Kuhn, Nicholas, et autres
Publié: (2026)

Massively Parallel Genetic Optimization through Asynchronous Propagation of Populations
par: Taubert, Oskar, et autres
Publié: (2023)

PETNet -- Coincident Particle Event Detection using Spiking Neural Networks
par: Debus, Jan, et autres
Publié: (2025)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
par: He, Wei, et autres
Publié: (2025)

Low-Rank Quantization-Aware Training for LLMs
par: Bondarenko, Yelysei, et autres
Publié: (2024)

Low-Rank Tensor Decompositions for the Theory of Neural Networks
par: Borsoi, Ricardo, et autres
Publié: (2025)

Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction
par: Hess, Florian, et autres
Publié: (2026)

Enhancing Trustworthiness of Graph Neural Networks with Rank-Based Conformal Training
par: Wang, Ting, et autres
Publié: (2025)

Training Neural Networks from Scratch with Parallel Low-Rank Adapters
par: Huh, Minyoung, et autres
Publié: (2024)

Inverse Design of Optical Multilayer Thin Films using Robust Masked Diffusion Models
par: Schaible, Jonas, et autres
Publié: (2026)

HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks
par: Ma, Yihong, et autres
Publié: (2023)

Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
par: Patankar, Dhruv, et autres
Publié: (2026)

A Convexity-dependent Two-Phase Training Algorithm for Deep Neural Networks
par: Hrycej, Tomas, et autres
Publié: (2025)

Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization
par: Hajimolahoseini, Habib, et autres
Publié: (2023)

Forward Only Learning for Orthogonal Neural Networks of any Depth
par: Caillon, Paul, et autres
Publié: (2025)

Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations
par: Xu, Alec S., et autres
Publié: (2026)

HD-PiSSA: High-Rank Distributed Orthogonal Adaptation
par: Wang, Yiding, et autres
Publié: (2025)

Joint Tensor-Train Parameterization for Efficient and Expressive Low-Rank Adaptation
par: Qi, Jun, et autres
Publié: (2025)

Low Tensor-Rank Adaptation of Kolmogorov--Arnold Networks
par: Gao, Yihang, et autres
Publié: (2025)

Approximated Orthogonal Projection Unit: Stabilizing Regression Network Training Using Natural Gradient
par: Wang, Shaoqi, et autres
Publié: (2024)

Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training
par: Emadi, Seyed Morteza
Publié: (2026)

Exploring Federated Learning for Thermal Urban Feature Segmentation -- A Comparison of Centralized and Decentralized Approaches
par: Duda, Leonhard, et autres
Publié: (2025)

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
par: Liu, Ziyue, et autres
Publié: (2025)

Q3R: Quadratic Reweighted Rank Regularizer for Effective Low-Rank Training
par: Ghosh, Ipsita, et autres
Publié: (2025)

LoRDO: Distributed Low-Rank Optimization with Infrequent Communication
par: Jovanović, Andrej, et autres
Publié: (2026)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
par: Su, DiJia, et autres
Publié: (2025)

Lotus: Efficient LLM Training by Randomized Low-Rank Gradient Projection with Adaptive Subspace Switching
par: Miao, Tianhao, et autres
Publié: (2026)

Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
par: Zhang, Chengqian, et autres
Publié: (2026)

MODE: Efficient Time Series Prediction with Mamba Enhanced by Low-Rank Neural ODEs
par: Chen, Xingsheng, et autres
Publié: (2026)

Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation
par: Zhang, Xiyang, et autres
Publié: (2026)