Cover Image

Efficient Resource-Constrained Training of Transformers via Subspace Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nguyen, Le-Trung, Tartaglione, Enzo, Nguyen, Van-Tam
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning
by: Nguyen, Le-Trung, et al.
Published: (2025)

Study of Training Dynamics for Memory-Constrained Fine-Tuning
by: Quélennec, Aël, et al.
Published: (2025)

Memory Constrained Dynamic Subnetwork Update for Transfer Learning
by: Quélennec, Aël, et al.
Published: (2025)

Activation Map Compression through Tensor Decomposition for Deep Learning
by: Nguyen, Le-Trung, et al.
Published: (2024)

Layer Collapse Can be Induced by Unstructured Pruning
by: Liao, Zhu, et al.
Published: (2024)

Memory-Optimized Once-For-All Network
by: Girard, Maxime, et al.
Published: (2024)

Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers
by: Liao, Zhu, et al.
Published: (2024)

Debiasing surgeon: fantastic weights and how to find them
by: Nahon, Rémi, et al.
Published: (2024)

DSD$^2$: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?
by: Quétu, Victor, et al.
Published: (2023)

Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees
by: Nguyen, Thien Hang, et al.
Published: (2024)

Optimizing Specific and Shared Parameters for Efficient Parameter Tuning
by: Nguyen, Van-Anh, et al.
Published: (2025)

Optimizing Multi-Stage Language Models for Effective Text Retrieval
by: Trung, Quang Hoang, et al.
Published: (2024)

Gap Safe Screening Rules for Fast Training of Robust Support Vector Machines under Feature Noise
by: Nguyen, Tan-Hau, et al.
Published: (2026)

High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces
by: Nguyen, Quoc-Anh Hoang, et al.
Published: (2024)

The Simpler The Better: An Entropy-Based Importance Metric To Reduce Neural Networks' Depth
by: Quétu, Victor, et al.
Published: (2024)

Sharpness-Guided Group Relative Policy Optimization via Probability Shaping
by: Le, Tue, et al.
Published: (2025)

Efficient Adaptation of Deep Neural Networks for Semantic Segmentation in Space Applications
by: Olivi, Leonardo, et al.
Published: (2025)

An Efficient Orlicz-Sobolev Approach for Transporting Unbalanced Measures on a Graph
by: Le, Tam, et al.
Published: (2025)

Structured Pruning for Diverse Best-of-N Reasoning Optimization
by: Nguyen, Hieu Trung, et al.
Published: (2025)

RepLoRA: Reparameterizing Low-Rank Adaptation via the Perspective of Mixture of Experts
by: Truong, Tuan, et al.
Published: (2025)

Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience
by: Huynh, Thanh Trung, et al.
Published: (2024)

SAVA: Scalable Learning-Agnostic Data Valuation
by: Kessler, Samuel, et al.
Published: (2024)

Generalized Sobolev Transport for Probability Measures on a Graph
by: Le, Tam, et al.
Published: (2024)

Optimal Transport for Measures with Noisy Tree Metric
by: Le, Tam, et al.
Published: (2023)

Feature Optimization for Time Series Forecasting via Novel Randomized Uphill Climbing
by: Van Thanh, Nguyen
Published: (2025)

Feature-Aware (Hyper)graph Generation via Next-Scale Prediction
by: Gailhard, Dorian, et al.
Published: (2025)

Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models
by: Pham, Cuong, et al.
Published: (2025)

Resource-Efficient Federated Multimodal Learning via Layer-wise and Progressive Training
by: Tun, Ye Lin, et al.
Published: (2024)

Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
by: Pham, Cuong, et al.
Published: (2025)

Safety Game: Inference-Time Alignment of Black-Box LLMs via Constrained Optimization
by: Nguyen, Tuan, et al.
Published: (2025)

Securing SIM-Assisted Wireless Networks via Quantum Reinforcement Learning
by: Hoang, Le-Hung, et al.
Published: (2026)

Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts
by: Le, Minh, et al.
Published: (2024)

Adaptive Two-Phase Finetuning LLMs for Japanese Legal Text Retrieval
by: Trung, Quang Hoang, et al.
Published: (2024)

Accelerating Transformers with Spectrum-Preserving Token Merging
by: Tran, Hoai-Chau, et al.
Published: (2024)

Risk Bounds for Mixture Density Estimation on Compact Domains via the $h$-Lifted Kullback--Leibler Divergence
by: Chong, Mark Chiu, et al.
Published: (2024)

MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment
by: Hong, Dang Nguyen, et al.
Published: (2026)

Fake Advertisements Detection Using Automated Multimodal Learning: A Case Study for Vietnamese Real Estate Data
by: Nguyen, Duy, et al.
Published: (2025)

Post-Transfer Learning Statistical Inference in High-Dimensional Regression
by: Tam, Nguyen Vu Khai, et al.
Published: (2025)

Guessing Efficiently for Constrained Subspace Approximation
by: Bhaskara, Aditya, et al.
Published: (2025)

CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition
by: Pham, Quang, et al.
Published: (2024)