Image de couverture de livre

PETRA: Parallel End-to-end Training with Reversible Architectures

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Rivaud, Stéphane, Fournier, Louis, Pumir, Thomas, Belilovsky, Eugene, Eickenberg, Michael, Oyallon, Edouard
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Cyclic Data Parallelism for Efficient Parallelism of Deep Neural Networks
par: Fournier, Louis, et autres
Publié: (2024)

ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
par: Nabli, Adel, et autres
Publié: (2024)

Stabilizing Native Low-Rank LLM Pretraining
par: Janson, Paul, et autres
Publié: (2026)

Model Parallelism With Subnetwork Data Parallelism
par: Singh, Vaibhav, et autres
Publié: (2025)

WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average
par: Fournier, Louis, et autres
Publié: (2024)

From Feature Visualization to Visual Circuits: Effect of Adversarial Model Manipulation
par: Nanfack, Geraldin, et autres
Publié: (2024)

$μ$LO: Compute-Efficient Meta-Generalization of Learned Optimizers
par: Thérien, Benjamin, et autres
Publié: (2024)

DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic prediction
par: Morel, Rudy, et autres
Publié: (2025)

Test-time Generalization for Physics through Neural Operator Splitting
par: Serrano, Louis, et autres
Publié: (2026)

Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
par: Miahi, Erfan, et autres
Publié: (2026)

FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups
par: Nanfack, Geraldin, et autres
Publié: (2025)

Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients
par: Legate, Gwen, et autres
Publié: (2025)

Unbiased Approximate Vector-Jacobian Products for Efficient Backpropagation
par: Bakong, Killian, et autres
Publié: (2026)

Celo: Training Versatile Learned Optimizers on a Compute Diet
par: Moudgil, Abhinav, et autres
Publié: (2025)

Model Breadcrumbs: Scaling Multi-Task Model Merging with Sparse Masks
par: Davari, MohammadReza, et autres
Publié: (2023)

Heterogeneous Low-Bandwidth Pre-Training of LLMs
par: Obeidi, Yazan, et autres
Publié: (2026)

Dual-Phase Continual Learning: Supervised Adaptation Meets Unsupervised Retention
par: Singh, Vaibhav, et autres
Publié: (2024)

When Data Falls Short: Grokking Below the Critical Threshold
par: Singh, Vaibhav, et autres
Publié: (2025)

Celo2: Towards Learned Optimization Free Lunch
par: Moudgil, Abhinav, et autres
Publié: (2026)

Efficient Refusal Ablation in LLM through Optimal Transport
par: Nanfack, Geraldin, et autres
Publié: (2026)

Accelerating Training with Neuron Interaction and Nowcasting Networks
par: Knyazev, Boris, et autres
Publié: (2024)

On Non-Linear operators for Geometric Deep Learning
par: Sergeant-Perthuis, Grégoire, et autres
Publié: (2022)

Communication Efficient LLM Pre-training with SparseLoCo
par: Sarfi, Amir, et autres
Publié: (2025)

Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting
par: Hameed, Humza Wajid, et autres
Publié: (2024)

DragD3D: Realistic Mesh Editing with Rigidity Control Driven by 2D Diffusion Priors
par: Xie, Tianhao, et autres
Publié: (2023)

End-to-end Planner Training for Language Modeling
par: Cornille, Nathan, et autres
Publié: (2024)

Rethinking Prompt Optimization: Reinforcement, Diversification, and Migration in Blackbox LLMs
par: Davari, MohammadReza, et autres
Publié: (2025)

DOGE-Train: Discrete Optimization on GPU with End-to-end Training
par: Abbas, Ahmed, et autres
Publié: (2022)

MuLoCo: Muon is a practical inner optimizer for DiLoCo
par: Thérien, Benjamin, et autres
Publié: (2025)

Less is More: Undertraining Experts Improves Model Upcycling
par: Horoi, Stefan, et autres
Publié: (2025)

End-to-end Training for Recommendation with Language-based User Profiles
par: Gao, Zhaolin, et autres
Publié: (2024)

Statistical Component Separation for Targeted Signal Recovery in Noisy Mixtures
par: Blancard, Bruno Régaldo-Saint, et autres
Publié: (2023)

Meta-learning Optimizers for Communication-Efficient Learning
par: Joseph, Charles-Étienne, et autres
Publié: (2023)

PETRA: Pretrained Evolutionary Transformer for SARS-CoV-2 Mutation Prediction
par: Zou, Xu
Publié: (2025)

Non-Uniform Parameter-Wise Model Merging
par: Camacho, Albert Manuel Orozco, et autres
Publié: (2024)

PyLO: Towards Accessible Learned Optimizers in PyTorch
par: Janson, Paul, et autres
Publié: (2025)

End-to-end Conditional Robust Optimization
par: Chenreddy, Abhilash, et autres
Publié: (2024)

AdaFisher: Adaptive Second Order Optimization via Fisher Information
par: Gomes, Damien Martins, et autres
Publié: (2024)

DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbone
par: Singh, Vaibhav, et autres
Publié: (2025)

Test Time Adaptation Using Adaptive Quantile Recalibration
par: Mehrbod, Paria, et autres
Publié: (2025)