Copertina

NoLoCo: No-all-reduce Low Communication Training Method for Large Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kolehmainen, Jari, Blagoev, Nikolay, Donaghy, John, Ersoy, Oğuzhan, Nies, Christopher
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

SkipPipe: Partial and Reordered Pipelining Framework for Training LLMs in Heterogeneous Networks
di: Blagoev, Nikolay, et al.
Pubblicazione: (2025)

Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO
di: Blagoev, Nikolay, et al.
Pubblicazione: (2025)

All is Not Lost: LLM Recovery without Checkpoints
di: Blagoev, Nikolay, et al.
Pubblicazione: (2025)

HDEE: Heterogeneous Domain Expert Ensemble
di: Ersoy, Oğuzhan, et al.
Pubblicazione: (2025)

F-TIS: Harnessing Diverse Models in Collaborative GRPO
di: Blagoev, Nikolay, et al.
Pubblicazione: (2026)

Backdoor Attacks on Decentralised Post-Training
di: Ersoy, Oğuzhan, et al.
Pubblicazione: (2026)

Training-Free Dynamic Upcycling of Expert Language Models
di: Fanì, Eros, et al.
Pubblicazione: (2026)

Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models
di: Blagoev, Nikolay, et al.
Pubblicazione: (2025)

Sharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharing
di: Amico, Jeffrey, et al.
Pubblicazione: (2025)

LoCo: Low-Bit Communication Adaptor for Large-scale Model Training
di: Xie, Xingyu, et al.
Pubblicazione: (2024)

DiLoCo: Distributed Low-Communication Training of Language Models
di: Douillard, Arthur, et al.
Pubblicazione: (2023)

DEI: Diversity in Evolutionary Inference for Quality-Diversity Search
di: Donaghy, John, et al.
Pubblicazione: (2026)

DiLoCoX: A Low-Communication Large-Scale Training Framework for Decentralized Cluster
di: Qi, Ji, et al.
Pubblicazione: (2025)

Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Data
di: Abad, Gorka, et al.
Pubblicazione: (2023)

OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
di: Jaghouar, Sami, et al.
Pubblicazione: (2024)

Robust Inference Methods for Latent Group Panel Models under Possible Group Non-Separation
di: Akgun, Oguzhan, et al.
Pubblicazione: (2025)

BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Training
di: Condat, Laurent, et al.
Pubblicazione: (2026)

LoLCATs: On Low-Rank Linearizing of Large Language Models
di: Zhang, Michael, et al.
Pubblicazione: (2024)

TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
di: Mu, Lin, et al.
Pubblicazione: (2026)

AdLoCo: adaptive batching significantly improves communications efficiency and convergence for Large Language Models
di: Kutuzov, Nikolay, et al.
Pubblicazione: (2025)

Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities
di: Hao, Zhiwei, et al.
Pubblicazione: (2025)

Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition
di: Yu, Yu, et al.
Pubblicazione: (2024)

Communication Efficient LLM Pre-training with SparseLoCo
di: Sarfi, Amir, et al.
Pubblicazione: (2025)

Towards a Small Language Model Lifecycle Framework
di: Miraghaei, Parsa, et al.
Pubblicazione: (2025)

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
di: Condat, Laurent, et al.
Pubblicazione: (2024)

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
di: Charles, Zachary, et al.
Pubblicazione: (2025)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
di: Yang, Yifan, et al.
Pubblicazione: (2024)

LoFT-LLM: Low-Frequency Time-Series Forecasting with Large Language Models
di: You, Jiacheng, et al.
Pubblicazione: (2025)

LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Models
di: Abdi, Hossein, et al.
Pubblicazione: (2024)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
di: Zhang, Jun, et al.
Pubblicazione: (2025)

LoRIF: Low-Rank Influence Functions for Scalable Training Data Attribution
di: Li, Shuangqi, et al.
Pubblicazione: (2026)

Sensitivity-LoRA: Low-Load Sensitivity-Based Fine-Tuning for Large Language Models
di: Zhang, Hao, et al.
Pubblicazione: (2025)

C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
di: Rahmati, Amir Hossein, et al.
Pubblicazione: (2025)

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
di: Zhang, Yilang, et al.
Pubblicazione: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
di: Hayou, Soufiane, et al.
Pubblicazione: (2024)

Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs
di: Defazio, Aaron, et al.
Pubblicazione: (2025)

LoRDO: Distributed Low-Rank Optimization with Infrequent Communication
di: Jovanović, Andrej, et al.
Pubblicazione: (2026)

BaLoRA: Bayesian Low-Rank Adaptation of Large Scale Models
di: Coscia, Dario, et al.
Pubblicazione: (2026)

DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models
di: Iacob, Alex, et al.
Pubblicazione: (2025)

MuLoCo: Muon is a practical inner optimizer for DiLoCo
di: Thérien, Benjamin, et al.
Pubblicazione: (2025)