Cover Image

Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lidin, Joel, Sarfi, Amir, Miahi, Erfan, Anthony, Quentin, Chauhan, Shivam, Pappas, Evangelos, Thérien, Benjamin, Belilovsky, Eugene, Dare, Samuel
Format:	Preprint
Published:	2026
Subjects:	Distributed, Parallel, and Cluster Computing Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Incentivizing Permissionless Distributed Learning of LLMs
by: Lidin, Joel, et al.
Published: (2025)

Communication Efficient LLM Pre-training with SparseLoCo
by: Sarfi, Amir, et al.
Published: (2025)

Heterogeneous Low-Bandwidth Pre-Training of LLMs
by: Obeidi, Yazan, et al.
Published: (2026)

Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
by: Miahi, Erfan, et al.
Published: (2026)

Validation of GPU Computation in Decentralized, Trustless Networks
by: Boniardi, Eric, et al.
Published: (2025)

TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference
by: Ong, Jack Min, et al.
Published: (2025)

PyLO: Towards Accessible Learned Optimizers in PyTorch
by: Janson, Paul, et al.
Published: (2025)

JSTprove: Pioneering Verifiable AI for a Trustless Future
by: Gold, Jonathan, et al.
Published: (2025)

ZipperChain: Transmuting Trusted Third-Party Services Into Trustless Atomic Broadcast
by: Bjornsson, Matteo, et al.
Published: (2025)

Self-Supervised Inference of Agents in Trustless Environments
by: Larin, Vladyslav, et al.
Published: (2024)

Secure Autonomous Agent Payments: Verifying Authenticity and Intent in a Trustless Environment
by: Acharya, Vivek
Published: (2025)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
by: Ibrahim, Adam, et al.
Published: (2024)

DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
by: Zhang, Yujie, et al.
Published: (2024)

Emergent Peer-to-Peer Multi-Hub Topology
by: Legheraba, Mohamed Amine, et al.
Published: (2024)

DWDP: Distributed Weight Data Parallelism for High-Performance LLM Inference on NVL72
by: Li, Wanqian, et al.
Published: (2026)

Charting the Uncharted: The Landscape of Monero Peer-to-Peer Network
by: Gao, Yu, et al.
Published: (2025)

PeerSwap: A Peer-Sampler with Randomness Guarantees
by: Guerraoui, Rachid, et al.
Published: (2024)

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
by: Shyam, Vasu, et al.
Published: (2026)

Fully-Distributed Construction of Byzantine-Resilient Dynamic Peer-to-Peer Networks
by: Gupta, Aayush, et al.
Published: (2025)

Maintaining a Bounded Degree Expander in Dynamic Peer-to-Peer Networks
by: Cruciani, Antonio
Published: (2025)

Optimizing Task Scheduling in Fog Computing with Deadline Awareness
by: Sirjani, Mohammad Sadegh, et al.
Published: (2025)

PeerFL: A Simulator for Peer-to-Peer Federated Learning at Scale
by: Luqman, Alka, et al.
Published: (2024)

Multiple Sides of 36 Coins: Measuring Peer-to-Peer Infrastructure Across Cryptocurrencies
by: Kiffer, Lucianna, et al.
Published: (2025)

Host-Side Telemetry for Performance Diagnosis in Cloud and HPC GPU Infrastructure
by: Darzi, Erfan, et al.
Published: (2025)

Towards Communication-Efficient Peer-to-Peer Networks
by: Hourani, Khalid, et al.
Published: (2024)

SoK: Consensus for Fair Message Ordering
by: Li, Zhuolun, et al.
Published: (2024)

Linear Search for Capturing an Oblivious Mobile Target in the Sender/Receiver Model
by: Jawhar, Khaled, et al.
Published: (2025)

Bike Assisted Evacuation on a Line of Robots with Communication Faults
by: Jawhar, Khaled, et al.
Published: (2023)

Comparative Study of Large Language Model Architectures on Frontier
by: Yin, Junqi, et al.
Published: (2024)

BlockDFL: A Blockchain-based Fully Decentralized Peer-to-Peer Federated Learning Framework
by: Qin, Zhen, et al.
Published: (2022)

Towards Learning-Augmented Peer-to-Peer Networks: Self-Stabilizing Graph Linearization with Untrusted Advice
by: Aradhya, Vijeth, et al.
Published: (2025)

Predictable LLM Serving on GPU Clusters
by: Darzi, Erfan, et al.
Published: (2025)

Peer-to-Peer Deep Learning for Beyond-5G IoT
by: Pranav, Srinivasa, et al.
Published: (2023)

A Survey of Recent Advancements in Secure Peer-to-Peer Networks
by: Patel, Raj, et al.
Published: (2025)

Capturing a Moving Target by Two Robots in the F2F Model
by: Jawhar, Khaled, et al.
Published: (2025)

AerialDB: A Federated Peer-to-Peer Spatio-temporal Edge Datastore for Drone Fleets
by: Jaiswal, Shashwat, et al.
Published: (2025)

Characterizing Communication Patterns in Distributed Large Language Model Inference
by: Xu, Lang, et al.
Published: (2025)

Blockchain based Secure Energy Marketplace Scheme to Motivate Peer to Peer Microgrids
by: Awais, Muhammad, et al.
Published: (2022)

Towards a Peer-to-Peer Data Distribution Layer for Efficient and Collaborative Resource Optimization of Distributed Dataflow Applications
by: Scheinert, Dominik, et al.
Published: (2023)

Scientific Workflow Scheduling in Cloud Considering Cold Start and Variable Pricing Model
by: Sarkar, Suvarthi, et al.
Published: (2025)