Copertina

Pretraining A Large Language Model using Distributed GPUs: A Memory-Efficient Decentralized Paradigm

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Jinrui, Xiao, Chaodong, Wu, Aoqi, Zhang, Xindong, Zhang, Lei
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MLP Memory: A Retriever-Pretrained Memory for Large Language Models
di: Wei, Rubin, et al.
Pubblicazione: (2025)

Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling
di: Yang, Linyao, et al.
Pubblicazione: (2023)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
di: Cao, Jiaqi, et al.
Pubblicazione: (2025)

DEP: A Decentralized Large Language Model Evaluation Protocol
di: Peng, Jianxiang, et al.
Pubblicazione: (2026)

MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models
di: Cheng, Hongrong, et al.
Pubblicazione: (2024)

Unifying Large Language Models and Knowledge Graphs: A Roadmap
di: Pan, Shirui, et al.
Pubblicazione: (2023)

UniVS: Unified and Universal Video Segmentation with Prompts as Queries
di: Li, Minghan, et al.
Pubblicazione: (2024)

CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning
di: Yu, Huimu, et al.
Pubblicazione: (2024)

Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration
di: Bai, Tianyi, et al.
Pubblicazione: (2024)

Knowledge Dependency Estimation for Reliable Question Answering
di: Tong, Chaodong, et al.
Pubblicazione: (2026)

HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering
di: Tong, Chaodong, et al.
Pubblicazione: (2025)

The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining
di: Shao, Jiandong, et al.
Pubblicazione: (2026)

Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective
di: Yu, Yiyao, et al.
Pubblicazione: (2025)

Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models
di: Shi, Luohe, et al.
Pubblicazione: (2024)

Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method
di: Zhang, Weichao, et al.
Pubblicazione: (2024)

On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
di: Zhang, Meishan, et al.
Pubblicazione: (2025)

LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary Captioning
di: Tang, Yolo Yunlong, et al.
Pubblicazione: (2023)

A Tutorial on the Pretrain-Finetune Paradigm for Natural Language Processing
di: Wang, Yu, et al.
Pubblicazione: (2024)

FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing
di: Liu, Xiao-Yang, et al.
Pubblicazione: (2024)

Enhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignment
di: Peng, Tianyu, et al.
Pubblicazione: (2024)

Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning
di: Zhuo, Xingrui, et al.
Pubblicazione: (2025)

INMS: Memory Sharing for Large Language Model based Agents
di: Gao, Hang, et al.
Pubblicazione: (2024)

Capybara-OMNI: An Efficient Paradigm for Building Omni-Modal Language Models
di: Ji, Xingguang, et al.
Pubblicazione: (2025)

Efficient Pretraining Length Scaling
di: Wu, Bohong, et al.
Pubblicazione: (2025)

SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models
di: Liu, Han, et al.
Pubblicazione: (2026)

A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models
di: Chen, Huiyao, et al.
Pubblicazione: (2025)

Benchmarking Gender and Political Bias in Large Language Models
di: Yang, Jinrui, et al.
Pubblicazione: (2025)

A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
di: Luo, Junyu, et al.
Pubblicazione: (2025)

DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimization
di: Deng, Hexuan, et al.
Pubblicazione: (2024)

Adapting Pretrained Language Models for Citation Classification via Self-Supervised Contrastive Learning
di: Li, Tong, et al.
Pubblicazione: (2025)

Semantic Reformulation Entropy for Robust Hallucination Detection in QA Tasks
di: Tong, Chaodong, et al.
Pubblicazione: (2025)

RECKON: Large-scale Reference-based Efficient Knowledge Evaluation for Large Language Model
di: Zhang, Lin, et al.
Pubblicazione: (2025)

PretrainRL: Alleviating Factuality Hallucination of Large Language Models at the Beginning
di: Liu, Langming, et al.
Pubblicazione: (2026)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
di: Guo, Ping, et al.
Pubblicazione: (2025)

Adaptive Draft-Verification for Efficient Large Language Model Decoding
di: Liu, Xukun, et al.
Pubblicazione: (2024)

FlashDecoding++: Faster Large Language Model Inference on GPUs
di: Hong, Ke, et al.
Pubblicazione: (2023)

MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models
di: Li, Jiazheng, et al.
Pubblicazione: (2025)

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
di: Liu, Jing, et al.
Pubblicazione: (2024)

The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis
di: Yang, Chen, et al.
Pubblicazione: (2024)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
di: Zhang, Hengxiang, et al.
Pubblicazione: (2024)