Copertina

Scalable Parameter and Memory Efficient Pretraining for LLM: Recent Algorithmic Advances and Benchmarking

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Glentis, Athanasios, Li, Jiaxiang, Shang, Qiulin, Han, Andi, Tsaknakis, Ioannis, Wei, Quan, Hong, Mingyi
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Memory-Efficient LLM Pretraining via Minimalist Optimizer Design
di: Glentis, Athanasios, et al.
Pubblicazione: (2025)

Do LLMs Recognize Your Latent Preferences? A Benchmark for Latent Information Discovery in Personalized Interaction
di: Tsaknakis, Ioannis, et al.
Pubblicazione: (2025)

Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
di: Li, Jiaxiang, et al.
Pubblicazione: (2024)

Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates
di: Glentis, Athanasios, et al.
Pubblicazione: (2026)

A Doubly Stochastically Perturbed Algorithm for Linearly Constrained Bilevel Optimization
di: Khanduri, Prashant, et al.
Pubblicazione: (2025)

A Discretization Approach for Bilevel Optimization with Low-Dimensional and Non-Convex Lower-Level
di: Jiang, Xiaotian, et al.
Pubblicazione: (2025)

A Framework for Quantifying How Pre-Training and Context Benefit In-Context Learning
di: Song, Bingqing, et al.
Pubblicazione: (2025)

KunServe: Parameter-centric Memory Management for Efficient Memory Overloading Handling in LLM Serving
di: Cheng, Rongxin, et al.
Pubblicazione: (2024)

Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
di: Huang, Shih-Cheng, et al.
Pubblicazione: (2022)

Muon Outperforms Adam in Tail-End Associative Memory Learning
di: Wang, Shuche, et al.
Pubblicazione: (2025)

A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation
di: Song, Yu, et al.
Pubblicazione: (2025)

RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
di: Wei, Quan, et al.
Pubblicazione: (2025)

SimpleMem: Efficient Lifelong Memory for LLM Agents
di: Liu, Jiaqi, et al.
Pubblicazione: (2026)

Hippocampus: An Efficient and Scalable Memory Module for Agentic AI
di: Li, Yi, et al.
Pubblicazione: (2026)

Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
di: Cao, Jiaqi, et al.
Pubblicazione: (2025)

Beluga: A CXL-Based Memory Architecture for Scalable and Efficient LLM KVCache Management
di: Yang, Xinjun, et al.
Pubblicazione: (2025)

Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment
di: Li, Chenliang, et al.
Pubblicazione: (2024)

ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimization
di: Tso, Joseph, et al.
Pubblicazione: (2026)

Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight
di: Ye, Junze, et al.
Pubblicazione: (2025)

Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
di: Wei, Tianxin, et al.
Pubblicazione: (2025)

BEAM: Bi-level Memory-adaptive Algorithmic Evolution for LLM-Powered Heuristic Design
di: Xiang, Chuyang, et al.
Pubblicazione: (2026)

CloneMem: Benchmarking Long-Term Memory for AI Clones
di: Hu, Sen, et al.
Pubblicazione: (2026)

Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning
di: Zhang, Yang, et al.
Pubblicazione: (2025)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
di: Chang, Aofei, et al.
Pubblicazione: (2024)

ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters
di: Hao, Zhiwei, et al.
Pubblicazione: (2025)

A Meta-Level Learning Algorithm for Sequential Hyper-Parameter Space Reduction in AutoML
di: Borboudakis, Giorgos, et al.
Pubblicazione: (2023)

Effectively Steer LLM To Follow Preference via Building Confident Directions
di: Song, Bingqing, et al.
Pubblicazione: (2025)

LLM Inference Serving: Survey of Recent Advances and Opportunities
di: Li, Baolin, et al.
Pubblicazione: (2024)

Efficient Detection of Bad Benchmark Items with Novel Scalability Coefficients
di: Hardy, Michael, et al.
Pubblicazione: (2026)

Towards Agentic OS: An LLM Agent Framework for Linux Schedulers
di: Zheng, Yusheng, et al.
Pubblicazione: (2025)

Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks
di: Baker, Nermeen Abou, et al.
Pubblicazione: (2026)

LLM-Inspired Pretrain-Then-Finetune for Small-Data, Large-Scale Optimization
di: Zhang, Zishi, et al.
Pubblicazione: (2026)

Beyond URLs: Metadata Diversity and Position for Efficient LLM Pretraining
di: Fan, Dongyang, et al.
Pubblicazione: (2025)

[Re] Benchmarking LLM Capabilities in Negotiation through Scoreable Games
di: Pollo, Jorge Carrasco, et al.
Pubblicazione: (2026)

Tuning LLMs by RAG Principles: Towards LLM-native Memory
di: Wei, Jiale, et al.
Pubblicazione: (2025)

A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems
di: Yi, Zihao, et al.
Pubblicazione: (2024)

Secondary Structure-Guided Novel Protein Sequence Generation with Latent Graph Diffusion
di: Hu, Yutong, et al.
Pubblicazione: (2024)

Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents
di: Wang, Zihao, et al.
Pubblicazione: (2025)

On Entropy Control in LLM-RL Algorithms
di: Shen, Han
Pubblicazione: (2025)

FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization
di: Kong, Minwei, et al.
Pubblicazione: (2026)