Copertina

nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yao, Yiqun, fan, Siqi, Huang, Xiusheng, Fang, Xuezhi, Li, Xiang, Ni, Ziyi, Jiang, Xin, Meng, Xuying, Han, Peng, Shang, Shuo, Liu, Kang, Sun, Aixin, Wang, Yequan
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

If an LLM Were a Character, Would It Know Its Own Story? Evaluating Lifelong Learning in LLMs
di: Fan, Siqi, et al.
Pubblicazione: (2025)

GCRE-GPT: A Generative Model for Comparative Relation Extraction
di: Wang, Yequan, et al.
Pubblicazione: (2023)

Not All Layers of LLMs Are Necessary During Inference
di: Fan, Siqi, et al.
Pubblicazione: (2024)

Open-domain Implicit Format Control for Large Language Model Generation
di: Yao, Yiqun, et al.
Pubblicazione: (2024)

EgoMem: Lifelong Memory Agent for Full-duplex Omnimodal Models
di: Yao, Yiqun, et al.
Pubblicazione: (2025)

RoboEgo System Card: An Omnimodal Model with Native Full Duplexity
di: Yao, Yiqun, et al.
Pubblicazione: (2025)

Sketch: A Toolkit for Streamlining LLM Operations
di: Jiang, Xin, et al.
Pubblicazione: (2024)

FLM-101B: An Open LLM and How to Train It with $100K Budget
di: Li, Xiang, et al.
Pubblicazione: (2023)

The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models
di: Fan, Siqi, et al.
Pubblicazione: (2025)

FLM-Audio: Natural Monologues Improves Native Full-Duplex Chatbots via Dual Training
di: Yao, Yiqun, et al.
Pubblicazione: (2025)

Position-Aware Depth Decay Decoding ($D^3$): Boosting Large Language Model Inference Efficiency
di: Fan, Siqi, et al.
Pubblicazione: (2025)

Reasons and Solutions for the Decline in Model Performance after Editing
di: Huang, Xiusheng, et al.
Pubblicazione: (2024)

Commonsense Knowledge Editing Based on Free-Text in LLMs
di: Huang, Xiusheng, et al.
Pubblicazione: (2024)

Masked Structural Growth for 2x Faster Language Model Pre-training
di: Yao, Yiqun, et al.
Pubblicazione: (2023)

Toward Embodied AGI: A Review of Embodied AI and the Road Ahead
di: Wang, Yequan, et al.
Pubblicazione: (2025)

Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
di: Luo, Haitong, et al.
Pubblicazione: (2025)

Capability Localization: Capabilities Can be Localized rather than Individual Knowledge
di: Huang, Xiusheng, et al.
Pubblicazione: (2025)

Mutual Enhancement Between Global Tokens and Patch Tokens: From Theory to Practice
di: Huang, Xiusheng, et al.
Pubblicazione: (2026)

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
di: Xing, Xingrun, et al.
Pubblicazione: (2024)

NetGPT: Generative Pretrained Transformer for Network Traffic
di: Meng, Xuying, et al.
Pubblicazione: (2023)

Nethira: A Heterogeneity-aware Hierarchical Pre-trained Model for Network Traffic Classification
di: Lin, Chungang, et al.
Pubblicazione: (2026)

SA-WavLM: Speaker-Aware Self-Supervised Pre-training for Mixture Speech
di: Lin, Jingru, et al.
Pubblicazione: (2024)

Theory-optimal Quantization Based on Flatness
di: Huang, Xiusheng, et al.
Pubblicazione: (2026)

A Contrastive Pre-trained Foundation Model for Deciphering Imaging Noisomics across Modalities
di: Gu, Yuanjie, et al.
Pubblicazione: (2026)

Benchmarking the Performance of Pre-trained LLMs across Urdu NLP Tasks
di: Tahir, Munief Hassan, et al.
Pubblicazione: (2024)

Hint Tuning: Less Data Makes Better Reasoners
di: Fan, Siqi, et al.
Pubblicazione: (2026)

Diverse Controllable Diffusion Policy with Signal Temporal Logic
di: Meng, Yue, et al.
Pubblicazione: (2025)

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
di: Yi, Rongjie, et al.
Pubblicazione: (2024)

UEPS: Robust and Efficient MRI Reconstruction (Pre-trained Model and Demo Data)
di: Zhou, Xiang, et al.
Pubblicazione: (2026)

Spectral-Based Graph Neural Networks for Complementary Item Recommendation
di: Luo, Haitong, et al.
Pubblicazione: (2024)

Are Transformers in Pre-trained LM A Good ASR Encoder? An Empirical Study
di: An, Keyu, et al.
Pubblicazione: (2024)

EAQEC codes from s-Galois hulls decomposition of linear codes
di: Li, Hui, et al.
Pubblicazione: (2024)

Linear complementary pairs of codes over a finite non-commutative Frobenius ring
di: Bhowmick, Sanjit, et al.
Pubblicazione: (2024)

B-cos LM: Efficiently Transforming Pre-trained Language Models for Improved Explainability
di: Wang, Yifan, et al.
Pubblicazione: (2025)

Practical Continual Forgetting for Pre-trained Vision Models
di: Zhao, Hongbo, et al.
Pubblicazione: (2025)

Home After Loss: Housing Relocation and Affordability Stress Following Spousal Loss
di: Gum‐Ryeong Park
Pubblicazione: (2026)

MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training
di: Zhang, Ruijie, et al.
Pubblicazione: (2026)

BiPFT: Binary Pre-trained Foundation Transformer with Low-rank Estimation of Binarization Residual Polynomials
di: Xing, Xingrun, et al.
Pubblicazione: (2023)

A Closer Look at the Explainability of Contrastive Language-Image Pre-training
di: Li, Yi, et al.
Pubblicazione: (2023)

Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue History
di: Zhong, Qishuai, et al.
Pubblicazione: (2025)