Copertina

Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhou, Jiecheng, Tang, Ding, Fu, Rong, Hu, Boni, Xu, Haoran, Wang, Yi, Pei, Zhilin, Su, Zhongling, Liu, Liang, Zhang, Xingcheng, Zhang, Weiming
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence I.2
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

PackMamba: Efficient Processing of Variable-Length Sequences in Mamba training
di: Xu, Haoran, et al.
Pubblicazione: (2024)

TMA-Adaptive FP8 Grouped GEMM: Eliminating Padding Requirements in Low-Precision Training and Inference on Hopper
di: Su, Zhongling, et al.
Pubblicazione: (2025)

H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
di: Tang, Ding, et al.
Pubblicazione: (2025)

Towards Effective and Efficient Continual Pre-training of Large Language Models
di: Chen, Jie, et al.
Pubblicazione: (2024)

Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
di: Yang, Qinyu, et al.
Pubblicazione: (2024)

TC-GS: A Faster Gaussian Splatting Module Utilizing Tensor Cores
di: Liao, Zimu, et al.
Pubblicazione: (2025)

ZeroPP: Unleashing Exceptional Parallelism Efficiency through Tensor-Parallelism-Free Methodology
di: Tang, Ding, et al.
Pubblicazione: (2024)

HySparK: Hybrid Sparse Masking for Large Scale Medical Image Pre-Training
di: Tang, Fenghe, et al.
Pubblicazione: (2024)

TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
di: Hsu, Shang-Ling, et al.
Pubblicazione: (2026)

FlowSteer: Towards Agents Designing Agentic Workflows via Reinforced Progressive Canvas Editing
di: Zhang, Mingda, et al.
Pubblicazione: (2026)

On Initializing Transformers with Pre-trained Embeddings
di: Kim, Ha Young, et al.
Pubblicazione: (2024)

New Skills or Sharper Primitives? A Probabilistic Perspective on the Emergence of Reasoning in RLVR
di: Wang, Zhilin, et al.
Pubblicazione: (2026)

Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion
di: Xu, Wenjie, et al.
Pubblicazione: (2023)

Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation
di: Chen, Mu-Chi, et al.
Pubblicazione: (2026)

Asset Pricing in Pre-trained Transformer
di: Lai, Shanyan
Pubblicazione: (2025)

Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing
di: Wang, Zhilin, et al.
Pubblicazione: (2025)

Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
di: He, Yanjin, et al.
Pubblicazione: (2025)

Zero-Shot Spam Email Classification Using Pre-trained Large Language Models
di: Rojas-Galeano, Sergio
Pubblicazione: (2024)

The Native Spiking Microarchitecture: From Iontronic Primitives to Bit-Exact FP8 Arithmetic
di: Tang, Zhengzheng
Pubblicazione: (2025)

Efficient Attention: Attention with Linear Complexities
di: Shen, Zhuoran, et al.
Pubblicazione: (2018)

Leveraging Personalized PageRank and Higher-Order Topological Structures for Heterophily Mitigation in Graph Neural Networks
di: Wang, Yumeng, et al.
Pubblicazione: (2025)

Universal Adversarial Perturbations for Vision-Language Pre-trained Models
di: Zhang, Peng-Fei, et al.
Pubblicazione: (2024)

Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras
di: Liao, Zimu, et al.
Pubblicazione: (2024)

FlashGS: Efficient 3D Gaussian Splatting for Large-scale and High-resolution Rendering
di: Feng, Guofeng, et al.
Pubblicazione: (2024)

FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning
di: Zhang, Su, et al.
Pubblicazione: (2026)

4D-PreNet: A Unified Preprocessing Framework for 4D-STEM Data Analysis
di: Liu, Mingyu, et al.
Pubblicazione: (2025)

CoMA: Complementary Masking and Hierarchical Dynamic Multi-Window Self-Attention in a Unified Pre-training Framework
di: Li, Jiaxuan, et al.
Pubblicazione: (2025)

FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
di: Liu, Fangxin, et al.
Pubblicazione: (2025)

DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features
di: Chen, Zhangquan, et al.
Pubblicazione: (2024)

DatUS^2: Data-driven Unsupervised Semantic Segmentation with Pre-trained Self-supervised Vision Transformer
di: Kumar, Sonal, et al.
Pubblicazione: (2024)

Hierarchical Pre-Training of Vision Encoders with Large Language Models
di: Lee, Eugene, et al.
Pubblicazione: (2026)

Effective and Efficient Schema-aware Information Extraction Using On-Device Large Language Models
di: Wen, Zhihao, et al.
Pubblicazione: (2025)

Efficient Image Pre-Training with Siamese Cropped Masked Autoencoders
di: Eymaël, Alexandre, et al.
Pubblicazione: (2024)

Towards Building Private LLMs: Exploring Multi-Node Expert Parallelism on Apple Silicon for Mixture-of-Experts Large Language Model
di: Chen, Mu-Chi, et al.
Pubblicazione: (2025)

On the Role of Pre-trained Embeddings in Binary Code Analysis
di: Maier, Alwin, et al.
Pubblicazione: (2025)

Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B
di: Zhang, Xue
Pubblicazione: (2025)

VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts
di: Qi, Peigui, et al.
Pubblicazione: (2026)

Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
di: Severin, Nikita, et al.
Pubblicazione: (2026)

Out-of-Sight Embodied Agents: Multimodal Tracking, Sensor Fusion, and Trajectory Forecasting
di: Zhang, Haichao, et al.
Pubblicazione: (2025)

A Multi-Pass Large Language Model Framework for Precise and Efficient Radiology Report Error Detection
di: Kim, Songsoo, et al.
Pubblicazione: (2025)