Copertina

Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Nielsen, Jacob, Schneider-Kamp, Peter, Galke, Lukas
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization
di: Nielsen, Jacob, et al.
Pubblicazione: (2024)

BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks
di: Nielsen, Jacob, et al.
Pubblicazione: (2024)

Sparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsity
di: Zhang, Di, et al.
Pubblicazione: (2026)

BitNet a4.8: 4-bit Activations for 1-bit LLMs
di: Wang, Hongyu, et al.
Pubblicazione: (2024)

1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
di: Wang, Jinheng, et al.
Pubblicazione: (2024)

BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
di: Wang, Hongyu, et al.
Pubblicazione: (2025)

DeToNATION: Decoupled Torch Network-Aware Training on Interlinked Online Nodes
di: From, Mogens Henrik, et al.
Pubblicazione: (2025)

BitNet b1.58 2B4T Technical Report
di: Ma, Shuming, et al.
Pubblicazione: (2025)

1.58-bit FLUX
di: Yang, Chenglin, et al.
Pubblicazione: (2024)

BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing
di: Kawamura, Masaya, et al.
Pubblicazione: (2025)

BitNet Distillation
di: Wu, Xun, et al.
Pubblicazione: (2025)

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
di: Ma, Shuming, et al.
Pubblicazione: (2024)

Training Language Models to Use Prolog as a Tool
di: Mellgren, Niklas, et al.
Pubblicazione: (2025)

SDUs DAISY: A Benchmark for Danish Culture
di: Nielsen, Jacob, et al.
Pubblicazione: (2026)

FlexMoRE: A Flexible Mixture of Rank-heterogeneous Experts for Efficient Federatedly-trained Large Language Models
di: Pirchert, Annemette Brok, et al.
Pubblicazione: (2026)

1-Bit FQT: Pushing the Limit of Fully Quantized Training to 1-bit
di: Gao, Chang, et al.
Pubblicazione: (2024)

FraQAT: Quantization Aware Training with Fractional bits
di: Morreale, Luca, et al.
Pubblicazione: (2025)

BitsFusion: 1.99 bits Weight Quantization of Diffusion Model
di: Sui, Yang, et al.
Pubblicazione: (2024)

1-bit Quantized Continuous Aperture Arrays
di: Ranasinghe, Kuranage Roche Rayan, et al.
Pubblicazione: (2026)

MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training
di: Kim, Yongwan, et al.
Pubblicazione: (2026)

Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
di: Torrielli, Federico, et al.
Pubblicazione: (2026)

BitROM: Weight Reload-Free CiROM Architecture Towards Billion-Parameter 1.58-bit LLM Inference
di: Zhang, Wenlun, et al.
Pubblicazione: (2025)

SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training
di: Jia, Jinda, et al.
Pubblicazione: (2024)

Bits and q-bits as versatility measures
di: José R.C. Piqueira
Pubblicazione: (2004)

ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
di: Yuan, Zhengqing, et al.
Pubblicazione: (2024)

Ultra-Quantisation: Efficient Embedding Search via 1.58-bit Encodings
di: Connor, Richard, et al.
Pubblicazione: (2025)

Towards One-bit ASR: Extremely Low-bit Conformer Quantization Using Co-training and Stochastic Precision
di: Li, Zhaoqing, et al.
Pubblicazione: (2025)

Latent-Space Mean-Field Theory for Deep BitNet-like Training: Constrained Gradient Flows with Smooth Quantization and STE Limits
di: Kim, Dongwon, et al.
Pubblicazione: (2025)

Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
di: Kim, Jinhee, et al.
Pubblicazione: (2025)

SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization
di: Bai, Runsheng, et al.
Pubblicazione: (2024)

When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
di: Zhang, Michael S., et al.
Pubblicazione: (2025)

Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
di: Geens, Robin, et al.
Pubblicazione: (2026)

BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deployment
di: Sajid, Md. Ashiq Ul Islam, et al.
Pubblicazione: (2026)

4bit-Quantization in Vector-Embedding for RAG
di: Jeong, Taehee
Pubblicazione: (2025)

Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
di: Tu, Zhijun, et al.
Pubblicazione: (2025)

2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution
di: Liu, Kai, et al.
Pubblicazione: (2024)

AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
di: IslamBouli, Beshr, et al.
Pubblicazione: (2026)

DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
di: Sharify, Sayeh, et al.
Pubblicazione: (2026)

Hybrid Gated Flow (HGF): Stabilizing 1.58-bit LLMs via Selective Low-Rank Correction
di: Pizzo, David Alejandro Trejo
Pubblicazione: (2026)

Stabilizing Backpropagation in 16-bit Neural Training with Modified Adam Optimizer
di: Yun, Juyoung
Pubblicazione: (2023)