Cover Image

BiPFT: Binary Pre-trained Foundation Transformer with Low-rank Estimation of Binarization Residual Polynomials

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xing, Xingrun, Du, Li, Wang, Xinyuan, Zeng, Xianlin, Wang, Yequan, Zhang, Zheng, Zhang, Jiajun
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
by: Xing, Xingrun, et al.
Published: (2024)

FedPFT: Federated Proxy Fine-Tuning of Foundation Models
by: Peng, Zhaopeng, et al.
Published: (2024)

Aligning Instruction Tuning with Pre-training
by: Liang, Yiming, et al.
Published: (2025)

Masked Structural Growth for 2x Faster Language Model Pre-training
by: Yao, Yiqun, et al.
Published: (2023)

BinaryDemoire: Moiré-Aware Binarization for Image Demoiréing
by: Chen, Zheng, et al.
Published: (2026)

Position-Aware Depth Decay Decoding ($D^3$): Boosting Large Language Model Inference Efficiency
by: Fan, Siqi, et al.
Published: (2025)

BiDense: Binarization for Dense Prediction
by: Yin, Rui, et al.
Published: (2024)

RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs
by: You, Youngcheon, et al.
Published: (2026)

MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training
by: Zhang, Ruijie, et al.
Published: (2026)

BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models
by: Zheng, Xingyu, et al.
Published: (2024)

KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Models
by: Ju, Yiming, et al.
Published: (2023)

LOST: Low-rank and Sparse Pre-training for Large Language Models
by: Li, Jiaxi, et al.
Published: (2025)

BinaryHPE: 3D Human Pose and Shape Estimation via Binarization
by: Li, Zhiteng, et al.
Published: (2023)

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking
by: Xing, Xingrun, et al.
Published: (2024)

PretrainZero: Reinforcement Active Pretraining
by: Xing, Xingrun, et al.
Published: (2025)

Revisiting Data Analysis with Pre-trained Foundation Models
by: Liang, Chen, et al.
Published: (2025)

BHViT: Binarized Hybrid Vision Transformer
by: Gao, Tian, et al.
Published: (2025)

Metis: A Foundation Speech Generation Model with Masked Generative Pre-training
by: Wang, Yuancheng, et al.
Published: (2025)

Binarized Low-light Raw Video Enhancement
by: Zhang, Gengchen, et al.
Published: (2024)

Equip Pre-ranking with Target Attention by Residual Quantization
by: Li, Yutong, et al.
Published: (2025)

Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation
by: Dong, Wei, et al.
Published: (2024)

Estimating Difficulty Levels of Programming Problems with Pre-trained Model
by: Wang, Zhiyuan, et al.
Published: (2024)

Efficient Unsupervised Community Search with Pre-trained Graph Transformer
by: Wang, Jianwei, et al.
Published: (2024)

BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillation
by: Wang, Chen, et al.
Published: (2024)

Graph Generative Pre-trained Transformer
by: Chen, Xiaohui, et al.
Published: (2025)

An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training
by: Nie, Yuxiang, et al.
Published: (2025)

NetGPT: Generative Pretrained Transformer for Network Traffic
by: Meng, Xuying, et al.
Published: (2023)

CAPformer: Compression-Aware Pre-trained Transformer for Low-Light Image Enhancement
by: Wang, Wei, et al.
Published: (2024)

BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models
by: Qiang, Rushi, et al.
Published: (2024)

BiVM: Accurate Binarized Neural Network for Efficient Video Matting
by: Qin, Haotong, et al.
Published: (2025)

BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge
by: Ji, Yuhao, et al.
Published: (2024)

Temporal Adaptation of Pre-trained Foundation Models for Music Structure Analysis
by: Zhang, Yixiao, et al.
Published: (2025)

RET-CLIP: A Retinal Image Foundation Model Pre-trained with Clinical Diagnostic Reports
by: Du, Jiawei, et al.
Published: (2024)

Progressive Residual Extraction based Pre-training for Speech Representation Learning
by: Wang, Tianrui, et al.
Published: (2024)

Compact Binary Systems Waveform Generation with Generative Pre-trained Transformer
by: Shi, Ruijun, et al.
Published: (2023)

ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
by: Wang, Yiming, et al.
Published: (2023)

Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging
by: Ju, Yiming, et al.
Published: (2024)

EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models
by: Xing, Xingrun, et al.
Published: (2025)

MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
by: Su, Yupeng, et al.
Published: (2026)

RmGPT: A Foundation Model with Generative Pre-trained Transformer for Fault Diagnosis and Prognosis in Rotating Machinery
by: Wang, Yilin, et al.
Published: (2024)