Copertina

DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Qi, Xianbiao, Chen, Marco, Xiao, Wenjie, Ye, Jiaquan, He, Yelin, Li, Chun-Guang, Lin, Zhouchen
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

SimpleGPT: Improving GPT via A Simple Normalization Strategy
di: Chen, Marco, et al.
Pubblicazione: (2026)

Taming Transformer Without Using Learning Rate Warmup
di: Qi, Xianbiao, et al.
Pubblicazione: (2025)

Delving into Muon and Beyond: Deep Analysis and Extensions
di: Qi, Xianbiao, et al.
Pubblicazione: (2026)

Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery
di: He, Wei, et al.
Pubblicazione: (2026)

Exploring a Principled Framework for Deep Subspace Clustering
di: Meng, Xianghan, et al.
Pubblicazione: (2025)

Transformers without Normalization
di: Zhu, Jiachen, et al.
Pubblicazione: (2025)

GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
di: Xia, Renqiu, et al.
Pubblicazione: (2024)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
di: Guo, Jialong, et al.
Pubblicazione: (2024)

MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models
di: Wang, Shengkang, et al.
Pubblicazione: (2024)

Sigma: Semantically Informative Pre-training for Skeleton-based Sign Language Understanding
di: Pu, Muxin, et al.
Pubblicazione: (2025)

LLMs can see and hear without any training
di: Ashutosh, Kumar, et al.
Pubblicazione: (2025)

Pseudo-Prompt Generating in Pre-trained Vision-Language Models for Multi-Label Medical Image Classification
di: Ye, Yaoqin, et al.
Pubblicazione: (2024)

Stronger Normalization-Free Transformers
di: Chen, Mingzhi, et al.
Pubblicazione: (2025)

HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding
di: Yang, Rui, et al.
Pubblicazione: (2025)

CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers
di: Shi, Dachuan, et al.
Pubblicazione: (2023)

On Pre-training of Multimodal Language Models Customized for Chart Understanding
di: Fan, Wan-Cyuan, et al.
Pubblicazione: (2024)

MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts
di: Teo, Rachel S. Y., et al.
Pubblicazione: (2024)

Compensating Distribution Drifts in Class-incremental Learning of Pre-trained Vision Transformers
di: Rao, Xuan, et al.
Pubblicazione: (2025)

Let the Target Select for Itself: Data Selection via Target-Aligned Paths
di: Yang, Huitao, et al.
Pubblicazione: (2026)

ROAP: A Reading-Order and Attention-Prior Pipeline for Optimizing Layout Transformers in Key Information Extraction
di: Xie, Tingwei, et al.
Pubblicazione: (2026)

VLMT: Vision-Language Multimodal Transformer for Multimodal Multi-hop Question Answering
di: Lim, Qi Zhi, et al.
Pubblicazione: (2025)

LION: Implicit Vision Prompt Tuning
di: Wang, Haixin, et al.
Pubblicazione: (2023)

Real-time Transformer-based Open-Vocabulary Detection with Efficient Fusion Head
di: Zhao, Tiancheng, et al.
Pubblicazione: (2024)

Vision-Braille: A Curriculum Learning Toolkit and Braille-Chinese Corpus for Braille Translation
di: Wu, Alan, et al.
Pubblicazione: (2024)

TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies
di: Liang, Guang, et al.
Pubblicazione: (2025)

In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understanding
di: Fan, Wan-Cyuan, et al.
Pubblicazione: (2025)

Exploiting GPT-4 Vision for Zero-shot Point Cloud Understanding
di: Sun, Qi, et al.
Pubblicazione: (2024)

VLP: A Survey on Vision-Language Pre-training
di: Chen, Feilong, et al.
Pubblicazione: (2022)

Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching ability
di: Yoon, Yejun, et al.
Pubblicazione: (2024)

An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance
di: Khanuja, Simran, et al.
Pubblicazione: (2024)

LOCR: Location-Guided Transformer for Optical Character Recognition
di: Sun, Yu, et al.
Pubblicazione: (2024)

Cross-attention for State-based model RWKV-7
di: Xiao, Liu, et al.
Pubblicazione: (2025)

Uni-SMART: Universal Science Multimodal Analysis and Research Transformer
di: Cai, Hengxing, et al.
Pubblicazione: (2024)

An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training
di: Nie, Yuxiang, et al.
Pubblicazione: (2025)

Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding
di: Xu, Hongshen, et al.
Pubblicazione: (2024)

Anatomical Structure-Guided Medical Vision-Language Pre-training
di: Li, Qingqiu, et al.
Pubblicazione: (2024)

Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matching
di: Wang, Bin, et al.
Pubblicazione: (2024)

Integrating Fine-Grained Audio-Visual Evidence for Robust Multimodal Emotion Reasoning
di: Zhao, Zhixian, et al.
Pubblicazione: (2026)

Winner Team Mia at TextVQA Challenge 2021: Vision-and-Language Representation Learning with Pre-trained Sequence-to-Sequence Model
di: Qiao, Yixuan, et al.
Pubblicazione: (2021)

Visually Guided Generative Text-Layout Pre-training for Document Intelligence
di: Mao, Zhiming, et al.
Pubblicazione: (2024)