Image de couverture de livre

CoMA: Complementary Masking and Hierarchical Dynamic Multi-Window Self-Attention in a Unified Pre-training Framework

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Jiaxuan, Xu, Qing, He, Xiangjian, Liu, Ziyu, Xing, Chang, Chen, Zhen, Zhang, Daokun, Qu, Rong, Chen, Chang Wen
Format:	Preprint
Publié:	2025
Sujets:	Computer Vision and Pattern Recognition Artificial Intelligence I.2.0
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

CFFormer: Cross CNN-Transformer Channel Attention and Spatial Feature Fusion for Improved Segmentation of Heterogeneous Medical Images
par: Li, Jiaxuan, et autres
Publié: (2025)

Equip Pre-ranking with Target Attention by Residual Quantization
par: Li, Yutong, et autres
Publié: (2025)

WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis
par: Lyu, Xinheng, et autres
Publié: (2025)

Hierarchical Pre-Training of Vision Encoders with Large Language Models
par: Lee, Eugene, et autres
Publié: (2026)

CoMA: Compositional Human Motion Generation with Multi-modal Agents
par: Sun, Shanlin, et autres
Publié: (2024)

Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning
par: Ge, Shiping, et autres
Publié: (2024)

Domain-Specific Self-Supervised Pre-training for Agricultural Disease Classification: A Hierarchical Vision Transformer Study
par: Sonavane, Arnav S.
Publié: (2026)

Towards Effective and Efficient Continual Pre-training of Large Language Models
par: Chen, Jie, et autres
Publié: (2024)

TKwinFormer: Top k Window Attention in Vision Transformers for Feature Matching
par: Liao, Yun, et autres
Publié: (2023)

Efficient Image Pre-Training with Siamese Cropped Masked Autoencoders
par: Eymaël, Alexandre, et autres
Publié: (2024)

DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features
par: Chen, Zhangquan, et autres
Publié: (2024)

DatUS^2: Data-driven Unsupervised Semantic Segmentation with Pre-trained Self-supervised Vision Transformer
par: Kumar, Sonal, et autres
Publié: (2024)

TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
par: Hsu, Shang-Ling, et autres
Publié: (2026)

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
par: Wang, Zhengyi, et autres
Publié: (2024)

Arabic Hate Speech Identification and Masking in Social Media using Deep Learning Models and Pre-trained Models Fine-tuning
par: Doghmash, Salam Thabet, et autres
Publié: (2025)

The Unified Cognitive Consciousness Theory for Language Models: Anchoring Semantics, Thresholds of Activation, and Emergent Reasoning
par: Chang, Edward Y., et autres
Publié: (2025)

Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
par: Yang, Qinyu, et autres
Publié: (2024)

CAMAL: Improving Attention Alignment and Faithfulness with Segmentation Masks
par: Hundal, Rajdeep Singh, et autres
Publié: (2026)

An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs
par: Rai, Daking, et autres
Publié: (2024)

On Initializing Transformers with Pre-trained Embeddings
par: Kim, Ha Young, et autres
Publié: (2024)

GPTON: Generative Pre-trained Transformers enhanced with Ontology Narration for accurate annotation of biological data
par: Li, Rongbin, et autres
Publié: (2024)

Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion
par: Xu, Wenjie, et autres
Publié: (2023)

HySparK: Hybrid Sparse Masking for Large Scale Medical Image Pre-Training
par: Tang, Fenghe, et autres
Publié: (2024)

Haze-Aware Attention Network for Single-Image Dehazing
par: Tong, Lihan, et autres
Publié: (2024)

LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting
par: Yang, Yu-Jie, et autres
Publié: (2026)

Self-Attention And Beyond the Infinite: Towards Linear Transformers with Infinite Self-Attention
par: Roffo, Giorgio, et autres
Publié: (2026)

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration
par: Cai, Gengyuan, et autres
Publié: (2024)

Unified Local and Global Attention Interaction Modeling for Vision Transformers
par: Nguyen, Tan, et autres
Publié: (2024)

Asset Pricing in Pre-trained Transformer
par: Lai, Shanyan
Publié: (2025)

Design of an basis-projected layer for sparse datasets in deep learning training using gc-ms spectra as a case study
par: Chang, Yu Tang, et autres
Publié: (2024)

Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
par: Zhou, Jiecheng, et autres
Publié: (2025)

Neural Image Compression Using Masked Sparse Visual Representation
par: Jiang, Wei, et autres
Publié: (2023)

Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
par: He, Yanjin, et autres
Publié: (2025)

Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility
par: Szilvasy, Gergely, et autres
Publié: (2026)

Tool-RoCo: An Agent-as-Tool Self-organization Large Language Model Benchmark in Multi-robot Cooperation
par: Zhang, Ke, et autres
Publié: (2025)

Unified Auto-Encoding with Masked Diffusion
par: Hansen-Estruch, Philippe, et autres
Publié: (2024)

The Origin of Self-Attention: Pairwise Affinity Matrices in Feature Selection and the Emergence of Self-Attention
par: Roffo, Giorgio
Publié: (2025)

A Hierarchical Self-Consistent Regularization Approach to Satellite Image Time Series Classification
par: Weikmann, Giulio, et autres
Publié: (2025)

When Does Global Attention Help? A Unified Empirical Study on Atomistic Graph Learning
par: Chowdhury, Arindam, et autres
Publié: (2025)

ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model
par: Pospieszny, Przemek, et autres
Publié: (2025)