Image de couverture de livre

Nexus: Higher-Order Attention Mechanisms in Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chen, Hanting, Zhu, Chong, Han, Kai, Tian, Yuchuan, Liang, Yuchen, Guo, Tianyu, Chen, Xinghao, Tao, Dacheng, Wang, Yunhe
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
par: Tian, Yuchuan, et autres
Publié: (2025)

DiJiang: Efficient Large Language Models through Compact Kernelization
par: Chen, Hanting, et autres
Publié: (2024)

Deferred Commitment Decoding for Diffusion Language Models
par: Shu, Yingte, et autres
Publié: (2026)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
par: Guo, Jialong, et autres
Publié: (2024)

Unshackling Context Length: An Efficient Selective Attention Approach through Query-Key Compression
par: Wang, Haoyu, et autres
Publié: (2025)

Near-Policy: Accelerating On-Policy Distillation via Asynchronous Generation and Selective Packing
par: Rang, Miao, et autres
Publié: (2026)

U-REPA: Aligning Diffusion U-Nets to ViTs
par: Tian, Yuchuan, et autres
Publié: (2025)

VersatileFFN: Achieving Parameter Efficiency in LLMs via Adaptive Wide-and-Deep Reuse
par: Nie, Ying, et autres
Publié: (2025)

A Survey on Transformer Compression
par: Tang, Yehui, et autres
Publié: (2024)

Multiscale Positive-Unlabeled Detection of AI-Generated Texts
par: Tian, Yuchuan, et autres
Publié: (2023)

DLLM Agent: See Farther, Run Faster
par: Zhen, Huiling, et autres
Publié: (2026)

U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers
par: Tian, Yuchuan, et autres
Publié: (2024)

Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants
par: Wang, Yunhe, et autres
Publié: (2026)

Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
par: Rang, Miao, et autres
Publié: (2025)

Transferable text data distillation by trajectory matching
par: Yao, Rong, et autres
Publié: (2025)

Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition
par: Chen, Hanting, et autres
Publié: (2025)

DiC: Rethinking Conv3x3 Designs in Diffusion Models
par: Tian, Yuchuan, et autres
Publié: (2024)

C-MOP: Integrating Momentum and Boundary-Aware Clustering for Enhanced Prompt Evolution
par: Yan, Binwei, et autres
Publié: (2026)

Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation
par: Tian, Yuchuan, et autres
Publié: (2024)

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
par: Tang, Yehui, et autres
Publié: (2025)

Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing
par: Wu, Yuchen, et autres
Publié: (2025)

Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
par: Wu, Yuchen, et autres
Publié: (2025)

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
par: Han, Jing, et autres
Publié: (2025)

PanGu-$π$: Enhancing Language Model Architectures via Nonlinearity Compensation
par: Wang, Yunhe, et autres
Publié: (2023)

Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
par: Wu, Yuchen, et autres
Publié: (2025)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
par: He, Wei, et autres
Publié: (2024)

Pangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
par: Chen, Hanting, et autres
Publié: (2025)

Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
par: Tu, Zhijun, et autres
Publié: (2025)

CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models
par: Nie, Ying, et autres
Publié: (2024)

Skip-Layer Attention: Bridging Abstract and Detailed Dependencies in Transformers
par: Chen, Qian, et autres
Publié: (2024)

Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models
par: Guo, Jianyuan, et autres
Publié: (2024)

Entropy-Guided Watermarking for LLMs: A Test-Time Framework for Robust and Traceable Text Generation
par: Cai, Shizhan, et autres
Publié: (2025)

IPT-V2: Efficient Image Processing Transformer using Hierarchical Attentions
par: Tu, Zhijun, et autres
Publié: (2024)

NoVo: Norm Voting off Hallucinations with Attention Heads in Large Language Models
par: Ho, Zheng Yi, et autres
Publié: (2024)

Learning Quantized Adaptive Conditions for Diffusion Models
par: Liang, Yuchen, et autres
Publié: (2024)

PanGu-$π$ Pro:Rethinking Optimization and Architecture for Tiny Language Models
par: Tang, Yehui, et autres
Publié: (2024)

Multi-Granularity Semantic Revision for Large Language Model Distillation
par: Liu, Xiaoyu, et autres
Publié: (2024)

EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models
par: Ni, Yunsheng, et autres
Publié: (2024)

WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
par: Wang, Wenbin, et autres
Publié: (2024)

SPLA: Block Sparse Plus Linear Attention for Long Context Modeling
par: Wang, Bailin, et autres
Publié: (2026)