:: Library Catalog

Copertina

Salvato in:

Dettagli Bibliografici
Autore principale:	Wong, Liang Ze
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Computation and Language
Accesso online:	https://arxiv.org/abs/2502.05076
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank
di: Roy, Debjyoti Saha, et al.
Pubblicazione: (2024)

Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
di: Zhuang, Xialie, et al.
Pubblicazione: (2025)

Pay Less Attention to Function Words for Free Robustness of Vision-Language Models
di: Tian, Qiwei, et al.
Pubblicazione: (2025)

ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
di: Ye, Lu, et al.
Pubblicazione: (2024)

High-Layer Attention Pruning with Rescaling
di: Liu, Songtao, et al.
Pubblicazione: (2025)

LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs
di: Souibgui, Mohamed Ali, et al.
Pubblicazione: (2026)

Multi-Layer Attention is the Amplifier of Demonstration Effectiveness
di: Wang, Dingzirui, et al.
Pubblicazione: (2025)

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
di: Bozic, Vukasin, et al.
Pubblicazione: (2023)

Empirical Capacity Model for Self-Attention Neural Networks
di: Härmä, Aki, et al.
Pubblicazione: (2024)

PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention
di: Chen, Lida, et al.
Pubblicazione: (2025)

Inducing Meaningful Units from Character Sequences with Dynamic Capacity Slot Attention
di: Behjati, Melika, et al.
Pubblicazione: (2021)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
di: Fu, Zizhuo, et al.
Pubblicazione: (2026)

CLAA: Cross-Layer Attention Aggregation for Accelerating LLM Prefill
di: McDanel, Bradley, et al.
Pubblicazione: (2026)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
di: Musat, Tiberiu
Pubblicazione: (2024)

Wave-PDE Nets: Trainable Wave-Equation Layers as an Alternative to Attention
di: Vejendla, Harshil
Pubblicazione: (2025)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
di: Brandon, William, et al.
Pubblicazione: (2024)

Why Softmax Attention Outperforms Linear Attention
di: Deng, Yichuan, et al.
Pubblicazione: (2023)

DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning
di: Zarch, Hossein Entezari, et al.
Pubblicazione: (2025)

IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
di: Bai, Yushi, et al.
Pubblicazione: (2026)

On the Existence and Behavior of Secondary Attention Sinks
di: Wong, Jeffrey T. H., et al.
Pubblicazione: (2025)

SEA: Sparse Linear Attention with Estimated Attention Mask
di: Lee, Heejun, et al.
Pubblicazione: (2023)

Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features
di: Bombari, Simone, et al.
Pubblicazione: (2024)

Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification
di: ElNokrashy, Muhammad, et al.
Pubblicazione: (2022)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
di: Yuan, Jingyang, et al.
Pubblicazione: (2025)

Instruction Following by Principled Boosting Attention of Large Language Models
di: Guardieiro, Vitoria, et al.
Pubblicazione: (2025)

From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning
di: Zhao, Eric, et al.
Pubblicazione: (2025)

Simulating Hard Attention Using Soft Attention
di: Yang, Andy, et al.
Pubblicazione: (2024)

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
di: Qiu, Quantong, et al.
Pubblicazione: (2026)

Fact or Fiction? Improving Fact Verification with Knowledge Graphs through Simplified Subgraph Retrievals
di: Opsahl, Tobias A.
Pubblicazione: (2024)

SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models
di: He, Jinghan, et al.
Pubblicazione: (2024)

Learning to Attribute with Attention
di: Cohen-Wang, Benjamin, et al.
Pubblicazione: (2025)

Scale-invariant Attention
di: Anson, Ben, et al.
Pubblicazione: (2025)

Exclusive Self Attention
di: Zhai, Shuangfei
Pubblicazione: (2026)

Breaking the Attention Bottleneck
di: Hilsenbek, Kalle
Pubblicazione: (2024)

Stochastic Attention: Connectome-Inspired Randomized Routing for Expressive Linear-Time Attention
di: Jin, Zehao, et al.
Pubblicazione: (2026)

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
di: Shyam, Vasudev, et al.
Pubblicazione: (2024)

Fast Multipole Attention: A Scalable Multilevel Attention Mechanism for Text and Images
di: Kang, Yanming, et al.
Pubblicazione: (2023)

HSR-Enhanced Sparse Attention Acceleration
di: Chen, Bo, et al.
Pubblicazione: (2024)

AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
di: Song, Dinghong, et al.
Pubblicazione: (2025)

Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
di: He, Mutian, et al.
Pubblicazione: (2025)