:: Library Catalog

Image de couverture de livre

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Chen, Lei, Bruna, Joan, Bietti, Alberto
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	https://arxiv.org/abs/2406.03068
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Compositional Reasoning with Transformers, RNNs, and Chain of Thought
par: Yehudai, Gilad, et autres
Publié: (2025)

Scaling Laws for Associative Memories
par: Cabannes, Vivien, et autres
Publié: (2023)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
par: Ling Team, et autres
Publié: (2025)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
par: Zhu, Kan, et autres
Publié: (2025)

FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning
par: Luo, Haozheng, et autres
Publié: (2026)

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention
par: Guo, Zhenyu, et autres
Publié: (2025)

Scaling Reasoning without Attention
par: Zhao, Xueliang, et autres
Publié: (2025)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
par: Chen, Yingfa, et autres
Publié: (2025)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
par: Zhu, Qianchao, et autres
Publié: (2024)

Decomposing Attention To Find Context-Sensitive Neurons
par: Gibson, Alex
Publié: (2025)

Which Attention Heads Matter for In-Context Learning?
par: Yin, Kayo, et autres
Publié: (2025)

Memorization vs. Reasoning: Updating LLMs with New Knowledge
par: Li, Aochong Oliver, et autres
Publié: (2025)

Compute Optimal Scaling of Skills: Knowledge vs Reasoning
par: Roberts, Nicholas, et autres
Publié: (2025)

Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves
par: Knupp, Jonas, et autres
Publié: (2026)

TIDE: Every Layer Knows the Token Beneath the Context
par: Jaiswal, Ajay, et autres
Publié: (2026)

MoBA: Mixture of Block Attention for Long-Context LLMs
par: Lu, Enzhe, et autres
Publié: (2025)

Learning Associative Memories with Gradient Descent
par: Cabannes, Vivien, et autres
Publié: (2024)

SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
par: Lee, Changhun, et autres
Publié: (2025)

Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
par: Yang, Wang, et autres
Publié: (2025)

Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation
par: Li, Ruizhe, et autres
Publié: (2025)

What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
par: Li, Ming, et autres
Publié: (2024)

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
par: Hsieh, Cheng-Yu, et autres
Publié: (2024)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
par: Collins, Liam, et autres
Publié: (2024)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
par: Zeng, Xiangyu, et autres
Publié: (2026)

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
par: Chen, Yuhan, et autres
Publié: (2023)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
par: Neo, Clement, et autres
Publié: (2024)

MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning
par: Xi, Ningyuan, et autres
Publié: (2024)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
par: Chen, Yingfa, et autres
Publié: (2026)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
par: Vasudeva, Bhavya, et autres
Publié: (2026)

Contextual Drag: How Errors in the Context Affect LLM Reasoning
par: Cheng, Yun, et autres
Publié: (2026)

Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models
par: Mąka, Paweł, et autres
Publié: (2024)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
par: MiniCPM Team, et autres
Publié: (2026)

Dynamic sparsity in tree-structured feed-forward layers at scale
par: Sedghi, Reza, et autres
Publié: (2026)

LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
par: Wang, Guangtao, et autres
Publié: (2025)

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
par: Sanyal, Sunny, et autres
Publié: (2024)

xVal: A Continuous Numerical Tokenization for Scientific Language Models
par: Golkar, Siavash, et autres
Publié: (2023)

FlowRL: Matching Reward Distributions for LLM Reasoning
par: Zhu, Xuekai, et autres
Publié: (2025)

Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities
par: Hua, Wenyue, et autres
Publié: (2024)

DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning
par: Wang, Yaxuan, et autres
Publié: (2025)

When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
par: Jeong, Soyeong, et autres
Publié: (2025)