Image de couverture de livre

Top-Theta Attention: Sparsifying Transformers by Compensated Thresholding

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Berestizshevsky, Konstantin, Andri, Renzo, Cavigelli, Lukas
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence 68T01 I.2
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Compressible Softmax-Attended Language under Incompressible Attention
par: Lee, Wonsuk
Publié: (2026)

Transformadores: Fundamentos teoricos y Aplicaciones
par: de la Torre, Jordi
Publié: (2023)

Can Out-of-Distribution Evaluations Uncover Reliance on Shortcuts? A Case Study in Question Answering
par: Štefánik, Michal, et autres
Publié: (2025)

SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions
par: Xu, Weijie, et autres
Publié: (2025)

Controlling Long-Horizon Behavior in Language Model Agents with Explicit State Dynamics
par: Subaharan, Sukesh
Publié: (2026)

An Automatic Text Classification Method Based on Hierarchical Taxonomies, Neural Networks and Document Embedding: The NETHIC Tool
par: Lomasto, Luigi, et autres
Publié: (2026)

Data and AI governance: Promoting equity, ethics, and fairness in large language models
par: Abhishek, Alok, et autres
Publié: (2025)

BEATS: Bias Evaluation and Assessment Test Suite for Large Language Models
par: Abhishek, Alok, et autres
Publié: (2025)

SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models
par: Abhishek, Alok, et autres
Publié: (2026)

TSDS: Data Selection for Task-Specific Model Finetuning
par: Liu, Zifan, et autres
Publié: (2024)

Reasoning Promotes Robustness in Theory of Mind Tasks
par: de Haan, Ian B., et autres
Publié: (2026)

ConSensus: Multi-Agent Collaboration for Multimodal Sensing
par: Yoon, Hyungjun, et autres
Publié: (2026)

Morphological Synthesizer for Ge'ez Language: Addressing Morphological Complexity and Resource Limitations
par: Gebremariam, Gebrearegawi, et autres
Publié: (2025)

Large Language Models are Inconsistent and Biased Evaluators
par: Stureborg, Rickard, et autres
Publié: (2024)

On the Invariants of Softmax Attention
par: Lee, Wonsuk
Publié: (2026)

Attention Please: What Transformer Models Really Learn for Process Prediction
par: Käppel, Martin, et autres
Publié: (2024)

On Adversarial Examples for Text Classification by Perturbing Latent Representations
par: Sooksatra, Korn, et autres
Publié: (2024)

Tailoring Vaccine Messaging with Common-Ground Opinions
par: Stureborg, Rickard, et autres
Publié: (2024)

An Epidemiological Knowledge Graph extracted from the World Health Organization's Disease Outbreak News
par: Consoli, Sergio, et autres
Publié: (2025)

HAAP: Vision-context Hierarchical Attention Autoregressive with Adaptive Permutation for Scene Text Recognition
par: Chen, Honghui, et autres
Publié: (2024)

Instilling Organisational Values in Firefighters through Simulation-Based Training
par: Osman, Nardine, et autres
Publié: (2025)

Feature Relevancy, Necessity and Usefulness: Complexity and Algorithms
par: Capdevielle, Tomás, et autres
Publié: (2025)

A Taxonomy of Omnicidal Futures Involving Artificial Intelligence
par: Critch, Andrew, et autres
Publié: (2025)

A Theoretical Framework for Adaptive Utility-Weighted Benchmarking
par: Waggoner, Philip
Publié: (2026)

From Language Models to Practical Self-Improving Computer Agents
par: Sheng, Alex
Publié: (2024)

Hilbert-Geo: Solving Solid Geometric Problems by Neural-Symbolic Reasoning
par: Xu, Ruoran, et autres
Publié: (2026)

Revenue-Sharing as Infrastructure: A Distributed Business Model for Generative AI Platforms
par: Mondjo, Ghislain Dorian Tchuente
Publié: (2026)

Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture
par: Burns, Thomas F, et autres
Publié: (2024)

CAG: Chunked Augmented Generation for Google Chrome's Built-in Gemini Nano
par: Surulimuthu, Vivek Vellaiyappan, et autres
Publié: (2024)

Graph Transformers: A Survey
par: Shehzad, Ahsan, et autres
Publié: (2024)

Abductive explanations of classifiers under constraints: Complexity and properties
par: Cooper, Martin, et autres
Publié: (2024)

AI Education in Higher Education: A Taxonomy for Curriculum Reform and the Mission of Knowledge
par: Zheng, Tian
Publié: (2025)

Reducing Instability in Synthetic Data Evaluation with a Super-Metric in MalDataGen
par: da Silva, Anna Luiza Gomes, et autres
Publié: (2025)

AI Consciousness is Inevitable: A Theoretical Computer Science Perspective
par: Blum, Lenore, et autres
Publié: (2024)

Intelligence as Computation
par: Brock, Oliver
Publié: (2024)

Quantifying Behavioral Dissimilarity Between Mathematical Expressions
par: Mežnar, Sebastian, et autres
Publié: (2024)

ORPHEAS: A Cross-Lingual Greek-English Embedding Model for Retrieval-Augmented Generation
par: Livieris, Ioannis E., et autres
Publié: (2026)

On Privacy Leakage in Tabular Diffusion Models: Influential Factors, Attacker Knowledge, and Metrics
par: Shafieinejad, Masoumeh, et autres
Publié: (2026)

On measuring grounding and generalizing grounding problems
par: Quigley, Daniel, et autres
Publié: (2025)

Project Synapse: A Hierarchical Multi-Agent Framework with Hybrid Memory for Autonomous Resolution of Last-Mile Delivery Disruptions
par: Yadav, Arin Gopalan, et autres
Publié: (2026)