Imagen de Portada

Attention Is Not All You Need: The Importance of Feedforward Networks in Transformer Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Gerber, Isaac
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
por: Tyukin, Georgy, et al.
Publicado: (2024)

Forget Attention: Importance-Aware Attention Is All You Need
por: Shin, Soohyeong, et al.
Publicado: (2026)

Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
por: Guo, Zhiyu, et al.
Publicado: (2024)

Tensor Product Attention Is All You Need
por: Zhang, Yifan, et al.
Publicado: (2025)

Attention Smoothing Is All You Need For Unlearning
por: Zade, Saleh Zare, et al.
Publicado: (2026)

Attention Is All You Need for KV Cache in Diffusion LLMs
por: Nguyen-Tri, Quan, et al.
Publicado: (2025)

What Matters in Transformers? Not All Attention is Needed
por: He, Shwai, et al.
Publicado: (2024)

You Need Better Attention Priors
por: Litman, Elon, et al.
Publicado: (2026)

RecurFormer: Not All Transformer Heads Need Self-Attention
por: Yan, Ruiqing, et al.
Publicado: (2024)

More Agents Is All You Need
por: Li, Junyou, et al.
Publicado: (2024)

Block Rotation is All You Need for MXFP4 Quantization
por: Shao, Yuantian, et al.
Publicado: (2025)

Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
por: Li, Pengyi, et al.
Publicado: (2025)

One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks
por: Nehrdich, Sebastian, et al.
Publicado: (2024)

Memory Is All You Need: Testing How Model Memory Affects LLM Performance in Annotation Tasks
por: Timoneda, Joan C., et al.
Publicado: (2025)

Guidance is All You Need: Temperature-Guided Reasoning in Large Language Models
por: Gomaa, Eyad, et al.
Publicado: (2024)

Synthetic Data RL: Task Definition Is All You Need
por: Guo, Yiduo, et al.
Publicado: (2025)

One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels
por: Seshadri, Amrit Diggavi
Publicado: (2025)

Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model!
por: Yoon, Do-hyeon, et al.
Publicado: (2025)

Reformulation is All You Need: Addressing Malicious Text Features in DNNs
por: Jiang, Yi, et al.
Publicado: (2025)

Evidence Is All You Need: Ordering Imaging Studies via Language Model Alignment with the ACR Appropriateness Criteria
por: Yao, Michael S., et al.
Publicado: (2024)

Attention Mechanisms Don't Learn Additive Models: Rethinking Feature Importance for Transformers
por: Leemann, Tobias, et al.
Publicado: (2024)

Are Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
por: Potamitis, Nearchos, et al.
Publicado: (2025)

SecEncoder: Logs are All You Need in Security
por: Bulut, Muhammed Fatih, et al.
Publicado: (2024)

An Extra RMSNorm is All You Need for Fine Tuning to 1.58 Bits
por: Steinmetz, Cody, et al.
Publicado: (2025)

All You Need is One: Capsule Prompt Tuning with a Single Vector
por: Liu, Yiyang, et al.
Publicado: (2025)

Reinforcement Learning is all You Need
por: Lian, Yongsheng
Publicado: (2025)

Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts
por: Gan, Chunjing, et al.
Publicado: (2024)

Geometry is All You Need: A Unified Taxonomy of Matrix and Tensor Factorization for Compression of Generative Language Models
por: Xu, Mingxue, et al.
Publicado: (2024)

Evaluation is All You Need: Strategic Overclaiming of LLM Reasoning Capabilities Through Evaluation Design
por: Sun, Lin, et al.
Publicado: (2025)

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking
por: Kulkarni, Atharva, et al.
Publicado: (2024)

Response-Level Rewards Are All You Need for Online Reinforcement Learning in LLMs: A Mathematical Perspective
por: He, Shenghua, et al.
Publicado: (2025)

All You Need is "Leet": Evading Hate-speech Detection AI
por: Kahu, Sampanna Yashwant, et al.
Publicado: (2025)

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
por: Bozic, Vukasin, et al.
Publicado: (2023)

More Compute Is What You Need
por: Guo, Zhen
Publicado: (2024)

Logits are All We Need to Adapt Closed Models
por: Hiranandani, Gaurush, et al.
Publicado: (2025)

Relevance Isn't All You Need: Scaling RAG Systems With Inference-Time Compute Via Multi-Criteria Reranking
por: LeVine, Will, et al.
Publicado: (2025)

Do Enterprise Systems Need Learned World Models? The Importance of Context to Infer Dynamics
por: Nair, Jishnu Sethumadhavan, et al.
Publicado: (2026)

Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
por: Chen, Lingjiao, et al.
Publicado: (2024)

SAP: Syntactic Attention Pruning for Transformer-based Language Models
por: Lee, Tzu-Yun, et al.
Publicado: (2025)

LASER: Attention with Exponential Transformation
por: Duvvuri, Sai Surya, et al.
Publicado: (2024)