Cover Image

Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	ElNokrashy, Muhammad, AlKhamissi, Badr, Diab, Mona
Format:	Preprint
Published:	2022
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Context-Contrastive Inference Approach To Partial Diacritization
by: ElNokrashy, Muhammad, et al.
Published: (2024)

Investigating Cultural Alignment of Large Language Models
by: AlKhamissi, Badr, et al.
Published: (2024)

Rosetta Stone at KSAA-RD Shared Task: A Hop From Language Modeling To Word--Definition Alignment
by: ElBakry, Ahmed, et al.
Published: (2023)

Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lens
by: AlKhamissi, Mai, et al.
Published: (2025)

Brain-Like Language Processing via a Shallow Untrained Multihead Attention Network
by: AlKhamissi, Badr, et al.
Published: (2024)

The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units
by: AlKhamissi, Badr, et al.
Published: (2024)

Inducing Dyslexia in Vision Language Models
by: Honarmand, Melika, et al.
Published: (2025)

Khattat: Enhancing Readability and Concept Representation of Semantic Typography
by: Hussein, Ahmed, et al.
Published: (2024)

MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding
by: Gokce, Abdulkadir, et al.
Published: (2026)

Rational Metareasoning for Large Language Models
by: De Sabbata, C. Nicolò, et al.
Published: (2024)

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data
by: AlKhamissi, Badr, et al.
Published: (2024)

Evaluating Contrast Localizer for Identifying Causal Units in Social & Mathematical Tasks in Language Models
by: Jamaa, Yassine, et al.
Published: (2025)

"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models
by: Halevy, Karina, et al.
Published: (2024)

Emotion Classification in Low and Moderate Resource Languages
by: Tafreshi, Shabnam, et al.
Published: (2024)

DSPA: Dynamic SAE Steering for Data-Efficient Preference Alignment
by: Wedgwood, James, et al.
Published: (2026)

Instruction-tuning Aligns LLMs to the Human Brain
by: Aw, Khai Loong, et al.
Published: (2023)

From Language to Cognition: How LLMs Outgrow the Human Language Network
by: AlKhamissi, Badr, et al.
Published: (2025)

DWTSumm: Discrete Wavelet Transform for Document Summarization
by: Salama, Rana, et al.
Published: (2026)

TopoLM: brain-like spatio-functional organization in a topographic language model
by: Rathi, Neil, et al.
Published: (2024)

MaxPoolBERT: Enhancing BERT Classification via Layer- and Token-Wise Aggregation
by: Behrendt, Maike, et al.
Published: (2025)

Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models
by: Muhamed, Aashiq, et al.
Published: (2024)

SimBA: Simplifying Benchmark Analysis Using Performance Matrices Alone
by: Subramani, Nishant, et al.
Published: (2025)

Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specialization
by: AlKhamissi, Badr, et al.
Published: (2025)

LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs
by: Souibgui, Mohamed Ali, et al.
Published: (2026)

Dual Path Attribution: Efficient Attribution for SwiGLU-Transformers through Layer-Wise Target Propagation
by: Jantsch, Lasse Marten, et al.
Published: (2026)

CoRAG: Collaborative Retrieval-Augmented Generation
by: Muhamed, Aashiq, et al.
Published: (2025)

Personal Information Parroting in Language Models
by: Subramani, Nishant, et al.
Published: (2026)

AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
by: Achtibat, Reduan, et al.
Published: (2024)

KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing
by: Yang, Yifei, et al.
Published: (2024)

OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling
by: Lou, Yuxuan, et al.
Published: (2026)

Adaptive Layer Selection for Layer-Wise Token Pruning in LLM Inference
by: Taniguchi, Rei, et al.
Published: (2026)

Listening to the Wise Few: Select-and-Copy Attention Heads for Multiple-Choice QA
by: Tulchinskii, Eduard, et al.
Published: (2024)

LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
by: Xi, Haocheng, et al.
Published: (2026)

SAEs $\textit{Can}$ Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
by: Muhamed, Aashiq, et al.
Published: (2025)

DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning
by: Zarch, Hossein Entezari, et al.
Published: (2025)

LPCD: Unified Framework from Layer-Wise to Submodule Quantization
by: Ichikawa, Yuma, et al.
Published: (2025)

Block-Wise Differentiable Sinkhorn Attention: Tail-Refinement Gradients with a Gap-Aware Dustbin Bridge
by: Forde, Dylan
Published: (2026)

KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
by: Li, Xing, et al.
Published: (2025)

Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers
by: Wong, Liang Ze
Published: (2025)

RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Models
by: Muhamed, Aashiq, et al.
Published: (2025)