Image de couverture de livre

Transformers with Selective Access to Early Representations

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Gunasekaran, Skye, Wright, Téa, Zhu, Rui-Jie, Eshraghian, Jason
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
par: Zhu, Rui-Jie, et autres
Publié: (2023)

A Predictive Approach To Enhance Time-Series Forecasting
par: Gunasekaran, Skye, et autres
Publié: (2024)

HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving
par: Kumar, Avinash, et autres
Publié: (2025)

Mechanistic Interpretability of Binary and Ternary Transformers
par: Li, Jason
Publié: (2024)

Early Transformers: A study on Efficient Training of Transformer Models through Early-Bird Lottery Tickets
par: Cheekati, Shravan
Publié: (2024)

Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2
par: Abreu, Steven, et autres
Publié: (2025)

Probabilistic Topic Modelling with Transformer Representations
par: Reuter, Arik, et autres
Publié: (2024)

Disentangling the Roles of Representation and Selection in Data Pruning
par: Du, Yupei, et autres
Publié: (2025)

VITRO: Vocabulary Inversion for Time-series Representation Optimization
par: Bellos, Filippos, et autres
Publié: (2024)

Selective Neuron Amplification in Transformer Language Models
par: Akhtar, Ryyan, et autres
Publié: (2026)

You Do Not Fully Utilize Transformer's Representation Capacity
par: Gerasimov, Gleb, et autres
Publié: (2025)

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers
par: Ren, Yunwei, et autres
Publié: (2024)

When Reward Hacking Rebounds: Understanding and Mitigating It with Representation-Level Signals
par: Wu, Rui, et autres
Publié: (2026)

Learning to Focus: Focal Attention for Selective and Scalable Transformers
par: Ram, Dhananjay, et autres
Publié: (2025)

Emergent Stack Representations in Modeling Counter Languages Using Transformers
par: Tiwari, Utkarsh, et autres
Publié: (2025)

Adapting Abstract Meaning Representation Parsing to the Clinical Narrative -- the SPRING THYME parser
par: Cai, Jon Z., et autres
Publié: (2024)

Machine Understanding of Scientific Language
par: Wright, Dustin
Publié: (2025)

DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System
par: Zhu, Yunying, et autres
Publié: (2026)

Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning
par: Arefin, Md Rifat, et autres
Publié: (2024)

Memory-Efficient Fine-Tuning of Transformers via Token Selection
par: Simoulin, Antoine, et autres
Publié: (2025)

Selective Attention: Enhancing Transformer through Principled Context Control
par: Zhang, Xuechen, et autres
Publié: (2024)

SENTRA: Selected-Next-Token Transformer for LLM Text Detection
par: Plyler, Mitchell, et autres
Publié: (2025)

Large Language Model-Enhanced Algorithm Selection: Towards Comprehensive Algorithm Representation
par: Wu, Xingyu, et autres
Publié: (2023)

A Comparative Analysis of Contextual Representation Flow in State-Space and Transformer Architectures
par: Hoang, Nhat M., et autres
Publié: (2025)

Zero-shot data citation function classification using transformer-based large language models (LLMs)
par: Byers, Neil, et autres
Publié: (2025)

Selective Attention Improves Transformer
par: Leviathan, Yaniv, et autres
Publié: (2024)

Converting Transformers into DGNNs Form
par: Zhang, Jie, et autres
Publié: (2025)

SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networks
par: Venkatesh, Sreyes, et autres
Publié: (2024)

HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processing
par: He, Zifan, et autres
Publié: (2024)

Differential Transformer
par: Ye, Tianzhu, et autres
Publié: (2024)

Group-Level Data Selection for Efficient Pretraining
par: Yu, Zichun, et autres
Publié: (2025)

Encoding Agent Trajectories as Representations with Sequence Transformers
par: Tsiligkaridis, Athanasios, et autres
Publié: (2024)

Rank, Head-Channel Non-Identifiability, and Symmetry Breaking: A Precise Analysis of Representational Collapse in Transformers
par: Cirrincione, Giansalvo
Publié: (2026)

Subjective Perspectives within Learned Representations Predict High-Impact Innovation
par: Cao, Likun, et autres
Publié: (2025)

A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representations
par: Kowtal, Nidhi, et autres
Publié: (2024)

Multi-modal Anchor Gated Transformer with Knowledge Distillation for Emotion Recognition in Conversation
par: Li, Jie, et autres
Publié: (2025)

Learning Novel Transformer Architecture for Time-series Forecasting
par: Zhang, Juyuan, et autres
Publié: (2025)

Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs
par: Qin, Yijian, et autres
Publié: (2023)

Sparse Transformer with Local and Seasonal Adaptation for Multivariate Time Series Forecasting
par: Zhang, Yifan, et autres
Publié: (2023)

General Transform: A Unified Framework for Adaptive Transform to Enhance Representations
par: Budiutama, Gekko, et autres
Publié: (2025)