Image de couverture de livre

Understanding Transformers via N-gram Statistics

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Nguyen, Timothy
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Transformers Can Represent $n$-gram Language Models
par: Svete, Anej, et autres
Publié: (2024)

Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descend
par: Lu, Ning, et autres
Publié: (2023)

An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks
par: Boreiko, Valentyn, et autres
Publié: (2024)

Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data
par: Havrilla, Alex, et autres
Publié: (2024)

CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functions
par: Fu, Zihao, et autres
Publié: (2025)

RCStat: A Statistical Framework for using Relative Contextualization in Transformers
par: Mahapatra, Debabrata, et autres
Publié: (2025)

Understanding Dynamic Compute Allocation in Recurrent Transformers
par: Moosa, Ibraheem Muhammad, et autres
Publié: (2026)

Legal2LogicICL: Improving Generalization in Transforming Legal Cases to Logical Formulas via Diverse Few-Shot Learning
par: Xue, Jieying, et autres
Publié: (2026)

ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Tasks
par: Van Huynh, Tin, et autres
Publié: (2026)

RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
par: Wang, Zihan, et autres
Publié: (2025)

Understanding In-context Learning of Addition via Activation Subspaces
par: Hu, Xinyan, et autres
Publié: (2025)

TableDART: Dynamic Adaptive Multi-Modal Routing for Table Understanding
par: Xing, Xiaobo, et autres
Publié: (2025)

Faster Transformer Decoding: N-gram Masked Self-Attention
par: Chelba, Ciprian, et autres
Publié: (2020)

Understanding Generalization in Role-Playing Models via Information Theory
par: Li, Yongqi, et autres
Publié: (2025)

Evaluating LLM Understanding via Structured Tabular Decision Simulations
par: Li, Sichao, et autres
Publié: (2025)

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text
par: Arbel, Iftach, et autres
Publié: (2024)

The Importance of Online Data: Understanding Preference Fine-tuning via Coverage
par: Song, Yuda, et autres
Publié: (2024)

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
par: Wang, Xu, et autres
Publié: (2025)

Teaching Your Models to Understand Code via Focal Preference Alignment
par: Wu, Jie, et autres
Publié: (2025)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
par: Qiu, Zeju, et autres
Publié: (2025)

Accelerating Transformer Inference for Translation via Parallel Decoding
par: Santilli, Andrea, et autres
Publié: (2023)

The Foundations of Tokenization: Statistical and Computational Concerns
par: Gastaldi, Juan Luis, et autres
Publié: (2024)

Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anything
par: Lin, Huawei, et autres
Publié: (2025)

The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
par: Shojaee, Parshin, et autres
Publié: (2025)

Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLMs
par: Zhou, Kuan Lok, et autres
Publié: (2025)

On Efficient and Statistical Quality Estimation for Data Annotation
par: Klie, Jan-Christoph, et autres
Publié: (2024)

Explain Less, Understand More: Jargon Detection via Personalized Parameter-Efficient Fine-tuning
par: Wu, Bohao, et autres
Publié: (2025)

Multi-Attribute Steering of Language Models via Targeted Intervention
par: Nguyen, Duy, et autres
Publié: (2025)

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
par: Zhuo, Zhijian, et autres
Publié: (2025)

ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization
par: Shopkhoev, Dmitriy, et autres
Publié: (2025)

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
par: Xiao, Da, et autres
Publié: (2025)

Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
par: Chen, Yilong, et autres
Publié: (2026)

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
par: Tian, Yuandong, et autres
Publié: (2023)

Understanding Understanding: A Pragmatic Framework Motivated by Large Language Models
par: Leyton-Brown, Kevin, et autres
Publié: (2024)

IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inference
par: Liu, Guanming, et autres
Publié: (2026)

Scalable Best-of-N Selection for Large Language Models via Self-Certainty
par: Kang, Zhewei, et autres
Publié: (2025)

Mission Impossible: A Statistical Perspective on Jailbreaking LLMs
par: Su, Jingtong, et autres
Publié: (2024)

Mechanistic Interpretability as Statistical Estimation: A Variance Analysis
par: Méloux, Maxime, et autres
Publié: (2025)

TPTT: Transforming Pretrained Transformers into Titans
par: Furfaro, Fabien
Publié: (2025)

Out-of-distribution generalization via composition: a lens through induction heads in Transformers
par: Song, Jiajun, et autres
Publié: (2024)