Imagen de Portada

Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chang, Tyler A., Bergen, Benjamin K.
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Explaining and Mitigating Crosslingual Tokenizer Inequities
por: Arnett, Catherine, et al.
Publicado: (2025)

Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability
por: Chang, Tyler A., et al.
Publicado: (2023)

Goldfish: Monolingual Language Models for 350 Languages
por: Chang, Tyler A., et al.
Publicado: (2024)

A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
por: Arnett, Catherine, et al.
Publicado: (2024)

On the Acquisition of Shared Grammatical Representations in Bilingual Language Models
por: Arnett, Catherine, et al.
Publicado: (2025)

Improbable Bigrams Expose Vulnerabilities of Incomplete Tokens in Byte-Level Tokenizers
por: Jang, Eugene, et al.
Publicado: (2024)

Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics
por: Michaelov, James A., et al.
Publicado: (2024)

Learning and Transferring Sparse Contextual Bigrams with Linear Transformers
por: Ren, Yunwei, et al.
Publicado: (2024)

Do Large Language Models Exhibit Spontaneous Rational Deception?
por: Taylor, Samuel M., et al.
Publicado: (2025)

Large Language Models Pass the Turing Test
por: Jones, Cameron R., et al.
Publicado: (2025)

BLooP: Zero-Shot Abstractive Summarization using Large Language Models with Bigram Lookahead Promotion
por: Iyer, Varun, et al.
Publicado: (2026)

Language Detection by Means of the Minkowski Norm: Identification Through Character Bigrams and Frequency Analysis
por: Pogăcean, Paul-Andrei, et al.
Publicado: (2025)

Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models
por: Gurgurov, Daniil, et al.
Publicado: (2025)

Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
por: Michaelov, James A., et al.
Publicado: (2025)

Evolving Subnetwork Training for Large Language Models
por: Li, Hanqi, et al.
Publicado: (2024)

Bolmo: Byteifying the Next Generation of Language Models
por: Minixhofer, Benjamin, et al.
Publicado: (2025)

Your Language Model Secretly Contains Personality Subnetworks
por: Ye, Ruimeng, et al.
Publicado: (2026)

Emergent inabilities? Inverse scaling over the course of pretraining
por: Michaelov, James A., et al.
Publicado: (2023)

Why do language models perform worse for morphologically complex languages?
por: Arnett, Catherine, et al.
Publicado: (2024)

Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models
por: Jones, Cameron R., et al.
Publicado: (2024)

The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
por: Bhaskar, Adithya, et al.
Publicado: (2024)

Discovering Knowledge-Critical Subnetworks in Pretrained Language Models
por: Bayazit, Deniz, et al.
Publicado: (2023)

Moving Beyond Next-Token Prediction: Transformers are Context-Sensitive Language Generators
por: Rhee, Phill Kyu
Publicado: (2025)

Better Language Model Inversion by Compactly Representing Next-Token Distributions
por: Nazir, Murtaza, et al.
Publicado: (2025)

Multimodal Latent Language Modeling with Next-Token Diffusion
por: Sun, Yutao, et al.
Publicado: (2024)

EVOKE: Emotion Vocabulary Of Korean and English
por: Jung, Yoonwon, et al.
Publicado: (2026)

How Open Must Language Models be to Enable Reliable Scientific Inference?
por: Michaelov, James A., et al.
Publicado: (2026)

Does GPT-4 pass the Turing test?
por: Jones, Cameron R., et al.
Publicado: (2023)

Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events
por: Michaelov, James A., et al.
Publicado: (2025)

ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams
por: Anand, Srija, et al.
Publicado: (2024)

SENTRA: Selected-Next-Token Transformer for LLM Text Detection
por: Plyler, Mitchell, et al.
Publicado: (2025)

Efficient Training of Language Models with Compact and Consistent Next Token Distributions
por: Sathe, Ashutosh, et al.
Publicado: (2024)

Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Model
por: Futami, Hayato, et al.
Publicado: (2024)

Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation
por: Zheng, Kening, et al.
Publicado: (2026)

Retrofitting Large Language Models with Dynamic Tokenization
por: Feher, Darius, et al.
Publicado: (2024)

A Law of Next-Token Prediction in Large Language Models
por: He, Hangfeng, et al.
Publicado: (2024)

Differentially Private Next-Token Prediction of Large Language Models
por: Flemings, James, et al.
Publicado: (2024)

Distribution Prompting: Understanding the Expressivity of Language Models Through the Next-Token Distributions They Can Produce
por: Wang, Haojin, et al.
Publicado: (2025)

Different Tokenization Schemes Lead to Comparable Performance in Spanish Number Agreement
por: Arnett, Catherine, et al.
Publicado: (2024)

From Next-Token to Mathematics: The Learning Dynamics of Mathematical Reasoning in Language Models
por: Mishra, Shubhra, et al.
Publicado: (2024)