Cover Image

A Primer on the Inner Workings of Transformer-based Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ferrando, Javier, Sarti, Gabriele, Bisazza, Arianna, Costa-jussà, Marta R.
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task
by: Ferrando, Javier, et al.
Published: (2024)

Non Verbis, Sed Rebus: Large Language Models are Weak Solvers of Italian Rebuses
by: Sarti, Gabriele, et al.
Published: (2024)

Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation
by: Qi, Jirui, et al.
Published: (2024)

Steering Large Language Models for Machine Translation Personalization
by: Scalena, Daniel, et al.
Published: (2025)

Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement
by: Sarti, Gabriele, et al.
Published: (2025)

Quantifying the Plausibility of Context Reliance in Neural Machine Translation
by: Sarti, Gabriele, et al.
Published: (2023)

Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation
by: Edman, Lukas, et al.
Published: (2023)

QE4PE: Word-level Quality Estimation for Human Post-Editing
by: Sarti, Gabriele, et al.
Published: (2025)

Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Models
by: Hirak, Vitalii, et al.
Published: (2026)

Improving Language and Modality Transfer in Translation by Character-level Modeling
by: Tsiamas, Ioannis, et al.
Published: (2025)

NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication
by: Lian, Yuchen, et al.
Published: (2024)

Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models
by: Padovani, Francesca, et al.
Published: (2025)

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
by: Qi, Jirui, et al.
Published: (2023)

MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs
by: Neplenbroek, Vera, et al.
Published: (2024)

Is Child-Directed Language Optimized for Word Learning? A Computational Study of Verb Meaning Acquisition
by: Padovani, Francesca, et al.
Published: (2026)

A Unified Assessment of the Poverty of the Stimulus Argument for Neural Language Models
by: Yang, Xiulin, et al.
Published: (2026)

Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation
by: Neplenbroek, Vera, et al.
Published: (2024)

Gender-specific Machine Translation with Large Language Models
by: Sánchez, Eduardo, et al.
Published: (2023)

Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization
by: Neplenbroek, Vera, et al.
Published: (2025)

Simulating the Emergence of Differential Case Marking with Communicating Neural-Network Agents
by: Lian, Yuchen, et al.
Published: (2025)

BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency
by: Haga, Akari, et al.
Published: (2024)

When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy
by: Qi, Jirui, et al.
Published: (2025)

IT5: Text-to-text Pretraining for Italian Language Understanding and Generation
by: Sarti, Gabriele, et al.
Published: (2022)

On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation
by: Qi, Jirui, et al.
Published: (2025)

NeLLCom-Lex: A Neural-agent Framework to Study the Interplay between Lexical Systems and Language Use
by: Zhang, Yuqing, et al.
Published: (2025)

TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairs
by: Başar, Ezgi, et al.
Published: (2025)

MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs
by: Jumelet, Jaap, et al.
Published: (2025)

Unveiling the Role of Pretraining in Direct Speech Translation
by: Alastruey, Belen, et al.
Published: (2024)

LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models
by: Tufanov, Igor, et al.
Published: (2024)

Post-Training Language Models for Crosslingual Consistency
by: Liu, Tianyu, et al.
Published: (2026)

Modeling Human-Like Color Naming Behavior in Context
by: Zhang, Yuqing, et al.
Published: (2026)

Interference Matrix: Quantifying Cross-Lingual Interference in Transformer Encoders
by: Alastruey, Belen, et al.
Published: (2025)

Analyzing the Inner Workings of Transformers in Compositional Generalization
by: Kumon, Ryoma, et al.
Published: (2025)

Information Flow Routes: Automatically Interpreting Language Models at Scale
by: Ferrando, Javier, et al.
Published: (2024)

The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models
by: Chen, Xinyi, et al.
Published: (2024)

Multi-property Steering of Large Language Models with Dynamic Activation Composition
by: Scalena, Daniel, et al.
Published: (2024)

Reveal-Bangla: A Dataset for Cross-Lingual Multi-Step Reasoning Evaluation
by: Islam, Khondoker Ittehadul, et al.
Published: (2025)

Automated Interpretability and Feature Discovery in Language Models with Agents
by: Marin-Llobet, Arnau, et al.
Published: (2026)

Pushing the Limits of Zero-shot End-to-End Speech Translation
by: Tsiamas, Ioannis, et al.
Published: (2024)

DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers
by: Langedijk, Anna, et al.
Published: (2023)