Cover Image

RELIC: Evaluating Complex Reasoning via the Recognition of Languages In-Context

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Petty, Jackson, Hu, Michael Y., Wang, Wentao, Ravfogel, Shauli, Merrill, William, Linzen, Tal
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction
by: Petty, Jackson, et al.
Published: (2026)

Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases
by: Hu, Michael Y., et al.
Published: (2025)

Emergence of Linear Truth Encodings in Language Models
by: Ravfogel, Shauli, et al.
Published: (2025)

How Does Code Pretraining Affect Language Model Task Performance?
by: Petty, Jackson, et al.
Published: (2024)

Entailment Semantics Can Be Extracted from an Ideal Language Model
by: Merrill, William, et al.
Published: (2022)

In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax
by: Mueller, Aaron, et al.
Published: (2023)

Can LLMs Introspect? A Reality Check
by: Singh, Shashwat, et al.
Published: (2026)

The Illusion of State in State-Space Models
by: Merrill, William, et al.
Published: (2024)

The Impact of Depth on Compositional Generalization in Transformer Language Models
by: Petty, Jackson, et al.
Published: (2023)

State over Tokens: Characterizing the Role of Reasoning Tokens
by: Levy, Mosh, et al.
Published: (2025)

Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment
by: Merrill, William, et al.
Published: (2024)

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
by: Ben-Zaken, Elad, et al.
Published: (2021)

Do Language Models' Words Refer?
by: Mandelkern, Matthew, et al.
Published: (2023)

Log-linear Guardedness and its Implications
by: Ravfogel, Shauli, et al.
Published: (2022)

Rapid Word Learning Through Meta In-Context Learning
by: Wang, Wentao, et al.
Published: (2025)

From Directions to Regions: Decomposing Activations in Language Models via Local Geometry
by: Shafran, Or, et al.
Published: (2026)

Language Models Struggle to Use Representations Learned In-Context
by: Lepori, Michael A., et al.
Published: (2026)

Gumbel Counterfactual Generation From Language Models
by: Ravfogel, Shauli, et al.
Published: (2024)

Linear Adversarial Concept Erasure
by: Ravfogel, Shauli, et al.
Published: (2022)

Geometric Factual Recall in Transformers
by: Ravfogel, Shauli, et al.
Published: (2026)

IQ Test for LLMs: An Evaluation Framework for Uncovering Core Skills in LLMs
by: Maimon, Aviya, et al.
Published: (2025)

SPAWNing Structural Priming Predictions from a Cognitively Motivated Parser
by: Prasad, Grusha, et al.
Published: (2024)

Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis
by: Timkey, William, et al.
Published: (2026)

A Practical Method for Generating String Counterfactuals
by: Avitan, Matan, et al.
Published: (2024)

Kernelized Concept Erasure
by: Ravfogel, Shauli, et al.
Published: (2022)

The Role of Language Imbalance in Cross-lingual Generalisation: Insights from Cloned Language Experiments
by: Schäfer, Anton, et al.
Published: (2024)

The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasure
by: Fan, Yu, et al.
Published: (2025)

Beyond Single Embeddings: Capturing Diverse Targets with Multi-Query Retrieval
by: Chen, Hung-Ting, et al.
Published: (2025)

Diversity Over Quantity: A Lesson From Few Shot Relation Classification
by: Cohen, Amir DN, et al.
Published: (2024)

Intrinsic Test of Unlearning Using Parametric Knowledge Traces
by: Hong, Yihuai, et al.
Published: (2024)

Deconstructing sentence disambiguation by joint latent modeling of reading paradigms: LLM surprisal is not enough
by: Paape, Dario, et al.
Published: (2026)

Manipulating language models' training data to study syntactic constraint learning: the case of English passivization
by: Leong, Cara Su-Yi, et al.
Published: (2024)

Context-Free Recognition with Transformers
by: Jerad, Selim, et al.
Published: (2026)

RELIC: Investigating Large Language Model Responses using Self-Consistency
by: Cheng, Furui, et al.
Published: (2023)

Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
by: Qiu, Linlu, et al.
Published: (2025)

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
by: Hu, Michael Y., et al.
Published: (2026)

Multilingual Prompting for Improving LLM Generation Diversity
by: Wang, Qihan, et al.
Published: (2025)

Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment
by: Rassin, Royi, et al.
Published: (2023)

Representation Surgery: Theory and Practice of Affine Steering
by: Singh, Shashwat, et al.
Published: (2024)

Description-Based Text Similarity
by: Ravfogel, Shauli, et al.
Published: (2023)