Cover Image

The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bigoulaeva, Irina, Madabushi, Harish Tayyar, Gurevych, Iryna
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Are Emergent Abilities in Large Language Models just In-Context Learning?
by: Lu, Sheng, et al.
Published: (2023)

Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
by: Puerto, Haritz, et al.
Published: (2024)

Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning
by: Niu, Jingcheng, et al.
Published: (2025)

Patches of Nonlinearity: Instruction Vectors in Large Language Models
by: Bigoulaeva, Irina, et al.
Published: (2026)

FS-RAG: A Frame Semantics Based Approach for Improved Factual Accuracy in Large Language Models
by: Madabushi, Harish Tayyar
Published: (2024)

SpeciaLex: A Benchmark for In-Context Specialized Lexicon Learning
by: Imperial, Joseph Marvin, et al.
Published: (2024)

Evaluating CxG Generalisation in LLMs via Construction-Based NLI Fine Tuning
by: Mackintosh, Tom, et al.
Published: (2025)

Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
by: Madabushi, Harish Tayyar, et al.
Published: (2025)

Dancing with Deer: A Constructional Perspective on MWEs in the Era of LLMs
by: Bonial, Claire, et al.
Published: (2025)

Hypothesis-Driven Feature Manifold Analysis in LLMs via Supervised Multi-Dimensional Scaling
by: Tiblias, Federico, et al.
Published: (2025)

Scaling Policy Compliance Assessment in Language Models with Policy Reasoning Traces
by: Imperial, Joseph Marvin, et al.
Published: (2025)

Safer Policy Compliance with Dynamic Epistemic Fallback
by: Imperial, Joseph Marvin, et al.
Published: (2026)

Pre-Trained Language Models Represent Some Geographic Populations Better Than Others
by: Dunn, Jonathan, et al.
Published: (2024)

Evaluating Large Language Models on Multiword Expressions in Multilingual and Code-Switched Contexts
by: De Leon, Frances Laureano, et al.
Published: (2025)

Standardize: Aligning Language Models with Expert-Defined Standards for Content Generation
by: Imperial, Joseph Marvin, et al.
Published: (2024)

Code-Mixed Probes Show How Pre-Trained Models Generalise On Code-Switched Text
by: De Leon, Frances A. Laureano, et al.
Published: (2024)

Standardizing Intelligence: Aligning Generative AI for Regulatory and Operational Compliance
by: Imperial, Joseph Marvin, et al.
Published: (2025)

Adapting Whisper for Regional Dialects: Enhancing Public Services for Vulnerable Populations in the United Kingdom
by: Torgbi, Melissa, et al.
Published: (2025)

Word Boundary Information Isn't Useful for Encoder Language Models
by: Gow-Smith, Edward, et al.
Published: (2024)

Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions
by: Scivetti, Wesley, et al.
Published: (2025)

Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework
by: Dycke, Nils, et al.
Published: (2025)

Citation Failure: Definition, Analysis and Efficient Mitigation
by: Buchmann, Jan, et al.
Published: (2025)

Like a Good Nearest Neighbor: Practical Content Moderation and Text Classification
by: Bates, Luke, et al.
Published: (2023)

Systematic Task Exploration with LLMs: A Study in Citation Text Generation
by: Şahinuç, Furkan, et al.
Published: (2024)

Enabling Natural Zero-Shot Prompting on Encoder Models via Statement-Tuning
by: Elshabrawy, Ahmed, et al.
Published: (2024)

SciCoQA: Quality Assurance for Scientific Paper--Code Alignment
by: Baumgärtner, Tim, et al.
Published: (2026)

Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings
by: Liu, Chen Cecilia, et al.
Published: (2023)

Cultural Learning-Based Culture Adaptation of Language Models
by: Liu, Chen Cecilia, et al.
Published: (2025)

LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction
by: Tonga, Junior Cedric, et al.
Published: (2026)

SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models
by: Rizvi, Md Imbesat Hassan, et al.
Published: (2024)

Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs
by: Puerto, Haritz, et al.
Published: (2024)

ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation
by: Ortiz-Barajas, Jesus-German, et al.
Published: (2026)

Commitment Checklist: Auditing Author Commitments in Peer Review
by: Chen, Chung-Chi, et al.
Published: (2026)

Learning from Implicit User Feedback, Emotions and Demographic Information in Task-Oriented and Document-Grounded Dialogues
by: Petrak, Dominic, et al.
Published: (2024)

Identifying Aspects in Peer Reviews
by: Lu, Sheng, et al.
Published: (2025)

Token Weighting for Long-Range Language Modeling
by: Helm, Falko, et al.
Published: (2025)

COVE: COntext and VEracity prediction for out-of-context images
by: Tonglet, Jonathan, et al.
Published: (2025)

Turning Logic Against Itself : Probing Model Defenses Through Contrastive Questions
by: Sachdeva, Rachneet, et al.
Published: (2025)

Expert Preference-based Evaluation of Automated Related Work Generation
by: Şahinuç, Furkan, et al.
Published: (2025)

M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
by: Geng, Jiahui, et al.
Published: (2025)