Cover Image

Updating Parametric Knowledge with Context Distillation Retains Post-Training Capabilities

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Padmanabhan, Shankar, Gul, Mustafa Omer, Goyal, Tanya
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Memorization vs. Reasoning: Updating LLMs with New Knowledge
by: Li, Aochong Oliver, et al.
Published: (2025)

MASH: Modeling Abstention via Selective Help-Seeking
by: Gul, Mustafa Omer, et al.
Published: (2025)

How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models
by: Kim, Minsung, et al.
Published: (2025)

Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention
by: Lin, Wenye, et al.
Published: (2026)

CoGen: Learning from Feedback with Coupled Comprehension and Generation
by: Gul, Mustafa Omer, et al.
Published: (2024)

DCRM: A Heuristic to Measure Response Pair Quality in Preference Optimization
by: Huang, Chengyu, et al.
Published: (2025)

KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
by: Xu, Hongling, et al.
Published: (2025)

From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascades
by: Wu, Yu, et al.
Published: (2025)

Spectrum Tuning: Post-Training for Distributional Coverage and In-Context Steerability
by: Sorensen, Taylor, et al.
Published: (2025)

Can QE-informed (Re)Translation lead to Error Correction?
by: Padmanabhan, Govardhan
Published: (2025)

EstLLM: Enhancing Estonian Capabilities in Multilingual LLMs via Continued Pretraining and Post-Training
by: Dorkin, Aleksei, et al.
Published: (2026)

Breaking the Capability Ceiling of LLM Post-Training by Reintroducing Markov States
by: Yuan, Yurun, et al.
Published: (2026)

EMCEE: Improving Multilingual Capability of LLMs via Bridging Knowledge and Reasoning with Extracted Synthetic Multilingual Context
by: Koo, Hamin, et al.
Published: (2025)

Distilling Mathematical Reasoning Capabilities into Small Language Models
by: Zhu, Xunyu, et al.
Published: (2024)

One Thousand and One Pairs: A "novel" challenge for long-context language models
by: Karpinska, Marzena, et al.
Published: (2024)

LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement
by: Ye, Zhifan, et al.
Published: (2025)

PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving
by: Parmar, Mihir, et al.
Published: (2025)

Intrinsic Test of Unlearning Using Parametric Knowledge Traces
by: Hong, Yihuai, et al.
Published: (2024)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
by: Tian, Yijun, et al.
Published: (2024)

IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages
by: Dawar, Aviral, et al.
Published: (2026)

BooookScore: A systematic exploration of book-length summarization in the era of LLMs
by: Chang, Yapei, et al.
Published: (2023)

ReAD: Reinforcement-Guided Capability Distillation for Large Language Models
by: Cheng, Xueqi, et al.
Published: (2026)

Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning
by: Kim, Minwu, et al.
Published: (2025)

Diagnosing Retrieval Bias Under Multiple In-Context Knowledge Updates in Large Language Models
by: Qiao, Boyu, et al.
Published: (2026)

Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
by: Bertsch, Amanda, et al.
Published: (2025)

When Context Leads but Parametric Memory Follows in Large Language Models
by: Tao, Yufei, et al.
Published: (2024)

Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning
by: Wang, Jianing, et al.
Published: (2023)

Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval
by: Vazhentsev, Artem, et al.
Published: (2026)

Distilling Monolingual and Crosslingual Word-in-Context Representations
by: Arase, Yuki, et al.
Published: (2024)

Decoder-based Sense Knowledge Distillation
by: Wang, Qitong, et al.
Published: (2026)

Self-Knowledge Distillation for Learning Ambiguity
by: Park, Hancheol, et al.
Published: (2024)

Training the Knowledge Base through Evidence Distillation and Write-Back Enrichment
by: Lu, Yuxing, et al.
Published: (2026)

Do Large Language Models Know Conflict? Investigating Parametric vs. Non-Parametric Knowledge of LLMs for Conflict Forecasting
by: Nemkova, Apollinaire Poli, et al.
Published: (2025)

On the Transformations across Reward Model, Parameter Update, and In-Context Prompt
by: Cai, Deng, et al.
Published: (2024)

LeKUBE: A Legal Knowledge Update BEnchmark
by: Wang, Changyue, et al.
Published: (2024)

Post-Trained MoE Can Skip Half Experts via Self-Distillation
by: Lv, Xingtai, et al.
Published: (2026)

Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
by: Bandarkar, Lucas, et al.
Published: (2026)

SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment
by: Liu, Qin, et al.
Published: (2024)

Retrieval-Constrained Decoding Reveals Underestimated Parametric Knowledge in Language Models
by: Hamdani, Rajaa El, et al.
Published: (2025)

Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement
by: Tan, Yuqiao, et al.
Published: (2025)