Cover Image

From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Eric, Awasthi, Pranjal, Haghtalab, Nika
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
by: Halawi, Danny, et al.
Published: (2024)

The Limits of Preference Data for Post-Training
by: Zhao, Eric, et al.
Published: (2025)

Learning With Multi-Group Guarantees For Clusterable Subpopulations
by: Dai, Jessica, et al.
Published: (2024)

On-Demand Sampling: Learning Optimally from Multiple Distributions
by: Haghtalab, Nika, et al.
Published: (2022)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
by: Feng, Jiahai, et al.
Published: (2024)

Subliminal Effects in Your Data: A General Mechanism via Log-Linearity
by: Aden-Ali, Ishaq, et al.
Published: (2026)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Diffusion Language Models are Provably Optimal Parallel Samplers
by: Jiang, Haozhe, et al.
Published: (2025)

Is Knowledge Power? On the (Im)possibility of Learning from Strategic Interactions
by: Ananthakrishnan, Nivasini, et al.
Published: (2024)

KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation
by: Luo, Xindi, et al.
Published: (2024)

On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
by: Jiang, Haozhe, et al.
Published: (2025)

Understanding Finetuning for Factual Knowledge Extraction
by: Ghosal, Gaurav, et al.
Published: (2024)

Task Matrices: Linear Maps for Cross-Model Finetuning Transfer
by: Brien, Darrin O', et al.
Published: (2025)

Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?
by: Gölz, Paul, et al.
Published: (2025)

Predicting Emergent Capabilities by Finetuning
by: Snell, Charlie, et al.
Published: (2024)

Algorithmic Content Selection and the Impact of User Disengagement
by: Calvano, Emilio, et al.
Published: (2024)

From Data to Knowledge: Evaluating How Efficiently Language Models Learn Facts
by: Christoph, Daniel, et al.
Published: (2025)

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
by: Cho, Hanseul, et al.
Published: (2024)

Truthfulness of Calibration Measures
by: Haghtalab, Nika, et al.
Published: (2024)

Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers
by: Wong, Liang Ze
Published: (2025)

Time Sensitive Knowledge Editing through Efficient Finetuning
by: Ge, Xiou, et al.
Published: (2024)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
by: Zhang, Biao, et al.
Published: (2024)

Panprediction: Optimal Predictions for Any Downstream Task and Loss
by: Balakrishnan, Sivaraman, et al.
Published: (2025)

Efficient Knowledge Injection in LLMs via Self-Distillation
by: Kujanpää, Kalle, et al.
Published: (2024)

Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
by: Shi, Taiwei, et al.
Published: (2025)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
by: Vasudeva, Bhavya, et al.
Published: (2026)

Fact or Fiction? Improving Fact Verification with Knowledge Graphs through Simplified Subgraph Retrievals
by: Opsahl, Tobias A.
Published: (2024)

Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification
by: Zhao, Eric, et al.
Published: (2025)

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
by: Gupta, Prakhar, et al.
Published: (2026)

Improving Sparse Memory Finetuning
by: Goyal, Satyam, et al.
Published: (2026)

Dialectical Behavior Therapy Approach to LLM Prompting
by: Vitman, Oxana, et al.
Published: (2024)

From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge
by: Chowdhury, Nafis, et al.
Published: (2025)

Can Probabilistic Feedback Drive User Impacts in Online Platforms?
by: Dai, Jessica, et al.
Published: (2024)

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
by: Guo, Han, et al.
Published: (2023)

Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition
by: Jagadeesan, Meena, et al.
Published: (2023)

Large Language Models to Diffusion Finetuning
by: Cetin, Edoardo, et al.
Published: (2025)

Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction
by: Yang, Yuheng, et al.
Published: (2026)

From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs
by: Yu, Stanley, et al.
Published: (2025)

Sparse Upcycling: Inference Inefficient Finetuning
by: Doubov, Sasha, et al.
Published: (2024)

Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgments
by: Li, Jiaxi, et al.
Published: (2025)