Imagen de Portada

From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Ruiqi, Wang, Lingxiang, Zhang, Hainan, Zheng, Zhiming, Lan, Yanyan
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation
por: Zhang, Ruiqi, et al.
Publicado: (2026)

Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Models
por: Chen, Jinwen, et al.
Publicado: (2025)

AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models
por: Zhang, Qianchi, et al.
Publicado: (2024)

MaFeRw: Query Rewriting with Multi-Aspect Feedbacks for Retrieval-Augmented Large Language Models
por: Wang, Yujing, et al.
Publicado: (2024)

Learning to Erase Private Knowledge from Multi-Documents for Retrieval-Augmented Large Language Models
por: Wang, Yujing, et al.
Publicado: (2025)

Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
por: Zhang, Yufeng, et al.
Publicado: (2024)

Parameter Importance-Driven Continual Learning for Foundation Models
por: Wang, Lingxiang, et al.
Publicado: (2025)

Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens
por: Zhang, Anqi, et al.
Publicado: (2024)

FedDTRE: Federated Dialogue Generation Models Powered by Trustworthiness Evaluation
por: Lu, Shule, et al.
Publicado: (2025)

Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generation
por: Zhang, Qianchi, et al.
Publicado: (2026)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
por: Wang, Shaobo, et al.
Publicado: (2026)

Inside the Black Box: Detecting Data Leakage in Pre-trained Language Encoders
por: Xin, Yuan, et al.
Publicado: (2024)

FedSEA-LLaMA: A Secure, Efficient and Adaptive Federated Splitting Framework for Large Language Models
por: Zhang, Zishuai, et al.
Publicado: (2025)

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
por: Yang, Lei, et al.
Publicado: (2025)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
por: Zhang, Haode, et al.
Publicado: (2022)

Less is More: Compact Clue Selection for Efficient Retrieval-Augmented Generation Reasoning
por: Zhang, Qianchi, et al.
Publicado: (2025)

Probing Language Models for Pre-training Data Detection
por: Liu, Zhenhua, et al.
Publicado: (2024)

Masked Structural Growth for 2x Faster Language Model Pre-training
por: Yao, Yiqun, et al.
Publicado: (2023)

LocalSUG: City-Preference-Enhanced LLM for Query Suggestion in Local-Life Services
por: Chen, Jinwen, et al.
Publicado: (2026)

DataMan: Data Manager for Pre-training Large Language Models
por: Peng, Ru, et al.
Publicado: (2025)

HSF: Defending against Jailbreak Attacks with Hidden State Filtering
por: Qian, Cheng, et al.
Publicado: (2024)

DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning
por: Qian, Yanyu, et al.
Publicado: (2026)

Model Merging in Pre-training of Large Language Models
por: Li, Yunshui, et al.
Publicado: (2025)

Pre-trained Language Models and Few-shot Learning for Medical Entity Extraction
por: Wang, Xiaokai, et al.
Publicado: (2025)

Understanding Data Temporality Impact on Large Language Models Pre-training
por: Pilchen, Hippolyte, et al.
Publicado: (2026)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
por: Ma, Shengjie, et al.
Publicado: (2025)

Measuring Stereotype and Deviation Biases in Large Language Models
por: Wang, Daniel, et al.
Publicado: (2025)

On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena
por: Naous, Tarek, et al.
Publicado: (2025)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
por: Kang, Katie, et al.
Publicado: (2024)

CodeBC: A More Secure Large Language Model for Smart Contract Code Generation in Blockchain
por: Wang, Lingxiang, et al.
Publicado: (2025)

Bridging the Knowledge Void: Inference-time Acquisition of Unfamiliar Programming Languages for Coding Tasks
por: Shen, Chen, et al.
Publicado: (2026)

Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding
por: Wang, Cheng, et al.
Publicado: (2024)

Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
por: Luo, Haitong, et al.
Publicado: (2025)

Cross-layer Attention Sharing for Pre-trained Large Language Models
por: Mu, Yongyu, et al.
Publicado: (2024)

Quantifying Memorization and Detecting Training Data of Pre-trained Language Models using Japanese Newspaper
por: Ishihara, Shotaro, et al.
Publicado: (2024)

GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics
por: Wang, Yujing, et al.
Publicado: (2026)

Enhancing Text-to-SQL Capabilities of Large Language Models via Domain Database Knowledge Injection
por: Ma, Xingyu, et al.
Publicado: (2024)

An Explainable Biomedical Foundation Model via Large-Scale Concept-Enhanced Vision-Language Pre-training
por: Nie, Yuxiang, et al.
Publicado: (2025)

RegMix: Data Mixture as Regression for Language Model Pre-training
por: Liu, Qian, et al.
Publicado: (2024)

Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models
por: Zhuang, Xinlin, et al.
Publicado: (2025)