Cover Image

Approximating Language Model Training Data from Weights

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Morris, John X., Yin, Junjie Oscar, Kim, Woojeong, Shmatikov, Vitaly, Rush, Alexander M.
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning to Detect Language Model Training Data via Active Reconstruction
by: Yin, Junjie Oscar, et al.
Published: (2026)

Universal Zero-shot Embedding Inversion
by: Zhang, Collin, et al.
Published: (2025)

Extracting Prompts by Inverting LLM Outputs
by: Zhang, Collin, et al.
Published: (2024)

Compute-Constrained Data Selection
by: Yin, Junjie Oscar, et al.
Published: (2024)

MillStone: How Open-Minded Are LLMs?
by: Triedman, Harold, et al.
Published: (2025)

Contextual Document Embeddings
by: Morris, John X., et al.
Published: (2024)

Adversarial Decoding: Generating Readable Documents for Adversarial Objectives
by: Zhang, Collin, et al.
Published: (2024)

Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents
by: Shafran, Avital, et al.
Published: (2024)

Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents
by: Jha, Rishi, et al.
Published: (2026)

How to Steal Reasoning Without Reasoning Traces
by: Zhang, Tingwei, et al.
Published: (2026)

From Chat Logs to Collective Insights: Aggregative Question Answering
by: Zhang, Wentao, et al.
Published: (2025)

DICE-BENCH: Evaluating the Tool-Use Capabilities of Large Language Models in Multi-Round, Multi-Party Dialogues
by: Jang, Kyochul, et al.
Published: (2025)

How much do language models memorize?
by: Morris, John X., et al.
Published: (2025)

Harnessing the Universal Geometry of Embeddings
by: Jha, Rishi, et al.
Published: (2025)

Better Language Model Inversion by Compactly Representing Next-Token Distributions
by: Nazir, Murtaza, et al.
Published: (2025)

Cascade-Aware Training of Language Models
by: Wang, Congchao, et al.
Published: (2024)

Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts
by: Ye, Jiayuan, et al.
Published: (2026)

Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
by: Na, Clara, et al.
Published: (2024)

Nomic Embed: Training a Reproducible Long Context Text Embedder
by: Nussbaum, Zach, et al.
Published: (2024)

Scaling Data-Constrained Language Models
by: Muennighoff, Niklas, et al.
Published: (2023)

WinoViz: Probing Visual Properties of Objects Under Different States
by: Jin, Woojeong, et al.
Published: (2024)

Evaluating Text Style Transfer: A Nine-Language Benchmark for Text Detoxification
by: Protasov, Vitaly, et al.
Published: (2025)

Balanced Training Data Augmentation for Aspect-Based Sentiment Analysis
by: Liu, Junjie, et al.
Published: (2025)

Data-free Weight Compress and Denoise for Large Language Models
by: Peng, Runyu, et al.
Published: (2024)

Offline Training of Language Model Agents with Functions as Learnable Weights
by: Zhang, Shaokun, et al.
Published: (2024)

PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery
by: He, Bowei, et al.
Published: (2025)

Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
by: Yu, Le, et al.
Published: (2024)

Self-interpreting Adversarial Images
by: Zhang, Tingwei, et al.
Published: (2024)

Great Memory, Shallow Reasoning: Limits of $k$NN-LMs
by: Geng, Shangyi, et al.
Published: (2024)

QuRating: Selecting High-Quality Data for Training Language Models
by: Wettig, Alexander, et al.
Published: (2024)

How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models
by: Kim, Minsung, et al.
Published: (2025)

MambaByte: Token-free Selective State Space Model
by: Wang, Junxiong, et al.
Published: (2024)

Transfer Learning for Automated Feedback Generation on Small Datasets
by: Morris, Oscar
Published: (2025)

Scalable Vision Language Model Training via High Quality Data Curation
by: Dong, Hongyuan, et al.
Published: (2025)

OverFill: Two-Stage Models for Efficient Language Model Decoding
by: Kim, Woojeong, et al.
Published: (2025)

A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
by: Luo, Junyu, et al.
Published: (2025)

OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
by: Lee, Changhun, et al.
Published: (2023)

DIRI: Adversarial Patient Reidentification with Large Language Models for Evaluating Clinical Text Anonymization
by: Morris, John X., et al.
Published: (2024)

Revealing the Inherent Instructability of Pre-Trained Language Models
by: An, Seokhyun, et al.
Published: (2024)

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
by: Charles, Zachary, et al.
Published: (2025)