Copertina

Organize the Web: Constructing Domains Enhances Pre-Training Data Curation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wettig, Alexander, Lo, Kyle, Min, Sewon, Hajishirzi, Hannaneh, Chen, Danqi, Soldaini, Luca
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
di: Cao, Qingqing, et al.
Pubblicazione: (2023)

Learning to Detect Language Model Training Data via Active Reconstruction
di: Yin, Junjie Oscar, et al.
Pubblicazione: (2026)

Olmix: A Framework for Data Mixing Throughout LM Development
di: Chen, Mayee F., et al.
Pubblicazione: (2026)

QuRating: Selecting High-Quality Data for Training Language Models
di: Wettig, Alexander, et al.
Pubblicazione: (2024)

How to Train Long-Context Language Models (Effectively)
di: Gao, Tianyu, et al.
Pubblicazione: (2024)

Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens
di: Liu, Jiacheng, et al.
Pubblicazione: (2024)

olmOCR 2: Unit Test Rewards for Document OCR
di: Poznanski, Jake, et al.
Pubblicazione: (2025)

Metadata Conditioning Accelerates Language Model Pre-training
di: Gao, Tianyu, et al.
Pubblicazione: (2025)

Finding Transformer Circuits with Edge Pruning
di: Bhaskar, Adithya, et al.
Pubblicazione: (2024)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
di: Zhao, Bowen, et al.
Pubblicazione: (2024)

Extracting Rule-based Descriptions of Attention Features in Transformers
di: Friedman, Dan, et al.
Pubblicazione: (2025)

Establishing Task Scaling Laws via Compute-Efficient Model Ladders
di: Bhagia, Akshita, et al.
Pubblicazione: (2024)

Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?
di: Bhaskar, Adithya, et al.
Pubblicazione: (2025)

SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature
di: Wadden, David, et al.
Pubblicazione: (2024)

SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore
di: Min, Sewon, et al.
Pubblicazione: (2023)

Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations
di: Chen, Tong, et al.
Pubblicazione: (2025)

Automatic Detection of Research Values from Scientific Abstracts Across Computer Science Subfields
di: Jiang, Hang, et al.
Pubblicazione: (2025)

CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation
di: Chen, Tong, et al.
Pubblicazione: (2024)

Do Membership Inference Attacks Work on Large Language Models?
di: Duan, Michael, et al.
Pubblicazione: (2024)

OLMoE: Open Mixture-of-Experts Language Models
di: Muennighoff, Niklas, et al.
Pubblicazione: (2024)

KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions
di: Xu, Fangyuan, et al.
Pubblicazione: (2024)

ScienceMeter: Tracking Scientific Knowledge Updates in Language Models
di: Wang, Yike, et al.
Pubblicazione: (2025)

FlexOlmo: Open Language Models for Flexible Data Use
di: Shi, Weijia, et al.
Pubblicazione: (2025)

Reliable, Adaptable, and Attributable Language Models with Retrieval
di: Asai, Akari, et al.
Pubblicazione: (2024)

Toxicity of the Commons: Curating Open-Source Pre-Training Data
di: Arnett, Catherine, et al.
Pubblicazione: (2024)

Mathfish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula
di: Lucy, Li, et al.
Pubblicazione: (2024)

Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
di: Heineman, David, et al.
Pubblicazione: (2025)

HREF: Human Response-Guided Evaluation of Instruction Following in Language Models
di: Lyu, Xinxi, et al.
Pubblicazione: (2024)

ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data
di: Chen, Tong, et al.
Pubblicazione: (2025)

MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning
di: Wang, Haojin, et al.
Pubblicazione: (2026)

Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning
di: Kim, Joongwon, et al.
Pubblicazione: (2024)

EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Trees
di: Zeng, Zhiyuan, et al.
Pubblicazione: (2025)

DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images
di: Baral, Sami, et al.
Pubblicazione: (2025)

Paloma: A Benchmark for Evaluating Language Model Fit
di: Magnusson, Ian, et al.
Pubblicazione: (2023)

Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index
di: Xu, Hao, et al.
Pubblicazione: (2025)

Set the Clock: Temporal Alignment of Pretrained Language Models
di: Zhao, Bowen, et al.
Pubblicazione: (2024)

TurnWise: The Gap between Single- and Multi-turn Language Model Capabilities
di: Graf, Victoria, et al.
Pubblicazione: (2026)

Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions
di: Wiegreffe, Sarah, et al.
Pubblicazione: (2024)

Data Engineering for Scaling Language Models to 128K Context
di: Fu, Yao, et al.
Pubblicazione: (2024)

Steering off Course: Reliability Challenges in Steering Language Models
di: Da Silva, Patrick Queiroz, et al.
Pubblicazione: (2025)