Cover Image

Toxicity of the Commons: Curating Open-Source Pre-Training Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Arnett, Catherine, Jones, Eliot, Yamshchikov, Ivan P., Langlais, Pierre-Carl
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training
by: Langlais, Pierre-Carl, et al.
Published: (2025)

What the HellaSwag? On the Validity of Common-Sense Reasoning Benchmarks
by: Chizhov, Pavel, et al.
Published: (2025)

BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training
by: Chizhov, Pavel, et al.
Published: (2024)

Model in Distress: Sentiment Analysis on French Synthetic Social Media
by: Langlais, Pierre-Carl, et al.
Published: (2026)

Even Small Reasoners Should Quote Their Sources: Introducing the Pleias-RAG Model Family
by: Langlais, Pierre-Carl, et al.
Published: (2025)

From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution
by: Chizhov, Pavel, et al.
Published: (2026)

Knowledge Graph Representation for Political Information Sources
by: Osmonova, Tinatin, et al.
Published: (2024)

Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pre-trained Models
by: Purason, Taido, et al.
Published: (2025)

Disaggregation Reveals Hidden Training Dynamics: The Case of Agreement Attraction
by: Michaelov, James A., et al.
Published: (2025)

Organize the Web: Constructing Domains Enhances Pre-Training Data Curation
by: Wettig, Alexander, et al.
Published: (2025)

Do Data-based Curricula Work?
by: Surkov, Maxim K., et al.
Published: (2021)

Transfer of Structural Knowledge from Synthetic Languages
by: Budnikov, Mikhail, et al.
Published: (2025)

$\textit{BenchIE}^{FL}$ : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark
by: Lamarche, Fabrice, et al.
Published: (2024)

Sui Generis: Large Language Models for Authorship Attribution and Verification in Latin
by: Schmidt, Gleb, et al.
Published: (2024)

ComicScene154: A Scene Dataset for Comic Analysis
by: Paval, Sandro, et al.
Published: (2025)

Vocabulary Transfer for Biomedical Texts: Add Tokens if You Can Not Add Data
by: Singh, Priyanka, et al.
Published: (2022)

BPE Stays on SCRIPT: Structured Encoding for Robust Multilingual Pretokenization
by: Land, Sander, et al.
Published: (2025)

Why do language models perform worse for morphologically complex languages?
by: Arnett, Catherine, et al.
Published: (2024)

On Evaluation Protocols for Data Augmentation in a Limited Data Scenario
by: Piedboeuf, Frédéric, et al.
Published: (2024)

Surface Fairness, Deep Bias: A Comparative Study of Bias in Language Models
by: Sorokovikova, Aleksandra, et al.
Published: (2025)

What is Wrong with Language Models that Can Not Tell a Story?
by: Yamshchikov, Ivan P., et al.
Published: (2022)

GDC Cohort Copilot: An AI Copilot for Curating Cohorts from the Genomic Data Commons
by: Song, Steven, et al.
Published: (2025)

Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
by: Lupidi, Alisia, et al.
Published: (2024)

OpenGVL -- Benchmarking Visual Temporal Progress for Data Curation
by: Budzianowski, Paweł, et al.
Published: (2025)

ToxiLab: How Well Do Open-Source LLMs Generate Synthetic Toxicity Data?
by: Hui, Zheng, et al.
Published: (2024)

CleanComedy: Creating Friendly Humor through Generative Techniques
by: Vikhorev, Dmitry, et al.
Published: (2024)

Better Call Claude: Can LLMs Detect Changes of Writing Style?
by: Römisch, Johannes, et al.
Published: (2025)

Team "better_call_claude": Style Change Detection using a Sequential Sentence Pair Classifier
by: Schmidt, Gleb, et al.
Published: (2025)

On the importance of Data Scale in Pretraining Arabic Language Models
by: Ghaddar, Abbas, et al.
Published: (2024)

Evaluating Morphological Alignment of Tokenizers in 70 Languages
by: Arnett, Catherine, et al.
Published: (2025)

Neural Machine Translation for Malayalam Paraphrase Generation
by: Varghese, Christeena, et al.
Published: (2024)

OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data
by: Du, Yuwen, et al.
Published: (2026)

Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering
by: Kobeissi, Amine, et al.
Published: (2026)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
by: Guo, Xu, et al.
Published: (2026)

Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics
by: Michaelov, James A., et al.
Published: (2024)

A Bit of a Problem: Measurement Disparities in Dataset Sizes Across Languages
by: Arnett, Catherine, et al.
Published: (2024)

Weight Tying Biases Token Embeddings Towards the Output Space
by: Lopardo, Antonio, et al.
Published: (2026)

Re-defining Humor Data Objects for AI Humor Research
by: Arnett, Anna, et al.
Published: (2026)

ReGLA: Refining Gated Linear Attention
by: Lu, Peng, et al.
Published: (2025)

Echo-chambers and Idea Labs: Communication Styles on Twitter
by: Sorokovikova, Aleksandra, et al.
Published: (2024)