Cover Image

GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kargaran, Amir Hossein, Yvon, François, Schütze, Hinrich
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GlotScript: A Resource and Tool for Low Resource Writing System Identification
by: Kargaran, Amir Hossein, et al.
Published: (2023)

GlotLID: Language Identification for Low-Resource Languages
by: Kargaran, Amir Hossein, et al.
Published: (2023)

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts
by: Kargaran, Amir Hossein, et al.
Published: (2026)

MaskLID: Code-Switching Language Identification through Iterative Masking
by: Kargaran, Amir Hossein, et al.
Published: (2024)

MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
by: Kargaran, Amir Hossein, et al.
Published: (2024)

How Programming Concepts and Neurons Are Shared in Code Language Models
by: Kargaran, Amir Hossein, et al.
Published: (2025)

MenuCraft: Interactive Menu System Design with Large Language Models
by: Kargaran, Amir Hossein, et al.
Published: (2023)

Quantifying Geospatial in the Common Crawl Corpus
by: Ilyankou, Ilya, et al.
Published: (2024)

Tracing Multilingual Factual Knowledge Acquisition in Pretraining
by: Liu, Yihong, et al.
Published: (2025)

CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation
by: Ziegler, Ingo, et al.
Published: (2024)

On Relation-Specific Neurons in Large Language Models
by: Liu, Yihong, et al.
Published: (2025)

Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts
by: Ma, Chunlan, et al.
Published: (2024)

UnifiedCrawl: Aggregated Common Crawl for Affordable Adaptation of LLMs on Low-Resource Languages
by: Tessema, Bethel Melesse, et al.
Published: (2024)

How Transliterations Improve Crosslingual Alignment
by: Liu, Yihong, et al.
Published: (2024)

Relational Linearity is a Predictor of Hallucinations
by: Lu, Yuetian, et al.
Published: (2026)

GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network
by: Yuan, Shuzhou, et al.
Published: (2024)

Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
by: Yang, Han, et al.
Published: (2025)

MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer
by: Ye, Haotian, et al.
Published: (2024)

The Synergy of Automated Pipelines with Prompt Engineering and Generative AI in Web Crawling
by: Huang, Chau-Jian
Published: (2024)

Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection
by: Hakimi, Ahmad Dawar, et al.
Published: (2026)

ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge
by: Taghavi, Zeinab Sadat, et al.
Published: (2025)

On the Entity-Level Alignment in Crosslingual Consistency
by: Liu, Yihong, et al.
Published: (2025)

Derivational Morphology Reveals Analogical Generalization in Large Language Models
by: Hofmann, Valentin, et al.
Published: (2024)

A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities
by: Ye, Haotian, et al.
Published: (2024)

Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu
by: Pei, Renhao, et al.
Published: (2025)

LongForm: Effective Instruction Tuning with Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2023)

Tibyan Corpus: Balanced and Comprehensive Error Coverage Corpus Using ChatGPT for Arabic Grammatical Error Correction
by: Alrehili, Ahlam, et al.
Published: (2024)

Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models
by: Röttger, Paul, et al.
Published: (2024)

MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2024)

BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
by: Nie, Ercong, et al.
Published: (2024)

Fine-Tuning Over Architectural Complexity: Broad-Coverage PII Detection on PIIBench with DeBERTa
by: Jha, Pritesh
Published: (2026)

Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation
by: Wang, Qianli, et al.
Published: (2026)

OpenWHO: A Document-Level Parallel Corpus for Health Translation in Low-Resource Languages
by: Merx, Raphaël, et al.
Published: (2025)

Dataverse: Open-Source ETL (Extract, Transform, Load) Pipeline for Large Language Models
by: Park, Hyunbyung, et al.
Published: (2024)

WinoWhat: A Parallel Corpus of Paraphrased WinoGrande Sentences with Common Sense Categorization
by: Gevers, Ine, et al.
Published: (2025)

Insights from the ICLR Peer Review and Rebuttal Process
by: Kargaran, Amir Hossein, et al.
Published: (2025)

Health Insurance Coverage Rule Interpretation Corpus: Law, Policy, and Medical Guidance for Health Insurance Coverage Understanding
by: Gartner, Mike
Published: (2025)

Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena
by: Weissweiler, Leonie, et al.
Published: (2024)

CC-GPX: Extracting High-Quality Annotated Geospatial Data from Common Crawl
by: Ilyankou, Ilya, et al.
Published: (2024)

Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset
by: Su, Dan, et al.
Published: (2024)