Image de couverture de livre

On the importance of Data Scale in Pretraining Arabic Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Ghaddar, Abbas, Langlais, Philippe, Rezagholizadeh, Mehdi, Chen, Boxing
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems
par: Ghaddar, Abbas, et autres
Publié: (2024)

ReGLA: Refining Gated Linear Attention
par: Lu, Peng, et autres
Publié: (2025)

OTTAWA: Optimal TransporT Adaptive Word Aligner for Hallucination and Omission Translation Errors Detection
par: Huang, Chenyang, et autres
Publié: (2024)

CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search
par: Mo, Fengran, et autres
Publié: (2024)

Batch-Max: Higher LLM Throughput using Larger Batch Sizes and KV Cache Compression
par: Metel, Michael R., et autres
Publié: (2024)

Integral Transformer: Denoising Attention, Not Too Much Not Too Little
par: Kobyzev, Ivan, et autres
Publié: (2025)

BOSCH: Black-Box Binary Optimization for Short-Context Attention-Head Selection in LLMs
par: Ghaddar, Abbas, et autres
Publié: (2026)

LABO: Towards Learning Optimal Label Regularization via Bi-level Optimization
par: Lu, Peng, et autres
Publié: (2023)

Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference
par: Kavehzadeh, Parsa, et autres
Publié: (2023)

Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity
par: Metel, Michael R., et autres
Publié: (2024)

Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucination
par: Huang, Jerry, et autres
Publié: (2024)

S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Models
par: Kavehzadeh, Parsa, et autres
Publié: (2024)

EchoAtt: Attend, Copy, then Adjust for More Efficient Large Language Models
par: Rajabzadeh, Hossein, et autres
Publié: (2024)

Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models
par: Jamialahmadi, Benyamin, et autres
Publié: (2025)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
par: Rajabzadeh, Hossein, et autres
Publié: (2024)

On Evaluation Protocols for Data Augmentation in a Limited Data Scenario
par: Piedboeuf, Frédéric, et autres
Publié: (2024)

EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems
par: Dehghan, Mohammad, et autres
Publié: (2024)

$\textit{BenchIE}^{FL}$ : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark
par: Lamarche, Fabrice, et autres
Publié: (2024)

Part-Of-Speech Sensitivity of Routers in Mixture of Experts Models
par: Antoine, Elie, et autres
Publié: (2024)

Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering
par: Kobeissi, Amine, et autres
Publié: (2026)

Resonance RoPE: Improving Context Length Generalization of Large Language Models
par: Wang, Suyuchen, et autres
Publié: (2024)

DTRNet: Dynamic Token Routing Network to Reduce Quadratic Costs in Transformers
par: Sharma, Aman, et autres
Publié: (2025)

The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs
par: Guan, Bryan, et autres
Publié: (2025)

Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
par: Wang, Xindi, et autres
Publié: (2024)

AraPoemBERT: A Pretrained Language Model for Arabic Poetry Analysis
par: Qarah, Faisal
Publié: (2024)

Zebra-Llama: Towards Extremely Efficient Hybrid Models
par: Yang, Mingyu, et autres
Publié: (2025)

Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data
par: Zhou, Jiaming, et autres
Publié: (2024)

X-EcoMLA: Upcycling Pre-Trained Attention into MLA for Efficient and Extreme KV Compression
par: Li, Guihong, et autres
Publié: (2025)

A linguistically-motivated evaluation methodology for unraveling model's abilities in reading comprehension tasks
par: Antoine, Elie, et autres
Publié: (2025)

Towards Practical Tool Usage for Continually Learning LLMs
par: Huang, Jerry, et autres
Publié: (2024)

Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora
par: Abbas, Chaymaa, et autres
Publié: (2026)

"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation
par: Thakur, Nandan, et autres
Publié: (2023)

Curriculum-Guided Layer Scaling for Language Model Pretraining
par: Singh, Karanpartap, et autres
Publié: (2025)

More Data, Fewer Diacritics: Scaling Arabic TTS
par: Musleh, Ahmed, et autres
Publié: (2026)

Thinking Long, but Short: Stable Sequential Test-Time Scaling for Large Reasoning Models
par: Metel, Michael R., et autres
Publié: (2026)

Multilingual Language Model Pretraining using Machine-translated Data
par: Wang, Jiayi, et autres
Publié: (2025)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
par: Ali, Mehdi, et autres
Publié: (2025)

The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technology
par: Al-Khalifa, Shahad, et autres
Publié: (2025)

AceGPT, Localizing Large Language Models in Arabic
par: Huang, Huang, et autres
Publié: (2023)

BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
par: Manoj, Guduru, et autres
Publié: (2025)