Cover Image

Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Choe, Jaeyoung, Noh, Keonwoong, Kim, Nayeon, Ahn, Seyun, Jung, Woohwan
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Automatic Transmission for LLM Tiers: Optimizing Cost and Accuracy in Large Language Models
by: Na, Injae, et al.
Published: (2025)

A Single Model Ensemble Framework for Neural Machine Translation using Pivot Translation
by: Oh, Seokjin, et al.
Published: (2025)

Hierarchical Retrieval with Evidence Curation for Open-Domain Financial Question Answering on Standardized Documents
by: Choe, Jaeyoung, et al.
Published: (2025)

Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions
by: Suh, Jiwon, et al.
Published: (2024)

Mangosteen: An Open Thai Corpus for Language Model Pretraining
by: Phatthiyaphaibun, Wannaphong, et al.
Published: (2025)

Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization
by: Behnia, Tina, et al.
Published: (2025)

FuLG: 150B Romanian Corpus for Language Model Pretraining
by: Bădoiu, Vlad-Andrei, et al.
Published: (2024)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
by: Soldaini, Luca, et al.
Published: (2024)

Don't Let It Fade: Preserving Edits in Diffusion Language Models via Token Timestep Allocation
by: Kim, Woojin, et al.
Published: (2025)

Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding
by: Jung, Sungmok, et al.
Published: (2026)

Acoustic-based Gender Differentiation in Speech-aware Language Models
by: Choi, Junhyuk, et al.
Published: (2025)

Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis
by: Lee, Nayeon, et al.
Published: (2023)

Cleaner Pretraining Corpus Curation with Neural Web Scraping
by: Xu, Zhipeng, et al.
Published: (2024)

Measuring Political Bias in Large Language Models: What Is Said and How It Is Said
by: Bang, Yejin, et al.
Published: (2024)

GlossLM: A Massively Multilingual Corpus and Pretrained Model for Interlinear Glossed Text
by: Ginn, Michael, et al.
Published: (2024)

Improving Detail in Pluralistic Image Inpainting with Feature Dequantization
by: Park, Kyungri, et al.
Published: (2024)

DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models
by: Thorat, Shantanu, et al.
Published: (2025)

VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models
by: Kim, Woojin, et al.
Published: (2026)

Multilingual Pretraining Using a Large Corpus Machine-Translated from a Single Source Language
by: Wang, Jiayi, et al.
Published: (2024)

CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge
by: Bae, Seyun, et al.
Published: (2026)

Modeling Caption Diversity in Contrastive Vision-Language Pretraining
by: Lavoie, Samuel, et al.
Published: (2024)

Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition
by: Kim, Jiyeon, et al.
Published: (2024)

What Happens When Small Is Made Smaller? Exploring the Impact of Compression on Small Data Pretrained Language Models
by: Awobade, Busayo, et al.
Published: (2024)

ProtoMed-LLM: An Automatic Evaluation Framework for Large Language Models in Medical Protocol Formulation
by: Yi, Seungjun, et al.
Published: (2024)

Exploring the Readiness of Prominent Small Language Models for the Democratization of Financial Literacy
by: Kosireddy, Tagore Rao, et al.
Published: (2024)

Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models
by: Jung, Chani, et al.
Published: (2024)

Vacaspati: A Diverse Corpus of Bangla Literature
by: Bhattacharyya, Pramit, et al.
Published: (2023)

Exploring the Benefits of Domain-Pretraining of Generative Large Language Models for Chemistry
by: Acharya, Anurag, et al.
Published: (2024)

MathBridge: A Large Corpus Dataset for Translating Spoken Mathematical Expressions into $LaTeX$ Formulas for Improved Readability
by: Jung, Kyudan, et al.
Published: (2024)

SEC-QA: A Systematic Evaluation Corpus for Financial QA
by: Lai, Viet Dac, et al.
Published: (2024)

Exploring Pretraining via Active Forgetting for Improving Cross Lingual Transfer for Decoder Language Models
by: Aggarwal, Divyanshu, et al.
Published: (2024)

Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages
by: Gurgurov, Daniil, et al.
Published: (2025)

MITRA: A Large-Scale Parallel Corpus and Multilingual Pretrained Language Model for Machine Translation and Semantic Retrieval for Pāli, Sanskrit, Buddhist Chinese, and Tibetan
by: Nehrdich, Sebastian, et al.
Published: (2026)

DNACHUNKER: Learnable Tokenization for DNA Language Models
by: Kim, Taewon, et al.
Published: (2026)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
by: Guo, Ping, et al.
Published: (2025)

Pretraining Language Models Using Translationese
by: Doshi, Meet, et al.
Published: (2024)

Geographic Adaptation of Pretrained Language Models
by: Hofmann, Valentin, et al.
Published: (2022)

Code Pretraining Improves Entity Tracking Abilities of Language Models
by: Kim, Najoung, et al.
Published: (2024)

Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning
by: Kim, Nayeon, et al.
Published: (2025)

Beyond Line-Level Filtering for the Pretraining Corpora of LLMs
by: Park, Chanwoo, et al.
Published: (2025)