Cover Image

LP Data Pipeline: Lightweight, Purpose-driven Data Pipeline for Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kim, Yungi, Ha, Hyunsoo, Yang, Seonghoon, Lee, Sukyung, Kim, Jihoo, Park, Chanjun
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

1 Trillion Token (1TT) Platform: A Novel Framework for Efficient Data Sharing and Compensation in Large Language Models
by: Park, Chanjun, et al.
Published: (2024)

Rethinking KenLM: Good and Bad Model Ensembles for Efficient Text Quality Filtering in Large Web Corpora
by: Kim, Yungi, et al.
Published: (2024)

Dataverse: Open-Source ETL (Extract, Transform, Load) Pipeline for Large Language Models
by: Park, Hyunbyung, et al.
Published: (2024)

Open Ko-LLM Leaderboard2: Bridging Foundational and Practical Evaluation for Korean LLMs
by: Kim, Hyeonwoo, et al.
Published: (2024)

Representing the Under-Represented: Cultural and Core Capability Benchmarks for Developing Thai Large Language Models
by: Kim, Dahyun, et al.
Published: (2024)

Evalverse: Unified and Accessible Library for Large Language Model Evaluation
by: Kim, Jihoo, et al.
Published: (2024)

SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models
by: Kim, Hyeonwoo, et al.
Published: (2024)

Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark
by: Park, Chanjun, et al.
Published: (2024)

InstaTrans: An Instruction-Aware Translation Framework for Non-English Instruction Datasets
by: Kim, Yungi, et al.
Published: (2024)

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
by: Kim, Dahyun, et al.
Published: (2023)

sDPO: Don't Use Your Data All at Once
by: Kim, Dahyun, et al.
Published: (2024)

Model-Based Data-Centric AI: Bridging the Divide Between Academic Ideals and Industrial Pragmatism
by: Park, Chanjun, et al.
Published: (2024)

KITE: A Benchmark for Evaluating Korean Instruction-Following Abilities in Large Language Models
by: Kim, Dongjun, et al.
Published: (2025)

MultiDocFusion: Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents
by: Shin, Joongmin, et al.
Published: (2026)

Are Large Language Models the New Interface for Data Pipelines?
by: Junior, Sylvio Barbon, et al.
Published: (2024)

Understanding LLM Development Through Longitudinal Study: Insights from the Open Ko-LLM Leaderboard
by: Park, Chanjun, et al.
Published: (2024)

Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelines
by: Song, Hwanjun
Published: (2026)

Unveiling Imitation Learning: Exploring the Impact of Data Falsity to Large Language Model
by: Cho, Hyunsoo
Published: (2024)

Lost in the Pipeline: How Well Do Large Language Models Handle Data Preparation?
by: Spreafico, Matteo, et al.
Published: (2025)

ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction
by: Park, Jeiyoon, et al.
Published: (2024)

KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs
by: Kim, Haechan, et al.
Published: (2026)

FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models
by: Jung, Dahyun, et al.
Published: (2025)

M3-SLU: Evaluating Speaker-Attributed Reasoning in Multimodal Large Language Models
by: Kwon, Yejin, et al.
Published: (2025)

A Large Language Model Pipeline for Breast Cancer Oncology
by: Pool, Tristen, et al.
Published: (2024)

Translation of Multifaceted Data without Re-Training of Machine Translation Systems
by: Moon, Hyeonseok, et al.
Published: (2024)

Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learning
by: Kim, Jisu, et al.
Published: (2024)

The Moral Consistency Pipeline: Continuous Ethical Evaluation for Large Language Models
by: Jamshidi, Saeid, et al.
Published: (2025)

Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection
by: Kim, San, et al.
Published: (2025)

Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels
by: Cen, Zhepeng, et al.
Published: (2025)

Inertia in Moral and Value Judgments of Large Language Models
by: Lee, Bruce W., et al.
Published: (2024)

SALP-CG: Standard-Aligned LLM Pipeline for Classifying and Grading Large Volumes of Online Conversational Health Data
by: Yan, Yiwei, et al.
Published: (2025)

Synthetic Data Generation for Phrase Break Prediction with Large Language Model
by: Lee, Hoyeon, et al.
Published: (2025)

ZEBRA: Leveraging Model-Behavioral Knowledge for Zero-Annotation Preference Dataset Construction
by: Jung, Jeesu, et al.
Published: (2025)

Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks
by: Kim, Dongjun, et al.
Published: (2025)

Enhanced Urdu Intent Detection with Large Language Models and Prototype-Informed Predictive Pipelines
by: Hassan, Faiza, et al.
Published: (2025)

CharacterGPT: A Persona Reconstruction Framework for Role-Playing Agents
by: Park, Jeiyoon, et al.
Published: (2024)

Theme-Explanation Structure for Table Summarization using Large Language Models: A Case Study on Korean Tabular Data
by: Kwack, TaeYoon, et al.
Published: (2025)

CoME: An Unlearning-based Approach to Conflict-free Model Editing
by: Jung, Dahyun, et al.
Published: (2025)

PARM: Pipeline-Adapted Reward Model
by: Fan, Xingyu, et al.
Published: (2026)

DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines
by: Singhvi, Arnav, et al.
Published: (2023)