Image de couverture de livre

Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Ishibashi, Yoichi, Yano, Taro, Oyamada, Masafumi
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents
par: Yano, Taro, et autres
Publié: (2025)

Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning
par: Ishibashi, Yoichi, et autres
Publié: (2024)

Effective Harness Engineering for Algorithm Discovery with Coding Agents
par: Ishibashi, Yoichi, et autres
Publié: (2026)

An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability
par: Yamauchi, Yusuke, et autres
Publié: (2025)

Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance
par: Tamura, Takuya, et autres
Publié: (2025)

Jellyfish: A Large Language Model for Data Preprocessing
par: Zhang, Haochen, et autres
Publié: (2023)

DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Ability
par: He, Yunzhen, et autres
Publié: (2025)

Understanding Hidden Computations in Chain-of-Thought Reasoning
par: Bharadwaj, Aryasomayajula Ram
Publié: (2024)

Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization
par: Ishibashi, Yoichi, et autres
Publié: (2024)

LLM Pretraining with Continuous Concepts
par: Tack, Jihoon, et autres
Publié: (2025)

Subspace Representations for Soft Set Operations and Sentence Similarities
par: Ishibashi, Yoichi, et autres
Publié: (2022)

BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining
par: DatologyAI, et autres
Publié: (2025)

Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal
par: Yuan, Aojie, et autres
Publié: (2026)

Reasoning-Driven Synthetic Data Generation and Evaluation
par: Davidson, Tim R., et autres
Publié: (2026)

RingSQL: Generating Synthetic Data with Schema-Independent Templates for Text-to-SQL Reasoning Models
par: Sterbentz, Marko, et autres
Publié: (2026)

RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold
par: Setlur, Amrith, et autres
Publié: (2024)

Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Sampling
par: Guo, Jizhou, et autres
Publié: (2025)

Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimization
par: Kawakami, Wataru, et autres
Publié: (2025)

BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
par: Merzah, Baqer M., et autres
Publié: (2025)

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection
par: Messmer, Bettina, et autres
Publié: (2025)

Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pretraining
par: Li, Jeffrey, et autres
Publié: (2026)

Deep Active Learning for Data Mining from Conflict Text Corpora
par: Croicu, Mihai
Publié: (2024)

Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning
par: Jia, Jinghan, et autres
Publié: (2026)

TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
par: Li, Jeffrey, et autres
Publié: (2025)

ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
par: Yang, Ling, et autres
Publié: (2025)

MedSyn: LLM-based Synthetic Medical Text Generation Framework
par: Kumichev, Gleb, et autres
Publié: (2024)

Thought Branches: Interpreting LLM Reasoning Requires Resampling
par: Macar, Uzay, et autres
Publié: (2025)

Thought Anchors: Which LLM Reasoning Steps Matter?
par: Bogdan, Paul C., et autres
Publié: (2025)

Accelerating Unbiased LLM Evaluation via Synthetic Feedback
par: Zhou, Zhaoyi, et autres
Publié: (2025)

Ulterior Motives: Detecting Misaligned Reasoning in Continuous Thought Models
par: Ramjee, Sharan
Publié: (2026)

Weight-of-Thought Reasoning: Exploring Neural Network Weights for Enhanced LLM Reasoning
par: Punjwani, Saif, et autres
Publié: (2025)

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
par: Saada, Thiziri Nait, et autres
Publié: (2025)

Revisiting Observation Reduction for Web Agents: Comprehensive Evaluation with a Lightweight Framework
par: Enomoto, Masafumi, et autres
Publié: (2026)

Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detection
par: Naseh, Ali, et autres
Publié: (2025)

Automated Text Mining of Experimental Methodologies from Biomedical Literature
par: Guo, Ziqing
Publié: (2024)

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows
par: Patel, Ajay, et autres
Publié: (2024)

Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content
par: Touchent, Rian, et autres
Publié: (2025)

Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching
par: Aytes, Simon A., et autres
Publié: (2025)

Strong Teacher Not Needed? On Distillation in LLM Pretraining
par: Lu, Taiming, et autres
Publié: (2026)

Demystifying Long Chain-of-Thought Reasoning in LLMs
par: Yeo, Edward, et autres
Publié: (2025)