Cover Image

Improving Autoregressive Training with Dynamic Oracles

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Jianing, Visvanathan, Harshine, Wang, Yilin, Hu, Xinyi, Gormley, Matthew
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Mutually Informed Representations for Characters and Subwords
by: Wang, Yilin, et al.
Published: (2023)

Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
by: Karvonen, Adam, et al.
Published: (2025)

Rethinking Training Dynamics in Scale-wise Autoregressive Generation
by: Zhou, Gengze, et al.
Published: (2025)

MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
by: Zhang, Shiyue, et al.
Published: (2023)

ComPO: Preference Alignment via Comparison Oracles
by: Chen, Peter, et al.
Published: (2025)

Projected Autoregression: Autoregressive Language Generation in Continuous State Space
by: Naparstek, Oshri
Published: (2026)

Improving Academic Skills Assessment with NLP and Ensemble Learning
by: Huang, Xinyi, et al.
Published: (2024)

DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA
by: Yin, Jianing, et al.
Published: (2026)

Advancing Sequential Numerical Prediction in Autoregressive Models
by: Fei, Xiang, et al.
Published: (2025)

OUTLINEFORGE: Hierarchical Reinforcement Learning with Explicit States for Scientific Writing
by: Bao, Yilin, et al.
Published: (2026)

Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle
by: Dai, Hui, et al.
Published: (2024)

Continuous Autoregressive Language Models
by: Shao, Chenze, et al.
Published: (2025)

MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models
by: Wen, Yilin, et al.
Published: (2023)

Learning Dynamics in Continual Pre-Training for Large Language Models
by: Wang, Xingjin, et al.
Published: (2025)

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
by: Wang, Siwei, et al.
Published: (2024)

Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models
by: Zhu, Zhaowei, et al.
Published: (2023)

ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training
by: Liang, Yu, et al.
Published: (2026)

Continuous Approximations for Improving Quantization Aware Training of LLMs
by: Li, He, et al.
Published: (2024)

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
by: Zhuang, Yuchen, et al.
Published: (2025)

Context Dependence and Reliability in Autoregressive Language Models
by: Sengupta, Poushali, et al.
Published: (2026)

Enabling Autoregressive Models to Fill In Masked Tokens
by: Israel, Daniel, et al.
Published: (2025)

ToolACE-MT: Non-Autoregressive Generation for Agentic Multi-Turn Interaction
by: Zeng, Xingshan, et al.
Published: (2025)

Diagnosing Medical Datasets with Training Dynamics
by: Wenderoth, Laura
Published: (2024)

AR-Omni: A Unified Autoregressive Model for Any-to-Any Generation
by: Cheng, Dongjie, et al.
Published: (2026)

LLMs as High-Dimensional Nonlinear Autoregressive Models with Attention: Training, Alignment and Inference
by: Krishnamurthy, Vikram
Published: (2026)

MetaTool: Facilitating Large Language Models to Master Tools with Meta-task Augmentation
by: Wang, Xiaohan, et al.
Published: (2024)

Improving Steering Vectors by Targeting Sparse Autoencoder Features
by: Chalnev, Sviatoslav, et al.
Published: (2024)

Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
by: Sheshadri, Abhay, et al.
Published: (2024)

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
by: Wang, Xinyi, et al.
Published: (2023)

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction
by: Zaratiana, Urchade, et al.
Published: (2024)

A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints
by: Ahmed, Kareem, et al.
Published: (2023)

GPTQT: Quantize Large Language Models Twice to Push the Efficiency
by: Guo, Yipin, et al.
Published: (2024)

Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
by: Bergner, Benjamin, et al.
Published: (2024)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
by: Fu, Yonggan, et al.
Published: (2025)

Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoning
by: Berrayana, Lina, et al.
Published: (2025)

Position: The Turing-Completeness of Autoregressive Transformers Relies Heavily on Context Management
by: Cui, Guanyu, et al.
Published: (2026)

AdaFRUGAL: Adaptive Memory-Efficient Training with Dynamic Control
by: Bui, Quang-Hung, et al.
Published: (2025)

EvoLM: In Search of Lost Language Model Training Dynamics
by: Qi, Zhenting, et al.
Published: (2025)

Offline Reinforcement Learning for LLM Multi-Step Reasoning
by: Wang, Huaijie, et al.
Published: (2024)

Self-Training Meets Consistency: Improving LLMs' Reasoning with Consistency-Driven Rationale Evaluation
by: Lee, Jaehyeok, et al.
Published: (2024)