Cover Image

Pre-trained Large Language Models Learn Hidden Markov Models In-context

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dai, Yijia, Gao, Zhaolin, Sattar, Yahya, Dean, Sarah, Sun, Jennifer J.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Hidden Markov Models Using Conditional Samples
by: Kakade, Sham M., et al.
Published: (2023)

The Future of Large Language Model Pre-training is Federated
by: Sani, Lorenzo, et al.
Published: (2024)

Pre-training Limited Memory Language Models with Internal and External Knowledge
by: Zhao, Linxi, et al.
Published: (2025)

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing
by: Toral, Lukas, et al.
Published: (2025)

Ensemble Methods for Sequence Classification with Hidden Markov Models
by: Kawawa-Beaudan, Maxime, et al.
Published: (2024)

S$^2$ALM: Sequence-Structure Pre-trained Large Language Model for Comprehensive Antibody Representation Learning
by: Yin, Mingze, et al.
Published: (2024)

Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model
by: Niu, Peisong, et al.
Published: (2025)

Transfer Learning with Pre-trained Conditional Generative Models
by: Yamaguchi, Shin'ya, et al.
Published: (2022)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
by: Ibrahim, Adam, et al.
Published: (2024)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
by: Song, Weixi, et al.
Published: (2023)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
by: Samragh, Mohammad, et al.
Published: (2024)

Machine Unlearning of Pre-trained Large Language Models
by: Yao, Jin, et al.
Published: (2024)

An Efficient Replay for Class-Incremental Learning with Pre-trained Models
by: Yin, Weimin, et al.
Published: (2024)

IMU-1: Sample-Efficient Pre-training of Small Language Models
by: Grigorev, George
Published: (2026)

Transformers as Multi-task Learners: Decoupling Features in Hidden Markov Models
by: Hao, Yifan, et al.
Published: (2025)

A Vision-Language Pre-training Model-Guided Approach for Mitigating Backdoor Attacks in Federated Learning
by: Gai, Keke, et al.
Published: (2025)

Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model
by: Jin, Jiarui, et al.
Published: (2025)

Large Language Models as Markov Chains
by: Zekri, Oussama, et al.
Published: (2024)

Induced Numerical Instability: Hidden Costs in Multimodal Large Language Models
by: Wong, Wai Tuck, et al.
Published: (2026)

A Pre-trained Data Deduplication Model based on Active Learning
by: Shi, Haochen, et al.
Published: (2023)

Towards Efficient Pre-training: Exploring FP4 Precision in Large Language Models
by: Zhou, Jiecheng, et al.
Published: (2025)

Probing the Decision Boundaries of In-context Learning in Large Language Models
by: Zhao, Siyan, et al.
Published: (2024)

Revisiting In-context Learning Inference Circuit in Large Language Models
by: Cho, Hakaze, et al.
Published: (2024)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
by: Xia, Mengzhou, et al.
Published: (2023)

Representation Learning of Auxiliary Concepts for Improved Student Modeling and Exercise Recommendation
by: Badran, Yahya, et al.
Published: (2025)

Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models
by: Tang, Yuanbo, et al.
Published: (2025)

Investigating Data Contamination for Pre-training Language Models
by: Jiang, Minhao, et al.
Published: (2024)

Aligning Pre-trained Models for Spoken Language Translation
by: Sedláček, Šimon, et al.
Published: (2024)

Sequence-to-Sequence Spanish Pre-trained Language Models
by: Araujo, Vladimir, et al.
Published: (2023)

FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics
by: Duan, ChenRui, et al.
Published: (2024)

Backward-Friendly Optimization: Training Large Language Models with Approximate Gradients under Memory Constraints
by: Yang, Jing, et al.
Published: (2025)

Embedding Hidden Adversarial Capabilities in Pre-Trained Diffusion Models
by: Beerens, Lucas, et al.
Published: (2025)

Discovering and Reasoning of Causality in the Hidden World with Large Language Models
by: Liu, Chenxi, et al.
Published: (2024)

Markov Constraint as Large Language Model Surrogate
by: Bonlarron, Alexandre, et al.
Published: (2024)

Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models
by: Sun, Shaoning, et al.
Published: (2026)

Detecting Hidden Triggers: Mapping Non-Markov Reward Functions to Markov
by: Hyde, Gregory, et al.
Published: (2024)

OSF: On Pre-training and Scaling of Sleep Foundation Models
by: Shuai, Zitao, et al.
Published: (2026)

Scaling Laws for Pre-training Agents and World Models
by: Pearce, Tim, et al.
Published: (2024)

Integrating Pre-trained Language Model into Neural Machine Translation
by: Hwang, Soon-Jae, et al.
Published: (2023)

Learning Linear Dynamics from Bilinear Observations
by: Sattar, Yahya, et al.
Published: (2024)