Image de couverture de livre

How Does Controllability Emerge In Language Models During Pretraining?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	She, Jianshu, Li, Xinyue, Xing, Eric, Liu, Zhengzhong, Ho, Qirong
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LAPS: A Length-Aware-Prefill LLM Serving System
par: She, Jianshu, et autres
Publié: (2026)

Does Object Binding Naturally Emerge in Large Pretrained Vision Transformers?
par: Li, Yihao, et autres
Publié: (2025)

Hawkeye:Efficient Reasoning with Model Collaboration
par: She, Jianshu, et autres
Publié: (2025)

Instruct-Tuning Pretrained Causal Language Models for Ancient Greek Papyrology and Epigraphy
par: Cullhed, Eric
Publié: (2024)

From Supervision to Exploration: What Does Protein Language Model Learn During Reinforcement Learning?
par: Cao, Hanqun, et autres
Publié: (2025)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
par: Zheng, Wenhao, et autres
Publié: (2025)

Does Differential Privacy Impact Bias in Pretrained NLP Models?
par: Islam, Md. Khairul, et autres
Publié: (2024)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
par: Kang, Katie, et autres
Publié: (2024)

Continual Learning of Nonlinear Independent Representations
par: Sun, Boyang, et autres
Publié: (2024)

Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretraining
par: Yang, Yazheng, et autres
Publié: (2024)

Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models
par: Li, Yuan, et autres
Publié: (2025)

Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism
par: Bick, Aviv, et autres
Publié: (2025)

Pretrained Vision-Language-Action Models are Surprisingly Resistant to Forgetting in Continual Learning
par: Liu, Huihan, et autres
Publié: (2026)

Performance of Small Language Model Pretraining on FABRIC: An Empirical Study
par: Rao, Praveen
Publié: (2026)

Learning Transferable Sensor Models via Language-Informed Pretraining
par: Chen, Yuliang, et autres
Publié: (2026)

Ayn: A Tiny yet Competitive Indian Legal Language Model Pretrained from Scratch
par: Niyogi, Mitodru, et autres
Publié: (2024)

Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
par: Lu, Haoran, et autres
Publié: (2025)

Multiple Physics Pretraining for Physical Surrogate Models
par: McCabe, Michael, et autres
Publié: (2023)

DisCO: Reinforcing Large Reasoning Models with Discriminative Constrained Optimization
par: Li, Gang, et autres
Publié: (2025)

A Step Toward Federated Pretraining of Multimodal Large Language Models
par: Xiong, Baochen, et autres
Publié: (2026)

Does Spatial Cognition Emerge in Frontier Models?
par: Ramakrishnan, Santhosh Kumar, et autres
Publié: (2024)

Pretraining Large Language Models with NVFP4
par: NVIDIA, et autres
Publié: (2025)

Controlling the Risk of Corrupted Contexts for Language Models via Early-Exiting
par: Wynn, Andrea, et autres
Publié: (2025)

Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining
par: Sow, Daouda, et autres
Publié: (2025)

Fast Adaptive Anti-Jamming Channel Access via Deep Q Learning and Coarse-Grained Spectrum Prediction
par: Zhang, Jianshu, et autres
Publié: (2025)

State-Novelty Guided Action Persistence in Deep Reinforcement Learning
par: Hu, Jianshu, et autres
Publié: (2024)

How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not
par: Verdini, Francesco, et autres
Publié: (2024)

In-context Pretraining: Language Modeling Beyond Document Boundaries
par: Shi, Weijia, et autres
Publié: (2023)

GQA-μP: The maximal parameterization update for grouped query attention
par: Chickering, Kyle R., et autres
Publié: (2026)

Patent Language Model Pretraining with ModernBERT
par: Yousefiramandi, Amirhossein, et autres
Publié: (2025)

MPLite: Multi-Aspect Pretraining for Mining Clinical Health Records
par: Yang, Eric, et autres
Publié: (2024)

MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttraining
par: Xiaomi, LLM-Core, et autres
Publié: (2025)

A Robust PPO-optimized Tabular Transformer Framework for Intrusion Detection in Industrial IoT Systems
par: She, Yuanya
Publié: (2025)

Beyond Additivity: Sparse Isotonic Shapley Regression toward Nonlinear Explainability
par: She, Jialai
Publié: (2025)

State Space Models over Directed Graphs
par: She, Junzhi, et autres
Publié: (2025)

FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
par: Zhang, Yao, et autres
Publié: (2025)

Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
par: McLeish, Sean, et autres
Publié: (2025)

Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges
par: Liu, Xiaoqian, et autres
Publié: (2023)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
par: Somayajula, Sai Ashish, et autres
Publié: (2024)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
par: Foroutan, Negar, et autres
Publié: (2025)