Imagen de Portada

Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Xinyi, Tan, Shawn, Xu, Shenbo, Jin, Mingyu, Wang, William Yang, Panda, Rameswar, Shen, Yikang
Formato:	Preprint
Publicado:	2025
Materias:	Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Scattered Mixture-of-Experts Implementation
por: Tan, Shawn, et al.
Publicado: (2024)

Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study
por: Tan, Shawn, et al.
Publicado: (2024)

Gated Linear Attention Transformers with Hardware-Efficient Training
por: Yang, Songlin, et al.
Publicado: (2023)

API Pack: A Massive Multi-Programming Language Dataset for API Call Generation
por: Guo, Zhen, et al.
Publicado: (2024)

PaTH Attention: Position Encoding via Accumulating Householder Transformations
por: Yang, Songlin, et al.
Publicado: (2025)

Diversity Measurement and Subset Selection for Instruction Tuning Datasets
por: Wang, Peiqi, et al.
Publicado: (2024)

Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
por: Shen, Yikang, et al.
Publicado: (2024)

FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference
por: Nrusimha, Aniruddha, et al.
Publicado: (2025)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
por: Pan, Bowen, et al.
Publicado: (2024)

TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP Environments
por: Xu, Zhangchen, et al.
Publicado: (2025)

Data Engineering for Scaling Language Models to 128K Context
por: Fu, Yao, et al.
Publicado: (2024)

Distilling to Hybrid Attention Models via KL-Guided Layer Selection
por: Li, Yanhong, et al.
Publicado: (2025)

Unveiling Scaling Laws of Parameter Identifiability and Uncertainty Quantification in Data-Driven Biological Modeling
por: Wang, Shun, et al.
Publicado: (2026)

Provable Data Scaling Law for Meta Learning via Complexity Minimization
por: Fukuchi, Kazuto, et al.
Publicado: (2026)

Disentangling Memory and Reasoning Ability in Large Language Models
por: Jin, Mingyu, et al.
Publicado: (2024)

Exploring Efficiency Frontiers of Thinking Budget in Medical Reasoning: Scaling Laws between Computational Resources and Reasoning Quality
por: Bi, Ziqian, et al.
Publicado: (2025)

Scaling Laws for Linear Complexity Language Models
por: Shen, Xuyang, et al.
Publicado: (2024)

Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
por: Nrusimha, Aniruddha, et al.
Publicado: (2024)

PRISM: Demystifying Retention and Interaction in Mid-Training
por: Runwal, Bharat, et al.
Publicado: (2026)

Time Matters: Scaling Laws for Any Budget
por: Inbar, Itay, et al.
Publicado: (2024)

Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models
por: Agarwal, Mayank, et al.
Publicado: (2024)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
por: Brandon, William, et al.
Publicado: (2024)

LangNav: Language as a Perceptual Representation for Navigation
por: Pan, Bowen, et al.
Publicado: (2023)

Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models
por: Lin, Junhong, et al.
Publicado: (2025)

Indocyanine green localization for preoperative CT‐guided localization of multiple pulmonary nodules
por: Shenbo Zhang, et al.
Publicado: (2024)

Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
por: Wu, Xiaojun, et al.
Publicado: (2025)

Scaling Laws in Linear Regression: Compute, Parameters, and Data
por: Lin, Licong, et al.
Publicado: (2024)

Integrating Physics Inspired Features with Graph Convolution
por: Sahu, Rameswar
Publicado: (2024)

Probing the Inert Doublet Dark Matter with Stellar-Mass Black Hole Mini-Spikes
por: Sahu, Rameswar
Publicado: (2026)

Can DPO Learn Diverse Human Values? A Theoretical Scaling Law
por: Im, Shawn, et al.
Publicado: (2024)

Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization
por: Wei, Yikang, et al.
Publicado: (2024)

ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following
por: Yang, Yuancheng, et al.
Publicado: (2026)

Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information
por: Zhang, Zeyu, et al.
Publicado: (2025)

LLM-Generated Natural Language Meets Scaling Laws: New Explorations and Data Augmentation Methods
por: Wang, Zhenhua, et al.
Publicado: (2024)

A Survey of Scaling in Large Language Model Reasoning
por: Chen, Zihan, et al.
Publicado: (2025)

Data-Driven Parameter Identification for Tumor Growth Models
por: Liu, Liu, et al.
Publicado: (2025)

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
por: Wang, Xinyi, et al.
Publicado: (2023)

Understanding the Implicit User Intention via Reasoning with Large Language Model for Image Editing
por: Wang, Yijia, et al.
Publicado: (2025)

Advancements in hydrogel‐based embolic agents: Categorized by therapeutic mechanisms
por: Shenbo Zhang, et al.
Publicado: (2024)

TextPSG: Panoptic Scene Graph Generation from Textual Descriptions
por: Zhao, Chengyang, et al.
Publicado: (2023)