Cover Image

An Analysis for Reasoning Bias of Language Models with Small Initialization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yao, Junjie, Zhang, Zhongwang, Xu, Zhi-Qin John
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reasoning Bias of Next Token Prediction Training
by: Lin, Pengxiao, et al.
Published: (2025)

Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers
by: Zhang, Zhongwang, et al.
Published: (2025)

Anchor function: a type of benchmark functions for studying language models
by: Zhang, Zhongwang, et al.
Published: (2024)

Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism
by: Wang, Zhiwei, et al.
Published: (2024)

Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing
by: Zhang, Zhongwang, et al.
Published: (2024)

Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
by: Xu, Xin, et al.
Published: (2025)

Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
by: He, Junlin, et al.
Published: (2026)

BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responses
by: Xu, Xin, et al.
Published: (2025)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
by: Samragh, Mohammad, et al.
Published: (2024)

Towards Reasoning Ability of Small Language Models
by: Srivastava, Gaurav, et al.
Published: (2025)

Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models
by: Li, Zihao, et al.
Published: (2025)

Can Small Language Models be Good Reasoners for Sequential Recommendation?
by: Wang, Yuling, et al.
Published: (2024)

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
by: Hu, Shengding, et al.
Published: (2024)

Bias after Prompting: Persistent Discrimination in Large Language Models
by: Sivakumar, Nivedha, et al.
Published: (2025)

Benchmarking Large Language Models for Math Reasoning Tasks
by: Seßler, Kathrin, et al.
Published: (2024)

Probability Signature: Bridging Data Semantics and Embedding Structure in Language Models
by: Yao, Junjie, et al.
Published: (2025)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
by: Cao, Jinghan, et al.
Published: (2026)

Reasoning Towards Fairness: Mitigating Bias in Language Models through Reasoning-Guided Fine-Tuning
by: Kabra, Sanchit, et al.
Published: (2025)

BiasEdit: Debiasing Stereotyped Language Models via Model Editing
by: Xu, Xin, et al.
Published: (2025)

Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
by: Scaria, Nicy, et al.
Published: (2024)

An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
by: Mundra, Nandini, et al.
Published: (2024)

Eliminating Position Bias of Language Models: A Mechanistic Approach
by: Wang, Ziqi, et al.
Published: (2024)

Loss Spike in Training Neural Networks
by: Li, Xiaolong, et al.
Published: (2023)

Fox-1: Open Small Language Model for Cloud and Edge
by: Hu, Zijian, et al.
Published: (2024)

Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models
by: Jin, Rihui, et al.
Published: (2025)

How Quantization Shapes Bias in Large Language Models
by: Marcuzzi, Federico, et al.
Published: (2025)

Bias in Large Language Models: Origin, Evaluation, and Mitigation
by: Guo, Yufei, et al.
Published: (2024)

Race, Ethnicity and Their Implication on Bias in Large Language Models
by: Hu, Shiyue, et al.
Published: (2026)

Retrieval-Reasoning Large Language Model-based Synthetic Clinical Trial Generation
by: Xu, Zerui, et al.
Published: (2024)

More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models
by: Wang, Xiao
Published: (2026)

T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
by: Hou, Zhenyu, et al.
Published: (2025)

PARAMANU-GANITA: Can Small Math Language Models Rival with Large Language Models on Mathematical Reasoning?
by: Niyogi, Mitodru, et al.
Published: (2024)

FERA: Uncertainty-Aware Federated Reasoning for Large Language Models
by: Wang, Ruhan, et al.
Published: (2026)

Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language
by: Gnadt, Kristin, et al.
Published: (2025)

Bridging Human and Model Perspectives: A Comparative Analysis of Political Bias Detection in News Media Using Large Language Models
by: Banik, Shreya Adrita, et al.
Published: (2025)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
by: Singh, Joykirat, et al.
Published: (2025)

Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models
by: Jhaveri, Ayush Rajesh, et al.
Published: (2026)

Red-Teaming for Inducing Societal Bias in Large Language Models
by: Luo, Chu Fei, et al.
Published: (2024)

Small Language Models: Survey, Measurements, and Insights
by: Lu, Zhenyan, et al.
Published: (2024)

Mitigating Selection Bias in Large Language Models via Permutation-Aware GRPO
by: Zheng, Jinquan, et al.
Published: (2026)