Cover Image

Scale Determines Whether Language Models Organize Representation Geometry for Prediction

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Xu, Weilun
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations
by: Zhao, Yize, et al.
Published: (2024)

The Geometry of Tokens in Internal Representations of Large Language Models
by: Viswanathan, Karthik, et al.
Published: (2025)

The Linear Representation Hypothesis and the Geometry of Large Language Models
by: Park, Kiho, et al.
Published: (2023)

Geometry-Calibrated Conformal Abstention for Language Models
by: Xu, Rui, et al.
Published: (2026)

InnerThoughts: Disentangling Representations and Predictions in Large Language Models
by: Chételat, Didier, et al.
Published: (2025)

The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
by: Wollschläger, Tom, et al.
Published: (2025)

Tracing the Representation Geometry of Language Models from Pretraining to Post-training
by: Li, Melody Zixuan, et al.
Published: (2025)

Next Semantic Scale Prediction via Hierarchical Diffusion Language Models
by: Zhou, Cai, et al.
Published: (2025)

Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling
by: Tamoyan, Hovhannes, et al.
Published: (2025)

Shared Global and Local Geometry of Language Model Embeddings
by: Lee, Andrew, et al.
Published: (2025)

Improving LLM Final Representations with Inter-Layer Geometry
by: Ulanovski, Tom, et al.
Published: (2026)

Spherical Steering: Geometry-Aware Activation Rotation for Language Models
by: You, Zejia, et al.
Published: (2026)

Representation Deficiency in Masked Language Modeling
by: Meng, Yu, et al.
Published: (2023)

Observational Scaling Laws and the Predictability of Language Model Performance
by: Ruan, Yangjun, et al.
Published: (2024)

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
by: Gu, Jiawei, et al.
Published: (2024)

Scaling Laws for Post Training Quantized Large Language Models
by: Xu, Zifei, et al.
Published: (2024)

Spatial-Temporal Large Language Model for Traffic Prediction
by: Liu, Chenxi, et al.
Published: (2024)

On the Representational Capacity of Recurrent Neural Language Models
by: Nowak, Franz, et al.
Published: (2023)

On the Origins of Linear Representations in Large Language Models
by: Jiang, Yibo, et al.
Published: (2024)

Scaling Embedding Layers in Language Models
by: Yu, Da, et al.
Published: (2025)

Parallel Scaling Law for Language Models
by: Chen, Mouxiang, et al.
Published: (2025)

Scaling Laws for Multilingual Language Models
by: He, Yifei, et al.
Published: (2024)

Geometry of Decision Making in Language Models
by: Joshi, Abhinav, et al.
Published: (2025)

Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence
by: Nava, Andres, et al.
Published: (2026)

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
by: Liu, Shuo, et al.
Published: (2026)

Unified Scaling Laws for Compressed Representations
by: Panferov, Andrei, et al.
Published: (2025)

Entropic-Time Inference: Self-Organizing Large Language Model Decoding Beyond Attention
by: Kiruluta, Andrew
Published: (2026)

Multimodal Clinical Trial Outcome Prediction with Large Language Models
by: Zheng, Wenhao, et al.
Published: (2024)

Enhancing TCR-Peptide Interaction Prediction with Pretrained Language Models and Molecular Representations
by: Qi, Cong, et al.
Published: (2025)

Hierarchical Mamba Meets Hyperbolic Geometry: A New Paradigm for Structured Language Embeddings
by: Patil, Sarang, et al.
Published: (2025)

Making Pre-trained Language Models Great on Tabular Prediction
by: Yan, Jiahuan, et al.
Published: (2024)

Social Determinants of Health Prediction for ICD-9 Code with Reasoning Models
by: Khan, Sharim, et al.
Published: (2025)

Scaling Beyond Masked Diffusion Language Models
by: Sahoo, Subham Sekhar, et al.
Published: (2026)

Reusing Overtrained Language Models Saturates Scaling
by: Liew, Seng Pei, et al.
Published: (2025)

RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance
by: Chen, Tianlang, et al.
Published: (2025)

Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for Large Language Models
by: Davoodi, Arash Gholami, et al.
Published: (2026)

GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Models
by: Wan, Lipeng, et al.
Published: (2026)

Parallel Token Prediction for Language Models
by: Draxler, Felix, et al.
Published: (2025)

Can Large Language Models Generalize Procedures Across Representations?
by: Lin, Fangru, et al.
Published: (2026)

Emergent Stack Representations in Modeling Counter Languages Using Transformers
by: Tiwari, Utkarsh, et al.
Published: (2025)