Cover Image

Tracing the Representation Geometry of Language Models from Pretraining to Post-training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Melody Zixuan, Agrawal, Kumar Krishna, Ghosh, Arna, Teru, Komal Kumar, Santoro, Adam, Lajoie, Guillaume, Richards, Blake A.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Harnessing small projectors and multiple views for efficient vision pretraining
by: Agrawal, Kumar Krishna, et al.
Published: (2023)

Synaptic Weight Distributions Depend on the Geometry of Plasticity
by: Pogodin, Roman, et al.
Published: (2023)

Measuring Representation Robustness in Large Language Models for Geometry
by: Jawandhia, Vedant, et al.
Published: (2026)

Language Models' Factuality Depends on the Language of Inquiry
by: Aggarwal, Tushar, et al.
Published: (2025)

On Linear Representations and Pretraining Data Frequency in Language Models
by: Merullo, Jack, et al.
Published: (2025)

Gender Encoding Patterns in Pretrained Language Model Representations
by: Zakizadeh, Mahdi, et al.
Published: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
by: Foroutan, Negar, et al.
Published: (2025)

Inhibitory normalization of error signals improves learning in neural circuits
by: Eyono, Roy Henha, et al.
Published: (2026)

Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviour
by: Norris, Max, et al.
Published: (2025)

A Survey on Post-training of Large Language Models
by: Tie, Guiyao, et al.
Published: (2025)

Training Language Models via Neural Cellular Automata
by: Lee, Dan, et al.
Published: (2026)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
by: Wang, Xinyi, et al.
Published: (2024)

Context Structure Reshapes the Representational Geometry of Language Models
by: Hosseini, Eghbal A., et al.
Published: (2026)

Enhancing Cache-Augmented Generation (CAG) with Adaptive Contextual Compression for Scalable Knowledge Integration
by: Agrawal, Rishabh, et al.
Published: (2025)

Why all roads don't lead to Rome: Representation geometry varies across the human visual cortical hierarchy
by: Ghosh, Arna, et al.
Published: (2025)

A Complexity-Based Theory of Compositionality
by: Elmoznino, Eric, et al.
Published: (2024)

Tracing Persona Vectors Through LLM Pretraining
by: Moskvoretskii, Viktor, et al.
Published: (2026)

Post-training Large Language Models for Diverse High-Quality Responses
by: Chen, Yilei, et al.
Published: (2025)

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
by: Qian, Chen, et al.
Published: (2024)

"Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Models
by: Tao, Yufei, et al.
Published: (2025)

The Tokenization Bottleneck: How Vocabulary Extension Improves Chemistry Representation Learning in Pretrained Language Models
by: Kalamkar, Prathamesh, et al.
Published: (2025)

Predicting Compact Phrasal Rewrites with Large Language Models for ASR Post Editing
by: Zhang, Hao, et al.
Published: (2025)

The Linear Representation Hypothesis and the Geometry of Large Language Models
by: Park, Kiho, et al.
Published: (2023)

Do Language Models Know When They're Hallucinating References?
by: Agrawal, Ayush, et al.
Published: (2023)

Multilingual Pretraining for Pixel Language Models
by: Kesen, Ilker, et al.
Published: (2025)

Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
by: Kumar, Sachin
Published: (2026)

Demystifying Domain-adaptive Post-training for Financial LLMs
by: Ke, Zixuan, et al.
Published: (2025)

Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations
by: Banerjee, Somnath, et al.
Published: (2025)

Towards Safer Pretraining: Analyzing and Filtering Harmful Content in Webscale datasets for Responsible LLMs
by: Mendu, Sai Krishna, et al.
Published: (2025)

When Context Leads but Parametric Memory Follows in Large Language Models
by: Tao, Yufei, et al.
Published: (2024)

Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages
by: Samuel, David, et al.
Published: (2025)

The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
by: Wollschläger, Tom, et al.
Published: (2025)

Enhancing Financial VQA in Vision Language Models using Intermediate Structured Representations
by: Srivastava, Archita, et al.
Published: (2025)

The Representational Geometry of Number
by: Hu, Zhimin, et al.
Published: (2026)

MTQ-Eval: Multilingual Text Quality Evaluation for Language Models
by: Pokharel, Rhitabrat, et al.
Published: (2025)

Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels
by: Kumar, Anantha Padmanaban Krishna
Published: (2025)

Drop Dropout on Single-Epoch Language Model Pretraining
by: Liu, Houjun, et al.
Published: (2025)

Rethinking the Role of Text Complexity in Language Model Pretraining
by: Velasco, Dan John, et al.
Published: (2025)

Mechanism Shift During Post-training from Autoregressive to Masked Diffusion Language Models
by: Kong, Injin, et al.
Published: (2026)

Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
by: Tao, Yongding, et al.
Published: (2025)