Cover Image

Sequences of Logits Reveal the Low Rank Structure of Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Golowich, Noah, Liu, Allen, Shetty, Abhishek
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Provably Learning from Modern Language Models via Low Logit Rank
by: Golowich, Noah, et al.
Published: (2025)

Subliminal Effects in Your Data: A General Mechanism via Log-Linearity
by: Aden-Ali, Ishaq, et al.
Published: (2026)

Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation
by: Li, Jin, et al.
Published: (2025)

The Role of Sparsity for Length Generalization in Transformers
by: Golowich, Noah, et al.
Published: (2025)

Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inference
by: Golowich, Noah, et al.
Published: (2026)

CoRA: Optimizing Low-Rank Adaptation with Common Subspace of Large Language Models
by: Xiao, Xiaojun, et al.
Published: (2024)

Multiple Choice Learning of Low-Rank Adapters for Language Modeling
by: Letzelter, Victor, et al.
Published: (2025)

LoLCATs: On Low-Rank Linearizing of Large Language Models
by: Zhang, Michael, et al.
Published: (2024)

API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access
by: Su, Jiayuan, et al.
Published: (2024)

SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models
by: Zhang, Jianyi, et al.
Published: (2024)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
by: Shi, Haizhou, et al.
Published: (2024)

Physics of Language Models: Part 1, Learning Hierarchical Language Structures
by: Allen-Zhu, Zeyuan, et al.
Published: (2023)

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
by: Wang, Yibin, et al.
Published: (2024)

Logits are All We Need to Adapt Closed Models
by: Hiranandani, Gaurush, et al.
Published: (2025)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
by: Somayajula, Sai Ashish, et al.
Published: (2024)

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning
by: Tang, Chuanyu, et al.
Published: (2024)

Batched Low-Rank Adaptation of Foundation Models
by: Wen, Yeming, et al.
Published: (2023)

Sequence-to-Sequence Spanish Pre-trained Language Models
by: Araujo, Vladimir, et al.
Published: (2023)

PRECISE: Reducing the Bias of LLM Evaluations Using Prediction-Powered Ranking Estimation
by: Divekar, Abhishek, et al.
Published: (2026)

RadAnnotate: Large Language Models for Efficient and Reliable Radiology Report Annotation
by: Shetty, Saisha Pradeep, et al.
Published: (2026)

Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization
by: Hajimolahoseini, Habib, et al.
Published: (2023)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
by: Samplawski, Colin, et al.
Published: (2025)

Large Language Model Reasoning Failures
by: Song, Peiyang, et al.
Published: (2026)

Linear Bellman Completeness Suffices for Efficient Online Reinforcement Learning with Few Actions
by: Golowich, Noah, et al.
Published: (2024)

Is Efficient PAC Learning Possible with an Oracle That Responds 'Yes' or 'No'?
by: Daskalakis, Constantinos, et al.
Published: (2024)

The Role of Inherent Bellman Error in Offline Reinforcement Learning with Linear Function Approximation
by: Golowich, Noah, et al.
Published: (2024)

Localized LoRA: A Structured Low-Rank Approximation for Efficient Fine-Tuning
by: Barazandeh, Babak, et al.
Published: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

CoLa: Learning to Interactively Collaborate with Large Language Models
by: Sharma, Abhishek, et al.
Published: (2025)

Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation
by: Das, Devleena, et al.
Published: (2025)

Ensembles of Low-Rank Expert Adapters
by: Li, Yinghao, et al.
Published: (2025)

The Expressive Power of Low-Rank Adaptation
by: Zeng, Yuchen, et al.
Published: (2023)

LoRe: Personalizing LLMs via Low-Rank Reward Modeling
by: Bose, Avinandan, et al.
Published: (2025)

Toeplitz MLP Mixers are Low Complexity, Information-Rich Sequence Models
by: Badger, Benjamin L., et al.
Published: (2026)

AdaPreLoRA: Adafactor Preconditioned Low-Rank Adaptation
by: Liu, Ziyun, et al.
Published: (2026)

Ranking Large Language Models without Ground Truth
by: Dhurandhar, Amit, et al.
Published: (2024)

Uncertainty in Language Models: Assessment through Rank-Calibration
by: Huang, Xinmeng, et al.
Published: (2024)

AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models
by: Liu, Zeyu, et al.
Published: (2024)

Prescriptive Scaling Reveals the Evolution of Language Model Capabilities
by: Zhang, Hanlin, et al.
Published: (2026)