Cover Image

Unveiling the Mystery of Weight in Large Foundation Models: Gaussian Distribution Never Fades

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Si, Chongjie, Jiang, Jingjing, Shen, Wei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models
by: Bordt, Sebastian, et al.
Published: (2024)

Dual-Personalizing Adapter for Federated Foundation Models
by: Yang, Yiyuan, et al.
Published: (2024)

See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition
by: Si, Chongjie, et al.
Published: (2024)

Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distribution
by: Zhao, Haiyan, et al.
Published: (2024)

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
by: Wang, Siwei, et al.
Published: (2024)

Foundations of Large Language Models
by: Xiao, Tong, et al.
Published: (2025)

RevOrder: A Novel Method for Enhanced Arithmetic in Language Models
by: Shen, Si, et al.
Published: (2024)

Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models
by: Dai, Runpeng, et al.
Published: (2025)

Unveiling and Addressing Pseudo Forgetting in Large Language Models
by: Sun, Huashan, et al.
Published: (2024)

Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
by: Liu, Jun, et al.
Published: (2024)

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
by: Chi, Haoang, et al.
Published: (2025)

Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation
by: Nadeem, Mohammad, et al.
Published: (2024)

Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection
by: Wang, Haoming, et al.
Published: (2025)

LLaSO: A Foundational Framework for Reproducible Research in Large Language and Speech Model
by: Sun, Yirong, et al.
Published: (2025)

GWQ: Gradient-Aware Weight Quantization for Large Language Models
by: Shao, Yihua, et al.
Published: (2024)

In-context Autoencoder for Context Compression in a Large Language Model
by: Ge, Tao, et al.
Published: (2023)

Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models
by: Sim, Shamus, et al.
Published: (2024)

Unveiling Imitation Learning: Exploring the Impact of Data Falsity to Large Language Model
by: Cho, Hyunsoo
Published: (2024)

Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopses
by: Su, Hung-Ting, et al.
Published: (2024)

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
by: Si, Chongjie, et al.
Published: (2025)

Textual Aesthetics in Large Language Models
by: Jiang, Lingjie, et al.
Published: (2024)

Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation
by: Vu, Tu, et al.
Published: (2024)

Pruning Foundation Models for High Accuracy without Retraining
by: Zhao, Pu, et al.
Published: (2024)

Are Large Language Models Table-based Fact-Checkers?
by: Zhang, Hanwen, et al.
Published: (2024)

Heterogeneous Scientific Foundation Model Collaboration
by: Li, Zihao, et al.
Published: (2026)

WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
by: Yue, Yuxuan, et al.
Published: (2024)

How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not
by: Verdini, Francesco, et al.
Published: (2024)

Representation Learning of Structured Data for Medical Foundation Models
by: Dwivedi, Vijay Prakash, et al.
Published: (2024)

Configurable Foundation Models: Building LLMs from a Modular Perspective
by: Xiao, Chaojun, et al.
Published: (2024)

An Evaluation on Large Language Model Outputs: Discourse and Memorization
by: de Wynter, Adrian, et al.
Published: (2023)

Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
by: Hu, Xinyang, et al.
Published: (2024)

Tool Learning with Foundation Models
by: Qin, Yujia, et al.
Published: (2023)

AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
by: Lee, Sangjun, et al.
Published: (2025)

Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost
by: Cao, Sheng, et al.
Published: (2025)

Weaver: Foundation Models for Creative Writing
by: Wang, Tiannan, et al.
Published: (2024)

Unveiling Downstream Performance Scaling of LLMs: A Clustering-Based Perspective
by: Xu, Chengyin, et al.
Published: (2025)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
by: Zhu, Hourun, et al.
Published: (2025)

Foundational Challenges in Assuring Alignment and Safety of Large Language Models
by: Anwar, Usman, et al.
Published: (2024)

Reviving The Classics: Active Reward Modeling in Large Language Model Alignment
by: Shen, Yunyi, et al.
Published: (2025)

A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Models
by: Hegselmann, Stefan, et al.
Published: (2024)