Cover Image

Rethinking Weight Tying: Pseudo-Inverse Tying for LM Stable Training and Updates

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gu, Jian, Aleti, Aldeida, Chen, Chunyang, Zhang, Hongyu
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SeMe: Training-Free Language Model Merging via Semantic Alignment
by: Gu, Jian, et al.
Published: (2025)

Beyond Neural Incompatibility: Cross-Scale Knowledge Transfer in Language Models through Latent Semantic Alignment
by: Gu, Jian, et al.
Published: (2025)

Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learning
by: Gu, Jian, et al.
Published: (2024)

A Semantic-Aware Layer-Freezing Approach to Computation-Efficient Fine-Tuning of Language Models
by: Gu, Jian, et al.
Published: (2024)

Neuron Patching: Semantic-based Neuron-level Language Model Repair for Code Generation
by: Gu, Jian, et al.
Published: (2023)

A Semantic-based Optimization Approach for Repairing LLMs: Case Study on Code Generation
by: Gu, Jian, et al.
Published: (2025)

Set-based Neural Network Encoding Without Weight Tying
by: Andreis, Bruno, et al.
Published: (2023)

ReCellTy: Domain-Specific Knowledge Graph Retrieval-Augmented LLMs Reasoning Workflow for Single-Cell Annotation
by: Han, Dezheng, et al.
Published: (2025)

Weight Tying Biases Token Embeddings Towards the Output Space
by: Lopardo, Antonio, et al.
Published: (2026)

ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
by: Zheng, Kangjie, et al.
Published: (2025)

SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues
by: Kuo, Martin, et al.
Published: (2025)

Dynamic Layer Tying for Parameter-Efficient Transformers
by: Hay, Tamir David, et al.
Published: (2024)

Stable LM 2 1.6B Technical Report
by: Bellagente, Marco, et al.
Published: (2024)

DEPART: DEcomposing PARiTy across Multilingual LLMs
by: Uppadhyay, Manan, et al.
Published: (2026)

EvoLM: In Search of Lost Language Model Training Dynamics
by: Qi, Zhenting, et al.
Published: (2025)

Fast and Effective Weight Update for Pruned Large Language Models
by: Boža, Vladimír
Published: (2024)

Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification
by: Kuo, Hsun-Yu, et al.
Published: (2024)

SBoRA: Low-Rank Adaptation with Regional Weight Updates
by: Po, Lai-Man, et al.
Published: (2024)

CausalLM is not optimal for in-context learning
by: Ding, Nan, et al.
Published: (2023)

Identifying the Risks of LM Agents with an LM-Emulated Sandbox
by: Ruan, Yangjun, et al.
Published: (2023)

Efficient RLVR Training via Weighted Mutual Information Data Selection
by: Zhou, Xinyu, et al.
Published: (2026)

Transferable Post-training via Inverse Value Learning
by: Lu, Xinyu, et al.
Published: (2024)

Stability-Weighted Decoding for Diffusion Language Models
by: Wu, Yue, et al.
Published: (2026)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
by: Pan, Bowen, et al.
Published: (2024)

Rethinking Local Learning: A Cheaper and Faster Recipe for LLM Post-Training
by: Shi, Hengyu, et al.
Published: (2026)

Stopping Computation for Converged Tokens in Masked Diffusion-LM Decoding
by: Oba, Daisuke, et al.
Published: (2026)

DarwinLM: Evolutionary Structured Pruning of Large Language Models
by: Tang, Shengkun, et al.
Published: (2025)

Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models
by: Tian, Junjiao, et al.
Published: (2024)

UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models
by: Gu, Xiaojie, et al.
Published: (2025)

DataComp-LM: In search of the next generation of training sets for language models
by: Li, Jeffrey, et al.
Published: (2024)

SMEC: Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression
by: Zhang, Biao, et al.
Published: (2025)

VRPO: Rethinking Value Modeling for Robust RL Training under Noisy Supervision
by: Zhu, Dingwei, et al.
Published: (2025)

In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement
by: Zhang, Zhen-Yu, et al.
Published: (2024)

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
by: Zhuo, Zhijian, et al.
Published: (2025)

APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning
by: Sun, Jiashuo, et al.
Published: (2022)

TWISTED-RL: Hierarchical Skilled Agents for Knot-Tying without Human Demonstrations
by: Freund, Guy, et al.
Published: (2026)

Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
by: Hua, Andong, et al.
Published: (2025)

CodecLM: Aligning Language Models with Tailored Synthetic Data
by: Wang, Zifeng, et al.
Published: (2024)

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
by: Vashurin, Roman, et al.
Published: (2024)

WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling
by: Li, Jiacheng, et al.
Published: (2025)