Cover Image

KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Wuyang, Gu, Yuxuan, Iacovides, Giorgos, Mandic, Danilo
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs
by: Gu, Yuxuan, et al.
Published: (2025)

FinDPO: Financial Sentiment Analysis for Algorithmic Trading through Preference Optimization of LLMs
by: Iacovides, Giorgos, et al.
Published: (2025)

Domain-Aware Tensor Network Structure Search
by: Iacovides, Giorgos, et al.
Published: (2025)

TeRA: Vector-based Random Tensor Network for High-Rank Adaptation of Large Language Models
by: Gu, Yuxuan, et al.
Published: (2025)

Towards LLM-guided Efficient and Interpretable Multi-linear Tensor Network Rank Selection
by: Iacovides, Giorgos, et al.
Published: (2024)

mHC: Manifold-Constrained Hyper-Connections
by: Xie, Zhenda, et al.
Published: (2025)

go-$m$HC: Direct Parameterization of Manifold-Constrained Hyper-Connections via Generalized Orthostochastic Matrices
by: Dandachi, Torque, et al.
Published: (2026)

Understanding the Rank of Tensor Networks via an Intuitive Example-Driven Approach
by: Zhou, Wuyang, et al.
Published: (2025)

RefineBridge: Generative Bridge Models Improve Financial Forecasting by Foundation Models
by: Bolton, Anthony, et al.
Published: (2025)

FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications
by: Konstantinidis, Thanos, et al.
Published: (2024)

Targeted Angular Reversal of Weights (TARS) for Knowledge Removal in Large Language Models
by: Davies, Harry J., et al.
Published: (2024)

mHC-SSM: Manifold-Constrained Hyper-Connections for State Space Language Models with Stream-Specialized Adapters
by: Mutlu, Abdulvahap, et al.
Published: (2026)

mHC-GNN: Manifold-Constrained Hyper-Connections for Graph Neural Networks
by: Mishra, Subhankar
Published: (2026)

Frac-Connections: Fractional Extension of Hyper-Connections
by: Zhu, Defa, et al.
Published: (2025)

Hyper-Connections
by: Zhu, Defa, et al.
Published: (2024)

Krony-PT: GPT2 compressed with Kronecker Products
by: Ayad, Mohamed Ayoub Ben, et al.
Published: (2024)

Geometry is All You Need: A Unified Taxonomy of Matrix and Tensor Factorization for Compression of Generative Language Models
by: Xu, Mingxue, et al.
Published: (2024)

TensorSLM: Energy-efficient Embedding Compression of Sub-billion Parameter Language Models on Low-end Devices
by: Xu, Mingxue, et al.
Published: (2025)

MoKA: Mixture of Kronecker Adapters
by: Sadeghi, Mohammadreza, et al.
Published: (2025)

TensorGPT: Efficient Compression of Large Language Models based on Tensor-Train Decomposition
by: Xu, Mingxue, et al.
Published: (2023)

Investigating the Synergistic Effects of Dropout and Residual Connections on Language Model Training
by: Li, Qingyang, et al.
Published: (2024)

Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders
by: Kurochkin, Vadim, et al.
Published: (2025)

SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection
by: Joshi, Maithili, et al.
Published: (2025)

JPmHC Dynamical Isometry via Orthogonal Hyper-Connections
by: Sengupta, Biswa, et al.
Published: (2026)

Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimation
by: Vernikos, Giorgos, et al.
Published: (2024)

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
by: Xiao, Da, et al.
Published: (2025)

Residual Matrix Transformers: Scaling the Size of the Residual Stream
by: Mak, Brian, et al.
Published: (2025)

Inference-Time Hyper-Scaling with KV Cache Compression
by: Łańcucki, Adrian, et al.
Published: (2025)

HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks
by: Zeng, Yiming, et al.
Published: (2025)

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
by: Wang, Qibin, et al.
Published: (2024)

ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations
by: Grishina, Ekaterina, et al.
Published: (2025)

Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices
by: Zhao, Pengxiang, et al.
Published: (2024)

Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
by: Gu, Xiaojie, et al.
Published: (2026)

Functionality-Oriented LLM Merging on the Fisher--Rao Manifold
by: Wang, Jiayu, et al.
Published: (2026)

Residual-Mass Accounting for Partial-KV Decoding
by: Hoshi, Yasuto, et al.
Published: (2026)

Residual Stream Analysis with Multi-Layer SAEs
by: Lawson, Tim, et al.
Published: (2024)

HyperMLP: An Integrated Perspective for Sequence Modeling
by: Lu, Jiecheng, et al.
Published: (2026)

HyperSteer: Activation Steering at Scale with Hypernetworks
by: Sun, Jiuding, et al.
Published: (2025)

Fast Inference with Kronecker-Sparse Matrices
by: Gonon, Antoine, et al.
Published: (2024)

Constrained Discrete Diffusion
by: Cardei, Michael, et al.
Published: (2025)