Cover Image

TensorSLM: Energy-efficient Embedding Compression of Sub-billion Parameter Language Models on Low-end Devices

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Mingxue, Xu, Yao Lei, Mandic, Danilo P.
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning Numerical Analysis
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TensorGPT: Efficient Compression of Large Language Models based on Tensor-Train Decomposition
by: Xu, Mingxue, et al.
Published: (2023)

Geometry is All You Need: A Unified Taxonomy of Matrix and Tensor Factorization for Compression of Generative Language Models
by: Xu, Mingxue, et al.
Published: (2024)

UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization
by: Jiang, Yiming, et al.
Published: (2023)

Tensor BM-Decomposition for Compression and Analysis of Video Data
by: Tian, Fan, et al.
Published: (2023)

A Novel Adaptive Low-Rank Matrix Approximation Method for Image Compression and Reconstruction
by: Xu, Weiwei, et al.
Published: (2025)

Modified Levenberg-Marquardt Algorithm For Tensor CP Decomposition in Image Compression
by: Karim, Ramin Goudarzi, et al.
Published: (2024)

FEABench: Evaluating Language Models on Multiphysics Reasoning Ability
by: Mudur, Nayantara, et al.
Published: (2025)

Efficient Orthogonal Decomposition with Automatic Basis Extraction for Low-Rank Matrix Approximation
by: Shen, Weijie, et al.
Published: (2024)

Randomized Functional Sparse Tucker Tensor for Compression and Fast Visualization of Scientific Data
by: Rai, Prashant, et al.
Published: (2019)

TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compression
by: Li, Zhong-Zhi, et al.
Published: (2025)

Randomized Algorithms for Low-Rank Matrix and Tensor Decompositions
by: Pearce, Katherine J., et al.
Published: (2025)

Efficient Algorithms for Low Tubal Rank Tensor Approximation with Applications
by: Ahmadi-Asl, Salman, et al.
Published: (2024)

Mathematical Modeling of Boson-Fermion Stars in the Generalized Scalar-Tensor Theories of Gravity
by: Boyadjiev, T. L., et al.
Published: (1999)

Sample Efficient Learning of Factored Embeddings of Tensor Fields
by: Heo, Taemin, et al.
Published: (2022)

ALM-PINNs Algorithms for Solving Nonlinear PDEs and Parameter Inversion Problems
by: Tian, Yimeng, et al.
Published: (2024)

Randomized Block Low-Rank Matrix Compression by Tagging
by: Pearce, Katherine J., et al.
Published: (2025)

Sparsity-Guided Multi-Parameter Selection in $\ell_1$-Regularized Models via a Fixed-Point Proximity Approach
by: Liu, Qianru, et al.
Published: (2025)

Local Interpolation via Low-Rank Tensor Trains
by: Guzman, Siddhartha E., et al.
Published: (2026)

Low-rank Tree Tensor Network Operators for Long-Range Pairwise Interactions
by: Ceruti, Gianluca, et al.
Published: (2024)

Multivariate Rational Approximation via Low-Rank Tensors and the p-AAA Algorithm
by: Balicki, Linus, et al.
Published: (2025)

Infrared Small Target Detection via tensor $L_{2,1}$ norm minimization and ASSTV regularization: A Novel Tensor Recovery Approach
by: Zhao, Jiqian, et al.
Published: (2024)

Provable Low-Rank Tensor-Train Approximations in the Inverse of Large-Scale Structured Matrices
by: Xiao, Chuanfu, et al.
Published: (2025)

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
by: Liu, Zechun, et al.
Published: (2024)

A DEIM Tucker Tensor Cross Algorithm and its Application to Dynamical Low-Rank Approximation
by: Ghahremani, Behzad, et al.
Published: (2024)

HOQRI: Higher-order QR Iteration for Low Multilinear Rank Approximation of Large and Sparse Tensors
by: Sun, Yuchen, et al.
Published: (2025)

Precision-induced Adaptive Randomized Low-Rank Approximation for SVD and Matrix Inversion
by: Xu, Weiwei, et al.
Published: (2026)

Pass-efficient Randomized Algorithms for Low-rank Approximation of Quaternion Matrices
by: Ahmadi-Asl, Salman, et al.
Published: (2025)

Sub-Sequential Physics-Informed Learning with State Space Model
by: Xu, Chenhui, et al.
Published: (2025)

Low-rankness and Smoothness Meet Subspace: A Unified Tensor Regularization for Hyperspectral Image Super-resolution
by: Zhang, Jun, et al.
Published: (2025)

Cross Interpolation for Solving High-Dimensional Dynamical Systems on Low-Rank Tucker and Tensor Train Manifolds
by: Ghahremani, Behzad, et al.
Published: (2024)

Quasi-random splitting method for accurate and efficient multiphysics simulation
by: Li, Lei, et al.
Published: (2026)

Tensor Train Decomposition for Adversarial Attacks on Computer Vision Models
by: Chertkov, Andrei, et al.
Published: (2023)

Nonnegative Biquadratic Tensors
by: Cui, Chunfeng, et al.
Published: (2025)

Accelerated Tensor Completion via Trace-Regularized Fully-Connected Tensor Network
by: Xie, Wenchao, et al.
Published: (2025)

Guaranteed Sampling Flexibility for Low-tubal-rank Tensor Completion
by: Su, Bowen, et al.
Published: (2024)

Numerical Fuzz: A Type System for Rounding Error Analysis
by: Kellison, Ariel E., et al.
Published: (2024)

Opening the black box of language acquisition
by: Michaud, Jérôme, et al.
Published: (2024)

Step-based checkpointing with high-level algorithmic differentiation
by: Maddison, James R.
Published: (2023)

Active Libraries: Rethinking the roles of compilers and libraries
by: Veldhuizen, Todd L., et al.
Published: (1998)

KazByte: Adapting Qwen models to Kazakh via Byte-level Adapter
by: Akylzhanov, Rauan
Published: (2026)