Cover Image

Knowledge Offloading: Decomposing LLMs into Sparse Backbones and Memory Modules

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Galliamov, Karim, Choenni, Rochelle, Titov, Ivan
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Enhancing RLHF with Human Gaze Modeling
by: Galliamov, Karim, et al.
Published: (2025)

M-Wanda: Improving One-Shot Pruning for Multilingual LLMs
by: Choenni, Rochelle, et al.
Published: (2025)

Finding Culture-Sensitive Neurons in Vision-Language Models
by: Zhao, Xiutian, et al.
Published: (2025)

Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuning
by: Galliamov, Karim, et al.
Published: (2024)

[Re] FairDICE: A Fair Tradeoff in Multi-objective Offline RL
by: Adema, Peter, et al.
Published: (2026)

Concepts' Information Bottleneck Models
by: Galliamov, Karim, et al.
Published: (2026)

Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning
by: Choenni, Rochelle, et al.
Published: (2024)

Decomposing The Dark Matter of Sparse Autoencoders
by: Engels, Joshua, et al.
Published: (2024)

Unravelling the (In)compatibility of Statistical-Parity and Equalized-Odds
by: Bargh, Mortaza S., et al.
Published: (2026)

Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
by: Bazinska, Julia, et al.
Published: (2025)

SDQ: Sparse Decomposed Quantization for LLM Inference
by: Jeong, Geonhwa, et al.
Published: (2024)

Backbone-Equated Diffusion OOD via Sparse Internal Snapshots
by: Rouzoumka, Yadang Alexis, et al.
Published: (2026)

What's New in My Data? Novelty Exploration via Contrastive Generation
by: Isonuma, Masaru, et al.
Published: (2024)

Targeted Visualization of the Backbone of Encoder LLMs
by: Roberts, Isaac, et al.
Published: (2024)

Language Agents Meet Causality -- Bridging LLMs and Causal World Models
by: Gkountouras, John, et al.
Published: (2024)

Efficient In-Memory Acceleration of Sparse Block Diagonal LLMs
by: de Lima, João Paulo Cardoso, et al.
Published: (2025)

Metaphor Understanding Challenge Dataset for LLMs
by: Tong, Xiaoyu, et al.
Published: (2024)

Autoencoding Conditional Neural Processes for Representation Learning
by: Prokhorov, Victor, et al.
Published: (2023)

Mitigating Copy Bias in In-Context Learning through Neuron Pruning
by: Ali, Ameen, et al.
Published: (2024)

Formal Theorem Proving by Rewarding LLMs to Decompose Proofs Hierarchically
by: Dong, Kefan, et al.
Published: (2024)

DecoHD: Decomposed Hyperdimensional Classification under Extreme Memory Budgets
by: Yun, Sanggeon, et al.
Published: (2025)

Multi-Turn Reasoning LLMs for Task Offloading in Mobile Edge Computing
by: Yang, Ning, et al.
Published: (2026)

When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
by: Sun, Libo, et al.
Published: (2026)

HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading
by: Luo, Cheng, et al.
Published: (2025)

Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models
by: Kyriakou, Athina, et al.
Published: (2026)

Joint Localization and Activation Editing for Low-Resource Fine-Tuning
by: Lai, Wen, et al.
Published: (2025)

Towards true discovery of the differential equations
by: Hvatov, Alexander, et al.
Published: (2023)

Hierarchical Decomposed Dual-domain Deep Learning for Sparse-View CT Reconstruction
by: Han, Yoseob
Published: (2025)

Decomposing Uncertainty in Probabilistic Knowledge Graph Embeddings: Why Entity Variance Is Not Enough
by: Lee, Chorok
Published: (2025)

Ultra-Sparse Memory Network
by: Huang, Zihao, et al.
Published: (2024)

Backbone Augmented Training for Adaptations
by: Park, Jae Wan, et al.
Published: (2025)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
by: Liu, Yuxi, et al.
Published: (2025)

Cache & Distil: Optimising API Calls to Large Language Models
by: Ramírez, Guillem, et al.
Published: (2023)

Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models
by: Ali, Ameen, et al.
Published: (2025)

Drawing Pandas: A Benchmark for LLMs in Generating Plotting Code
by: Galimzyanov, Timur, et al.
Published: (2024)

Decomposing Behavioral Phase Transitions in LLMs: Order Parameters for Emergent Misalignment
by: Arnold, Julian, et al.
Published: (2025)

The Echoes of Multilinguality: Tracing Cultural Value Shifts during LM Fine-tuning
by: Choenni, Rochelle, et al.
Published: (2024)

How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning
by: Choenni, Rochelle, et al.
Published: (2023)

Decomposed Trust: Privacy, Adversarial Robustness, Ethics, and Fairness in Low-Rank LLMs
by: Asante, Daniel Agyei, et al.
Published: (2025)

From Tokenizer Bias to Backbone Capability: A Controlled Study of LLMs for Time Series Forecasting
by: Zhang, Xinyu, et al.
Published: (2025)