Cover Image

MemoryFormer: Minimize Transformer Computation by Removing Fully-Connected Layers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ding, Ning, Tang, Yehui, Qin, Haochen, Zhou, Zhenli, Xu, Chao, Li, Lin, Han, Kai, Liao, Heng, Wang, Yunhe
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning
by: Jie, Shibo, et al.
Published: (2024)

GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task
by: Ding, Ning, et al.
Published: (2023)

Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
by: Ding, Ning, et al.
Published: (2025)

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
by: Bi, Zhenni, et al.
Published: (2024)

Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
by: Zhou, Hang, et al.
Published: (2024)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
by: He, Wei, et al.
Published: (2024)

A Survey on Transformer Compression
by: Tang, Yehui, et al.
Published: (2024)

Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning
by: Jie, Shibo, et al.
Published: (2024)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
by: Guo, Jialong, et al.
Published: (2024)

GhostNetV3: Exploring the Training Strategies for Compact Models
by: Liu, Zhenhua, et al.
Published: (2024)

EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models
by: Ni, Yunsheng, et al.
Published: (2024)

Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
by: Liu, Fangcheng, et al.
Published: (2024)

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs
by: Han, Kai, et al.
Published: (2024)

Eve: Efficient Multimodal Vision Language Models with Elastic Visual Experts
by: Rang, Miao, et al.
Published: (2025)

LLM Data Selection and Utilization via Dynamic Bi-level Optimization
by: Yu, Yang, et al.
Published: (2025)

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking
by: Li, Wenshuo, et al.
Published: (2024)

EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
by: Fu, Zhongqian, et al.
Published: (2025)

ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters
by: Hao, Zhiwei, et al.
Published: (2025)

SlimLLM: Accurate Structured Pruning for Large Language Models
by: Guo, Jialong, et al.
Published: (2025)

Mixture of Lookup Experts
by: Jie, Shibo, et al.
Published: (2025)

SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution
by: Wang, Chengcheng, et al.
Published: (2024)

A robust audio deepfake detection system via multi-view feature
by: Yang, Yujie, et al.
Published: (2024)

No Time to Waste: Squeeze Time into Channel for Mobile Video Understanding
by: Zhai, Yingjie, et al.
Published: (2024)

Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation
by: Ni, Zhenliang, et al.
Published: (2024)

MixFormerV2: Efficient Fully Transformer Tracking
by: Cui, Yutao, et al.
Published: (2023)

Rethinking Transformer Connectivity: TLinFormer, A Path to Exact, Full Context-Aware Linear Attention
by: Tang, Zhongpan
Published: (2025)

MeKi: Memory-based Expert Knowledge Injection for Efficient LLM Scaling
by: Ding, Ning, et al.
Published: (2026)

Rethinking Cross-Layer Information Routing in Diffusion Transformers
by: Xu, Chao, et al.
Published: (2026)

Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation
by: Tian, Yuchuan, et al.
Published: (2024)

Zirconium‐Pentacarboxylate Frameworks with Coexisted 4‐ and 9‐Connected Zr6 Clusters for Detection and Removal of Tetracycline Hydrochloride
by: Tianyou Peng, et al.
Published: (2024)

Deep Photonic Reservoir Computer Based on Frequency Multiplexing with Fully Analog Connection Between Layers
by: Lupo, Alessandro, et al.
Published: (2023)

AesFormer: Transform Everyday Photos into Beautiful Memories
by: Du, Tianxiang, et al.
Published: (2026)

PocketLLM: Ultimate Compression of Large Language Models via Meta Networks
by: Tian, Ye, et al.
Published: (2025)

Saliency-driven Dynamic Token Pruning for Large Language Models
by: Tao, Yao, et al.
Published: (2025)

StateLinFormer: Stateful Training Enhancing Long-term Memory in Navigation
by: Chen, Zhiyuan, et al.
Published: (2026)

LoReTrack: Efficient and Accurate Low-Resolution Transformer Tracking
by: Dong, Shaohua, et al.
Published: (2024)

KuaiFormer: Transformer-Based Retrieval at Kuaishou
by: Liu, Chi, et al.
Published: (2024)

Fully Binarized Graph Convolutional Network Accelerator Based on In‐Memory Computing with Resistive Random‐Access Memory
by: Woyu Zhang, et al.
Published: (2024)

From TLinFormer to TConstFormer: The Leap to Constant-Time Transformer Attention: Achieving O(1) Computation and O(1) KV Cache during Autoregressive Inference
by: Tang, Zhongpan
Published: (2025)

WaveletFormerNet: A Transformer-based Wavelet Network for Real-world Non-homogeneous and Dense Fog Removal
by: Zhang, Shengli, et al.
Published: (2024)