Cover Image

ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	You, Haoran, Shi, Huihong, Guo, Yipin, Lin, Yingyan Celine
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks
by: You, Haoran, et al.
Published: (2022)

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
by: You, Haoran, et al.
Published: (2024)

ShiftAddAug: Augment Multiplication-Free Tiny Neural Network with Hybrid Computation
by: Guo, Yipin, et al.
Published: (2024)

When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
by: You, Haoran, et al.
Published: (2024)

Max-Affine Spline Insights Into Deep Network Pruning
by: You, Haoran, et al.
Published: (2021)

MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Accelerators
by: Wan, Cheng, et al.
Published: (2025)

ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design
by: You, Haoran, et al.
Published: (2022)

ShiftAddNet: A Hardware-Inspired Deep Network
by: You, Haoran, et al.
Published: (2020)

NASH: Neural Architecture and Accelerator Search for Multiplication-Reduced Hybrid Models
by: Xu, Yang, et al.
Published: (2024)

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
by: Wu, Xiaoyou, et al.
Published: (2026)

Where to Add PDE Diffusion in Transformers
by: Zhang, Yukun, et al.
Published: (2025)

Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning
by: Li, Haoran, et al.
Published: (2025)

MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation
by: Zhang, Yongan, et al.
Published: (2024)

Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers
by: You, Haoran, et al.
Published: (2024)

Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
by: Guo, Yongxin, et al.
Published: (2024)

Geometric and Dynamic Scaling in Deep Transformers
by: Su, Haoran, et al.
Published: (2026)

AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment
by: Fu, Yonggan, et al.
Published: (2024)

ViR: Towards Efficient Vision Retention Backbones
by: Hatamizadeh, Ali, et al.
Published: (2023)

M$^2$-ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantization
by: Liang, Yanbiao, et al.
Published: (2024)

Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer
by: Shi, Huihong, et al.
Published: (2024)

SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving
by: Guo, Yipin, et al.
Published: (2026)

LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
by: Shi, Dachuan, et al.
Published: (2025)

Expert Upcycling: Shifting the Compute-Efficient Frontier of Mixture-of-Experts
by: Dwivedi, Chaitanya, et al.
Published: (2026)

GPTQT: Quantize Large Language Models Twice to Push the Efficiency
by: Guo, Yipin, et al.
Published: (2024)

P$^2$-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
by: Shi, Huihong, et al.
Published: (2024)

EffGen: Enabling Small Language Models as Capable Autonomous Agents
by: Srivastava, Gaurav, et al.
Published: (2026)

An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViT
by: Shao, Haikuo, et al.
Published: (2024)

FasterViT: Fast Vision Transformers with Hierarchical Attention
by: Hatamizadeh, Ali, et al.
Published: (2023)

Dynamic Mixture of Experts Against Severe Distribution Shifts
by: Kim, Donghu
Published: (2025)

AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design
by: Liang, Yanbiao, et al.
Published: (2025)

Partial Domain Adaptation via Importance Sampling-based Shift Correction
by: Guo, Cheng-Jun, et al.
Published: (2025)

AdaptiveLoad: Towards Efficient Video Diffusion Transformer Training
by: Guo, Yucheng, et al.
Published: (2026)

SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules
by: Li, Suyi, et al.
Published: (2024)

Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques
by: He, Shwai, et al.
Published: (2024)

TPC-ViT: Token Propagation Controller for Efficient Vision Transformer
by: Zhu, Wentao
Published: (2024)

VariViT: A Vision Transformer for Variable Image Sizes
by: Varma, Aswathi, et al.
Published: (2026)

ScriptViT: Vision Transformer-Based Personalized Handwriting Generation
by: Acharya, Sajjan, et al.
Published: (2025)

Octic Vision Transformers: Quicker ViTs Through Equivariance
by: Nordström, David, et al.
Published: (2025)

Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers
by: Li, Zhengang, et al.
Published: (2024)

Neuro-inspired Ensemble-to-Ensemble Communication Primitives for Sparse and Efficient ANNs
by: Konstantaropoulos, Orestis, et al.
Published: (2025)