Cover Image

MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Choi, Kanghyun, Lee, Hye Yoon, Kwon, Dain, Park, SunJong, Kim, Kyuyeun, Park, Noseong, Choi, Jonghyun, Lee, Jinho
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic
by: Choi, Kanghyun, et al.
Published: (2025)

DataFreeShield: Defending Adversarial Attacks without Training Data
by: Lee, Hyeyoon, et al.
Published: (2024)

Activation Quantization of Vision Encoders Needs Prefixing Registers
by: Kim, Seunghyeon, et al.
Published: (2025)

Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization
by: Son, Seungwoo, et al.
Published: (2024)

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain
by: Wi, Hyowon, et al.
Published: (2025)

DCText: Scheduled Attention Masking for Visual Text Generation via Divide-and-Conquer Strategy
by: Song, Jaewoo, et al.
Published: (2025)

TextGuider: Training-Free Guidance for Text Rendering via Attention Alignment
by: Baek, Kanghyun, et al.
Published: (2025)

PAC-FNO: Parallel-Structured All-Component Fourier Neural Operators for Recognizing Low-Quality Images
by: Jeon, Jinsung, et al.
Published: (2024)

Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment
by: Park, Jonghyun, et al.
Published: (2025)

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention
by: Shin, Yehjin, et al.
Published: (2023)

Joint Laser Inter-Satellite Link Matching and Traffic Flow Routing in LEO Mega-Constellations via Lagrangian Duality
by: Gu, Zhouyou, et al.
Published: (2026)

First Detection and Genomic Characterization of Feline Orthopneumovirus From Domestic Cats in South Korea
by: Jonghyun Park, et al.
Published: (2025)

Low-Complexity Semantic Packet Aggregation for Token Communication via Lookahead Search
by: Lee, Seunghun, et al.
Published: (2025)

Semantic Packet Aggregation for Token Communication via Genetic Beam Search
by: Lee, Seunghun, et al.
Published: (2025)

Semantic Packet Aggregation and Repeated Transmission for Text-to-Image Generation
by: Lee, Seunghun, et al.
Published: (2025)

Enhancing Reliability in LEO Satellite Networks via High-Speed Inter-Satellite Links
by: Choi, Jinho
Published: (2024)

PIORF: Physics-Informed Ollivier-Ricci Flow for Long-Range Interactions in Mesh Graph Neural Networks
by: Yu, Youn-Yeol, et al.
Published: (2025)

Graph Convolutions Enrich the Self-Attention in Transformers!
by: Choi, Jeongwhan, et al.
Published: (2023)

QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs
by: Noh, Kanghyun, et al.
Published: (2026)

Impact of Regularization on Calibration and Robustness: from the Representation Space Perspective
by: Park, Jonghyun, et al.
Published: (2024)

Exploring Multimodal Perception in Large Language Models Through Perceptual Strength Ratings
by: Lee, Jonghyun, et al.
Published: (2025)

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
by: Park, Jungwoo, et al.
Published: (2025)

DefectFill: Realistic Defect Generation with Inpainting Diffusion Model for Visual Inspection
by: Song, Jaewoo, et al.
Published: (2025)

SVD-AE: Simple Autoencoders for Collaborative Filtering
by: Hong, Seoyoung, et al.
Published: (2024)

SCONE: A Novel Stochastic Sampling to Generate Contrastive Views and Hard Negative Samples for Recommendation
by: Lee, Chaejeong, et al.
Published: (2024)

Are Graph Transformers Necessary? Efficient Long-Range Message Passing with Fractal Nodes in MPNNs
by: Choi, Jeongwhan, et al.
Published: (2025)

RadZero: Similarity-Based Cross-Attention for Explainable Vision-Language Alignment in Chest X-ray with Zero-Shot Multi-Task Capability
by: Park, Jonggwon, et al.
Published: (2025)

Enhancing Generalization in Data-free Quantization via Mixup-class Prompting
by: Park, Jiwoong, et al.
Published: (2025)

How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts
by: Park, Sumin, et al.
Published: (2025)

Blockage-Aware UAV-Assisted Wireless Data Harvesting With Building Avoidance
by: Park, Gitae, et al.
Published: (2025)

On Correcting Errors in Existing Mathematical Approaches for UAV Trajectory Design Considering No-Fly-Zones
by: Heo, Kanghyun, et al.
Published: (2023)

TV-Rec: Time-Variant Convolutional Filter for Sequential Recommendation
by: Shin, Yehjin, et al.
Published: (2025)

Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors
by: Choi, Jeongwhan, et al.
Published: (2026)

Towards Unified and Adaptive Cross-Domain Collaborative Filtering via Graph Signal Processing
by: Lee, Jeongeun, et al.
Published: (2024)

RDGCL: Reaction-Diffusion Graph Contrastive Learning for Recommendation
by: Choi, Jeongwhan, et al.
Published: (2023)

Multiparametric myocardial mapping using cardiac magnetic resonance imaging in healthy dogs: Reproducibility, repeatability, and differences across slices, segments, and sequences
by: Dain Yun, et al.
Published: (2024)

Attention-based Iterative Decomposition for Tensor Product Representation
by: Park, Taewon, et al.
Published: (2024)

Joint Design of Power Control and Access Point Scheduling for Uplink Cell-Free Massive MIMO Networks
by: Yeom, Hyeonsik, et al.
Published: (2022)

Possibility for Proactive Anomaly Detection
by: Jeon, Jinsung, et al.
Published: (2025)

Anomaly Detection-Based UE-Centric Inter-Cell Interference Suppression
by: Park, Kwonyeol, et al.
Published: (2025)