Image de couverture de livre

Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yun, Sungmin, Kyung, Kwanhee, Cho, Juhwan, Choi, Jaewan, Kim, Jongmin, Kim, Byeongho, Lee, Sukhan, Sohn, Kyomin, Ahn, Jung Ho
Format:	Preprint
Publié:	2024
Sujets:	Hardware Architecture Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency
par: Kyung, Kwanhee, et autres
Publié: (2025)

Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
par: Yun, Sungmin, et autres
Publié: (2025)

LP5X-PIM Sim: A High-Fidelity HW/SW Integrated Simulator for LPDDR5X-PIM
par: Cha, SangHoon, et autres
Publié: (2026)

Per-Row Activation Counting on Real Hardware: Demystifying Performance Overheads
par: Kim, Jumin, et autres
Publié: (2025)

CiFHER: A Chiplet-Based FHE Accelerator with a Resizable Structure
par: Kim, Sangpyo, et autres
Publié: (2023)

IVE: An Accelerator for Single-Server Private Information Retrieval Using Versatile Processing Elements
par: Kim, Sangpyo, et autres
Publié: (2025)

Theodosian: A Deep Dive into Memory-Hierarchy-Centric FHE Acceleration
par: Choi, Wonseok, et autres
Publié: (2025)

GPIR: Enabling Practical Private Information Retrieval with GPUs
par: Ji, Hyesung, et autres
Publié: (2026)

Sieve: Dynamic Expert-Aware PIM Acceleration for Evolving Mixture-of-Experts Models
par: Kim, Jungwoo, et autres
Publié: (2026)

MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models
par: Kim, Taehyun, et autres
Publié: (2024)

RoMe: Row Granularity Access Memory System for Large Language Models
par: Nam, Hwayong, et autres
Publié: (2025)

NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencing
par: Heo, Guseul, et autres
Publié: (2024)

AME-PIM: Can Memory be Your Next Tensor Accelerator?
par: Venieri, Emanuele, et autres
Publié: (2026)

SoK: Systematizing a Decade of Architectural RowHammer Defenses Through the Lens of Streaming Algorithms
par: Kim, Michael Jaemin, et autres
Publié: (2025)

Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
par: Choi, Dawon, et autres
Publié: (2026)

PVAC: A RowHammer Mitigation Architecture Exploiting Per-victim-row Counting
par: Kim, Jumin, et autres
Publié: (2026)

SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference
par: Choi, Yuseon, et autres
Publié: (2025)

Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference
par: Skliar, Andrii, et autres
Publié: (2024)

Implementing and Optimizing the Scaled Dot-Product Attention on Streaming Dataflow
par: Sohn, Gina, et autres
Publié: (2024)

NASiC: 3D NAND-based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference
par: Xu, Weikai, et autres
Publié: (2026)

Cosmos: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
par: Ko, Seoyoung, et autres
Publié: (2025)

Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
par: Bambhaniya, Abhimanyu, et autres
Publié: (2026)

DRAMScope: Uncovering DRAM Microarchitecture and Characteristics by Issuing Memory Commands
par: Nam, Hwayong, et autres
Publié: (2024)

Area-Efficient In-Memory Computing for Mixture-of-Experts via Multiplexing and Caching
par: Gao, Hanyuan, et autres
Publié: (2026)

FlexNeRFer: A Multi-Dataflow, Adaptive Sparsity-Aware Accelerator for On-Device NeRF Rendering
par: Noh, Seock-Hwan, et autres
Publié: (2025)

SAL-PIM: A Subarray-level Processing-in-Memory Architecture with LUT-based Linear Interpolation for Transformer-based Text Generation
par: Han, Wontak, et autres
Publié: (2024)

A Host-SSD Collaborative Write Accelerator for LSM-Tree-Based Key-Value Stores
par: Kim, KiHwan, et autres
Publié: (2024)

AERO: Adaptive Erase Operation for Improving Lifetime and Performance of Modern NAND Flash-Based SSDs
par: Cho, Sungjun, et autres
Publié: (2024)

OpenCXD: An Open Real-Device-Guided Hybrid Evaluation Framework for CXL-SSDs
par: Chung, Hyunsun, et autres
Publié: (2025)

SeVeDo: A Heterogeneous Transformer Accelerator for Low-Bit Inference via Hierarchical Group Quantization and SVD-Guided Mixed Precision
par: Choi, Yuseon, et autres
Publié: (2025)

Bandwidth-Effective DRAM Cache for GPUs with Storage-Class Memory
par: Hong, Jeongmin, et autres
Publié: (2024)

Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling
par: Ma, Songchen, et autres
Publié: (2026)

Securing DRAM at Scale: ARFM-Driven Row Hammer Defense with Unveiling the Threat of Short tRC Patterns
par: Joo, Nogeun, et autres
Publié: (2025)

A 0.5V, 6.2$μ$W, 0.059mm$^{2}$ Sinusoidal Current Generator IC with 0.088% THD for Bio-Impedance Sensing
par: Kim, Kwantae, et autres
Publié: (2024)

AnalogToBi: Device-Level Analog Circuit Topology Generation via Bipartite Graph and Grammar Guided Decoding
par: Kim, Seungmin, et autres
Publié: (2026)

LPU: A Latency-Optimized and Highly Scalable Processor for Large Language Model Inference
par: Moon, Seungjae, et autres
Publié: (2024)

CD-PIM: A High-Bandwidth and Compute-Efficient LPDDR5-Based PIM for Low-Batch LLM Acceleration on Edge-Device
par: Lin, Ye, et autres
Publié: (2026)

CXL Topology-Aware and Expander-Driven Prefetching: Unlocking SSD Performance
par: Oh, Dongsuk, et autres
Publié: (2025)

CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGA
par: Dong, Jiale, et autres
Publié: (2025)

Sudoku: Decomposing DRAM Address Mapping into Component Functions
par: Wi, Minbok, et autres
Publié: (2025)