Copertina

DAPA: Distribution Aware Piecewise Activation Functions for On-Device Transformer Inference and Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xiang, Maoyang, Wang, Bo
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On-Device Qwen2.5: Efficient LLM Inference with Model Compression and Hardware Acceleration
di: Xiang, Maoyang, et al.
Pubblicazione: (2025)

GoQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantization
di: Xiang, Maoyang, et al.
Pubblicazione: (2026)

Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks
di: Patty, William H
Pubblicazione: (2025)

End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
di: Tan, Qitao, et al.
Pubblicazione: (2025)

EdgeFlex-Transformer: Transformer Inference for Edge Devices
di: Mohammad, Shoaib, et al.
Pubblicazione: (2025)

On the Expressive Power of Transformers for Maxout Networks and Continuous Piecewise Linear Functions
di: Gu, Linyan, et al.
Pubblicazione: (2026)

Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
di: Haziza, Daniel, et al.
Pubblicazione: (2025)

Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices
di: Gräfe, Alexander, et al.
Pubblicazione: (2026)

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference
di: Zhang, Zhenyu, et al.
Pubblicazione: (2025)

Privacy-Aware Multi-Device Cooperative Edge Inference with Distributed Resource Bidding
di: Zhuang, Wenhao, et al.
Pubblicazione: (2024)

Don't Forget the Nonlinearity: Unlocking Activation Functions in Efficient Fine-Tuning
di: Yin, Bo, et al.
Pubblicazione: (2025)

AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks
di: Wei, Yi, et al.
Pubblicazione: (2026)

Representing Piecewise-Linear Functions by Functions with Minimal Arity
di: Koutschan, Christoph, et al.
Pubblicazione: (2024)

Non-Singularity of the Gradient Descent map for Neural Networks with Piecewise Analytic Activations
di: Crăciun, Alexandru, et al.
Pubblicazione: (2025)

Piecewise Normalizing Flows
di: Bevins, Harry, et al.
Pubblicazione: (2023)

Region Seeding via Pre-Activation Regularization: A Geometric View of Piecewise Affine Neural Networks
di: Wei, Yi, et al.
Pubblicazione: (2026)

ASTRA: Communication-Efficient Acceleration for Multi-Device Transformer Inference
di: Liu, Xiao, et al.
Pubblicazione: (2025)

Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
di: Qi, Xuan, et al.
Pubblicazione: (2026)

Piecewise Deterministic Markov Processes for Bayesian Inference of PDE Coefficients
di: Riccius, Leon, et al.
Pubblicazione: (2026)

Hysteresis Activation Function for Efficient Inference
di: Kimhi, Moshe, et al.
Pubblicazione: (2024)

Secure Transformer Inference Protocol
di: Yuan, Mu, et al.
Pubblicazione: (2023)

WiSparse: Boosting LLM Inference Efficiency with Weight-Aware Mixed Activation Sparsity
di: Chen, Lei, et al.
Pubblicazione: (2026)

Beat the long tail: Distribution-Aware Speculative Decoding for RL Training
di: Shao, Zelei, et al.
Pubblicazione: (2025)

Topology-Aware Activation Functions in Neural Networks
di: Snopov, Pavel, et al.
Pubblicazione: (2025)

Piecewise deterministic generative models
di: Bertazzi, Andrea, et al.
Pubblicazione: (2024)

Distributed Sign Momentum with Local Steps for Training Transformers
di: Yu, Shuhua, et al.
Pubblicazione: (2024)

Flow-Transformed Implicit Processes for Function-Space Variational Inference
di: Ortega, Luis A., et al.
Pubblicazione: (2026)

Digi-Q: Learning Q-Value Functions for Training Device-Control Agents
di: Bai, Hao, et al.
Pubblicazione: (2025)

Transformer-Based Approach for Automated Functional Group Replacement in Chemical Compounds
di: Pan, Bo, et al.
Pubblicazione: (2026)

TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly
di: Koike-Akino, Toshiaki, et al.
Pubblicazione: (2026)

Sparsity-Aware Communication for Distributed Graph Neural Network Training
di: Mukhodopadhyay, Ujjaini, et al.
Pubblicazione: (2025)

DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training
di: Liu, Renyuan, et al.
Pubblicazione: (2025)

ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
di: Meng, Han, et al.
Pubblicazione: (2026)

Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptation
di: Whittle, George, et al.
Pubblicazione: (2025)

Symmetry-Aware Transformer Training for Automated Planning
di: Fritzsche, Markus, et al.
Pubblicazione: (2025)

Decomposition Polyhedra of Piecewise Linear Functions
di: Brandenburg, Marie-Charlotte, et al.
Pubblicazione: (2024)

Kraken: Inherently Parallel Transformers For Efficient Multi-Device Inference
di: Prabhakar, Rohan Baskar, et al.
Pubblicazione: (2024)

Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
di: Cheng, Xiang, et al.
Pubblicazione: (2023)

PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
di: Andronic, Marta, et al.
Pubblicazione: (2023)

Joint Optimization of Piecewise Linear Ensembles
di: Raymond, Matt, et al.
Pubblicazione: (2024)