Image de couverture de livre

VersatileFFN: Achieving Parameter Efficiency in LLMs via Adaptive Wide-and-Deep Reuse

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Nie, Ying, Han, Kai, Li, Hongguang, Zhou, Hang, Guo, Tianyu, Wu, Enhua, Chen, Xinghao, Wang, Yunhe
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

ParameterNet: Parameters Are All You Need
par: Han, Kai, et autres
Publié: (2023)

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs
par: Han, Kai, et autres
Publié: (2024)

Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
par: Rang, Miao, et autres
Publié: (2025)

An Empirical Study of World Model Quantization
par: Fu, Zhongqian, et autres
Publié: (2026)

A robust audio deepfake detection system via multi-view feature
par: Yang, Yujie, et autres
Publié: (2024)

ROOT: Robust Orthogonalized Optimizer for Neural Network Training
par: He, Wei, et autres
Publié: (2025)

Near-Policy: Accelerating On-Policy Distillation via Asynchronous Generation and Selective Packing
par: Rang, Miao, et autres
Publié: (2026)

Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
par: Wang, Chengcheng, et autres
Publié: (2023)

RevFFN: Memory-Efficient Full-Parameter Fine-Tuning of Mixture-of-Experts LLMs with Reversible Blocks
par: Liu, Ningyuan, et autres
Publié: (2025)

Nexus: Higher-Order Attention Mechanisms in Transformers
par: Chen, Hanting, et autres
Publié: (2025)

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking
par: Li, Wenshuo, et autres
Publié: (2024)

SlimLLM: Accurate Structured Pruning for Large Language Models
par: Guo, Jialong, et autres
Publié: (2025)

SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization
par: Guo, Jialong, et autres
Publié: (2024)

Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads
par: Song, Chendong, et autres
Publié: (2026)

FashionLens: Toward Versatile Fashion Image Retrieval via Task-Adaptive Learning
par: Wen, Haokun, et autres
Publié: (2026)

From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
par: Tian, Yuchuan, et autres
Publié: (2025)

MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles
par: Han, Jing, et autres
Publié: (2025)

Spark Transformer: Reactivating Sparsity in FFN and Attention
par: You, Chong, et autres
Publié: (2025)

UFineBench: Towards Text-based Person Retrieval with Ultra-fine Granularity
par: Zuo, Jialong, et autres
Publié: (2023)

Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs
par: Liu, Hongliang, et autres
Publié: (2026)

ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters
par: Hao, Zhiwei, et autres
Publié: (2025)

EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse
par: Guo, Tianyu, et autres
Publié: (2025)

How to Achieve the Intended Aim of Deep Clustering Now, without Deep Learning
par: Ting, Kai Ming, et autres
Publié: (2026)

Mask Is What DLLM Needs: A Masked Data Training Paradigm for Diffusion LLMs
par: Ma, Linrui, et autres
Publié: (2026)

On the Reuse Bias in Off-Policy Reinforcement Learning
par: Ying, Chengyang, et autres
Publié: (2022)

LLM Data Selection and Utilization via Dynamic Bi-level Optimization
par: Yu, Yang, et autres
Publié: (2025)

Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
par: li, Fei, et autres
Publié: (2026)

EnvSSLAM-FFN: Lightweight Layer-Fused System for ESDD 2026 Challenge
par: Guo, Xiaoxuan, et autres
Publié: (2025)

Adaptive Reuse
par: Freschi, Elisa, et autres
Publié: (2017)

Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse
par: Zhang, Yuheng, et autres
Publié: (2025)

DECO: Unleashing the Potential of ConvNets for Query-based Detection and Segmentation
par: Chen, Xinghao, et autres
Publié: (2023)

Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
par: Pei, Zehua, et autres
Publié: (2025)

Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
par: Liu, Guowei, et autres
Publié: (2026)

RankAdaptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned LLMs via Performance Model
par: Zhou, Changhai, et autres
Publié: (2024)

vmintf/WsFFN: WsFFN Pre Alpha v0.0.2a-e Experimental Version Release
par: 민성 Skystarry
Publié: (2025)

FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
par: Jaiswal, Ajay, et autres
Publié: (2024)

Exploring Bonding in Bioinorganic Chemistry: Ab Initio Valence Bond Calculations of Iron(II)‐CO and Iron(IV)‐Oxo Units in Cytochrome P450 under Oriented External Electric Fields
par: Enhua Zhang, et autres
Publié: (2025)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
par: He, Wei, et autres
Publié: (2024)

GhostRNN: Reducing State Redundancy in RNN with Cheap Operations
par: Zhou, Hang, et autres
Publié: (2024)

UMoE: Unifying Attention and FFN with Shared Experts
par: Yang, Yuanhang, et autres
Publié: (2025)