Cover Image

Incorporating Exponential Smoothing into MLP: A Simple but Effective Sequence Model

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chu, Jiqun, Lin, Zuoquan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

An Empirical Study on Context Length for Open-Domain Dialog Generation
by: Shen, Xinyi, et al.
Published: (2024)

HyperMLP: An Integrated Perspective for Sequence Modeling
by: Lu, Jiecheng, et al.
Published: (2026)

Toeplitz MLP Mixers are Low Complexity, Information-Rich Sequence Models
by: Badger, Benjamin L., et al.
Published: (2026)

Simple and Effective Masked Diffusion Language Models
by: Sahoo, Subham Sekhar, et al.
Published: (2024)

A Simple and Effective Pruning Approach for Large Language Models
by: Sun, Mingjie, et al.
Published: (2023)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
by: Zhu, Hourun, et al.
Published: (2025)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
by: Neo, Clement, et al.
Published: (2024)

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
by: Tian, Yuandong, et al.
Published: (2023)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
by: Kim, Gyeongman, et al.
Published: (2025)

Meta-DiffuB: A Contextualized Sequence-to-Sequence Text Diffusion Model with Meta-Exploration
by: Chuang, Yun-Yen, et al.
Published: (2024)

Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning
by: Song, Haobo, et al.
Published: (2024)

SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling
by: Kim, Dahyun, et al.
Published: (2023)

Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification
by: Kruse, Maya, et al.
Published: (2025)

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
by: Xiao, Guangxuan, et al.
Published: (2022)

Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn Search Agents
by: Wang, Guoqing, et al.
Published: (2025)

Sequence-to-Sequence Spanish Pre-trained Language Models
by: Araujo, Vladimir, et al.
Published: (2023)

An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling
by: Tang, Xuemei, et al.
Published: (2024)

CultureLLM: Incorporating Cultural Differences into Large Language Models
by: Li, Cheng, et al.
Published: (2024)

dLLM: Simple Diffusion Language Modeling
by: Zhou, Zhanhui, et al.
Published: (2026)

It's Not That Simple. An Analysis of Simple Test-Time Scaling
by: Wu, Guojun
Published: (2025)

Simple Policy Gradients for Reasoning with Diffusion Language Models
by: Zhan, Anthony
Published: (2025)

From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP
by: Ganesan, Adithya V, et al.
Published: (2026)

Native Hybrid Attention for Efficient Sequence Modeling
by: Du, Jusen, et al.
Published: (2025)

A Simple but Effective Closed-form Solution for Extreme Multi-label Learning
by: Onishi, Kazuma, et al.
Published: (2025)

Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models
by: Zhao, Siyan, et al.
Published: (2024)

Group Sequence Policy Optimization
by: Zheng, Chujie, et al.
Published: (2025)

How Well Can a Long Sequence Model Model Long Sequences? Comparing Architechtural Inductive Biases on Long-Context Abilities
by: Huang, Jerry
Published: (2024)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
by: Ibrahim, Adam, et al.
Published: (2024)

SiLQ: Simple Large Language Model Quantization-Aware Training
by: Esser, Steven K., et al.
Published: (2025)

Incorporating Attribution Importance for Improving Faithfulness Metrics
by: Zhao, Zhixue, et al.
Published: (2023)

Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures
by: Muchane, Mark, et al.
Published: (2025)

MoM: Linear Sequence Modeling with Mixture-of-Memories
by: Du, Jusen, et al.
Published: (2025)

Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling
by: Egli, Eric, et al.
Published: (2025)

Attention Smoothing Is All You Need For Unlearning
by: Zade, Saleh Zare, et al.
Published: (2026)

Linear Model Merging Unlocks Simple and Scalable Multimodal Data Mixture Optimization
by: Berasi, Davide, et al.
Published: (2026)

RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques
by: Tang, Zhengyang, et al.
Published: (2025)

FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version
by: Nguyen-Cong, Dat, et al.
Published: (2026)

FutureFill: Fast Generation from Convolutional Sequence Models
by: Agarwal, Naman, et al.
Published: (2024)

Sequences of Logits Reveal the Low Rank Structure of Language Models
by: Golowich, Noah, et al.
Published: (2025)

GLiClass: Generalist Lightweight Model for Sequence Classification Tasks
by: Stepanov, Ihor, et al.
Published: (2025)