Imagen de Portada

On the Power of Convolution Augmented Transformer

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Mingchen, Zhang, Xuechen, Huang, Yixiao, Oymak, Samet
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Computation and Language Neural and Evolutionary Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

BrainTransformers: SNN-LLM
por: Tang, Zhengzheng, et al.
Publicado: (2024)

SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
por: Csordás, Róbert, et al.
Publicado: (2023)

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
por: Wang, Hanrui, et al.
Publicado: (2020)

A Hormone-inspired Emotion Layer for Transformer language models (HELT)
por: Reda, Eslam, et al.
Publicado: (2026)

Sorbet: A Neuromorphic Hardware-Compatible Transformer-Based Spiking Language Model
por: Tang, Kaiwen, et al.
Publicado: (2024)

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking
por: Xing, Xingrun, et al.
Publicado: (2024)

SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
por: Shen, Shuaijie, et al.
Publicado: (2024)

SpikeLM: Towards General Spike-Driven Language Modeling via Elastic Bi-Spiking Mechanisms
por: Xing, Xingrun, et al.
Publicado: (2024)

Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models
por: Majumdar, Somshubra, et al.
Publicado: (2024)

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
por: Zhu, Rui-Jie, et al.
Publicado: (2023)

Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models
por: Dong, Peijie, et al.
Publicado: (2024)

GridPE: Unifying Positional Encoding in Transformers with a Grid Cell-Inspired Framework
por: Li, Boyang, et al.
Publicado: (2024)

EvoX: Meta-Evolution for Automated Discovery
por: Liu, Shu, et al.
Publicado: (2026)

GLU Attention Improve Transformer
por: Wang, Zehao
Publicado: (2025)

Hysteresis Activation Function for Efficient Inference
por: Kimhi, Moshe, et al.
Publicado: (2024)

Large Language Models for Tuning Evolution Strategies
por: Kramer, Oliver
Publicado: (2024)

An enhanced Teaching-Learning-Based Optimization (TLBO) with Grey Wolf Optimizer (GWO) for text feature selection and clustering
por: Azarshab, Mahsa, et al.
Publicado: (2024)

Improving Sequence-to-Sequence Models for Abstractive Text Summarization Using Meta Heuristic Approaches
por: Saxena, Aditya, et al.
Publicado: (2024)

B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory
por: Zancato, Luca, et al.
Publicado: (2024)

Neural Information Organizing and Processing -- Neural Machines
por: Petrila, Iosif Iulian
Publicado: (2024)

EvolKV: Evolutionary KV Cache Compression for LLM Inference
por: Yu, Bohan, et al.
Publicado: (2025)

Decomposing Evolutionary Mixture-of-LoRA Architectures: The Routing Lever, the Lifecycle Penalty, and a Substrate-Conditional Boundary
por: Kumaresan, Ramchand
Publicado: (2026)

An In-depth Walkthrough on Evolution of Neural Machine Translation
por: Jagtap, Rohan, et al.
Publicado: (2020)

Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop
por: Briesch, Martin, et al.
Publicado: (2023)

Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers
por: Kadlčík, Marek, et al.
Publicado: (2025)

AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging
por: Chen, Kesheng, et al.
Publicado: (2025)

Intelligent Neural Networks: From Layered Architectures to Graph-Organized Intelligence
por: Salomon, Antoine
Publicado: (2025)

ComplicaCode: Enhancing Disease Complication Detection in Electronic Health Records through ICD Path Generation
por: Zhou, Xiaofan
Publicado: (2023)

Why Prompt Optimization Works, and Why It Sometimes Doesn't: A Causal-Inspired Edit-Level Analysis
por: Gong, Shuzhi, et al.
Publicado: (2026)

Improving Language Plasticity via Pretraining with Active Forgetting
por: Chen, Yihong, et al.
Publicado: (2023)

Minimal Convolutional RNNs Accelerate Spatiotemporal Learning
por: Horuz, Coşku Can, et al.
Publicado: (2025)

A Transformer-based Neural Architecture Search Method
por: Wang, Shang, et al.
Publicado: (2025)

NOBLE: Accelerating Transformers with Nonlinear Low-Rank Branches
por: Smith, Ethan
Publicado: (2026)

Generalized Dynamic Brain Functional Connectivity Based on Random Convolutions
por: Duan, Yongjie, et al.
Publicado: (2024)

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
por: Munkhdalai, Tsendsuren, et al.
Publicado: (2024)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
por: Lee, Philip Heejun
Publicado: (2025)

A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends
por: Younesi, Abolfazl, et al.
Publicado: (2024)

Predicting the Stay Length of Patients in Hospitals using Convolutional Gated Recurrent Deep Learning Model
por: Neshat, Mehdi, et al.
Publicado: (2024)

GT-SNT: A Linear-Time Transformer for Large-Scale Graphs via Spiking Node Tokenization
por: Zhang, Huizhe, et al.
Publicado: (2025)

Interpretable Solutions for Breast Cancer Diagnosis with Grammatical Evolution and Data Augmentation
por: Hasan, Yumnah, et al.
Publicado: (2024)