Cover Image

Wave-PDE Nets: Trainable Wave-Equation Layers as an Alternative to Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Vejendla, Harshil
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

H1B-KV: Hybrid One-Bit Caches for Memory-Efficient Large Language Model Inference
by: Vejendla, Harshil
Published: (2025)

RewriteNets: End-to-End Trainable String-Rewriting for Generative Sequence Modeling
by: Vejendla, Harshil
Published: (2026)

SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling
by: Vejendla, Harshil
Published: (2025)

LATTA: Langevin-Anchored Test-Time Adaptation for Enhanced Robustness and Stability
by: Vejendla, Harshil
Published: (2025)

Learning to Predict Chaos: Curriculum-Driven Training for Robust Forecasting of Chaotic Dynamics
by: Vejendla, Harshil
Published: (2025)

Teaching by Failure: Counter-Example-Driven Curricula for Transformer Self-Improvement
by: Vejendla, Harshil
Published: (2025)

Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases
by: Vejendla, Harshil
Published: (2025)

Trainable Dynamic Mask Sparse Attention
by: Shi, Jingze, et al.
Published: (2025)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
by: Dong, Yihe, et al.
Published: (2025)

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
by: Bozic, Vukasin, et al.
Published: (2023)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
by: Yuan, Jingyang, et al.
Published: (2025)

Trainable Transformer in Transformer
by: Panigrahi, Abhishek, et al.
Published: (2023)

Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Models
by: Zhao, Yicong, et al.
Published: (2025)

LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs
by: Souibgui, Mohamed Ali, et al.
Published: (2026)

Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers
by: Wong, Liang Ze
Published: (2025)

Challenges and Considerations in Annotating Legal Data: A Comprehensive Overview
by: Darji, Harshil, et al.
Published: (2024)

High-Layer Attention Pruning with Rescaling
by: Liu, Songtao, et al.
Published: (2025)

Multi-Layer Attention is the Amplifier of Demonstration Effectiveness
by: Wang, Dingzirui, et al.
Published: (2025)

Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-Attention
by: Kiruluta, Andrew, et al.
Published: (2025)

RAM-Net: Expressive Linear Attention with Selectively Addressable Memory
by: Xiao, Kaicheng, et al.
Published: (2026)

Projected Compression: Trainable Projection for Efficient Transformer Compression
by: Stefaniak, Maciej, et al.
Published: (2025)

CLAA: Cross-Layer Attention Aggregation for Accelerating LLM Prefill
by: McDanel, Bradley, et al.
Published: (2026)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
by: Musat, Tiberiu
Published: (2024)

Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
by: Fu, Zizhuo, et al.
Published: (2026)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
by: Brandon, William, et al.
Published: (2024)

Supervised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns
by: Zhao, Yang, et al.
Published: (2024)

Segmentation and Processing of German Court Decisions from Open Legal Data
by: Darji, Harshil, et al.
Published: (2026)

DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning
by: Zarch, Hossein Entezari, et al.
Published: (2025)

IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
by: Bai, Yushi, et al.
Published: (2026)

MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection
by: Lin, Bokai, et al.
Published: (2024)

Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression
by: Peng, Liangzu, et al.
Published: (2025)

BeamLLM: Vision-Empowered mmWave Beam Prediction with Large Language Models
by: Zheng, Can, et al.
Published: (2025)

Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features
by: Bombari, Simone, et al.
Published: (2024)

Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification
by: ElNokrashy, Muhammad, et al.
Published: (2022)

SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
by: Wang, Zihao, et al.
Published: (2024)

CodePDE: An Inference Framework for LLM-driven PDE Solver Generation
by: Li, Shanda, et al.
Published: (2025)

BitNet Distillation
by: Wu, Xun, et al.
Published: (2025)

Why Softmax Attention Outperforms Linear Attention
by: Deng, Yichuan, et al.
Published: (2023)

Semantic Wave Functions: Exploring Meaning in Large Language Models Through Quantum Formalism
by: Laine, Timo Aukusti
Published: (2025)

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
by: Sanyal, Sunny, et al.
Published: (2024)