Image de couverture de livre

Constructing Efficient Fact-Storing MLPs for Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dugan, Owen, Garcia, Roberto, Junkins, Ronny, Liu, Jerry, Zinsley, Dylan, Eyuboglu, Sabri, Rudra, Atri, Ré, Chris
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Cartridges: Lightweight and general-purpose long context representations via self-study
par: Eyuboglu, Sabri, et autres
Publié: (2025)

Simple linear attention language models balance the recall-throughput tradeoff
par: Arora, Simran, et autres
Publié: (2024)

BWLer: Barycentric Weight Layer Elucidates a Precision-Conditioning Tradeoff for PINNs
par: Liu, Jerry, et autres
Publié: (2025)

Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters
par: Garcia, Roberto, et autres
Publié: (2025)

Towards Learning High-Precision Least Squares Algorithms with Sequence Models
par: Liu, Jerry, et autres
Publié: (2025)

Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models
par: Narayan, Avanika, et autres
Publié: (2025)

Just read twice: closing the recall gap for recurrent language models
par: Arora, Simran, et autres
Publié: (2024)

Weight-based Decomposition: A Case for Bilinear MLPs
par: Pearce, Michael T., et autres
Publié: (2024)

Mimetic Initialization of MLPs
par: Trockman, Asher, et autres
Publié: (2026)

Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations
par: Xu, Alec S., et autres
Publié: (2026)

Channel-Wise MLPs Improve the Generalization of Recurrent Convolutional Networks
par: Breslow, Nathan
Publié: (2025)

On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Dimensional Criticality at Grokking Across MLPs and Transformers
par: Wang, Ping
Publié: (2026)

Training MLPs on Graphs without Supervision
par: Wang, Zehong, et autres
Publié: (2024)

From Latent Space to Training Data: Explainable Specialization in Minimal MLPs
par: Alba, Enrique, et autres
Publié: (2026)

Heuristic Methods are Good Teachers to Distill MLPs for Graph Link Prediction
par: Qin, Zongyue, et autres
Publié: (2025)

Diffusion-Assisted Distillation for Self-Supervised Graph Representation Learning with MLPs
par: Ahn, Seong Jin, et autres
Publié: (2025)

Modular addition without black-boxes: Compressing explanations of MLPs that compute numerical integration
par: Yip, Chun Hei, et autres
Publié: (2024)

Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrasting
par: Wu, Lirong, et autres
Publié: (2024)

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
par: Wu, Taiqiang, et autres
Publié: (2023)

Outlier-Efficient Hopfield Layers for Large Transformer-Based Models
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

SimMLP: Training MLPs on Graphs without Supervision
par: Wang, Zehong, et autres
Publié: (2024)

MDMLP-EIA: Multi-domain Dynamic MLPs with Energy Invariant Attention for Time Series Forecasting
par: Zhang, Hu, et autres
Publié: (2025)

MLPMoE: Zero-Shot Architectural Metamorphosis of Dense LLM MLPs into Static Mixture-of-Experts
par: Novikov, Ivan
Publié: (2025)

In-Context Algorithm Emulation in Fixed-Weight Transformers
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2025)

VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs
par: Yang, Ling, et autres
Publié: (2023)

Secure and Storage-Efficient Deep Learning Models for Edge AI Using Automatic Weight Generation
par: Rahaman, Habibur, et autres
Publié: (2025)

Transformer Approximations from ReLUs
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2026)

Nutrition Facts, Drug Facts, and Model Facts: Putting AI Ethics into Practice in Gun Violence Research
par: Zhu, Jessica, et autres
Publié: (2024)

LLM-FS-Agent: A Deliberative Role-based Large Language Model Architecture for Transparent Feature Selection
par: Bal-Ghaoui, Mohamed, et autres
Publié: (2025)

Looped ReLU MLPs May Be All You Need as Practical Programmable Computers
par: Liang, Yingyu, et autres
Publié: (2024)

Efficient line search for optimizing Area Under the ROC Curve in gradient descent
par: Fowler, Jadon, et autres
Publié: (2024)

Runtime Detection of Adversarial Attacks in AI Accelerators Using Performance Counters
par: Rahaman, Habibur, et autres
Publié: (2025)

FactLens: Benchmarking Fine-Grained Fact Verification
par: Mitra, Kushan, et autres
Publié: (2024)

OccamLLM: Fast and Exact Language Model Arithmetic in a Single Step
par: Dugan, Owen, et autres
Publié: (2024)

Symmetry-Compatible Principle for Optimizer Design: Embeddings, LM Heads, SwiGLU MLPs, and MoE Routers
par: Lau, Tim Tsz-Kit, et autres
Publié: (2026)

Learning to Construct Practical Agentic Systems
par: Kumar, Aditya, et autres
Publié: (2026)

FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verification
par: Yue, Ling, et autres
Publié: (2026)

Human-like Forgetting Curves in Deep Neural Networks
par: Kline, Dylan
Publié: (2025)

Module-Aware Parameter-Efficient Machine Unlearning on Transformers
par: Bao, Wenjie, et autres
Publié: (2025)