Cover Image

Mimetic Initialization Helps State Space Models Learn to Recall

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Trockman, Asher, Harutyunyan, Hrayr, Kolter, J. Zico, Kumar, Sanjiv, Bhojanapalli, Srinadh
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mimetic Initialization of MLPs
by: Trockman, Asher, et al.
Published: (2026)

Antidistillation Fingerprinting
by: Xu, Yixuan Even, et al.
Published: (2026)

In-context Learning in Presence of Spurious Correlations
by: Harutyunyan, Hrayr, et al.
Published: (2024)

FUSE-ing Language Models: Zero-Shot Adapter Discovery for Prompt Optimization Across Tokenizers
by: Williams, Joshua Nathaniel, et al.
Published: (2024)

Predicting the Performance of Black-box LLMs through Follow-up Queries
by: Sam, Dylan, et al.
Published: (2025)

Massive Activations in Large Language Models
by: Sun, Mingjie, et al.
Published: (2024)

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
by: Rawat, Ankit Singh, et al.
Published: (2024)

HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference
by: L, Yashas Samaga B, et al.
Published: (2024)

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
by: Cho, Hanseul, et al.
Published: (2024)

Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
by: Bae, Sangmin, et al.
Published: (2024)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
by: Goyal, Sachin, et al.
Published: (2024)

Forcing Diffuse Distributions out of Language Models
by: Zhang, Yiming, et al.
Published: (2024)

A Simple and Effective Pruning Approach for Large Language Models
by: Sun, Mingjie, et al.
Published: (2023)

Scalable In-context Ranking with Generative Models
by: Gupta, Nilesh, et al.
Published: (2025)

On student-teacher deviations in distillation: does it pay to disobey?
by: Nagarajan, Vaishnavh, et al.
Published: (2023)

Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learning
by: Xu, Yixuan Even, et al.
Published: (2025)

Looking beyond the next token
by: Thankaraj, Abitha, et al.
Published: (2025)

TOFU: A Task of Fictitious Unlearning for LLMs
by: Maini, Pratyush, et al.
Published: (2024)

Rethinking LLM Memorization through the Lens of Adversarial Compression
by: Schwarzschild, Avi, et al.
Published: (2024)

Base Models Look Human To AI Detectors
by: Xu, Yixuan Even, et al.
Published: (2026)

T-MARS: Improving Visual Representations by Circumventing Text Feature Learning
by: Maini, Pratyush, et al.
Published: (2023)

AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs
by: Akinwande, Victor, et al.
Published: (2024)

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
by: Bae, Sangmin, et al.
Published: (2025)

Antidistillation Sampling
by: Savani, Yash, et al.
Published: (2025)

Efficient Language Model Architectures for Differentially Private Federated Learning
by: Ro, Jae Hun, et al.
Published: (2024)

In-Context Credit Assignment via the Core
by: Harris, Keegan, et al.
Published: (2026)

Existing Large Language Model Unlearning Evaluations Are Inconclusive
by: Feng, Zhili, et al.
Published: (2025)

One-Step Diffusion Distillation via Deep Equilibrium Models
by: Geng, Zhengyang, et al.
Published: (2023)

Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models
by: Zheng, Junhao, et al.
Published: (2023)

Training a Generally Curious Agent
by: Tajwar, Fahim, et al.
Published: (2025)

Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count
by: Cho, Hanseul, et al.
Published: (2024)

Learning Semantic Structure through First-Order-Logic Translation
by: Chaturvedi, Akshay, et al.
Published: (2024)

Diffusing Differentiable Representations
by: Savani, Yash, et al.
Published: (2024)

Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
by: Sam, Dylan, et al.
Published: (2025)

Equilibrium Reasoners: Learning Attractors Enables Scalable Reasoning
by: Huang, Benhao, et al.
Published: (2026)

Landscape-Aware Growing: The Power of a Little LAG
by: Karp, Stefani, et al.
Published: (2024)

LLM In-Context Recall is Prompt Dependent
by: Machlab, Daniel, et al.
Published: (2024)

StateX: Enhancing RNN Recall via Post-training State Expansion
by: Shen, Xingyu, et al.
Published: (2025)

Functional Interpolation for Relative Positions Improves Long Context Transformers
by: Li, Shanda, et al.
Published: (2023)

Can Stories Help LLMs Reason? Curating Information Space Through Narrative
by: Javadi, Vahid Sadiri, et al.
Published: (2024)