Imagen de Portada

Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Anand, Suraj, Lepori, Michael A., Merullo, Jack, Pavlick, Ellie
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Circuit Component Reuse Across Tasks in Transformer Language Models
por: Merullo, Jack, et al.
Publicado: (2023)

Language Models Implement Simple Word2Vec-style Vector Arithmetic
por: Merullo, Jack, et al.
Publicado: (2023)

Transferring Linear Features Across Language Models With Model Stitching
por: Chen, Alan, et al.
Publicado: (2025)

Talking Heads: Understanding Inter-layer Communication in Transformer Language Models
por: Merullo, Jack, et al.
Publicado: (2024)

$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
por: Khandelwal, Apoorv, et al.
Publicado: (2024)

Does Training on Synthetic Data Make Models Less Robust?
por: Zhang, Lingze, et al.
Publicado: (2025)

How Do Vision-Language Models Process Conflicting Information Across Modalities?
por: Hua, Tianze, et al.
Publicado: (2025)

Uncovering Intermediate Variables in Transformers using Circuit Probing
por: Lepori, Michael A., et al.
Publicado: (2023)

Instilling Inductive Biases with Subnetworks
por: Zhang, Enyan, et al.
Publicado: (2023)

I Have No Mouth, and I Must Rhyme: Uncovering Internal Phonetic Representations in LLaMA 3.2
por: McLaughlin, Oliver, et al.
Publicado: (2025)

From Memorization to Reasoning in the Spectrum of Loss Curvature
por: Merullo, Jack, et al.
Publicado: (2025)

Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities
por: Jobanputra, Mayank, et al.
Publicado: (2025)

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
por: Bigelow, Eric, et al.
Publicado: (2026)

Are LLMs Models of Distributional Semantics? A Case Study on Quantifiers
por: Enyan, Zhang, et al.
Publicado: (2024)

Are PPO-ed Language Models Hackable?
por: Anand, Suraj, et al.
Publicado: (2024)

Controlling Summarization Length Through EOS Token Weighting
por: Belligoli, Zeno, et al.
Publicado: (2025)

How Much Can We Forget about Data Contamination?
por: Bordt, Sebastian, et al.
Publicado: (2024)

Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMs
por: Hegazy, Amr, et al.
Publicado: (2025)

Variance Control via Weight Rescaling in LLM Pre-training
por: Owen, Louis, et al.
Publicado: (2025)

Learning to Forget: Continual Learning with Adaptive Weight Decay
por: Ramesh, Aditya A., et al.
Publicado: (2026)

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models
por: Lee, Chungpa, et al.
Publicado: (2026)

Learning Word Embedding with Better Distance Weighting and Window Size Scheduling
por: Yang, Chaohao, et al.
Publicado: (2024)

Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
por: Lu, Yining, et al.
Publicado: (2025)

ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
por: Zhang, Bonan, et al.
Publicado: (2025)

Steering Language Models with Weight Arithmetic
por: Fierro, Constanza, et al.
Publicado: (2025)

Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
por: Lewis, Martha, et al.
Publicado: (2022)

The dynamic interplay between in-context and in-weight learning in humans and neural networks
por: Russin, Jacob, et al.
Publicado: (2024)

SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
por: Zheng, Binbin, et al.
Publicado: (2026)

Learning to Interpret Weight Differences in Language Models
por: Goel, Avichal, et al.
Publicado: (2025)

Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
por: Abdi, Immanuel, et al.
Publicado: (2026)

Measuring Catastrophic Forgetting in Cross-Lingual Transfer Paradigms: Exploring Tuning Strategies
por: Koloski, Boshko, et al.
Publicado: (2023)

Shared Lexical Task Representations Explain Behavioral Variability In LLMs
por: Yang, Zhuonan, et al.
Publicado: (2026)

Stability-Weighted Decoding for Diffusion Language Models
por: Wu, Yue, et al.
Publicado: (2026)

SWSC: Shared Weight for Similar Channel in LLM
por: Zeng, Binrui, et al.
Publicado: (2025)

Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning
por: Wang, Xiaolei, et al.
Publicado: (2024)

How Do Language Models Compose Functions?
por: Khandelwal, Apoorv, et al.
Publicado: (2025)

Can LLMs subtract numbers?
por: Jobanputra, Mayank, et al.
Publicado: (2025)

Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
por: Lepori, Michael A., et al.
Publicado: (2025)

Weight-of-Thought Reasoning: Exploring Neural Network Weights for Enhanced LLM Reasoning
por: Punjwani, Saif, et al.
Publicado: (2025)

Model Assembly Learning with Heterogeneous Layer Weight Merging
por: Zhang, Yi-Kai, et al.
Publicado: (2025)