Imagen de Portada

Transformers Learn Low Sensitivity Functions: Investigations and Implications

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Vasudeva, Bhavya, Fu, Deqing, Zhou, Tianyi, Kau, Elliott, Huang, Youqi, Sharan, Vatsal
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Convergent Evolution: How Different Language Models Learn Similar Number Representations
por: Fu, Deqing, et al.
Publicado: (2026)

Latent Concept Disentanglement in Transformer-based Language Models
por: Hong, Guan Zhe, et al.
Publicado: (2025)

Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
por: Fu, Deqing, et al.
Publicado: (2023)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
por: Vasudeva, Bhavya, et al.
Publicado: (2026)

Pre-trained Large Language Models Use Fourier Features to Compute Addition
por: Zhou, Tianyi, et al.
Publicado: (2024)

In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
por: Deora, Puneesh, et al.
Publicado: (2025)

FoNE: Precise Single-Token Number Embeddings via Fourier Features
por: Zhou, Tianyi, et al.
Publicado: (2025)

Emotion Classification in Low and Moderate Resource Languages
por: Tafreshi, Shabnam, et al.
Publicado: (2024)

Limitations on Accurate, Trusted, Human-level Reasoning
por: Panigrahy, Rina, et al.
Publicado: (2025)

Ulterior Motives: Detecting Misaligned Reasoning in Continuous Thought Models
por: Ramjee, Sharan
Publicado: (2026)

Can GPT Redefine Medical Understanding? Evaluating GPT on Biomedical Machine Reading Comprehension
por: Vatsal, Shubham, et al.
Publicado: (2024)

DeLLMa: Decision Making Under Uncertainty with Large Language Models
por: Liu, Ollie, et al.
Publicado: (2024)

UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making
por: Duan, Jinhao, et al.
Publicado: (2025)

Low-rank finetuning for LLMs: A fairness perspective
por: Das, Saswat, et al.
Publicado: (2024)

Can GPT Improve the State of Prior Authorization via Guideline Based Automated Question Answering?
por: Vatsal, Shubham, et al.
Publicado: (2024)

Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks
por: Vatsal, Shubham, et al.
Publicado: (2025)

Do great minds think alike? Investigating Human-AI Complementarity in Question Answering with CAIMIRA
por: Gor, Maharshi, et al.
Publicado: (2024)

ATLaS: Agent Tuning via Learning Critical Steps
por: Chen, Zhixun, et al.
Publicado: (2025)

Sensitivity-Positional Co-Localization in GQA Transformers
por: Rao, Manoj Chandrashekar
Publicado: (2026)

On the Relation between Sensitivity and Accuracy in In-context Learning
por: Chen, Yanda, et al.
Publicado: (2022)

CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functions
por: Fu, Zihao, et al.
Publicado: (2025)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
por: Vasudeva, Bhavya, et al.
Publicado: (2025)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
por: Collins, Liam, et al.
Publicado: (2024)

Transformers Provably Learn Algorithmic Solutions for Graph Connectivity, But Only with the Right Data
por: Ye, Qilin, et al.
Publicado: (2025)

Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI
por: Maiya, Sharan, et al.
Publicado: (2025)

How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data
por: Vasudeva, Bhavya, et al.
Publicado: (2025)

Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
por: Mathewson, Kyle Elliott
Publicado: (2026)

What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
por: Li, Ming, et al.
Publicado: (2024)

Resa: Transparent Reasoning Models via SAEs
por: Wang, Shangshang, et al.
Publicado: (2025)

LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
por: Pal, Soumyadeep, et al.
Publicado: (2025)

Investigating the Transferability of Code Repair for Low-Resource Programming Languages
por: Wong, Kyle, et al.
Publicado: (2024)

Learning and Enforcing Context-Sensitive Control for LLMs
por: Albinhassan, Mohammad, et al.
Publicado: (2026)

Cluster-norm for Unsupervised Probing of Knowledge
por: Laurito, Walter, et al.
Publicado: (2024)

Compute Optimal Scaling of Skills: Knowledge vs Reasoning
por: Roberts, Nicholas, et al.
Publicado: (2025)

Enhancing In-context Learning via Linear Probe Calibration
por: Abbas, Momin, et al.
Publicado: (2024)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
por: Neo, Clement, et al.
Publicado: (2024)

When Domains Interact: Asymmetric and Order-Sensitive Cross-Domain Effects in Reinforcement Learning for Reasoning
por: Yang, Wang, et al.
Publicado: (2026)

Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statements
por: Li, Ming, et al.
Publicado: (2024)

Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
por: Fan, Chenrui, et al.
Publicado: (2025)

How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients
por: Li, Ming, et al.
Publicado: (2025)