Copertina

On the Robustness of Transformers against Context Hijacking for Linear Classification

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Tianle, Zhang, Chenyang, Chen, Xingwu, Cao, Yuan, Zou, Difan
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
di: Chen, Xingwu, et al.
Pubblicazione: (2025)

Towards Theoretical Understanding of Transformer Test-Time Computing: Investigation on In-Context Linear Regression
di: Chen, Xingwu, et al.
Pubblicazione: (2025)

In-Context Representation Hijacking
di: Yona, Itay, et al.
Pubblicazione: (2025)

A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models
di: Xie, Chengxing, et al.
Pubblicazione: (2024)

Breaking Contextual Inertia: Reinforcement Learning with Single-Turn Anchors for Stable Multi-Turn Interaction
di: Chen, Xingwu, et al.
Pubblicazione: (2026)

Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders
di: Wang, Xu, et al.
Pubblicazione: (2025)

Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
di: Fu, Deqing, et al.
Pubblicazione: (2023)

SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification
di: Jiao, Difan, et al.
Pubblicazione: (2023)

How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression
di: Chen, Xingwu, et al.
Pubblicazione: (2024)

Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation
di: Zhang, Ziniu, et al.
Pubblicazione: (2025)

Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation
di: Chen, Simin, et al.
Pubblicazione: (2025)

Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
di: Wang, Xu, et al.
Pubblicazione: (2025)

DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoders
di: Wang, Xu, et al.
Pubblicazione: (2026)

MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations
di: Huang, Kaixuan, et al.
Pubblicazione: (2025)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
di: MiniCPM Team, et al.
Pubblicazione: (2026)

Your Transformer is Secretly Linear
di: Razzhigaev, Anton, et al.
Pubblicazione: (2024)

Self-Distillation for Multi-Token Prediction
di: Zhao, Guoliang, et al.
Pubblicazione: (2026)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
di: Chen, Yingfa, et al.
Pubblicazione: (2025)

Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
di: Zhang, Qizheng, et al.
Pubblicazione: (2025)

Optimal Decay Spectra for Linear Recurrences
di: Cao, Yang
Pubblicazione: (2026)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
di: Chen, Yingfa, et al.
Pubblicazione: (2026)

Transformers Can Achieve Length Generalization But Not Robustly
di: Zhou, Yongchao, et al.
Pubblicazione: (2024)

Learning under Quantization for High-Dimensional Linear Regression
di: Zhang, Dechen, et al.
Pubblicazione: (2025)

Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
di: Chen, Taiye, et al.
Pubblicazione: (2025)

Language Models "Grok" to Copy
di: Lv, Ang, et al.
Pubblicazione: (2024)

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency
di: Hu, Jerry Yao-Chieh, et al.
Pubblicazione: (2024)

Exploring Explanations Improves the Robustness of In-Context Learning
di: Honda, Ukyo, et al.
Pubblicazione: (2025)

Differentiable Evolutionary Reinforcement Learning
di: Cheng, Sitao, et al.
Pubblicazione: (2025)

PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
di: Wang, Xuliang, et al.
Pubblicazione: (2026)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
di: Zhu, Shiyi, et al.
Pubblicazione: (2023)

Do pretrained Transformers Learn In-Context by Gradient Descent?
di: Shen, Lingfeng, et al.
Pubblicazione: (2023)

What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
di: Chen, Xingwu, et al.
Pubblicazione: (2024)

Transformer Circuit Faithfulness Metrics are not Robust
di: Miller, Joseph, et al.
Pubblicazione: (2024)

Large Language Models as Tool Makers
di: Cai, Tianle, et al.
Pubblicazione: (2023)

Linear Chain Transformation: Expanding Optimization Dynamics for Fine-Tuning Large Language Models
di: Wang, Yulong, et al.
Pubblicazione: (2024)

LLM-Based Robust Product Classification in Commerce and Compliance
di: Gholamian, Sina, et al.
Pubblicazione: (2024)

Proximal Supervised Fine-Tuning
di: Zhu, Wenhong, et al.
Pubblicazione: (2025)

Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
di: Choi, Sehyun
Pubblicazione: (2024)

ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RL
di: Lu, Xingyu, et al.
Pubblicazione: (2026)

Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks
di: Tang, Xuan, et al.
Pubblicazione: (2025)