Cover Image

When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Petrov, Aleksandar, Torr, Philip H. S., Bibi, Adel
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models
by: Eiras, Francisco, et al.
Published: (2024)

Universal In-Context Approximation By Prompting Fully Recurrent Models
by: Petrov, Aleksandar, et al.
Published: (2024)

Prompting a Pretrained Transformer Can Be a Universal Approximator
by: Petrov, Aleksandar, et al.
Published: (2024)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
by: Zhang, Wenxuan, et al.
Published: (2024)

L-TUNING: Synchronized Label Tuning for Prompt and Prefix in LLMs
by: Kowsher, Md., et al.
Published: (2023)

Mixture of Experts Made Intrinsically Interpretable
by: Yang, Xingyi, et al.
Published: (2025)

No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance
by: Udandarao, Vishaal, et al.
Published: (2024)

Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
by: Hammoud, Hasan Abed Al Kader, et al.
Published: (2024)

Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
by: Huang, Zeyu, et al.
Published: (2025)

Shh, don't say that! Domain Certification in LLMs
by: Emde, Cornelius, et al.
Published: (2025)

On the Coexistence and Ensembling of Watermarks
by: Petrov, Aleksandar, et al.
Published: (2025)

PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
by: Chen, Mengzhao, et al.
Published: (2024)

Detecting LLM Hallucination Through Layer-wise Information Deficiency: Analysis of Ambiguous Prompts and Unanswerable Questions
by: Kim, Hazel, et al.
Published: (2024)

Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentation
by: Eiras, Francisco, et al.
Published: (2023)

Fundamental Limits of Prompt Tuning Transformers: Universality, Capacity and Efficiency
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Causal Fine-Tuning under Latent Confounded Shift
by: Yu, Jialin, et al.
Published: (2024)

Reliable Chain-of-Thought via Prefix Consistency
by: Iwase, Naoto, et al.
Published: (2026)

Accelerating Direct Preference Optimization with Prefix Sharing
by: Wang, Franklin, et al.
Published: (2024)

Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
by: Soylu, Dilara, et al.
Published: (2024)

Plain Transformers Can be Powerful Graph Learners
by: Ma, Liheng, et al.
Published: (2025)

Continual Learning on a Diet: Learning from Sparsely Labeled Streams Under Constrained Computation
by: Zhang, Wenxuan, et al.
Published: (2024)

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities
by: Sun, Chung-En, et al.
Published: (2024)

SCULPT: Systematic Tuning of Long Prompts
by: Kumar, Shanu, et al.
Published: (2024)

Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models
by: Lan, Michael, et al.
Published: (2023)

When Prompts Interact: Assessing Prompt Arithmetic for Deconfounding under Distribution Shift
by: Sheng, Zhecheng, et al.
Published: (2026)

Prompt Tuning for Natural Language to SQL with Embedding Fine-Tuning and RAG
by: Jang, Jisoo, et al.
Published: (2025)

Efficient Lifelong Model Evaluation in an Era of Rapid Progress
by: Prabhu, Ameya, et al.
Published: (2024)

PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning
by: Zhang, Tianrong, et al.
Published: (2024)

Towards Certification of Uncertainty Calibration under Adversarial Attacks
by: Emde, Cornelius, et al.
Published: (2024)

Demystifying When Pruning Works via Representation Hierarchies
by: He, Shwai, et al.
Published: (2026)

When Do "More Contexts" Help with Sarcasm Recognition?
by: Nimase, Ojas, et al.
Published: (2024)

Towards Infinite-Long Prefix in Transformer
by: Liang, Yingyu, et al.
Published: (2024)

Assessing the Capabilities and Limitations of FinGPT Model in Financial NLP Applications
by: Djagba, Prudence, et al.
Published: (2025)

Beyond Linear Probes: Dynamic Safety Monitoring for Language Models
by: Oldfield, James, et al.
Published: (2025)

Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
by: Cohen, Eilam, et al.
Published: (2026)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
by: Lin, Yong, et al.
Published: (2024)

Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation
by: Yang, Yibo, et al.
Published: (2024)

Exploring Embedding Priors in Prompt-Tuning for Improved Interpretability and Control
by: Sedov, Sergey, et al.
Published: (2024)

Show Your Work with Confidence: Confidence Bands for Tuning Curves
by: Lourie, Nicholas, et al.
Published: (2023)

DynaPrompt: Dynamic Test-Time Prompt Tuning
by: Xiao, Zehao, et al.
Published: (2025)