Image de couverture de livre

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xu, Zhuoyan, Shi, Zhenmei, Wei, Junyi, Mu, Fangzhou, Li, Yin, Liang, Yingyu
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Why Larger Language Models Do In-context Learning Differently?
par: Shi, Zhenmei, et autres
Publié: (2024)

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability
par: Xu, Zhuoyan, et autres
Publié: (2024)

Towards Infinite-Long Prefix in Transformer
par: Liang, Yingyu, et autres
Publié: (2024)

Can Language Models Compose Skills In-Context?
par: Liu, Zidong, et autres
Publié: (2025)

Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers
par: Liang, Yingyu, et autres
Publié: (2024)

Training Tensor Attention Efficiently: From Cubic to Almost Linear Time
par: Cao, Yang, et autres
Publié: (2024)

The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
par: Chen, Yifang, et autres
Publié: (2024)

HSR-Enhanced Sparse Attention Acceleration
par: Chen, Bo, et autres
Publié: (2024)

Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
par: Liang, Yingyu, et autres
Publié: (2024)

Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Time
par: Liang, Yingyu, et autres
Publié: (2024)

Circuit Complexity Bounds for Visual Autoregressive Model
par: Ke, Yekun, et autres
Publié: (2025)

A Tighter Complexity Analysis of SparseGPT
par: Li, Xiaoyu, et autres
Publié: (2024)

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction
par: Shi, Zhenmei, et autres
Publié: (2024)

RoPE Attention Can Be Trained in Almost Linear Time
par: Cao, Yang, et autres
Publié: (2024)

Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$-based Tensor Attention Transformers
par: Li, Xiaoyu, et autres
Publié: (2024)

Neural Algorithmic Reasoning for Hypergraphs with Looped Transformers
par: Huang, Zekai, et autres
Publié: (2025)

Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond
par: Cao, Yang, et autres
Publié: (2024)

Modern Hopfield Networks Require Chain-of-Thought to Solve $\mathsf{NC}^1$-Hard Problems
par: Cao, Yang, et autres
Publié: (2024)

Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
par: Chen, Yifang, et autres
Publié: (2025)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
par: Chen, Bo, et autres
Publié: (2024)

On Fine-Grained I/O Complexity of Attention Backward Passes
par: Li, Xiaoyu, et autres
Publié: (2024)

Universal Approximation of Visual Autoregressive Transformers
par: Chen, Yifang, et autres
Publié: (2025)

PRISP: Privacy-Safe Few-Shot Personalization via Lightweight Adaptation
par: Park, Junho, et autres
Publié: (2026)

Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models
par: Kumar, Sachin
Publié: (2026)

RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation
par: Cao, Yuefan, et autres
Publié: (2025)

Out-of-distribution generalization via composition: a lens through induction heads in Transformers
par: Song, Jiajun, et autres
Publié: (2024)

When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?
par: Li, Chenyang, et autres
Publié: (2025)

Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective
par: Liang, Yingyu, et autres
Publié: (2024)

Meta-Semantics Augmented Few-Shot Relational Learning
par: Wu, Han, et autres
Publié: (2025)

Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent
par: Chen, Bo, et autres
Publié: (2024)

Kernel Regression in Structured Non-IID Settings: Theory and Implications for Denoising Score Learning
par: Zhang, Dechen, et autres
Publié: (2025)

Curse of Attention: A Kernel-Based Perspective for Why Transformers Fail to Generalize on Time Series Forecasting and Beyond
par: Ke, Yekun, et autres
Publié: (2024)

Few-Shot Recalibration of Language Models
par: Li, Xiang Lisa, et autres
Publié: (2024)

Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning
par: Chen, Yanda, et autres
Publié: (2024)

LoRA Users Beware: A Few Spurious Tokens Can Manipulate Your Finetuned Model
par: Salles, Marcel Mateos, et autres
Publié: (2025)

Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
par: Halawi, Danny, et autres
Publié: (2024)

Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare
par: Gondara, Lovedeep, et autres
Publié: (2025)

Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches
par: Chen, Yifang, et autres
Publié: (2025)

Advancing the Understanding of Fixed Point Iterations in Deep Neural Networks: A Detailed Analytical Study
par: Ke, Yekun, et autres
Publié: (2024)

Provable Differentially Private Computation of the Cross-Attention Mechanism
par: Ke, Yekun, et autres
Publié: (2024)