Cover Image

Breaking through the learning plateaus of in-context learning in Transformer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fu, Jingwen, Yang, Tao, Wang, Yuwang, Lu, Yan, Zheng, Nanning
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models
by: Bi, Tianci, et al.
Published: (2025)

Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting
by: Zhuo, Linhai, et al.
Published: (2024)

PromptTA: Prompt-driven Text Adapter for Source-free Domain Generalization
by: Zhang, Haoran, et al.
Published: (2024)

A Survey on Transformer Compression
by: Tang, Yehui, et al.
Published: (2024)

Not All Tokens Matter Equally: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation
by: Wu, Ning, et al.
Published: (2026)

What do vision-language models see in the context? Investigating multimodal in-context learning
by: Santos, Gabriel O. dos, et al.
Published: (2025)

The in-context inductive biases of vision-language models differ across modalities
by: Allen, Kelsey, et al.
Published: (2025)

VLLaVO: Mitigating Visual Gap through LLMs
by: Chen, Shuhao, et al.
Published: (2024)

What to align in multimodal contrastive learning?
by: Dufumier, Benoit, et al.
Published: (2024)

LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding
by: Wu, Haoning, et al.
Published: (2024)

Efficient Scaling of Diffusion Transformers for Text-to-Image Generation
by: Li, Hao, et al.
Published: (2024)

MoPD: Mixture-of-Prompts Distillation for Vision-Language Models
by: Chen, Yang, et al.
Published: (2024)

SpinQuant: LLM quantization with learned rotations
by: Liu, Zechun, et al.
Published: (2024)

Transformer-VQ: Linear-Time Transformers via Vector Quantization
by: Lingle, Lucas D.
Published: (2023)

VINCIE: Unlocking In-context Image Editing from Video
by: Qu, Leigang, et al.
Published: (2025)

SOLO: A Single Transformer for Scalable Vision-Language Modeling
by: Chen, Yangyi, et al.
Published: (2024)

General Transform: A Unified Framework for Adaptive Transform to Enhance Representations
by: Budiutama, Gekko, et al.
Published: (2025)

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models
by: Zheng, Kening, et al.
Published: (2024)

Fine-Grained Alignment in Vision-and-Language Navigation through Bayesian Optimization
by: Song, Yuhang, et al.
Published: (2024)

Sparrow: Data-Efficient Video-LLM with Text-to-Image Augmentation
by: Yin, Shukang, et al.
Published: (2024)

Diffusion-RPO: Aligning Diffusion Models through Relative Preference Optimization
by: Gu, Yi, et al.
Published: (2024)

MatFormer: Nested Transformer for Elastic Inference
by: Devvrit, et al.
Published: (2023)

Transformer with Controlled Attention for Synchronous Motion Captioning
by: Radouane, Karim, et al.
Published: (2024)

Enhancing Visual Question Answering through Ranking-Based Hybrid Training and Multimodal Fusion
by: Chen, Peiyuan, et al.
Published: (2024)

Text-centric Alignment for Multi-Modality Learning
by: Tsai, Yun-Da, et al.
Published: (2024)

Head Pursuit: Probing Attention Specialization in Multimodal Transformers
by: Basile, Lorenzo, et al.
Published: (2025)

HATFormer: Historic Handwritten Arabic Text Recognition with Transformers
by: Chan, Adrian, et al.
Published: (2024)

Matryoshka Query Transformer for Large Vision-Language Models
by: Hu, Wenbo, et al.
Published: (2024)

CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers
by: Yamada, Yoshihiro
Published: (2025)

Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers
by: Tas, Omer Sahin, et al.
Published: (2024)

A U-Net and Transformer Pipeline for Multilingual Image Translation
by: Sahay, Siddharth, et al.
Published: (2025)

Text Role Classification in Scientific Charts Using Multimodal Transformers
by: Kim, Hye Jin, et al.
Published: (2024)

Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs
by: Li, Shuo, et al.
Published: (2024)

Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehension
by: Wang, Xiyao, et al.
Published: (2024)

Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
by: Thangarasa, Vithursan, et al.
Published: (2023)

White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
by: Yu, Yaodong, et al.
Published: (2023)

DNT: a Deeply Normalized Transformer that can be trained by Momentum SGD
by: Qi, Xianbiao, et al.
Published: (2025)

Stronger Normalization-Free Transformers
by: Chen, Mingzhi, et al.
Published: (2025)

GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation
by: Sîrbu, Iustin, et al.
Published: (2025)

DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams
by: Picón, Ginés Carreto, et al.
Published: (2025)