Copertina

Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lee, Ivan, Jiang, Nan, Berg-Kirkpatrick, Taylor
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models
di: Lee, Ivan, et al.
Pubblicazione: (2025)

Optical Context Compression Is Just (Bad) Autoencoding
di: Lee, Ivan Yee, et al.
Pubblicazione: (2025)

ICL-Router: In-Context Learned Model Representations for LLM Routing
di: Wang, Chenxu, et al.
Pubblicazione: (2025)

VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning
di: Zong, Yongshuo, et al.
Pubblicazione: (2024)

CrystalICL: Enabling In-Context Learning for Crystal Generation
di: Wang, Ruobing, et al.
Pubblicazione: (2025)

Alt-Text with Context: Improving Accessibility for Images on Twitter
di: Srivatsan, Nikita, et al.
Pubblicazione: (2023)

TabICL: A Tabular Foundation Model for In-Context Learning on Large Data
di: Qu, Jingang, et al.
Pubblicazione: (2025)

Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning
di: Zhang, Kaiyi, et al.
Pubblicazione: (2024)

Auto-ICL: In-Context Learning without Human Supervision
di: Yang, Jinghan, et al.
Pubblicazione: (2023)

Parcae: Scaling Laws For Stable Looped Language Models
di: Prairie, Hayden, et al.
Pubblicazione: (2026)

IV-ICL: Bounding Causal Effects with Instrumental Variables via In-Context Learning
di: Balazadeh, Vahid, et al.
Pubblicazione: (2026)

Continuous Diffusion Models Can Obey Formal Syntax
di: Kim, Jinwoo, et al.
Pubblicazione: (2026)

RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning
di: Scarlatos, Alexander, et al.
Pubblicazione: (2023)

DP-TabICL: In-Context Learning with Differentially Private Tabular Data
di: Carey, Alycia N., et al.
Pubblicazione: (2024)

On the Relationship Between the Choice of Representation and In-Context Learning
di: Marinescu, Ioana, et al.
Pubblicazione: (2025)

Smaller Language Models are Better Black-box Machine-Generated Text Detectors
di: Mireshghallah, Niloofar, et al.
Pubblicazione: (2023)

Studying the Soupability of Documents in State Space Models
di: Jafari, Yasaman, et al.
Pubblicazione: (2025)

Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment
di: Zarbaft, Armin, et al.
Pubblicazione: (2026)

Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
di: Xu, Xiaoyue, et al.
Pubblicazione: (2024)

Exploring the Relationship Between Feature Attribution Methods and Model Performance
di: Silva, Priscylla, et al.
Pubblicazione: (2024)

Task Diversity Shortens the ICL Plateau
di: Kim, Jaeyeon, et al.
Pubblicazione: (2024)

BACHI: Boundary-Aware Symbolic Chord Recognition Through Masked Iterative Decoding on Pop and Classical Music
di: Yao, Mingyang, et al.
Pubblicazione: (2025)

DITTO-2: Distilled Diffusion Inference-Time T-Optimization for Music Generation
di: Novack, Zachary, et al.
Pubblicazione: (2024)

TabGen-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation
di: Fang, Liancheng, et al.
Pubblicazione: (2025)

Constrained Adaptive Rejection Sampling
di: Parys, Paweł, et al.
Pubblicazione: (2025)

CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations
di: Kothapalli, Vignesh, et al.
Pubblicazione: (2025)

Demonstrations, CoT, and Prompting: A Theoretical Analysis of ICL
di: Tong, Xuhan, et al.
Pubblicazione: (2026)

GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design
di: Sun, Yuanfu, et al.
Pubblicazione: (2025)

Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification
di: Liu, Shang, et al.
Pubblicazione: (2024)

Exploring the Relationships Between Physiological Signals During Automated Fatigue Detection
di: Kakhi, Kourosh, et al.
Pubblicazione: (2025)

PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processing
di: Long, Phillip, et al.
Pubblicazione: (2024)

Learning the Error Patterns of Language Models
di: Kim, Jinwoo, et al.
Pubblicazione: (2026)

Grammar-Aligned Decoding
di: Park, Kanghee, et al.
Pubblicazione: (2024)

Constrained Sampling for Language Models Should Be Easy: An MCMC Perspective
di: Gonzalez, Emmanuel Anaya, et al.
Pubblicazione: (2025)

DITTO: Diffusion Inference-Time T-Optimization for Music Generation
di: Novack, Zachary, et al.
Pubblicazione: (2024)

Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation
di: Chen, Ke, et al.
Pubblicazione: (2024)

Exploring Urban Factors with Autoencoders: Relationship Between Static and Dynamic Features
di: Pocco, Ximena, et al.
Pubblicazione: (2025)

ClimaQA: An Automated Evaluation Framework for Climate Question Answering Models
di: Manivannan, Veeramakali Vignesh, et al.
Pubblicazione: (2024)

Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
di: Okawa, Maya, et al.
Pubblicazione: (2023)

Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks
di: Campbell, Ryan, et al.
Pubblicazione: (2024)