Imagen de Portada

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xu, Zhuoyan, Shi, Zhenmei, Liang, Yingyu
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Why Larger Language Models Do In-context Learning Differently?
por: Shi, Zhenmei, et al.
Publicado: (2024)

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning
por: Xu, Zhuoyan, et al.
Publicado: (2024)

Can Language Models Compose Skills In-Context?
por: Liu, Zidong, et al.
Publicado: (2025)

The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
por: Chen, Yifang, et al.
Publicado: (2024)

Training Tensor Attention Efficiently: From Cubic to Almost Linear Time
por: Cao, Yang, et al.
Publicado: (2024)

Towards Infinite-Long Prefix in Transformer
por: Liang, Yingyu, et al.
Publicado: (2024)

Circuit Complexity Bounds for Visual Autoregressive Model
por: Ke, Yekun, et al.
Publicado: (2025)

HSR-Enhanced Sparse Attention Acceleration
por: Chen, Bo, et al.
Publicado: (2024)

Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
por: Liang, Yingyu, et al.
Publicado: (2024)

Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Time
por: Liang, Yingyu, et al.
Publicado: (2024)

Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction
por: Shi, Zhenmei, et al.
Publicado: (2024)

A Tighter Complexity Analysis of SparseGPT
por: Li, Xiaoyu, et al.
Publicado: (2024)

RoPE Attention Can Be Trained in Almost Linear Time
por: Cao, Yang, et al.
Publicado: (2024)

Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$-based Tensor Attention Transformers
por: Li, Xiaoyu, et al.
Publicado: (2024)

Neural Algorithmic Reasoning for Hypergraphs with Looped Transformers
por: Huang, Zekai, et al.
Publicado: (2025)

Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers
por: Liang, Yingyu, et al.
Publicado: (2024)

Modern Hopfield Networks Require Chain-of-Thought to Solve $\mathsf{NC}^1$-Hard Problems
por: Cao, Yang, et al.
Publicado: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
por: Chen, Bo, et al.
Publicado: (2024)

On Fine-Grained I/O Complexity of Attention Backward Passes
por: Li, Xiaoyu, et al.
Publicado: (2024)

Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
por: Chen, Yifang, et al.
Publicado: (2025)

Universal Approximation of Visual Autoregressive Transformers
por: Chen, Yifang, et al.
Publicado: (2025)

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
por: Dong, Guanting, et al.
Publicado: (2023)

Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Models
por: Cui, Andrea Yaoyun, et al.
Publicado: (2025)

When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?
por: Li, Chenyang, et al.
Publicado: (2025)

Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective
por: Liang, Yingyu, et al.
Publicado: (2024)

Deception Abilities Emerged in Large Language Models
por: Hagendorff, Thilo
Publicado: (2023)

RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation
por: Cao, Yuefan, et al.
Publicado: (2025)

Having Beer after Prayer? Measuring Cultural Bias in Large Language Models
por: Naous, Tarek, et al.
Publicado: (2023)

Self-Evolving Critique Abilities in Large Language Models
por: Tang, Zhengyang, et al.
Publicado: (2025)

Emergent Abilities in Large Language Models: A Survey
por: Berti, Leonardo, et al.
Publicado: (2025)

Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond
por: Cao, Yang, et al.
Publicado: (2024)

Re-Tuning: Overcoming the Compositionality Limits of Large Language Models with Recursive Tuning
por: Pasewark, Eric, et al.
Publicado: (2024)

On Computational Limits and Provably Efficient Criteria of Visual Autoregressive Models: A Fine-Grained Complexity Analysis
por: Ke, Yekun, et al.
Publicado: (2025)

Out-of-distribution generalization via composition: a lens through induction heads in Transformers
por: Song, Jiajun, et al.
Publicado: (2024)

Learning to Inference Adaptively for Multimodal Large Language Models
por: Xu, Zhuoyan, et al.
Publicado: (2025)

Psychological Counseling Ability of Large Language Models
por: Peng, Fangyu, et al.
Publicado: (2025)

Hallucination is Inevitable: An Innate Limitation of Large Language Models
por: Xu, Ziwei, et al.
Publicado: (2024)

Towards Trustable Language Models: Investigating Information Quality of Large Language Models
por: Rejeleene, Rick, et al.
Publicado: (2024)

Looped ReLU MLPs May Be All You Need as Practical Programmable Computers
por: Liang, Yingyu, et al.
Publicado: (2024)

Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models
por: Sun, Haoran, et al.
Publicado: (2024)