Image de couverture de livre

Unlocking the Theory Behind Scaling 1-Bit Neural Networks

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Daliri, Majid, Song, Zhao, Yang, Chiwun
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Artificial Intelligence Computational Complexity Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead
par: Zandieh, Amir, et autres
Publié: (2024)

Unifying Learning Dynamics and Generalization in Transformers Scaling Law
par: Yang, Chiwun
Publié: (2025)

Modern Hopfield Networks Require Chain-of-Thought to Solve $\mathsf{NC}^1$-Hard Problems
par: Cao, Yang, et autres
Publié: (2024)

Neural Algorithmic Reasoning for Hypergraphs with Looped Transformers
par: Huang, Zekai, et autres
Publié: (2025)

RoPE Attention Can Be Trained in Almost Linear Time
par: Cao, Yang, et autres
Publié: (2024)

On Fine-Grained I/O Complexity of Attention Backward Passes
par: Li, Xiaoyu, et autres
Publié: (2024)

The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
par: Chen, Yifang, et autres
Publié: (2024)

Theoretical Constraints on the Expressive Power of $\mathsf{RoPE}$-based Tensor Attention Transformers
par: Li, Xiaoyu, et autres
Publié: (2024)

Circuit Complexity Bounds for Visual Autoregressive Model
par: Ke, Yekun, et autres
Publié: (2025)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
par: Chen, Bo, et autres
Publié: (2024)

Time and Memory Trade-off of KV-Cache Compression in Tensor Transformer Decoding
par: Chen, Yifang, et autres
Publié: (2025)

How Sparse Attention Approximates Exact Attention? Your Attention is Naturally $n^C$-Sparse
par: Deng, Yichuan, et autres
Publié: (2024)

Towards Infinite-Long Prefix in Transformer
par: Liang, Yingyu, et autres
Publié: (2024)

On the Computational Capability of Graph Neural Networks: A Circuit Complexity Bound Perspective
par: Li, Xiaoyu, et autres
Publié: (2025)

A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits
par: Zhang, Yuyang, et autres
Publié: (2026)

NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes
par: Fan, Lizhou, et autres
Publié: (2023)

Demystifying the unreasonable effectiveness of online alignment methods
par: Kang, Enoch Hyunwook
Publié: (2026)

Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives
par: Meyerson, Elliot, et autres
Publié: (2025)

Provable Failure of Language Models in Learning Majority Boolean Logic via Gradient Descent
par: Chen, Bo, et autres
Publié: (2025)

When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?
par: Li, Chenyang, et autres
Publié: (2025)

A Theory of Learning with Autoregressive Chain of Thought
par: Joshi, Nirmit, et autres
Publié: (2025)

Looped ReLU MLPs May Be All You Need as Practical Programmable Computers
par: Liang, Yingyu, et autres
Publié: (2024)

Computational Limits of Low-Rank Adaptation (LoRA) Fine-Tuning for Transformer Models
par: Hu, Jerry Yao-Chieh, et autres
Publié: (2024)

Transformers Can Represent $n$-gram Language Models
par: Svete, Anej, et autres
Publié: (2024)

Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't
par: Svete, Anej, et autres
Publié: (2026)

AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
par: Liu, Dongrui, et autres
Publié: (2026)

Video Latent Flow Matching: Optimal Polynomial Projections for Video Interpolation and Extrapolation
par: Cao, Yang, et autres
Publié: (2025)

Infinite Time Turing Machines and their Applications
par: Weerawarana, Rukmal, et autres
Publié: (2025)

Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspective
par: Cao, Yang, et autres
Publié: (2025)

CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models
par: Liang, Xuechen, et autres
Publié: (2024)

Journalists, Emotions, and the Introduction of Generative AI Chatbots: A Large-Scale Analysis of Tweets Before and After the Launch of ChatGPT
par: Lewis, Seth C., et autres
Publié: (2024)

Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving
par: AbdElhameed, Marwan, et autres
Publié: (2024)

Lossless Model Compression via Joint Low-Rank Factorization Optimization
par: Zhang, Boyang, et autres
Publié: (2024)

Mathematical Formalism for Memory Compression in Selective State Space Models
par: Bhat, Siddhanth
Publié: (2024)

Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirement
par: Boche, Holger, et autres
Publié: (2024)

Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transition
par: Hazra, Rishi, et autres
Publié: (2025)

Learning to Think from Multiple Thinkers
par: Joshi, Nirmit, et autres
Publié: (2026)

Provably Overwhelming Transformer Models with Designed Inputs
par: Stambler, Lev, et autres
Publié: (2025)

A Provable Expressiveness Hierarchy in Hybrid Linear-Full Attention
par: Ye, Xiaowei, et autres
Publié: (2026)

On the Expressive Power and Limitations of Multi-Layer SSMs
par: Zubić, Nikola, et autres
Publié: (2026)