Cover Image

Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Cho, Hanseul, Cha, Jaeyoung, Bhojanapalli, Srinadh, Yun, Chulhee
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure
by: Cho, Hanseul, et al.
Published: (2024)

DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticity
by: Shin, Baekrok, et al.
Published: (2024)

Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems
by: Kim, Yujun, et al.
Published: (2025)

On student-teacher deviations in distillation: does it pay to disobey?
by: Nagarajan, Vaishnavh, et al.
Published: (2023)

Transformers Can Achieve Length Generalization But Not Robustly
by: Zhou, Yongchao, et al.
Published: (2024)

Fundamental Benefit of Alternating Updates in Minimax Optimization
by: Lee, Jaewook, et al.
Published: (2024)

Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
by: Jung, Hyunji, et al.
Published: (2025)

On Vanishing Variance in Transformer Length Generalization
by: Li, Ruining, et al.
Published: (2025)

Provable Benefit of Cutout and CutMix for Feature Learning
by: Oh, Junsoo, et al.
Published: (2024)

Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertainty
by: Cho, Yeseul, et al.
Published: (2025)

The Role of Sparsity for Length Generalization in Transformers
by: Golowich, Noah, et al.
Published: (2025)

HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference
by: L, Yashas Samaga B, et al.
Published: (2024)

Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
by: Lee, Nayoung, et al.
Published: (2025)

Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
by: Moon, Chaewon, et al.
Published: (2026)

The Cost of Robustness: Tighter Bounds on Parameter Complexity for Robust Memorization in ReLU Nets
by: Kim, Yujun, et al.
Published: (2025)

Transformers Can Do Arithmetic with the Right Embeddings
by: McLeish, Sean, et al.
Published: (2024)

Mamba Modulation: On the Length Generalization of Mamba
by: Lu, Peng, et al.
Published: (2025)

Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization
by: Huang, Yu, et al.
Published: (2025)

On the Limitations and Capabilities of Position Embeddings for Length Generalization
by: Chen, Yang, et al.
Published: (2025)

Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders
by: Rozonoyer, Benjamin, et al.
Published: (2026)

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
by: Kim, Jihwan, et al.
Published: (2026)

Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
by: Baek, Beomhan, et al.
Published: (2025)

Length-Aware Adversarial Training for Variable-Length Trajectories: Digital Twins for Mall Shopper Paths
by: Sun, He, et al.
Published: (2026)

When Can Transformers Count to n?
by: Yehudai, Gilad, et al.
Published: (2024)

Length Optimization in Conformal Prediction
by: Kiyani, Shayan, et al.
Published: (2024)

Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo
by: Kim, Hyunsu, et al.
Published: (2025)

Can an LLM Induce a Graph? Investigating Memory Drift and Context Length
by: Yousuf, Raquib Bin, et al.
Published: (2025)

ParaDySe: A Parallel-Strategy Switching Framework for Dynamic Sequence Lengths in Transformer
by: Ou, Zhixin, et al.
Published: (2025)

On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
by: Kim, Sunghwan, et al.
Published: (2026)

Excess Description Length of Learning Generalizable Predictors
by: Donoway, Elizabeth, et al.
Published: (2026)

A Swap-Adversarial Framework for Improving Domain Generalization in Electroencephalography-Based Parkinson's Disease Prediction
by: Jin, Seongwon, et al.
Published: (2026)

Principled Understanding of Generalization for Generative Transformer Models in Arithmetic Reasoning Tasks
by: Xu, Xingcheng, et al.
Published: (2024)

Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
by: Song, Minhak, et al.
Published: (2025)

LengthLogD: A Length-Stratified Ensemble Framework for Enhanced Peptide Lipophilicity Prediction via Multi-Scale Feature Integration
by: Wu, Shuang, et al.
Published: (2025)

DeciMamba: Exploring the Length Extrapolation Potential of Mamba
by: Ben-Kish, Assaf, et al.
Published: (2024)

An Empirical Study on Context Length for Open-Domain Dialog Generation
by: Shen, Xinyi, et al.
Published: (2024)

Provable Length Generalization in Sequence Prediction via Spectral Filtering
by: Marsden, Annie, et al.
Published: (2024)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
by: Leng, Jiaqi, et al.
Published: (2025)

Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization
by: Urrutia, Felipe, et al.
Published: (2026)

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
by: Zhao, Kangwen, et al.
Published: (2025)