Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.21800 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912926522146816 |
|---|---|
| author | Ghosh, Madhusudan Gupta, Rishabh |
| author_facet | Ghosh, Madhusudan Gupta, Rishabh |
| contents | The rapid advancement of large language models (LLMs) has led to a significant increase in automated tools in the software engineering, capable of performing various code-related tasks such as code generation, completion, and translation. Despite these advancements, its effectiveness is constrained by fixed context lengths, limiting its ability to generalize across long, domain-specific code sequences. To address this challenge, we investigate zero-shot, inference-only methods aimed at improving position encodings and optimizing attention mechanisms. Our goal is to provide a thorough analysis of current approaches that facilitate context length extrapolation in code, particularly in the context of long code completion tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21800 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | An Evaluation of Context Length Extrapolation in Long Code via Positional Embeddings and Efficient Attention Ghosh, Madhusudan Gupta, Rishabh Software Engineering Artificial Intelligence The rapid advancement of large language models (LLMs) has led to a significant increase in automated tools in the software engineering, capable of performing various code-related tasks such as code generation, completion, and translation. Despite these advancements, its effectiveness is constrained by fixed context lengths, limiting its ability to generalize across long, domain-specific code sequences. To address this challenge, we investigate zero-shot, inference-only methods aimed at improving position encodings and optimizing attention mechanisms. Our goal is to provide a thorough analysis of current approaches that facilitate context length extrapolation in code, particularly in the context of long code completion tasks. |
| title | An Evaluation of Context Length Extrapolation in Long Code via Positional Embeddings and Efficient Attention |
| topic | Software Engineering Artificial Intelligence |
| url | https://arxiv.org/abs/2602.21800 |