Copertina

Length Generalization of Causal Transformers without Position Encoding

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Jie, Ji, Tao, Wu, Yuanbin, Yan, Hang, Gui, Tao, Zhang, Qi, Huang, Xuanjing, Wang, Xiaoling
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis
di: Wang, Siyin, et al.
Pubblicazione: (2024)

Generation with Dynamic Vocabulary
di: Liu, Yanting, et al.
Pubblicazione: (2024)

Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
di: Lu, Yi, et al.
Pubblicazione: (2025)

DVAGen: Dynamic Vocabulary Augmented Generation
di: Du, Wei, et al.
Pubblicazione: (2025)

AntLM: Bridging Causal and Masked Language Models
di: Yu, Xinru, et al.
Pubblicazione: (2024)

Unveiling Linguistic Regions in Large Language Models
di: Zhang, Zhihao, et al.
Pubblicazione: (2024)

LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition
di: Ye, Junjie, et al.
Pubblicazione: (2024)

Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
di: Ji, Tao, et al.
Pubblicazione: (2025)

RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
di: Zhang, Yuansen, et al.
Pubblicazione: (2024)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
di: Lu, Yi, et al.
Pubblicazione: (2024)

Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding
di: Hayakawa, Daichi, et al.
Pubblicazione: (2024)

Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding
di: Zhao, Liang, et al.
Pubblicazione: (2023)

LLaMA Beyond English: An Empirical Study on Language Capability Transfer
di: Zhao, Jun, et al.
Pubblicazione: (2024)

CCTU: A Benchmark for Tool Use under Complex Constraints
di: Ye, Junjie, et al.
Pubblicazione: (2026)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
di: Wang, Zhenghua, et al.
Pubblicazione: (2025)

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
di: Zheng, Chuanyang, et al.
Pubblicazione: (2024)

Improving RL Exploration for LLM Reasoning through Retrospective Replay
di: Dou, Shihan, et al.
Pubblicazione: (2025)

Fusion Matters: Length-Aware Analysis of Positional-Encoding Fusion in Transformers
di: Hallam, Mohamed Amine, et al.
Pubblicazione: (2026)

Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control
di: Wang, Xinyu, et al.
Pubblicazione: (2026)

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
di: Zheng, Rui, et al.
Pubblicazione: (2024)

Are Large Language Models Good Prompt Optimizers?
di: Ma, Ruotian, et al.
Pubblicazione: (2024)

60 Data Points are Sufficient to Fine-Tune LLMs for Question-Answering
di: Ye, Junjie, et al.
Pubblicazione: (2024)

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
di: Ye, Junjie, et al.
Pubblicazione: (2024)

Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings
di: Zuo, Chunsheng, et al.
Pubblicazione: (2024)

ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
di: Ye, Junjie, et al.
Pubblicazione: (2024)

Advancing Block Diffusion Language Models for Test-Time Scaling
di: Lu, Yi, et al.
Pubblicazione: (2026)

CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
di: Lv, Huijie, et al.
Pubblicazione: (2024)

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
di: Dou, Shihan, et al.
Pubblicazione: (2024)

Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric
di: Yang, Yuming, et al.
Pubblicazione: (2025)

PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness
di: Wang, Zekun, et al.
Pubblicazione: (2024)

Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
di: Xi, Zhiheng, et al.
Pubblicazione: (2023)

FinToolSyn: A forward synthesis Framework for Financial Tool-Use Dialogue Data with Dynamic Tool Retrieval
di: Huang, Caishuang, et al.
Pubblicazione: (2026)

Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems
di: da Silva, Italo Luis, et al.
Pubblicazione: (2024)

Distill Visual Chart Reasoning Ability from LLMs to MLLMs
di: He, Wei, et al.
Pubblicazione: (2024)

On the Geometry of Positional Encodings in Transformers
di: Cirrincione, Giansalvo
Pubblicazione: (2026)

MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
di: Lin, Jiahang, et al.
Pubblicazione: (2026)

Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation
di: Wang, Yikun, et al.
Pubblicazione: (2023)

LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding
di: Zhang, Shen, et al.
Pubblicazione: (2025)

Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control
di: Jiang, Changhao, et al.
Pubblicazione: (2026)

RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
di: Ye, Junjie, et al.
Pubblicazione: (2024)