Imagen de Portada

Length Generalization of Causal Transformers without Position Encoding

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Jie, Ji, Tao, Wu, Yuanbin, Yan, Hang, Gui, Tao, Zhang, Qi, Huang, Xuanjing, Wang, Xiaoling
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis
por: Wang, Siyin, et al.
Publicado: (2024)

Generation with Dynamic Vocabulary
por: Liu, Yanting, et al.
Publicado: (2024)

Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
por: Lu, Yi, et al.
Publicado: (2025)

DVAGen: Dynamic Vocabulary Augmented Generation
por: Du, Wei, et al.
Publicado: (2025)

AntLM: Bridging Causal and Masked Language Models
por: Yu, Xinru, et al.
Publicado: (2024)

Unveiling Linguistic Regions in Large Language Models
por: Zhang, Zhihao, et al.
Publicado: (2024)

LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition
por: Ye, Junjie, et al.
Publicado: (2024)

Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
por: Ji, Tao, et al.
Publicado: (2025)

RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
por: Zhang, Yuansen, et al.
Publicado: (2024)

LongHeads: Multi-Head Attention is Secretly a Long Context Processor
por: Lu, Yi, et al.
Publicado: (2024)

Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding
por: Hayakawa, Daichi, et al.
Publicado: (2024)

Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding
por: Zhao, Liang, et al.
Publicado: (2023)

LLaMA Beyond English: An Empirical Study on Language Capability Transfer
por: Zhao, Jun, et al.
Publicado: (2024)

CCTU: A Benchmark for Tool Use under Complex Constraints
por: Ye, Junjie, et al.
Publicado: (2026)

Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling
por: Wang, Zhenghua, et al.
Publicado: (2025)

DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
por: Zheng, Chuanyang, et al.
Publicado: (2024)

Improving RL Exploration for LLM Reasoning through Retrospective Replay
por: Dou, Shihan, et al.
Publicado: (2025)

Fusion Matters: Length-Aware Analysis of Positional-Encoding Fusion in Transformers
por: Hallam, Mohamed Amine, et al.
Publicado: (2026)

Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control
por: Wang, Xinyu, et al.
Publicado: (2026)

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
por: Zheng, Rui, et al.
Publicado: (2024)

Are Large Language Models Good Prompt Optimizers?
por: Ma, Ruotian, et al.
Publicado: (2024)

60 Data Points are Sufficient to Fine-Tune LLMs for Question-Answering
por: Ye, Junjie, et al.
Publicado: (2024)

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
por: Ye, Junjie, et al.
Publicado: (2024)

Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings
por: Zuo, Chunsheng, et al.
Publicado: (2024)

ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
por: Ye, Junjie, et al.
Publicado: (2024)

Advancing Block Diffusion Language Models for Test-Time Scaling
por: Lu, Yi, et al.
Publicado: (2026)

CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
por: Lv, Huijie, et al.
Publicado: (2024)

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
por: Dou, Shihan, et al.
Publicado: (2024)

Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric
por: Yang, Yuming, et al.
Publicado: (2025)

PositionID: LLMs can Control Lengths, Copy and Paste with Explicit Positional Awareness
por: Wang, Zekun, et al.
Publicado: (2024)

Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
por: Xi, Zhiheng, et al.
Publicado: (2023)

FinToolSyn: A forward synthesis Framework for Financial Tool-Use Dialogue Data with Dynamic Tool Retrieval
por: Huang, Caishuang, et al.
Publicado: (2026)

Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems
por: da Silva, Italo Luis, et al.
Publicado: (2024)

Distill Visual Chart Reasoning Ability from LLMs to MLLMs
por: He, Wei, et al.
Publicado: (2024)

On the Geometry of Positional Encodings in Transformers
por: Cirrincione, Giansalvo
Publicado: (2026)

MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
por: Lin, Jiahang, et al.
Publicado: (2026)

Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation
por: Wang, Yikun, et al.
Publicado: (2023)

LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding
por: Zhang, Shen, et al.
Publicado: (2025)

Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control
por: Jiang, Changhao, et al.
Publicado: (2026)

RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
por: Ye, Junjie, et al.
Publicado: (2024)