Copertina

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Lee, Philip Heejun
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language Neural and Evolutionary Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space
di: Liu, Houjun, et al.
Pubblicazione: (2025)

GLU Attention Improve Transformer
di: Wang, Zehao
Pubblicazione: (2025)

A Transformer-based Neural Architecture Search Method
di: Wang, Shang, et al.
Pubblicazione: (2025)

NOBLE: Accelerating Transformers with Nonlinear Low-Rank Branches
di: Smith, Ethan
Pubblicazione: (2026)

Vector Policy Optimization: Training for Diversity Improves Test-Time Search
di: Bahlous-Boldi, Ryan, et al.
Pubblicazione: (2026)

Accelerating Training Speed of Tiny Recursive Models with Curriculum Guided Adaptive Recursion
di: Qasim, Kaleem Ullah, et al.
Pubblicazione: (2025)

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
di: Munkhdalai, Tsendsuren, et al.
Pubblicazione: (2024)

AgenticGEO: A Self-Evolving Agentic System for Generative Engine Optimization
di: Yuan, Jiaqi, et al.
Pubblicazione: (2026)

Towards evolution of Deep Neural Networks through contrastive Self-Supervised learning
di: Vinhas, Adriano, et al.
Pubblicazione: (2024)

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
di: Miranda, Brando, et al.
Pubblicazione: (2023)

Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives
di: Meyerson, Elliot, et al.
Pubblicazione: (2025)

Traveling Waves Encode the Recent Past and Enhance Sequence Learning
di: Keller, T. Anderson, et al.
Pubblicazione: (2023)

Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
di: Wang, Jianyu, et al.
Pubblicazione: (2025)

Position: Leverage Foundational Models for Black-Box Optimization
di: Song, Xingyou, et al.
Pubblicazione: (2024)

CAPO: Cost-Aware Prompt Optimization
di: Zehle, Tom, et al.
Pubblicazione: (2025)

W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models
di: Wang, Shang
Pubblicazione: (2025)

Topic Modelling Black Box Optimization
di: Akramov, Roman, et al.
Pubblicazione: (2025)

Language Models Learn Universal Representations of Numbers and Here's Why You Should Care
di: Štefánik, Michal, et al.
Pubblicazione: (2025)

Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
di: Parab, Aishni, et al.
Pubblicazione: (2025)

LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers
di: Abhyankar, Nikhil, et al.
Pubblicazione: (2025)

Benchmarking Randomized Optimization Algorithms on Binary, Permutation, and Combinatorial Problem Landscapes
di: Odeyemi, Jethro, et al.
Pubblicazione: (2025)

Elastic Architecture Search for Efficient Language Models
di: Wang, Shang
Pubblicazione: (2025)

Large Language Models and Emergence: A Complex Systems Perspective
di: Krakauer, David C., et al.
Pubblicazione: (2025)

Understanding Textual Emotion Through Emoji Prediction
di: Gordon, Ethan, et al.
Pubblicazione: (2025)

Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
di: Liang, Haoyu, et al.
Pubblicazione: (2025)

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
di: Bergsma, Shane, et al.
Pubblicazione: (2025)

Comply: Learning Sentences with Complex Weights inspired by Fruit Fly Olfaction
di: Figueroa, Alexei, et al.
Pubblicazione: (2025)

A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations
di: Javidnia, Hossein
Pubblicazione: (2026)

Interlocking-free Selective Rationalization Through Genetic-based Learning
di: Ruggeri, Federico, et al.
Pubblicazione: (2024)

Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces
di: Javidnia, Hossein
Pubblicazione: (2026)

Evolutionary Multi-Objective Optimization of Large Language Model Prompts for Balancing Sentiments
di: Baumann, Jill, et al.
Pubblicazione: (2024)

H-Node Attack and Defense in Large Language Models
di: Yocam, Eric, et al.
Pubblicazione: (2026)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
di: Zhang, Buze, et al.
Pubblicazione: (2026)

EvoGPT-f: An Evolutionary GPT Framework for Benchmarking Formal Math Languages
di: Mercer, Johnathan
Pubblicazione: (2024)

Compute Allocation in Evolutionary Search: From Depth-Breadth to Multi-Armed Bandits
di: Xing, Sixue, et al.
Pubblicazione: (2026)

Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
di: Qu, Youyang, et al.
Pubblicazione: (2024)

Hyperbolic Fine-Tuning for Large Language Models
di: Yang, Menglin, et al.
Pubblicazione: (2024)

MAR: Efficient Large Language Models via Module-aware Architecture Refinement
di: Cai, Junhong, et al.
Pubblicazione: (2026)

Fleet of Agents: Coordinated Problem Solving with Large Language Models
di: Klein, Lars, et al.
Pubblicazione: (2024)

Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
di: Petruzzellis, Flavio, et al.
Pubblicazione: (2024)