Cover Image

SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dragunov, Nikita, Rahmatullaev, Temurbek, Goncharova, Elizaveta, Kurdiukov, Nikita, Mirzoeva, Aysel, Borisiuk, Anna, Kuznetsov, Andrey, Razzhigaev, Anton
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
by: Razzhigaev, Anton, et al.
Published: (2025)

Universal Adversarial Attack on Aligned Multimodal LLMs
by: Rahmatullaev, Temurbek, et al.
Published: (2025)

Logit-KL Flow Matching: Non-Autoregressive Text Generation via Sampling-Hybrid Inference
by: Sevriugov, Egor, et al.
Published: (2024)

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
by: Chekalina, Viktoriia, et al.
Published: (2024)

Sentence-Anchored Gist Compression for Long-Context LLMs
by: Tarasov, Dmitrii, et al.
Published: (2025)

The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
by: Razzhigaev, Anton, et al.
Published: (2023)

Your Transformer is Secretly Linear
by: Razzhigaev, Anton, et al.
Published: (2024)

NoReGeo: Non-Reasoning Geometry Benchmark
by: Abdullaeva, Irina, et al.
Published: (2026)

Back to Basics: Revisiting Exploration in Reinforcement Learning for LLM Reasoning via Generative Probabilities
by: Li, Pengyi, et al.
Published: (2026)

Heterogeneous LLM Methods for Ontology Learning (Few-Shot Prompting, Ensemble Typing, and Attention-Based Taxonomies)
by: Beliaeva, Aleksandra, et al.
Published: (2025)

Calibri: Enhancing Diffusion Transformers via Parameter-Efficient Calibration
by: Tokhchukov, Danil, et al.
Published: (2026)

When Less is Enough: Adaptive Token Reduction for Efficient Image Representation
by: Allakhverdov, Eduard, et al.
Published: (2025)

MindShift: Analyzing Language Models' Reactions to Psychological Prompts
by: Vasiliuk, Anton, et al.
Published: (2025)

Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuning
by: Borisiuk, Anna, et al.
Published: (2026)

Simple Vision-Language Math Reasoning via Rendered Text
by: Skripkin, Matvey, et al.
Published: (2025)

Polyhedral products, graph products and p-central series
by: Panov, Taras, et al.
Published: (2024)

SONAR-SLT: Multilingual Sign Language Translation via Language-Agnostic Sentence Embedding Supervision
by: Hamidullah, Yasser, et al.
Published: (2025)

Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech
by: Omnilingual SONAR Team, et al.
Published: (2026)

OmniFusion Technical Report
by: Goncharova, Elizaveta, et al.
Published: (2024)

Image Reconstruction as a Tool for Feature Analysis
by: Allakhverdov, Eduard, et al.
Published: (2025)

MOVE: A Mixture-of-Vision-Encoders Approach for Domain-Focused Vision-Language Processing
by: Skripkin, Matvey, et al.
Published: (2025)

MERA: A Comprehensive LLM Evaluation in Russian
by: Fenogenova, Alena, et al.
Published: (2024)

Incremental Sentence Processing Mechanisms in Autoregressive Transformer Language Models
by: Hanna, Michael, et al.
Published: (2024)

Quantum-Inspired Self-Attention in a Large Language Model
by: Kuznetsov, Nikita, et al.
Published: (2026)

LLM-Independent Adaptive RAG: Let the Question Speak for Itself
by: Marina, Maria, et al.
Published: (2025)

Real-World Transferable Adversarial Attack on Face-Recognition Systems
by: Kaznacheev, Andrey, et al.
Published: (2025)

Hierarchical Embedding Fusion for Retrieval-Augmented Code Generation
by: Sorokin, Nikita, et al.
Published: (2026)

Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding
by: Salama, Rana, et al.
Published: (2025)

GigaEmbeddings: Efficient Russian Language Embedding Model
by: Kolodin, Egor, et al.
Published: (2025)

When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning
by: Wei, Jiaqi, et al.
Published: (2026)

SCI-IDEA: Context-Aware Scientific Ideation Using Token and Sentence Embeddings
by: Keya, Farhana, et al.
Published: (2025)

I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
by: Galichin, Andrey, et al.
Published: (2025)

Feature-Level Insights into Artificial Text Detection with Sparse Autoencoders
by: Kuznetsov, Kristian, et al.
Published: (2025)

Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform
by: Salama, Rana Aref, et al.
Published: (2025)

Memory Tokens: Large Language Models Can Generate Reversible Sentence Embeddings
by: Sastre, Ignacio, et al.
Published: (2025)

Think in Sentences: Explicit Sentence Boundaries Enhance Language Model's Capabilities
by: Liu, Zhichen, et al.
Published: (2026)

Space Decomposition for Sentence Embedding
by: Ponwitayarat, Wuttikorn, et al.
Published: (2024)

Text Simplification with Sentence Embeddings
by: Shardlow, Matthew
Published: (2025)

Combating data scarcity in recommendation services: Integrating cognitive types of VARK and neural network technologies (LLM)
by: Zmanovskii, Nikita
Published: (2026)

Scalable Parameter-Light Spectral Method for Clustering Short Text Embeddings with a Cohesion-Based Evaluation Metric
by: Neveditsin, Nikita, et al.
Published: (2025)