Cover Image

SampoNLP: A Self-Referential Toolkit for Morphological Analysis of Subword Tokenizers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chelombitko, Iaroslav, Chelombitko, Ekaterina, Komissarov, Aleksey
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Information Retrieval Machine Learning I.2.7; I.2.6; H.3.1
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer Quality in Large Language Models
by: Chelombitko, Iaroslav, et al.
Published: (2024)

Subword-Based Comparative Linguistics across 242 Languages Using Wikipedia Glottosets
by: Chelombitko, Iaroslav, et al.
Published: (2026)

Compressed code: the hidden effects of quantization and distillation on programming tokens
by: Siniaev, Viacheslav, et al.
Published: (2026)

What Matters in LLM-Based Feature Extractor for Recommender? A Systematic Analysis of Prompts, Models, and Adaptation
by: Shi, Kainan, et al.
Published: (2025)

IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text
by: Pall, Rajveer Singh
Published: (2026)

ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection
by: Guo, Dongxin, et al.
Published: (2026)

LLM Reasoning for Cold-Start Item Recommendation
by: Li, Shijun, et al.
Published: (2025)

Tokens with Meaning: A Hybrid Tokenization Approach for Turkish
by: Bayram, M. Ali, et al.
Published: (2025)

DEUCE: Dual-diversity Enhancement and Uncertainty-awareness for Cold-start Active Learning
by: Guo, Jiaxin, et al.
Published: (2025)

ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks
by: Tanguturi, Samuel Sameer
Published: (2026)

ATANT: An Evaluation Framework for AI Continuity
by: Tanguturi, Samuel Sameer
Published: (2026)

LLMs as Architects and Critics for Multi-Source Opinion Summarization
by: Attri, Anuj, et al.
Published: (2025)

Why We Feel What We Feel: Joint Detection of Emotions and Their Opinion Triggers in E-commerce
by: Attri, Arnav, et al.
Published: (2025)

HiPS: Hierarchical PDF Segmentation of Textbooks
by: Wehnert, Sabine, et al.
Published: (2025)

IMDMR: An Intelligent Multi-Dimensional Memory Retrieval System for Enhanced Conversational AI
by: Pawar, Tejas, et al.
Published: (2025)

Improving Large-Scale k-Nearest Neighbor Text Categorization with Label Autoencoders
by: Ribadas-Pena, Francisco J., et al.
Published: (2024)

MoVoC: Morphology-Aware Subword Construction for Geez Script Languages
by: Teklehaymanot, Hailay Kidu, et al.
Published: (2025)

HySemRAG: A Hybrid Semantic Retrieval-Augmented Generation Framework for Automated Literature Synthesis and Methodological Gap Analysis
by: Godinez, Alejandro
Published: (2025)

PRISM: LLM-Guided Semantic Clustering for High-Precision Topics
by: Douglas, Connor, et al.
Published: (2026)

Leveraging Large Language Models to Extract and Translate Medical Information in Doctors' Notes for Health Records and Diagnostic Billing Codes
by: Hartnett, Peter, et al.
Published: (2026)

The Syntactic Acceptability Dataset (Preview): A Resource for Machine Learning and Linguistic Analysis of English
by: Juzek, Tom S
Published: (2025)

Suppressing Domain-Specific Hallucination in Construction LLMs: A Knowledge Graph Foundation for GraphRAG and QLoRA on River and Sediment Control Technical Standards
by: Yasuno, Takato
Published: (2026)

Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems
by: Hua, Kai, et al.
Published: (2025)

Improving the Performance of Sequential Recommendation Systems with an Extended Large Language Model
by: Choi, Sinnyum, et al.
Published: (2025)

PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text Embedding
by: Ayaou, Iliass, et al.
Published: (2025)

Real-Time RAG for the Identification of Supply Chain Vulnerabilities
by: Ponnock, Jesse, et al.
Published: (2025)

Mind the Gap: A Generalized Approach for Cross-Modal Embedding Alignment
by: Yadav, Arihan, et al.
Published: (2024)

Survey Transfer Learning: Recycling Data with Silicon Responses
by: Amini, Ali
Published: (2025)

Tensor Manifold-Based Graph-Vector Fusion for AI-Native Academic Literature Retrieval
by: Wei, Xing, et al.
Published: (2026)

Tokenization Standards for Linguistic Integrity: Turkish as a Benchmark
by: Bayram, M. Ali, et al.
Published: (2025)

SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation
by: Qiu, Jingxi, et al.
Published: (2026)

CogCanvas: Verbatim-Grounded Artifact Extraction for Long LLM Conversations
by: An, Tao
Published: (2025)

Graph Your Way to Inspiration: Integrating Co-Author Graphs with Retrieval-Augmented Generation for Large Language Model Based Scientific Idea Generation
by: Xie, Pengzhen, et al.
Published: (2025)

AI-Friendly LaTeX: Using LaTeX Code as a Knowledge Source for Retrieval-Augmented Generation
by: Verhoeff, Tom
Published: (2026)

On Self-improving Token Embeddings
by: Kubek, Mario M., et al.
Published: (2025)

A Scalable and High Availability Solution for Recommending Resolutions to Problem Tickets
by: Saragadam, Harish, et al.
Published: (2025)

Retrieval Augmented Classification for Confidential Documents
by: Chang, Yeseul E., et al.
Published: (2026)

TrueReason: An Exemplar Personalised Learning System Integrating Reasoning with Foundational Models
by: Bulathwela, Sahan, et al.
Published: (2025)

Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems
by: Meghwani, Hansa, et al.
Published: (2025)

Combating data scarcity in recommendation services: Integrating cognitive types of VARK and neural network technologies (LLM)
by: Zmanovskii, Nikita
Published: (2026)