Copertina

How Much is Enough? The Diminishing Returns of Tokenization Training Data

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Reddy, Varshini, Schmidt, Craig W., Pinter, Yuval, Tanner, Chris
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Computational Engineering, Finance, and Science
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Faster Superword Tokenization
di: Schmidt, Craig W., et al.
Pubblicazione: (2026)

Boundless Byte Pair Encoding: Breaking the Pre-tokenization Barrier
di: Schmidt, Craig W., et al.
Pubblicazione: (2025)

Tokenization with Split Trees
di: Schmidt, Craig W., et al.
Pubblicazione: (2026)

The Effect of Scripts and Formats on LLM Numeracy
di: Reddy, Varshini, et al.
Pubblicazione: (2026)

Greed is All You Need: An Evaluation of Tokenizer Inference Methods
di: Uzan, Omri, et al.
Pubblicazione: (2024)

Tokenization Is More Than Compression
di: Schmidt, Craig W., et al.
Pubblicazione: (2024)

Computerized Modeling of Electrophysiology and Pathoelectrophysiology of the Atria -- How Much Detail is Needed?
di: Dössel, Olaf, et al.
Pubblicazione: (2025)

Leveraging Large Language Models for Sentiment Analysis: Multi-Modal Analysis of Decentraland's MANA Token
di: Wu, Xintong, et al.
Pubblicazione: (2026)

Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
di: Huang, Jimin, et al.
Pubblicazione: (2024)

No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks
di: Hu, Gang, et al.
Pubblicazione: (2024)

All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection
di: Jiang, Yuechen, et al.
Pubblicazione: (2026)

UCFE: A User-Centric Financial Expertise Benchmark for Large Language Models
di: Yang, Yuzhe, et al.
Pubblicazione: (2024)

PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
di: Xiao, Hengbo, et al.
Pubblicazione: (2025)

When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs
di: Zagribelnyy, Bogdan, et al.
Pubblicazione: (2026)

SusGen-GPT: A Data-Centric LLM for Financial NLP and Sustainability Report Generation
di: Wu, Qilong, et al.
Pubblicazione: (2024)

Data-Driven Information Extraction and Enrichment of Molecular Profiling Data for Cancer Cell Lines
di: Smith, Ellery, et al.
Pubblicazione: (2023)

EXFormer: A Multi-Scale Trend-Aware Transformer with Dynamic Variable Selection for Foreign Exchange Returns Prediction
di: Liu, Dinggao, et al.
Pubblicazione: (2025)

Extracting the Structure of Press Releases for Predicting Earnings Announcement Returns
di: Wu, Yuntao, et al.
Pubblicazione: (2025)

The Hype Index: an NLP-driven Measure of Market News Attention
di: Cao, Zheng, et al.
Pubblicazione: (2025)

BERTopic-Driven Stock Market Predictions: Unraveling Sentiment Insights
di: Zhu, Enmin, et al.
Pubblicazione: (2024)

NumLLM: Numeric-Sensitive Large Language Model for Chinese Finance
di: Su, Huan-Yi, et al.
Pubblicazione: (2024)

Tokenizing Stock Prices for Enhanced Multi-Step Forecast and Prediction
di: Zhu, Zhuohang, et al.
Pubblicazione: (2025)

Tokenized but Illiquid? Evidence from Real-World Asset Markets
di: Mafrur, Rischan
Pubblicazione: (2026)

Decentralized Token Economy Theory (DeTEcT)
di: Sadykhov, Rem, et al.
Pubblicazione: (2023)

Impacts of Economic Policies on Wealth Distribution in Token Economies
di: Sadykhov, Rem, et al.
Pubblicazione: (2026)

Beyond TVL: An Explainable Risk Scoring Framework for Tokenized Real-World Assets
di: Mafrur, Rischan, et al.
Pubblicazione: (2026)

MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier
di: Yang, Zonglin, et al.
Pubblicazione: (2026)

Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow
di: Zhang, Wenqi, et al.
Pubblicazione: (2023)

A Role-Aware Multi-Agent Framework for Financial Education Question Answering with LLMs
di: Zhu, Andy, et al.
Pubblicazione: (2025)

Improved Methods for Model Pruning and Knowledge Distillation
di: Jiang, Wei, et al.
Pubblicazione: (2025)

FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
di: Tang, Zichen, et al.
Pubblicazione: (2025)

(G)I-DLE: Generative Inference via Distribution-preserving Logit Exclusion with KL Divergence Minimization for Constrained Decoding
di: Lee, Hanwool
Pubblicazione: (2025)

Cross-Asset Risk Management: Integrating LLMs for Real-Time Monitoring of Equity, Fixed Income, and Currency Markets
di: Yang, Jie, et al.
Pubblicazione: (2025)

Dynamic Hedging Strategies in Derivatives Markets with LLM-Driven Sentiment and News Analytics
di: Yang, Jie, et al.
Pubblicazione: (2025)

ZiGong 1.0: A Large Language Model for Financial Credit
di: Lei, Yu, et al.
Pubblicazione: (2025)

ERA-IT: Aligning Semantic Models with Revealed Economic Preference for Real-Time and Explainable Patent Valuation
di: Yoo, Yongmin, et al.
Pubblicazione: (2025)

FinGEAR: Financial Mapping-Guided Enhanced Answer Retrieval
di: Li, Ying, et al.
Pubblicazione: (2025)

Dissecting the Ledger: Locating and Suppressing "Liar Circuits" in Financial Large Language Models
di: Mirajkar, Soham
Pubblicazione: (2025)

Aligning Language Models with Real-time Knowledge Editing
di: Tang, Chenming, et al.
Pubblicazione: (2025)

SciNet: Evaluating AI Agents in Relation-Aware Scientific Literature Retrieval
di: Shao, Chenyang, et al.
Pubblicazione: (2025)