Cover Image

Revisiting subword tokenization: A case study on affixal negation in large language models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Truong, Thinh Hung, Otmakhova, Yulia, Verspoor, Karin, Cohn, Trevor, Baldwin, Timothy
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning Robust Negation Text Representations
by: Truong, Thinh Hung, et al.
Published: (2025)

Comparative analysis of subword tokenization approaches for Indian languages
by: Das, Sudhansu Bala, et al.
Published: (2025)

FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluation
by: Otmakhova, Yulia, et al.
Published: (2025)

Narrative Media Framing in Political Discourse
by: Otmakhova, Yulia, et al.
Published: (2025)

Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation
by: Dasgupta, Sayantan, et al.
Published: (2026)

Zero‐ and few‐shot prompting of generative large language models provides weak assessment of risk of bias in clinical trials
by: Simon Šuster, et al.
Published: (2024)

Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval
by: Yang, Jinrui, et al.
Published: (2023)

Not all ANIMALs are equal: metaphorical framing through source domains and semantic frames
by: Otmakhova, Yulia, et al.
Published: (2026)

Morphological evaluation of subwords vocabulary used by BETO language model
by: García-Sierra, Óscar, et al.
Published: (2024)

Retain or Reframe? A Computational Framework for the Analysis of Framing in News Articles and Reader Comments
by: Guida, Matteo, et al.
Published: (2025)

Collaborative decoding of critical tokens for boosting factuality of large language models
by: Jin, Lifeng, et al.
Published: (2024)

EMBRE: Entity-aware Masking for Biomedical Relation Extraction
by: Li, Mingjie, et al.
Published: (2024)

LLMs for Argument Mining: Detection, Extraction, and Relationship Classification of pre-defined Arguments in Online Comments
by: Guida, Matteo, et al.
Published: (2025)

Article and Comment Frames Shape the Quality of Online Comments
by: Guida, Matteo, et al.
Published: (2026)

Generative Debunking of Climate Misinformation
by: Zanartu, Francisco, et al.
Published: (2024)

AtteSTNet -- An attention and subword tokenization based approach for code-switched text hate speech detection
by: Shingi, Geet, et al.
Published: (2021)

Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation
by: Huang, Sukai, et al.
Published: (2024)

Retrieval augmentation of large language models for lay language generation
by: Guo, Yue, et al.
Published: (2022)

On state complexity for subword-closed languages
by: Guyot, Jérôme
Published: (2024)

Do language models plan ahead for future tokens?
by: Wu, Wilson, et al.
Published: (2024)

Principles from Clinical Research for NLP Model Generalization
by: Elangovan, Aparna, et al.
Published: (2023)

Visualizing token importance for black-box language models
by: Rauba, Paulius, et al.
Published: (2025)

LoraxBench: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages
by: Aji, Alham Fikri, et al.
Published: (2025)

How Robust Are Large Language Models for Clinical Numeracy? An Empirical Study on Numerical Reasoning Abilities in Clinical Contexts
by: Nguyen, Minh-Vuong, et al.
Published: (2026)

On the scaling relationship between cloze probabilities and language model next-token prediction
by: Jacobs, Cassandra L., et al.
Published: (2026)

Gemination and degemination in English affixation
by: Ben Hedia, Sonia
Published: (2020)

DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysis
by: Georgiou, Efthymios, et al.
Published: (2025)

RADS: Reinforcement Learning-Based Sample Selection Improves Transfer Learning in Low-resource and Imbalanced Clinical Settings
by: Han, Wei, et al.
Published: (2026)

Explanation sensitivity to the randomness of large language models: the case of journalistic text classification
by: Bogaert, Jeremie, et al.
Published: (2024)

Disambiguating Complexity: From CAF to CAFIC: A Commentary on “Complexity and Difficulty in Second Language Acquisition: A Theoretical and Methodological Overview”
by: Marjolijn Verspoor
Published: (2024)

Are LLM-generated plain language summaries truly understandable? A large-scale crowdsourced evaluation
by: Guo, Yue, et al.
Published: (2025)

Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited
by: Cohn, Anthony G, et al.
Published: (2025)

Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision
by: Jiang, Fan, et al.
Published: (2024)

Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
by: Bandarkar, Lucas, et al.
Published: (2026)

Few-Shot Multilingual Open-Domain QA from 5 Examples
by: Jiang, Fan, et al.
Published: (2025)

Do large language models resemble humans in language use?
by: Cai, Zhenguang G., et al.
Published: (2023)

A Little Leak Will Sink a Great Ship: Survey of Transparency for Large Language Models from Start to Finish
by: Kaneko, Masahiro, et al.
Published: (2024)

Dissociating language and thought in large language models
by: Mahowald, Kyle, et al.
Published: (2023)

Is Sanskrit the most token-efficient language? A quantitative study using GPT, Gemini, and SentencePiece
by: Kumar, Anshul
Published: (2026)

Language-Specific Latent Process Hinders Cross-Lingual Performance
by: Lim, Zheng Wei, et al.
Published: (2025)