Cover Image

Unsupervised Morphological Tree Tokenizer

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Qingyang, Hu, Xiang, Ji, Pengyu, Wu, Wei, Tu, Kewei
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale
by: Hu, Xiang, et al.
Published: (2024)

Augmenting Transformers with Recursively Composed Multi-grained Representations
by: Hu, Xiang, et al.
Published: (2023)

Flash Multi-Head Feed-Forward Network
by: Zhang, Minshen, et al.
Published: (2025)

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
by: Lou, Chao, et al.
Published: (2024)

Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
by: Wang, Guoli, et al.
Published: (2026)

D$^{3}$ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMs
by: Chang, Shuochen, et al.
Published: (2025)

Investigating the Synergistic Effects of Dropout and Residual Connections on Language Model Training
by: Li, Qingyang, et al.
Published: (2024)

From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Prediction
by: Zhu, Mingcheng, et al.
Published: (2026)

Rethinking Token Prediction: Tree-Structured Diffusion Language Model
by: Wu, Zihao, et al.
Published: (2026)

Zero-Shot Morphological Discovery in Low-Resource Bantu Languages via Cross-Lingual Transfer and Unsupervised Clustering
by: Mutisya, Hillary, et al.
Published: (2026)

DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards
by: Zhang, Kaiyi, et al.
Published: (2026)

TokenShapley: Token Level Context Attribution with Shapley Value
by: Xiao, Yingtai, et al.
Published: (2025)

Replacing Language Model for Style Transfer
by: Cheng, Pengyu, et al.
Published: (2022)

Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
by: Huang, Hongzhi, et al.
Published: (2025)

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models
by: Zhang, Yang, et al.
Published: (2025)

Not All Tokens Matter Equally: Dynamic In-context Vector Distillation with Decisive-Token Supervision for Long-form Medical Report Generation
by: Wu, Ning, et al.
Published: (2026)

Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
by: Hu, Zhengmian, et al.
Published: (2023)

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability
by: Lin, Zicheng, et al.
Published: (2024)

STS: Efficient Sparse Attention with Speculative Token Sparsity
by: Xu, Ceyu, et al.
Published: (2026)

Attention with Trained Embeddings Provably Selects Important Tokens
by: Wu, Diyuan, et al.
Published: (2025)

$\textbf{AGT$^{AO}$}$: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality
by: Li, Pengyu, et al.
Published: (2026)

Scaling Multi-Hop Training Data via Graph-Constrained Path Selection
by: Chen, Pengyu, et al.
Published: (2026)

Token Distillation: Attention-aware Input Embeddings For New Tokens
by: Dobler, Konstantin, et al.
Published: (2025)

X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
by: Sreenivas, Sharath Turuvekere, et al.
Published: (2026)

SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularization
by: Wu, Jianghao, et al.
Published: (2025)

Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization
by: Gong, Zixuan, et al.
Published: (2025)

Rethinking Token Reduction for State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
by: Qin, Zongyue, et al.
Published: (2024)

Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning
by: Feng, Weitao, et al.
Published: (2025)

ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding
by: Zhong, Shuzhang, et al.
Published: (2024)

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
by: He, Yinhan, et al.
Published: (2026)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
by: Cao, Jinghan, et al.
Published: (2026)

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
by: Jo, Dongwon, et al.
Published: (2026)

Unsupervised Location Mapping for Narrative Corpora
by: Wagner, Eitan, et al.
Published: (2025)

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
by: Bae, Sangmin, et al.
Published: (2025)

TIDE: Token-Informed Depth Execution for Per-Token Early Exit in LLM Inference
by: Jaber, Jaber, et al.
Published: (2026)

Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
by: Luo, Xianzhen, et al.
Published: (2024)

How Far Can Unsupervised RLVR Scale LLM Training?
by: He, Bingxiang, et al.
Published: (2026)

ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Models
by: Zheng, Kangjie, et al.
Published: (2025)

Ensemble Distillation for Unsupervised Constituency Parsing
by: Shayegh, Behzad, et al.
Published: (2023)