DNACHUNKER: Learnable Tokenization for DNA Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914581084897280 |
|---|---|
| author | Kim, Taewon Shin, Jihwan Kim, Hyomin Jung, Youngmok Lee, Jonghoon Lee, Won-Chul Ahn, Sungsoo Han, Insu |
| author_facet | Kim, Taewon Shin, Jihwan Kim, Hyomin Jung, Youngmok Lee, Jonghoon Lee, Won-Chul Ahn, Sungsoo Han, Insu |
| contents | DNA language models are increasingly used to represent genomic sequence, yet their effectiveness depends critically on how raw nucleotides are converted into model inputs. Unlike natural language, DNA offers no canonical boundaries, making fixed tokenizations a brittle design choice under shifts, indels, and local repeats. We introduce DNAChunker, a masked DNA language model that incorporates a learnable adaptive segmentation module to produce context-dependent, variable-length units. Building on a dynamic segmentation procedure, DNAChunker learns to allocate finer granularity to functionally enriched regions while compressing repetitive or redundant sequence. We pretrain DNAChunker on the human reference genome and evaluate it across five benchmarks, where it consistently improves over strong fixed-tokenization baselines. Further analyses and ablations indicate that unlike fixed tokenizations, segmentation is learned in a biologically-informed, mutation-resilient manner. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03019 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DNACHUNKER: Learnable Tokenization for DNA Language Models Kim, Taewon Shin, Jihwan Kim, Hyomin Jung, Youngmok Lee, Jonghoon Lee, Won-Chul Ahn, Sungsoo Han, Insu Genomics Computation and Language DNA language models are increasingly used to represent genomic sequence, yet their effectiveness depends critically on how raw nucleotides are converted into model inputs. Unlike natural language, DNA offers no canonical boundaries, making fixed tokenizations a brittle design choice under shifts, indels, and local repeats. We introduce DNAChunker, a masked DNA language model that incorporates a learnable adaptive segmentation module to produce context-dependent, variable-length units. Building on a dynamic segmentation procedure, DNAChunker learns to allocate finer granularity to functionally enriched regions while compressing repetitive or redundant sequence. We pretrain DNAChunker on the human reference genome and evaluate it across five benchmarks, where it consistently improves over strong fixed-tokenization baselines. Further analyses and ablations indicate that unlike fixed tokenizations, segmentation is learned in a biologically-informed, mutation-resilient manner. |
| title | DNACHUNKER: Learnable Tokenization for DNA Language Models |
| topic | Genomics Computation and Language |
| url | https://arxiv.org/abs/2601.03019 |