Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.15575 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909357223968768 |
|---|---|
| author | Zhang, Yanxiang Zhang, Yuanbo Sun, Haicheng Wang, Yun Dou, Billy Sivek, Gary Zhai, Shumin |
| author_facet | Zhang, Yanxiang Zhang, Yuanbo Sun, Haicheng Wang, Yun Dou, Billy Sivek, Gary Zhai, Shumin |
| contents | Gboard Decoder produces suggestions by looking for paths that best match input touch points on the context aware search space, which is backed by the language Finite State Transducers (FST). The language FST is currently an N-gram language model (LM). However, N-gram LMs, limited in context length, are known to have sparsity problem under device model size constraint. In this paper, we propose \textbf{Neural Search Space} which substitutes the N-gram LM with a Neural Network LM (NN-LM) and dynamically constructs the search space during decoding. Specifically, we integrate the long range context awareness of NN-LM into the search space by converting its outputs given context, into the language FST at runtime. This involves language FST structure redesign, pruning strategy tuning, and data structure optimizations. Online experiments demonstrate improved quality results, reducing Words Modified Ratio by [0.26\%, 1.19\%] on various locales with acceptable latency increases. This work opens new avenues for further improving keyboard decoding quality by enhancing neural LM more directly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15575 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Neural Search Space in Gboard Decoder Zhang, Yanxiang Zhang, Yuanbo Sun, Haicheng Wang, Yun Dou, Billy Sivek, Gary Zhai, Shumin Computation and Language Gboard Decoder produces suggestions by looking for paths that best match input touch points on the context aware search space, which is backed by the language Finite State Transducers (FST). The language FST is currently an N-gram language model (LM). However, N-gram LMs, limited in context length, are known to have sparsity problem under device model size constraint. In this paper, we propose \textbf{Neural Search Space} which substitutes the N-gram LM with a Neural Network LM (NN-LM) and dynamically constructs the search space during decoding. Specifically, we integrate the long range context awareness of NN-LM into the search space by converting its outputs given context, into the language FST at runtime. This involves language FST structure redesign, pruning strategy tuning, and data structure optimizations. Online experiments demonstrate improved quality results, reducing Words Modified Ratio by [0.26\%, 1.19\%] on various locales with acceptable latency increases. This work opens new avenues for further improving keyboard decoding quality by enhancing neural LM more directly. |
| title | Neural Search Space in Gboard Decoder |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.15575 |