Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912010684334080 |
|---|---|
| author | Mayr, Martin Dreier, Marcel Kordon, Florian Seuret, Mathias Zöllner, Jochen Wu, Fei Maier, Andreas Christlein, Vincent |
| author_facet | Mayr, Martin Dreier, Marcel Kordon, Florian Seuret, Mathias Zöllner, Jochen Wu, Fei Maier, Andreas Christlein, Vincent |
| contents | The imitation of cursive handwriting is mainly limited to generating handwritten words or lines. Multiple synthetic outputs must be stitched together to create paragraphs or whole pages, whereby consistency and layout information are lost. To close this gap, we propose a method for imitating handwriting at the paragraph level that also works for unseen writing styles. Therefore, we introduce a modified latent diffusion model that enriches the encoder-decoder mechanism with specialized loss functions that explicitly preserve the style and content. We enhance the attention mechanism of the diffusion model with adaptive 2D positional encoding and the conditioning mechanism to work with two modalities simultaneously: a style image and the target text. This significantly improves the realism of the generated handwriting. Our approach sets a new benchmark in our comprehensive evaluation. It outperforms all existing imitation methods at both line and paragraph levels, considering combined style and content preservation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_00786 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models Mayr, Martin Dreier, Marcel Kordon, Florian Seuret, Mathias Zöllner, Jochen Wu, Fei Maier, Andreas Christlein, Vincent Computer Vision and Pattern Recognition The imitation of cursive handwriting is mainly limited to generating handwritten words or lines. Multiple synthetic outputs must be stitched together to create paragraphs or whole pages, whereby consistency and layout information are lost. To close this gap, we propose a method for imitating handwriting at the paragraph level that also works for unseen writing styles. Therefore, we introduce a modified latent diffusion model that enriches the encoder-decoder mechanism with specialized loss functions that explicitly preserve the style and content. We enhance the attention mechanism of the diffusion model with adaptive 2D positional encoding and the conditioning mechanism to work with two modalities simultaneously: a style image and the target text. This significantly improves the realism of the generated handwriting. Our approach sets a new benchmark in our comprehensive evaluation. It outperforms all existing imitation methods at both line and paragraph levels, considering combined style and content preservation. |
| title | Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.00786 |