Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917800741699584 |
|---|---|
| author | Gao, Yingqiang Fischer, Lukas Lintner, Alexa Ebling, Sarah |
| author_facet | Gao, Yingqiang Fischer, Lukas Lintner, Alexa Ebling, Sarah |
| contents | Audio descriptions (ADs) function as acoustic commentaries designed to assist blind persons and persons with visual impairments in accessing digital media content on television and in movies, among other settings. As an accessibility service typically provided by trained AD professionals, the generation of ADs demands significant human effort, making the process both time-consuming and costly. Recent advancements in natural language processing (NLP) and computer vision (CV), particularly in large language models (LLMs) and vision-language models (VLMs), have allowed for getting a step closer to automatic AD generation. This paper reviews the technologies pertinent to AD generation in the era of LLMs and VLMs: we discuss how state-of-the-art NLP and CV technologies can be applied to generate ADs and identify essential research directions for the future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08860 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies Gao, Yingqiang Fischer, Lukas Lintner, Alexa Ebling, Sarah Computation and Language Computer Vision and Pattern Recognition Audio descriptions (ADs) function as acoustic commentaries designed to assist blind persons and persons with visual impairments in accessing digital media content on television and in movies, among other settings. As an accessibility service typically provided by trained AD professionals, the generation of ADs demands significant human effort, making the process both time-consuming and costly. Recent advancements in natural language processing (NLP) and computer vision (CV), particularly in large language models (LLMs) and vision-language models (VLMs), have allowed for getting a step closer to automatic AD generation. This paper reviews the technologies pertinent to AD generation in the era of LLMs and VLMs: we discuss how state-of-the-art NLP and CV technologies can be applied to generate ADs and identify essential research directions for the future. |
| title | Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies |
| topic | Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.08860 |