Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2211.13087 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911139632250880 |
|---|---|
| author | Zhang, Mengmi Pavarino, Elisa Liu, Xiao Dellaferrera, Giorgia Sikarwar, Ankur Chen, Caishun Armendariz, Marcelo Mudrik, Noga Agrawal, Prachi Madan, Spandan Shetty, Mranmay Barbu, Andrei Yang, Haochen Kumar, Tanishq Han, Shui'Er Singh, Aman Raj Sadwani, Meghna Dellaferrera, Stella Pizzochero, Michele Tang, Brandon Ong, Yew Soon Pfister, Hanspeter Kreiman, Gabriel |
| author_facet | Zhang, Mengmi Pavarino, Elisa Liu, Xiao Dellaferrera, Giorgia Sikarwar, Ankur Chen, Caishun Armendariz, Marcelo Mudrik, Noga Agrawal, Prachi Madan, Spandan Shetty, Mranmay Barbu, Andrei Yang, Haochen Kumar, Tanishq Han, Shui'Er Singh, Aman Raj Sadwani, Meghna Dellaferrera, Stella Pizzochero, Michele Tang, Brandon Ong, Yew Soon Pfister, Hanspeter Kreiman, Gabriel |
| contents | As AI becomes increasingly embedded in daily life, ascertaining whether an agent is human is critical. We systematically benchmark AI's ability to imitate humans in three language tasks (image captioning, word association, conversation) and three vision tasks (color estimation, object detection, attention prediction), collecting data from 636 humans and 37 AI agents. Next, we conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges. Current AIs are approaching the ability to convincingly impersonate humans and deceive human judges in both language and vision. Even simple AI judges outperformed humans in distinguishing AI from human responses. Imitation ability showed minimal correlation with conventional AI performance metrics, suggesting that passing as human is an important independent evaluation criterion. The large-scale Turing datasets and metrics introduced here offer valuable benchmarks for assessing human-likeness in AI and highlight the importance of rigorous, quantitative imitation tests for AI development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_13087 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap Zhang, Mengmi Pavarino, Elisa Liu, Xiao Dellaferrera, Giorgia Sikarwar, Ankur Chen, Caishun Armendariz, Marcelo Mudrik, Noga Agrawal, Prachi Madan, Spandan Shetty, Mranmay Barbu, Andrei Yang, Haochen Kumar, Tanishq Han, Shui'Er Singh, Aman Raj Sadwani, Meghna Dellaferrera, Stella Pizzochero, Michele Tang, Brandon Ong, Yew Soon Pfister, Hanspeter Kreiman, Gabriel Computer Vision and Pattern Recognition Artificial Intelligence As AI becomes increasingly embedded in daily life, ascertaining whether an agent is human is critical. We systematically benchmark AI's ability to imitate humans in three language tasks (image captioning, word association, conversation) and three vision tasks (color estimation, object detection, attention prediction), collecting data from 636 humans and 37 AI agents. Next, we conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges. Current AIs are approaching the ability to convincingly impersonate humans and deceive human judges in both language and vision. Even simple AI judges outperformed humans in distinguishing AI from human responses. Imitation ability showed minimal correlation with conventional AI performance metrics, suggesting that passing as human is an important independent evaluation criterion. The large-scale Turing datasets and metrics introduced here offer valuable benchmarks for assessing human-likeness in AI and highlight the importance of rigorous, quantitative imitation tests for AI development. |
| title | Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2211.13087 |