ChatGPT, Bard, Bing Chat, and Claude generate feedback for Chinese as foreign language writing: A comparative case study
Fuente:
Wiley Open Access
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Artículo Open Access |
| Publicado: |
Wiley
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1867008330465017857 |
|---|---|
| author | Saleh Obaidoon Haiping Wei |
| author_facet | Saleh Obaidoon Haiping Wei Saleh Obaidoon Haiping Wei |
| collection | Wiley Open Access |
| contents | ChatGPT, Bard, Bing Chat, and Claude generate feedback for Chinese as foreign language writing: A comparative case study Saleh Obaidoon Haiping Wei Future in Educational Research AbstractThis comparative case study analyzes and evaluates the performance of four prevalent artificial intelligence (AI) models―ChatGPT, Google Bard, Microsoft Bing, and Claude―in generating feedback on Chinese as a Foreign Language writing. The study assessed the models' effectiveness, accuracy, alignment with pedagogical principles, and cultural appropriateness through a multi‐faceted data collection process involving student article writing, chatbot feedback, and teacher evaluation. The quantitative analysis of teacher ratings indicates that Claude demonstrated the highest average alignment with human instructor scores across the four articles, followed by Google Bard. Qualitative examination reveals differences in the types of feedback provided, with models excelling at surface‐level vocabulary, grammar, and mechanics critiques but limited in providing rhetorical, pragmatic, and structural feedback compared to teachers. While showing potential benefits, judicious integration of AI writing feedback tools upholding academic integrity is advised. This paper utilizes non‐Pro subscription plans for its research, ensuring accessibility by teachers or students without any cost. The date of access for these chatbots was September 20, 2023. The AI models used include ChatGPT based on OpenAI's GPT‐3.5 architecture with a knowledge cut‐off in January 2022, without Internet browsing capabilities; Google Bard from the Gemini family, version 1.0, which integrates internet‐based search; Microsoft Copilot (Balanced mode), which evolved from Bing Chat, providing information and content generation; and Claude version 2. This approach ensures the study's findings are applicable and replicable for educators and students utilizing freely available resources. 10.1002/fer3.39 http://creativecommons.org/licenses/by/4.0/ |
| doi_str_mv | 10.1002/fer3.39 |
| format | Artículo Open Access |
| id | wiley_oa_10_1002_fer3_39 |
| institution | Wiley Open Access |
| license_str_mv | http://creativecommons.org/licenses/by/4.0/ |
| publishDate | 2024 |
| publisher | Wiley |
| record_format | wiley_oa |
| spellingShingle | ChatGPT, Bard, Bing Chat, and Claude generate feedback for Chinese as foreign language writing: A comparative case study Saleh Obaidoon Haiping Wei Future in Educational Research ChatGPT, Bard, Bing Chat, and Claude generate feedback for Chinese as foreign language writing: A comparative case study Saleh Obaidoon Haiping Wei Future in Educational Research AbstractThis comparative case study analyzes and evaluates the performance of four prevalent artificial intelligence (AI) models―ChatGPT, Google Bard, Microsoft Bing, and Claude―in generating feedback on Chinese as a Foreign Language writing. The study assessed the models' effectiveness, accuracy, alignment with pedagogical principles, and cultural appropriateness through a multi‐faceted data collection process involving student article writing, chatbot feedback, and teacher evaluation. The quantitative analysis of teacher ratings indicates that Claude demonstrated the highest average alignment with human instructor scores across the four articles, followed by Google Bard. Qualitative examination reveals differences in the types of feedback provided, with models excelling at surface‐level vocabulary, grammar, and mechanics critiques but limited in providing rhetorical, pragmatic, and structural feedback compared to teachers. While showing potential benefits, judicious integration of AI writing feedback tools upholding academic integrity is advised. This paper utilizes non‐Pro subscription plans for its research, ensuring accessibility by teachers or students without any cost. The date of access for these chatbots was September 20, 2023. The AI models used include ChatGPT based on OpenAI's GPT‐3.5 architecture with a knowledge cut‐off in January 2022, without Internet browsing capabilities; Google Bard from the Gemini family, version 1.0, which integrates internet‐based search; Microsoft Copilot (Balanced mode), which evolved from Bing Chat, providing information and content generation; and Claude version 2. This approach ensures the study's findings are applicable and replicable for educators and students utilizing freely available resources. 10.1002/fer3.39 http://creativecommons.org/licenses/by/4.0/ |
| title | ChatGPT, Bard, Bing Chat, and Claude generate feedback for Chinese as foreign language writing: A comparative case study |
| topic | Future in Educational Research |
| url | https://onlinelibrary.wiley.com/doi/10.1002/fer3.39 |