Saved in:
| Main Author: | |
|---|---|
| Format: | Artículo científico |
| Language: | en |
| Published: |
Universidad de Costa Rica
2024
|
| Subjects: | |
| Online Access: | https://www.redalyc.org/articulo.oa?id=44776288019 https://www.redalyc.org/journal/447/44776288019/ https://www.redalyc.org/journal/447/44776288019/html/ https://www.redalyc.org/journal/447/44776288019/44776288019.epub https://www.redalyc.org/journal/447/44776288019/movil https://doi.org/10.15517/aie.v24i1.55612 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866812430346092544 |
|---|---|
| author | William Charpentier-Jiménez |
| author_facet | William Charpentier-Jiménez |
| contents | Assessing artificial intelligence and professors’ calibration in English as a foreign language writing courses at a Costa Rican public university William Charpentier-Jiménez Educación assessment higher education writing (composition) Artificial intelligence second language instruction This article paper explores the evaluation of artificial intelligence (AI) in English as a Foreign Language (EFL) writing courses and the importance of calibration in writing evaluations. The role of calibration has received little attention in language contexts, while the role of artificial intelligence has gained increased attention in the last couple of years. This investigation, conducted from August 2022 to March 2023, involved eight TESOL students enrolled in an English as a Foreign Language (EFL) major at a Costa Rican public university, ten TESOL university professors, and one AI piece of software. It used a quantitative, quasi-experimental design, and a language elicitation data collection process. Data was collected by means of a rubric-based writing assessment. Quantitative data were analyzed using descriptive statistics. Data analyses indicate that: 1) human-created paragraphs (X̄ = 7,56) and AI writing (X̄ = 7,61) yield similar results when evaluated; 2) some criteria may favor human creativity or computer, rule-oriented writing; and 3) professors’ ratings reveal inconsistencies when grading human writing in particular. These findings demonstrate that AI matches, at least to a basic level, human writing skills. Furthermore, data show that students may be falling behind in aspects such as grammar, vocabulary, and mechanics. Finally, the analysis indicates that professors’ grading lacks consistency, and a calibration model should be incorporated as part of regular training workshops. 2024 artículo científico 1409-4703 https://www.redalyc.org/articulo.oa?id=44776288019 https://www.redalyc.org/journal/447/44776288019/ https://www.redalyc.org/journal/447/44776288019/html/ https://www.redalyc.org/journal/447/44776288019/44776288019.epub https://www.redalyc.org/journal/447/44776288019/movil https://doi.org/10.15517/aie.v24i1.55612 en http://www.redalyc.org/revista.oa?id=447 Revista Electrónica "Actualidades Investigativas en Educación" application/pdf Universidad de Costa Rica Revista Electrónica "Actualidades Investigativas en Educación" (Costa Rica) Num.1 Vol.24 |
| format | Artículo científico |
| id | redalyc_44776288019 |
| language | en |
| publishDate | 2024 |
| publisher | Universidad de Costa Rica |
| spellingShingle | Assessing artificial intelligence and professors’ calibration in English as a foreign language writing courses at a Costa Rican public university William Charpentier-Jiménez Educación assessment higher education writing (composition) Artificial intelligence second language instruction Assessing artificial intelligence and professors’ calibration in English as a foreign language writing courses at a Costa Rican public university William Charpentier-Jiménez Educación assessment higher education writing (composition) Artificial intelligence second language instruction This article paper explores the evaluation of artificial intelligence (AI) in English as a Foreign Language (EFL) writing courses and the importance of calibration in writing evaluations. The role of calibration has received little attention in language contexts, while the role of artificial intelligence has gained increased attention in the last couple of years. This investigation, conducted from August 2022 to March 2023, involved eight TESOL students enrolled in an English as a Foreign Language (EFL) major at a Costa Rican public university, ten TESOL university professors, and one AI piece of software. It used a quantitative, quasi-experimental design, and a language elicitation data collection process. Data was collected by means of a rubric-based writing assessment. Quantitative data were analyzed using descriptive statistics. Data analyses indicate that: 1) human-created paragraphs (X̄ = 7,56) and AI writing (X̄ = 7,61) yield similar results when evaluated; 2) some criteria may favor human creativity or computer, rule-oriented writing; and 3) professors’ ratings reveal inconsistencies when grading human writing in particular. These findings demonstrate that AI matches, at least to a basic level, human writing skills. Furthermore, data show that students may be falling behind in aspects such as grammar, vocabulary, and mechanics. Finally, the analysis indicates that professors’ grading lacks consistency, and a calibration model should be incorporated as part of regular training workshops. 2024 artículo científico 1409-4703 https://www.redalyc.org/articulo.oa?id=44776288019 https://www.redalyc.org/journal/447/44776288019/ https://www.redalyc.org/journal/447/44776288019/html/ https://www.redalyc.org/journal/447/44776288019/44776288019.epub https://www.redalyc.org/journal/447/44776288019/movil https://doi.org/10.15517/aie.v24i1.55612 en http://www.redalyc.org/revista.oa?id=447 Revista Electrónica "Actualidades Investigativas en Educación" application/pdf Universidad de Costa Rica Revista Electrónica "Actualidades Investigativas en Educación" (Costa Rica) Num.1 Vol.24 |
| title | Assessing artificial intelligence and professors’ calibration in English as a foreign language writing courses at a Costa Rican public university |
| topic | Educación assessment higher education writing (composition) Artificial intelligence second language instruction |
| url | https://www.redalyc.org/articulo.oa?id=44776288019 https://www.redalyc.org/journal/447/44776288019/ https://www.redalyc.org/journal/447/44776288019/html/ https://www.redalyc.org/journal/447/44776288019/44776288019.epub https://www.redalyc.org/journal/447/44776288019/movil https://doi.org/10.15517/aie.v24i1.55612 |