Automated speech analysis using machine learning methods for language competencies assessment in Russian as a foreign language exam
https://doi.org/10.32517/0234-0453-2025-40-1-82-94
Abstract
The article explores the application of computer and digital technologies in assessing students’ speech and communication skills in the Russian as a Foreign Language (RFL) exam, particularly in the “Speaking” subtest. The study analyzes existing approaches to automated speech assessment, identifying their advantages and limitations. Based on empirical research conducted on a sample of 120 students in 2022–2023, the effectiveness of using computer algorithms for analyzing prosodic, lexical, and grammatical characteristics of examinees’ speech is demonstrated. The application of machine learning methods allowed achieving a high correlation (r = 0,87; p < 0,01) between expert assessments and automated analysis results. Particular attention is paid to the creation of a three-factor model that takes into account speech fluency, lexical-grammatical and phonetic-prosodic characteristics. This model demonstrates high prediction accuracy (R2 = 0.79) and has successfully undergone cross-validation on an independent sample. The data obtained indicate the promising potential of integrating computer technologies into the system of speech skills assessment while maintaining the key role of expert evaluation. The research results are significant for improving RFL teaching methods and developing tools for objective assessment of communicative competence.
About the Authors
E. L. CherkashinaRussian Federation
Elena L. Cherkashina, Candidate of Sciences (Philology), Docent, Associate Professor at the Department of Russian as a Foreign Language and General Theoretical Subjects; Associate Professor at the Department of “Russian as a Foreign Language”, Faculty “Linguistics”
Moscow
R. I. Zhamaletdinov
Russian Federation
Rustam I. Zhamaletdinov, Lecturer at the Department of “Russian as a Foreign Language”, Faculty of International Educational Programs
Moscow
References
1. Shibaeva N. B. Information and communication technologies as a means of control in teaching Russian language to foreign students. Current Issues of Foreign Language Education in the Context of Intercultural Interaction. Collection of Scientific Articles of the 36th Int. Scientific and Practical Conf. Cheboksary, Chuvash State Pedagogical University; 2024:242–248. (In Russian.) EDN: CSEQLV.
2. Beloglazova L. B., Bragina M. A., Konyrbayev N. B., Beloglazov A. A. Digital and computer technologies as a means of intensifying the process of teaching russian language to foreign students. MCU Journal of Informatics and Informatization of Education. 2024;(1(67)):18–27. (In Russian.) EDN: KKMUJG. DOI: 10.25688/2072-9014.2024.67.1.02.
3. Romanova T. V., Kulikova V. A., Klimova M. A. Testing communicative competencies of Russian language proficiency. Scientific Research and Development. Modern Communication Studies. 2024;13(2):12–21. (In Russian.) EDN: JOCMFX. DOI: 10.12737/2587-9103-2024-13-2-12-21. Available at: https://naukaru.ru/ru/storage/download/154110
4. Benevolenskaya N. P. The influence of storytelling method on the development of communicative competence of foreign students studying in Russian as a foreign language (RFL). CITISE. 2024;(1(39)):456–464. DOI: 10.15350/2409- 7616.2024.1.39. Available at: https://ma123.ru/wp-content/uploads/2024/03/Benevolenskaya_CITISE_1-2024-1.pdf
5. Jung Sang-Im, Lim Sunghee, Jo Eunha, Sim Hyun Sub, Sung Jee Eun, Kim Young Tae. The efficacy of telepractice intervention for children & adolescents with speech, language & hearing impairments: A meta-analysis. Communication Sciences & Disorders. 2020;25(4):976–986. DOI: 10.12963/csd.20756.
6. Akhutina T. Luria’s classification of aphasias and its theoretical basis. Aphasiology. 2016;30(8):878–897. DOI: 10.1080/02687038.2015.1070950.
7. Sakbaeva M. A. The role of multimedia and computer technologies in teaching foreign languages. Advances of Science and Education. 2020;(8(62)):44–46. (In Russian.) EDN: YXXHFB.]
8. Trizano-Hermosilla I., Gálvez-Nieto J. L., Alvarado J. M., Saiz J. L., Salvo-Garrido S. Reliability estimation in multidimensional scales: Comparing the bias of six estimators in measures with a bifactor structure. Frontiers in Psychology. 2021;12:1–7. DOI: 10.3389/fpsyg.2021.508.
9. Yang J., Nizkoshapkina O. V. Using the storytelling method in the teaching of Russian as a foreign language to form Russian-speaking communicative competence in students Moscow Pedagogical Journal. 2023;(4):93–107. (In Russian.) EDN: VKEPIO. DOI: 10.18384/2949-4974-2023-4-93-107.
10. Hecht A., Wargers I., Leinweber J., Borgetto B. Videotherapie in der ambulanten logopädischen Versorgung — empirisch begründete Handlungsempfehlungen. 21. Deutscher Kongress für Versorgungsforschung (DKVF). Potsdam, 05.- 07.10.2022. Düsseldorf, German Medical Science GMS Publishing House; 2022:Doc22dkvf073. DOI: 10.3205/22dkvf073.
11. Kireytseva A. N. ABC of testing. A practical guide for teachers of Russian as a foreign language. Saint Petersburg, Zlatoust; 2013. 184 p. (In Russian.) Available at: https://www.ros-edu.ru/book?id=81348
12. Gačnik M., Istenič Starčič A., Zaletelj J., Zajc M. Usercentred app design for speech sound disorders interventions with tablet computers. Universal Access in the Information Society. 2018;17:821–832. DOI: 10.1007/s10209-017-0545-9.
13. International cooperation in education and science. Proc. 9th Int. Conf. “International Cooperation in Education and Science: Current Issues of Teaching Foreign Students in Russian Universities. 50 years of the Preparatory Faculty” (St. Petersburg, May 20–23, 2015). Vol. 7. Modern Technologies in Teaching Russian as a Foreign Language. Saint Petersburg, Polytechnic University Publishing House; 2015. 148 p. (In Russian.) Available at: https://iep.spbstu.ru/userfiles/files/pdf/Konf-2015-t-7.pdf
14. The Russian Aphasia Test: The first comprehensive, quantitative, standardized, and computerized aphasia language battery in Russian. PLoS ONE. 2021;16(11):e0258946. DOI: 10.1371/journal.pone.0258946.
15. Wang Yameng. Modern approaches to teaching Russian language in higher educational institutions of China. Modern Science: Actual Problems of Theory and Practice. Series “Humanities”. 2019;(8):55–58. (In Russian.) EDN: MOSMAF.
16. Eberhard D. M., Simons G. F., Fennig C. D. (eds.) Ethnologue: Languages of the world. 24th edition. Dallas TX, SIL International; 2021. Available at: https://www.ethnologue.com/
17. Methods of teaching foreign languages and Russian as a foreign language: Traditions and innovations. Collection of Scientific Papers of the 8th Int. Scientific and Methodological Online Conf. (Kursk, April 11, 2023). Kursk, Kursk State Medical University; 2023. 521 p. (In Russian.) EDN: LUCCHF.
18. Chesnokova M. P. Methods of teaching Russian as a foreign language. 2nd edition, revised. Moscow, MADI; 2015. 132 p. (In Russian.) Available at: https://lib.madi.ru/fel/fel1/fel15E327.pdf
19. Speech processing for language learning: A practical approach to computer-assisted pronunciation teaching. Electronics. 2021;10(3):235. DOI: 10.3390/electronics10030235.
20. Batraeva O. M., Bimurzina I. V. Intensive methods and technologies in teaching Russian as a foreign language. Pedagogy: traditions and innovations. Proc. Int. Scientific Conf. (Chelyabinsk, October 20–23, 2011). Chelyabinsk, Dva komsomol’tsa; 2011:62–64. (In Russian.) EDN: VKTKUT.
21. Maior N. A., Ovcharenko N. N. Academic major language mastering by foreign students at pre-university course at electrotechnical university. Modern Education: Content, Technologies, Quality. Proc. 29th Int. Scientific and Metodological Conf. In two volumes. Vol. 1. Saint Petersburg, Saint Petersburg Electrotechnical University; 2023:363–365. (In Russian.) EDN: XGNXPL.
22. Krasman V. A., Rusinova N. S. Specifics of teaching Russian as a foreign language to students at the preparatory department planning to enter master’s programs in Russian. Russian as a Foreign Language: History, Present and Future. Collection of Scientific Articles of the Int. Scientific and Practical Conf. Kazan, Kazan University Publishing House; 2023:461–465. (In Russian.) EDN: PUIUKO. Available at: https://dspace.kpfu.ru/xmlui/handle/net/177147
23. Makarova T. S., Matveeva E. E., Molchanova M. A. Teaching language for specific purposes. Three “L” in the Paradigm of Modern Humanitarian Knowledge: Linguistics, Literary Studies, Linguodidactics. Collection of Scientific Articles of the All-Russian Scientific and Practical Conf. Moscow, Yazyki Narodov Mira; 2024:424–430. (In Russian.) EDN: WGGHNX.
24. Burukina I. P., Mamelina Yu. V. Services for determining the tone of speech based on neural networks. New Information Technologies and Systems (NITiS-2023). Collection of Scientific Articles on the Materials of the 20th Int. Scientific and Technical Conf. Penza, Penza State University; 2023:130–135. (In Russian.) Available at: https://elib.pnzgu.ru/files/eb/1oDBoQhhB8bv.pdf
25. Maslova I. B. Communication skills of foreign studentstranslators: Modern forms of control and assessment models (training course “Russian language and culture of speech”). Modern Rhetoric and Speech Disciplines: From Oral Dialogue to Computer-Digital Interaction. Proc. 26th Int. Scientific and Practical Conf. Moscow, Moscow State Linguistic University; 2023:197–203. (In Russian.) EDN: SCNGYE.
Review
For citations:
Cherkashina E.L., Zhamaletdinov R.I. Automated speech analysis using machine learning methods for language competencies assessment in Russian as a foreign language exam. Informatics and education. 2025;40(1):82-94. (In Russ.) https://doi.org/10.32517/0234-0453-2025-40-1-82-94