infojournalИнформатика и образованиеInformatics and education0234-04532658-7769Publishing House Education and Informatics10.32517/0234-0453-2020-35-5-56-63infojournal-543Research ArticleПЕДАГОГИЧЕСКИЕ ИЗМЕРЕНИЯ И ТЕСТЫPEDAGOGICAL ASSESSMENTS AND TESTSПрименение технологий компьютерного зрения для автоматического сбора данных об эмоциях обучающихся во время групповой работыApplication of computer vision technologies for automatic data collection about emotions of students during group workhttps://orcid.org/0000-0001-5153-4334КуприяновР. Б.KupriyanovR. B.

Куприянов Роман Борисович, зам. начальника управления информационных технологий

129226, г. Москва, 2-й Сельскохозяйственный проезд, д. 4

Roman B. Kupriyanov, Deputy Head of the IT Department

129226, Moscow, Vtoroy Selskohoziajstvenny proezd, 4

kupriyanovrb@mgpu.ruМосковский городской педагогический университетРоссияMoscow City UniversityRussian Federation202004072020055663Copyright © Куприянов Р.Б., 20202020Куприянов Р.Б.Kupriyanov R.B.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://info.infojournal.ru/jour/article/view/543

Сегодня мировое научное сообщество активно обсуждает вопросы применения искусственного интеллекта в образовании. Одна из наименее изученных технологий, в части ее применения в сфере образования, — компьютерное зрение. Разработка и внедрение интеллектуальных систем на основе алгоритмов анализа видео и машинного обучения предоставляют преподавателям и сотрудникам администрации образовательной организации новые возможности для понимания образовательного процесса и его трансформации. В статье рассматривается применение технологий анализа видео с камер с углом обзора, равным 360 градусам, для сбора данных об эмоциональном состоянии обучающихся во время групповой работы на учебных занятиях. В ходе описываемого исследования было разработано программное решение для автоматического сбора данных об эмоциях обучающихся во время работы в группах на учебных занятиях, которое может быть применено для проведения дальнейших исследований, направленных на изучение влияния эмоционального состояния обучающихся на их образовательные успехи. Результаты исследования могут быть использованы для формирования повестки научных исследований российских вузов в целях реализации задач раздела «Образование и кадры» программы «Цифровая экономика Российской Федерации», утвержденной Правительством Российской Федерации.

Today the global scientific community is actively discussing the issues on the application of artificial intelligence in education. One of the least studied technologies in terms of its application in education is computer vision. The development and implementation of intelligent systems based on video analysis and machine learning algorithms provide new opportunities for teachers and staff of the educational organization administration to understand and transform the educational process. The article discusses the use of video analysis technologies from cameras with a 360-degree view to collect data on the emotional state of students during group work in the classroom. In the course of the described research, a software solution for automatic emotions data collection during students’ teamwork leaning was developed. This solution can be used for future research aimed at studying the impact of emotional state on students’ educational success. The results of the study can be used to form the research agenda of Russian universities in order to implement the objectives of the section “Education and personnel” of the program “Digital economy of the Russian Federation”, approved by the Government of the Russian Federation.

цифровая трансформация образовательного процессаискусственный интеллекткомпьютерное зрениевидеоанализанализ данныхинтеллектуальный анализ данныхинтеллектуальный анализ образовательных данныхdigital transformation of educational processartificial intelligencecomputer visionvideo analysisdata analysisdata miningeducational data miningReferencesAfsar P., Cortez P., Santos H. Automatic visual detection of human behavior: A review from 2000 to 2014 // Expert Systems with Applications. 2015. Vol. 42. Is. 20. P. 6935–6956. DOI: 10.1016/j.eswa.2015.05.023Afsar P., Cortez P., Santos H. Automatic visual detection of human behavior: A review from 2000 to 2014. Expert Systems with Applications, 2015, vol. 42, is. 20, p. 6935–6956. DOI: 10.1016/j.eswa.2015.05.023Уваров А. Ю., Гейбл Э., Дворецкая И. В., Заславский И. М., Карлов И. А., Мерцалов Т. А., Сергоманов П. А., Фрумин И. Д. Трудности и перспективы цифровой трансформации образования. М.: ВШЭ, 2019. 343 с. DOI: 10.17323/978-5-7598-1990-5UvarovA.Yu., GableE., DvoretskayaI.V., ZaslavskyI.M., Karlov I. A., Mertsalov T. A., Sergomanov P. A., Frumin I. D. Trudnosti i perspektivy tsifrovoj transformatsii obrazovaniya [Difficulties and prospects of the digital transformation of education]. Moscow, HSE, 2019. 343 p. (In Russian.) DOI: 10.17323/978-5-7598-1990-5Rizzotto L. The future of education: How A.I. and immersive tech will reshape learning forever. Medium, 2017. https://medium.com/futurepi/a-vision-for-education-andits-immersive-a-i-driven-future-b5a9d34ce26dRizzotto L. The future of education: How A.I. and immersive tech will reshape learning forever. Medium, 2017. Available at: https://medium.com/futurepi/a-vision-foreducation-and-its-immersive-a-i-driven-future-b5a9d34ce26dFaggella D. Examples of artificial intelligence in education. Emerj, 2019. https://www.techemergence.com/examples-of-artificial-intelligence-in-educationFaggella D. Examples of artificial intelligence in education. Emerj, 2019. Available at: https://www.techemergence.com/examples-of-artificial-intelligence-in-educationDaria R. AI in education or How to create an advanced artificial intelligence program. Cleveroad, 2017. https://www.cleveroad.com/blog/ai-in-education-or-whatadvantages-of-artificial-intelligence-in-education-you-cangain-Daria R. AI in education or How to create an advanced artificial intelligence program. Cleveroad, 2017. Available at: https://www.cleveroad.com/blog/ai-in-education-orwhat-advantages-of-artificial-intelligence-in-education-youcan-gain-Уваров А. Ю. Технологии искусственного интеллекта в образовании // Информатика и образование. 2018. № 4. С. 14–22.Uvarov A. Yu. Tekhnologii iskusstvennogo intellekta v obrazovanii [Artificial intelligence technologies in education]. Informatika i obrazovanie — Informatics and Education, 2018, no. 4, p. 14–22. (In Russian.)Wagner W., Göllner R., Werth S., Voss T., Schmitz B., Trautwein U. Student and teacher ratings of instructional quality: Consistency of ratings over time, agreement, and predictive power // Journal of Educational Psychology. 2016. Vol. 108. No. 5. P. 705–721. DOI: 10.1037/edu0000075Wagner W., Göllner R., Werth S., Voss T., Schmitz B., Trautwein U. Student and teacher ratings of instructional quality: Consistency of ratings over time, agreement, and predictive power. Journal of Educational Psychology, 2016, vol. 108, no. 5, p. 705–721. DOI: 10.1037/edu0000075Kunter M., Klusmann U., Baumert J., Richter D., Voss T., Hachfeld A. Professional competence of teachers: Effects on instructional quality and student development // Journal of Educational Psychology. 2013. Vol. 105. Is. 3. P. 805–820. DOI: 10.1037/a0032583Kunter M., Klusmann U., Baumert J., Richter D., Voss T., Hachfeld A. Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 2013, vol. 105, is. 3, p. 805–820. DOI: 10.1037/a0032583Fox A. S., Lapate R. C., Shackman A. J., Davidson R. J. The nature of emotion: Fundamental questions. Oxford: Oxford University Press, 2018. 632 p.62Fox A. S., Lapate R. C., Shackman A. J., Davidson R. J. The nature of emotion: Fundamental questions. Oxford, Oxford University Press, 2018. 632 p.Bidwell J., Fuchs H. Classroom analytics: Measuring student engagement with automated gaze tracking. 2011. 17 p. DOI: 10.13140/RG.2.1.4865.6242Bidwell J., Fuchs H. Classroom analytics: Measuring student engagement with automated gaze tracking. 2011. 17 p. DOI: 10.13140/RG.2.1.4865.6242Helmke A., Helmke T., Heyne N., Hosenfeld A., Schrader F.-W., Wagner W. Allgemeine beobachtungsinstrumente der unterrichtsstudie “VERA — gute unterrichtspraxis”. Landau: Universität Koblenz-Landau, 2007. (In German.)Helmke A., Helmke T., Heyne N., Hosenfeld A., Schrader F.-W., Wagner W. Allgemeine beobachtungsinstrumente der unterrichtsstudie “VERA — gute unterrichtspraxis”. Landau, Universität Koblenz-Landau, 2007. (In German.)Klieme E., Eichler W., Helmke A., Lehmann R. H., Nold G., Rolff H.-G., Schröder K., Thomé G., Willenberg H. Unterricht und kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie. Weinheim: Beltz, 2008. 459 p. (In German.)Klieme E., Eichler W., Helmke A., Lehmann R. H., Nold G., Rolff H.-G., Schröder K., Thomé G., Willenberg H. Unterricht und kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie. Weinheim, Beltz, 2008. 459 p. (In German.)Ainley M. Connecting with learning: Motivation, affect and cognition in interest processes // Educational Psychology Review. 2006. Vol. 18. P. 391–405. DOI: 10.1007/s10648-006-9033-0Ainley M. Connecting with learning: Motivation, affect and cognition in interest processes. Educational Psychology Review, 2006, vol. 18, p. 391–405. DOI: 10.1007/s10648-006-9033-0Appleton J. J., Lawrenz F. Student and teacher perspectives across mathematics and science classrooms: The importance of engaging contexts // School Science and Mathematics. 2011. Vol. 111. Is. 4. P. 143–155. DOI: 10.1111/j.1949-8594.2011.00072.xAppleton J. J., Lawrenz F. Student and teacher perspectives across mathematics and science classrooms: The importance of engaging contexts. School Science and Mathematics, 2011, vol. 111, is. 4, p. 143–155. DOI: 10.1111/j.1949-8594.2011.00072.xAung A. M., Ramakrishnan A., Whitehill J. R. Who are they looking at? Automatic eye gaze following for classroom observation video analysis // Proc. 11th Int. Conf. on Educational Data Mining. 2018. P. 252–258.Aung A. M., Ramakrishnan A., Whitehill J. R. Who are they looking at? Automatic eye gaze following for classroom observation video analysis. Proc. 11th Int. Conf. on Educational Data Mining. 2018, p. 252–258.Santini T., Kubler T., Draghetti L., Gerjets P., Wagner W., Trautwein U., Kasneci E. Automatic mapping of remote crowd gaze to stimuli in the classroom. https://pdfs.semanticscholar.org/581e/920ddb6ecfc2a313a3aa6fed3d933b917ab0.pdfSantini T., Kubler T., Draghetti L., Gerjets P., Wagner W., Trautwein U., Kasneci E. Automatic mapping of remote crowd gaze to stimuli in the classroom. Available at: https://pdfs.semanticscholar.org/581e/920ddb6ecfc2a313a3aa6fed3d933b917ab0.pdfSoloviev V. Machine learning approach for student engagement automatic recognition from facial expressions // Scientific Publications of the State University of Novi Pazar. Series A: Applied Mathematics, Informatics and Mechanics. 2018. Vol. 10. Is. 2. P. 79–86. DOI: 10.5937/SPSUNP1802079SSoloviev V. Machine learning approach for student engagement automatic recognition from facial expressions. Scientific Publications of the State University of Novi Pazar. Series A: Applied Mathematics, Informatics and Mechanics, 2018, vol. 10, is. 2, p. 79–86. DOI: 10.5937/SPSUNP1802079SReilly J. M., Ravenell M., Schneider B. Exploring collaboration using motion sensors and multi-modal learning analytics // Proc. 11th Int. Conf. on Educational Data Mining. 2018. P. 333–341.Reilly J. M., Ravenell M., Schneider B. Exploring collaboration using motion sensors and multi-modal learning analytics. Proc. 11th Int. Conf. on Educational Data Mining. 2018, p. 333–341.Rajendran Ramkumar, Kümar Anurag, Carter Kelly E., Levin Daniel T., Biswas Gautam. Predicting learning by analyzing eye-gaze data of reading behavior // International Conference on Educational Data Mining (EDM) 2018. Proceedings of the 11th International Conference on Educational Data Mining, Kristy Elizabeth Boyer and Michael Yudelson (Eds.). Buffalo, NY, July 16–20, 2018, С. 455–462. https://educationaldatamining.org/files/conferences/EDM2018/papers/EDM2018_paper_205.pdfRajendran Ramkumar, Kümar Anurag, Carter Kelly E., Levin Daniel T., Biswas Gautam. Predicting learning by analyzing eye-gaze data of reading behavior. International Conference on Educational Data Mining (EDM) 2018. Proceedings of the 11th International Conference on Educational Data Mining, Kristy Elizabeth Boyer and Michael Yudelson (Eds.). Buffalo, NY, July 16–20, 2018, С. 455–462. https://educationaldatamining.org/files/conferences/EDM2018/papers/EDM2018_paper_205.pdfD’Mello S., Graesser A. Dynamics of affective states during complex learning // Learning and Instruction. 2012. Vol. 22. Is. 2. P. 145–157. DOI: 10.1016/j.learninstruc.D’Mello S., Graesser A. Dynamics of affective states during complex learning. Learning and Instruction, 2012, vol. 22, is. 2, p. 145–157. DOI: 10.1016/j.learninstruc.2011.10.00110.001 21. Bosch N. Detecting student engagement: Human versus machine // UMAP ‘16: Proc. 2016 Conf. on User Modeling Adaptation and Personalization. New York: ACM, 2016. P. 317–320. DOI: 10.1145/2930238.2930371Bosch N. Detecting student engagement: Human versus machine. UMAP ‘16: Proc. 2016 Conf. on User Modeling Adaptation and Personalization. New York, ACM, 2016, p. 317–320. DOI: 10.1145/2930238.2930371Hallion L. S., Steinman S. A., Kusmierski S. N. Difficulty concentrating in generalized anxiety disorder: An evaluation of incremental utility and relationship to worry // Journal of Anxiety Disorders. 2018. Vol. 53. P. 39–45. DOI: 10.1016/j.janxdis.2017.10.007Hallion L. S., Steinman S. A., Kusmierski S. N. Difficulty concentrating in generalized anxiety disorder: An evaluation of incremental utility and relationship to worry. Journal of Anxiety Disorders, 2018, vol. 53, p. 39–45. DOI: 10.1016/j.janxdis.2017.10.007Whitehill J., Serpell Z., Lin Y.-C., Foster A., Movellan J. R. The faces of engagement: Automatic recognition of student engagement from facial expressions // IEEE Transactions on Affective Computing. 2014. Vol. 5. Is. 1. P. 86–98. DOI: 10.1109/TAFFC.2014.2316163Whitehill J., Serpell Z., Lin Y.-C., Foster A., Movellan J. R. The faces of engagement: Automatic recognition of student engagement from facial expressions. IEEE Transactions on Affective Computing, 2014, vol. 5, is. 1, p. 86–98. DOI: 10.1109/TAFFC.2014.2316163Shuman V., Clark-Polner E., Meuleman B., Sander D., Scherer K. Emotion perception from a componential perspective // Cognition and Emotion. 2017. Vol. 31. Is. 1. P. 47–56. DOI: 10.1080/02699931.2015.1075964Shuman V., Clark-Polner E., Meuleman B., Sander D., Scherer K. Emotion perception from a componential perspective. Cognition and Emotion, 2017, vol. 31, is. 1, p. 47–56. DOI: 10.1080/02699931.2015.1075964Распоряжение Правительства Российской Федерации от 28 июля 2017 года № 1632-р «Об утверждении программы “Цифровая экономика Российской Федерации”». http://www.consultant.ru/document/cons_doc_LAW_221756/Rasporyazhenie Pravitel’stva Rossijskoj Federatsii ot 28 iyulya 2017 goda № 1632-r “Ob utverzhdenii programmy “Tsifrovaya ehkonomika Rossijskoj Federatsii”” [Order of the Government of the Russian Federation dated July 28, 2017 No. 1632-r “On approval of the Program “Digital Economy of the Russian Federation””]. (In Russian.) Available at: http://www.consultant.ru/document/cons_doc_LAW_221756/

The authors declare that there are no conflicts of interest present.