Predicting academic performance in a course by universal features of LMS Moodle digital footprint
https://doi.org/10.32517/0234-0453-2023-38-3-31-41
Abstract
Student retention prediction is one of the most important problems of learning analytics. In the global scope research on the topic for higher education is rather extensive, there are cases of successful implementation of education support services in universities. The literature analysis shows of the growing interest in this problem in the Russian scientific and pedagogical community. At the same time, the specifics of Russian education does not allow direct transfer of foreign experience into the domestic educational system.
The study reveals that a significant contribution to predicting student retention can be made by models for predicting academic performance in educational courses of the curriculum. The authors propose a structural model of a system for predicting academic performance, which includes a universal model based on generalized indicators of the digital footprint, a course-based model that takes into account the specifics of learning in a particular discipline, and a model based on the student’s educational profile.
In the empirical study we trained 5 models for early prediction of interim assessment grades based on the universal indicators of the LMS Moodle student digital footprint. The most accurate model, especially in the first half of the semester, turned out to be ensemble-averaging models of logistic regression, random forest and gradient boosting. It was found that universal models are effective for detection of at-risk students in the discipline, the directions for further improvement of the universal model of performance prediction were determined and conditions for scaling the proposed approach to create a prognostic system for student retention to other educational institutions were formulated.
Keywords
About the Authors
R. V. EsinRussian Federation
Roman V. Esin, Candidate of Sciences (Education), Associate Professor at the Department of Applied Mathematics and Data Analysis, School of Space and Information Technology
Krasnoyarsk
T. A. Kustitskaya
Russian Federation
Tatiana A. Kustitskaya, Candidate of Sciences (Physics and Mathematics), Associate Professor at the Department of Applied Mathematics and Data Analysis, School of Space and Information Technology
Krasnoyarsk
M. V. Noskov
Russian Federation
Mihail V. Noskov, Doctor of Sciences (Physics and Mathematics), Professor, Professor at the Department of Applied Mathematics and Data Analysis, School of Space and Information Technologies
Krasnoyarsk
References
1. Siemens G., Gašević D. Guest editorial — learning and knowledge analytics. Educational Technology & Society. 2012;15(3):1–2. Available at: https://drive.google.com/file/d/1SJQZSFOrix9_WZTvBtzvUL70bsLa_eqQ/view
2. Arifulina R. U., Katushenko O. A. Analysis of domestic and foreign trends of individualization of the educational process in the university: analytical report. Vestnik of Minin University. 2021;9(4):2. (In Russian.) EDN: CMEEMZ. DOI: 10.26795/2307-1281-2021-9-4-2
3. Gordeeva T. O., Sychev O. A., Osin E. N. “Academic motivation scales” questionnaire. Psikhologicheskii zhurnal. 2014;35(4):96–107. (In Russian.). EDN: SJVWLN
4. Nur N., Park N., Dorodchi M., Dou W., Mahzoon M. J., Niu X., Maher M. L. Student network analysis: A novel way to predict delayed graduation in higher education. Artificial Intelligence in Education. AIED 2019. Lecture Notes in Computer Science. 2019;11625:370–382. Springer, Cham. DOI: 10.1007/978-3-030-23204-7_31
5. York T. T., Gibson C., Rankin S. Defining and measuring academic success. Practical Assessment, Research, and Evaluation. 2015;20:5. DOI: 10.7275/hz5x-tx03
6. Hommel M., Egetenmeier A., Maier U. Supporting stakeholders with learning analytics to increase study success. Utilizing Learning Analytics to Support Study Success. Springer, Cham; 2019:37–60. DOI: 10.1007/978-3-319-64792-0_3
7. Shaporov A. M. Forecasting the risk of expulsion from the university as a result of academic failure of the student. Yaroslavl Pedagogical Bulletin. 2022;(1(124)):48–55. (In Russian.) EDN: NSUCPN. DOI: 10.20323/1813-145X-2022-1-124-48-55
8. Bravo-Agapito J., Romero S. J., Pamplona S. Early prediction of undergraduate Student’s academic performance in completely online learning: A five-year study. Computers in Human Behavior. 2021;115:106595. DOI: 10.1016/j.chb.2020.106595
9. Luis R. M. M. F., Llamas-Nistal M., Iglesias M. J. F. On the introduction of intelligent alerting systems to reduce e-learning dropout: A case study. Smart Learning Environments. 2022;9:29. DOI: 10.1186/s40561-022-00210-0
10. Moreira da Silva D. E., Solteiro Pires E. J., Reis A., de Moura Oliveira P. B., Barroso J. Forecasting Students Dropout: A UTAD University Study. Future Internet. 2022;14(3):76. DOI: 10.3390/fi14030076
11. Gaševič D., Dawson S., Rogers T., Gasevic D. Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education. 2016;28:68–84. DOI: 10.1016/j.iheduc.2015.10.002
12. Knowles J. E. Of needles and haystacks: Building an accurate statewide dropout early warning system in Wisconsin. Journal of Educational Data Mining. 2015;7(3):18–67. DOI: 10.5281/zenodo.3554725
13. Ifenthaler D., Mah D.-K., Yau J.Y.-K. Utilising learning analytics for study success: Reflections on current empirical findings. Utilizing Learning Analytics to Support Study Success. Springer, Cham; 2019:27–36. DOI: 10.1007/978-3-319-64792-0_2
14. Ortigosa A., Carro R. M., Bravo-Agapito J., Lizcano D., Alcolea J. J. Blanco Ó. From lab to production: Lessons learnt and real-life challenges of an early student-dropout prevention system. IEEE Transactions on Learning Technologies. 2019;12(2):264–277. DOI: 10.1109/TLT.2019.2911608
15. Mussida P., Lanzi P. L. A computational tool for engineer dropout prediction. 2022 IEEE Global Engineering Education Conference (EDUCON). 2022:1571–1576. DOI: 10.1109/EDUCON52537.2022.9766632
16. Silva J., Matos L. F. A., Mosquera C. M., Mercado C. V., González R. B., Llinás N. O., Lezama O. B. P. Prediction of academic dropout in university students using data mining: Engineering case. Advances in Cybernetics, Cognition, and Machine Learning for Communication Technologies. Springer, Singapore; 2020:459–500. DOI: 10.1007/978-981-15-3125-5_49
17. Esin P. V., Zykova T. V., Kustitskaya T. A., Kytmanov A. A. Digital educational history as a component of the digital student’s profile in the context of education transformation. Perspectives of Science and Education. 2022;(5):566–584. (In Russian.) EDN: OMTHCN. DOI: 10.32744/ pse.2022.5.34
18. Shaimov N. D., Lomazova I. A., Mitsyuk A. A., Samonenko I. Y. Analysis of students’ academic performance using LMS event logs. Modeling and Analysis of Information Systems. 2022;29(4):286–314. (In Russian.) EDN: STNXMO. DOI: 10.18255/1818-1015-2022-4-286-314
19. Tamada M. M., Giusti R., Netto J. F. d. M. Predicting students at risk of dropout in technical course using LMS logs. Electronics. 2022;11(3):468. DOI: 10.3390/electronics11030468
20. Toktarova V. I., Popova O. G. An analysis of educational data on the correlation between learning success and students’behavior in the university digital educational environment. Informatics and Education. 2022;37(4):54–63. (In Russian.) EDN: ESVZJD. DOI: 10.32517/0234-0453-2022-37-4-54-63
21. Kustitskaya T. A., Kytmanov A. A., Noskov M. V. Early student-at-risk detection by current learning performance and learning behavior indicators. Cybernetics and Information Technologies. 2022;22(1):117–133. EDN: UDNYJC. DOI: 10.2478/cait-2022-0008
22. Udzhukhu I. A., Meshvez R. K., Manchenko Yu. V., Galyunko T. E. Electronic information and educational environment of a modern university: Concept, structure, application. Vestnik Maykopskogo Gosudarstvennogo Technologicheskogo Universiteta. 2020;1(44):113–121. (In Russian.) EDN: IXROHD. DOI: 10.24411/2078-1024-2020-11011
23. Haakman M., Cruz L., Huijgens H., van Deursen A. AI lifecycle models need to be revised. Empirical Software Engineering. 2021;26(5):95. DOI: 10.1007/s10664-021-09993-1
24. Susnjak T., Ramaswami G. S., Mathrani A. Learning analytics dashboard: A tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education. 2022;19(1):12. DOI: 10.1186/s41239-021-00313-7
25. Akcayir G., Farias Wanderley L., Demmans Epp C., Hewitt J., Mahmoudi-Nejad A. Learning analytics dashboard use in online courses: Why and how instructors interpret discussion data. Visualizations and Dashboards for Learning Analytics. Springer, Cham; 2021:371–397. DOI: 10.1007/978-3-030-81222-5_17
Review
For citations:
Esin R.V., Kustitskaya T.A., Noskov M.V. Predicting academic performance in a course by universal features of LMS Moodle digital footprint. Informatics and education. 2023;38(3):31-41. (In Russ.) https://doi.org/10.32517/0234-0453-2023-38-3-31-41