Multi-criteria methodology for LMS quality assessment within the framework of emergent learning system
https://doi.org/10.32517/0234-0453-2021-36-5-4-11
Abstract
In the article, using the case of Learning Management System (LMS), we consider a methodology for assessing the quality of education, taking into account the requests of experts, lecturers, and students. There is a growing relevance of such research due to the digitalization of education, the transition to distance learning as the main or auxiliary form of organizing the educational process. The applied methodology relies on the earlier developed concept of emergent learning and the system for the assessment of the quality of education, using the conception of negentropy. By adapting this methodology to the LMS assessment task, we show the importance of creating a hierarchical scheme of modules and functional elements necessary for an effective organization of the learning process. In order to provide an integral assessment, it is necessary, according to our position, to determine the weighting factor of each module and of each criterion by which this module is assessed and then to determine the values of each criterion for a specific LMS. A proposed flexible assessment system that takes into account the requests of all aspects of the educational process, allows, by modifying the weighting factors, to adapt the equation in accordance with the specificity of the subject area and educational institution within which the LMS is used. As an advantage of this methodology, it makes it possible not only to compare different LMSs in terms of quality but also to identify areas in which it is possible to improve and refine existing LMSs. We analyze the risks associated with the transformation of assessment systems from an auxiliary tool into an end in itself. In this regard, we propose an algorithm for separating qualitative and quantitative factors accounting, which helps to avoid the negative effect of the absolutization of assessment systems. In the conclusion, we outline the ways of developing the design of such a system of the management of educational process in which quality requirements prevail over circumstantial ones.
About the Authors
A. Kh. MarinosyanRussian Federation
Andreas Kh. Marinosyan, Specialist, 119049, Russia, Moscow, Maronovskiy per., 26
O. V. Andryushkova
Russian Federation
Olga V. Andryushkova, Candidate of Sciences (Chemistry), Docent, Head of the Laboratory of Methods of Teaching Chemistry of the Department of General Chemistry, 119991, Russia, Moscow, Leninskie gory, 1, building 3
References
1. Andryushkova O. V., Grigoriev S. G. Ehmergentnoe obuchenie v informatsionno-obrazovatel’noj srede [Emergent training in the information educational environment]. Moscow, Obrazovanie i Informatika, 2018. 104 p. (In Russian.)
2. Kellaghan T., Stufflebeam D. L. International handbook of educational evaluation. Dordrecht, Springer, 2003. 1060 p. DOI: 10.1007/978-94-010-0309-4
3. Dunn L., Morgan C., O’Reilly M., Parry S. The student assessment handbook: New directions in traditional and online assessment. London, Routledge, 2003. 320 p. DOI: 10.4324/9780203416518
4. Cox K., Imrie B. W., Miller A. H. Student assessment in higher education: A handbook for assessing performance. London, Routledge, 1998. 292 p. Available at: https://www.routledge.com/Student-Assessment-in-Higher-Education-A-Handbook-for-Assessing-Performance/Cox-Imrie-Miller/p/book/9780749427979
5. Andryushkova O., Grigoriev S. The influence online learning quality criteria selection on negentropy. Proc. 4th Int. Conf. on Informatization of Education and E-learning Methodology: Digital Technologies in Education. 2020, vol. 2770, p. 127–139. Available at: http://ceur-ws.org/Vol-2770/paper17.pdf
6. Andreev A. A. Otsenka kachestva onlajn kursov [Evaluation of the quality of online courses]. Territoriya nauki — Territory of Science, 2015, no. 1, p. 20–26. (In Russian.)
7. Horvat A., Dobrota M., Krsmanovic M., Cudanov M. Student perception of Moodle learning management system: a satisfaction and significance analysis. Interactive Learning Environments, 2015, vol. 23, is. 4, p. 515–527. DOI: 10.1080/10494820.2013.788033
8. Han I., Shin W. S. The use of a mobile learning management system and academic achievement of online students. Computers & Education, 2016, vol. 102, p. 79–89. DOI: 10.1016/j.compedu.2016.07.003
9. Farid S., Ahmad R., Alam M., Akbar A., Chang V. A sustainable quality assessment model for the information delivery in E-learning systems. Information Discovery and Delivery, 2018, vol. 46, is. 1, p. 1–25. DOI: 10.1108/IDD-11-2016-0047
10. Cerezo R., Bogarín A., Esteban M., Romero C. Process mining for self-regulated learning assessment in e-learning. Journal of Computing in Higher Education, 2020, vol. 32, is. 1, p. 74–88. DOI: 10.1007/s12528-019-09225-y
11. Henriksen D., Creely E., Henderson M. Folk pedagogies for teacher transitions: Approaches to synchronous online learning in the wake of COVID-19. Journal of Technology and Teacher Education, 2020, vol. 28, is. 2, p. 201–209.
12. Gillett-Swan J. The challenges of online learning: Supporting and engaging the isolated learner. Journal of Learning Design, 2017, vol. 10, no. 1, p. 20–30. DOI: 10.5204/jld.v9i3.293
13. Kara M., Erdoğdu F., Kokoç M., Cagiltay K. Challenges faced by adult learners in online distance education: A literature review. Open Praxis, 2019, vol. 11, no. 1, p. 5–22. DOI: 10.5944/openpraxis.11.1.929
14. Fresen J. W. Embracing distance education in a blended learning model: challenges and prospects. Distance Education, 2018, vol. 39, is. 2, p. 224–240. DOI: 10.1080/01587919.2018.1457949
15. Damary R., Markova T., Pryadilina N. Key challenges of on-line education in multi-cultural context. Procedia — Social and Behavioral Sciences, 2017, vol. 237, p. 83–89. DOI: 10.1016/j.sbspro.2017.02.034
16. Ouma R., Nkuyubwatsi B. Transforming university learner support in open and distance education: Staff and students perceived challenges and prospects. Cogent Education, 2019, vol. 6, is. 1. DOI: 10.1080/2331186X.2019.1658934
17. Nage-Sibande B., Morolong B. L. A trend analysis of opportunities and challenges of open and distance learning provision in dual-mode institutions. Distance Education, 2018, vol. 39, is. 4, p. 495–510. DOI: 10.1080/01587919.2018.1457951
18. Palvia S., Aeron P., Gupta P., Mahapatra D., Parida R., Rosner R., Sindhi S. Online education: Worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 2018, vol. 21, is. 4, p. 233–241. DOI: 10.1080/1097198X.2018.1542262
19. Kebritchi M., Lipschuetz A., Santiague L. Issues and challenges for teaching successful online courses in higher education: A literature review. Journal of Educational Technology Systems, 2017, vol. 46, is. 1, p. 4–29. DOI: 10.1177/0047239516661713
20. Chen S.-Y., Basma D., Ju J., Ng K.-M. Opportunities and challenges of multicultural and international online education. Professional Counselor, 2020, vol. 10, no. 1, p. 120–132. Available at: https://eric.ed.gov/?id=EJ1251004
21. Cloete A. L. Technology and education: Challenges and opportunities. Theological Studies, 2017, vol. 73, no. 3, p. 1–7. DOI: 10.4102/hts.v73i3.4589
22. Yusuf B. N. Are we prepared enough? A case study of challenges in online learning in a private higher learning institution during the COVID-19 outbreaks. Advances in Social Sciences Research Journal, 2020, vol. 7, no. 5, p. 205–212. DOI: 10.14738/assrj.75.8211
23. Kear K., Rosewell J., Williams K., Ossiannilsson E., Rodrigo C., Sánchez-Elvira Paniagua Á., Santamaría Lancho M., Vyt A., Mellar H. Quality assessment for E-learning: A benchmarking approach. Maastricht: EADTU, 2016. Available at: http://oro.open.ac.uk/47597/
24. Xu H., Mahenthiran S. Factors that influence online learning assessment and satisfaction: Using Moodle as a Learning Management System. International Business Research, 2016, vol. 9, no. 2, p. 1–18. DOI: 10.5539/ibr.v9n2p1
25. Andryushkova O. V., Gorbunov M. A., Kozlova A. V. Learning Management System kak neobkhodimyj ehlement Blended Learning [Learning Management System as a necessary element of Blended Learning]. Otkrytoe obrazovanie — Open Education, 2017, vol. 21, no. 3, p. 80–88. (In Russian.) DOI: 10.21686/1818-4243-2017-3-80-88
26. Stobart G. Testing times: The uses and abuses of assessment. London, Routledge, 2008. 224 p. DOI: 10.4324/9780203930502
Review
For citations:
Marinosyan A.Kh., Andryushkova O.V. Multi-criteria methodology for LMS quality assessment within the framework of emergent learning system. Informatics and education. 2021;(5):4-11. (In Russ.) https://doi.org/10.32517/0234-0453-2021-36-5-4-11