Preview

Informatics and education

Advanced search

Multi-criteria methodology for LMS quality assessment within the framework of emergent learning system

https://doi.org/10.32517/0234-0453-2021-36-5-4-11

Abstract

In the article, using the case of Learning Management System (LMS), we consider a methodology for assessing the quality of education, taking into account the requests of experts, lecturers, and students. There is a growing relevance of such research due to the digitalization of education, the transition to distance learning as the main or auxiliary form of organizing the educational process. The applied methodology relies on the earlier developed concept of emergent learning and the system for the assessment of the quality of education, using the conception of negentropy. By adapting this methodology to the LMS assessment task, we show the importance of creating a hierarchical scheme of modules and functional elements necessary for an effective organization of the learning process. In order to provide an integral assessment, it is necessary, according to our position, to determine the weighting factor of each module and of each criterion by which this module is assessed and then to determine the values of each criterion for a specific LMS. A proposed flexible assessment system that takes into account the requests of all aspects of the educational process, allows, by modifying the weighting factors, to adapt the equation in accordance with the specificity of the subject area and educational institution within which the LMS is used. As an advantage of this methodology, it makes it possible not only to compare different LMSs in terms of quality but also to identify areas in which it is possible to improve and refine existing LMSs. We analyze the risks associated with the transformation of assessment systems from an auxiliary tool into an end in itself. In this regard, we propose an algorithm for separating qualitative and quantitative factors accounting, which helps to avoid the negative effect of the absolutization of assessment systems. In the conclusion, we outline the ways of developing the design of such a system of the management of educational process in which quality requirements prevail over circumstantial ones.

About the Authors

A. Kh. Marinosyan
State Academic University for the Humanities
Russian Federation

Andreas Kh. Marinosyan, Specialist, 119049, Russia, Moscow, Maronovskiy per., 26



O. V. Andryushkova
Lomonosov Moscow State University, Faculty of Chemistry
Russian Federation

Olga V. Andryushkova, Candidate of Sciences (Chemistry), Docent, Head of the Laboratory of Methods of Teaching Chemistry of the Department of General Chemistry, 119991, Russia, Moscow, Leninskie gory, 1, building 3



References

1. Andryushkova O. V., Grigoriev S. G. Ehmergentnoe obuchenie v informatsionno-obrazovatel’noj srede [Emergent training in the information educational environment]. Moscow, Obrazovanie i Informatika, 2018. 104 p. (In Russian.)

2. Kellaghan T., Stufflebeam D. L. International handbook of educational evaluation. Dordrecht, Springer, 2003. 1060 p. DOI: 10.1007/978-94-010-0309-4

3. Dunn L., Morgan C., O’Reilly M., Parry S. The student assessment handbook: New directions in traditional and online assessment. London, Routledge, 2003. 320 p. DOI: 10.4324/9780203416518

4. Cox K., Imrie B. W., Miller A. H. Student assessment in higher education: A handbook for assessing performance. London, Routledge, 1998. 292 p. Available at: https://www.routledge.com/Student-Assessment-in-Higher-Education-A-Handbook-for-Assessing-Performance/Cox-Imrie-Miller/p/book/9780749427979

5. Andryushkova O., Grigoriev S. The influence online learning quality criteria selection on negentropy. Proc. 4th Int. Conf. on Informatization of Education and E-learning Methodology: Digital Technologies in Education. 2020, vol. 2770, p. 127–139. Available at: http://ceur-ws.org/Vol-2770/paper17.pdf

6. Andreev A. A. Otsenka kachestva onlajn kursov [Evaluation of the quality of online courses]. Territoriya nauki — Territory of Science, 2015, no. 1, p. 20–26. (In Russian.)

7. Horvat A., Dobrota M., Krsmanovic M., Cudanov M. Student perception of Moodle learning management system: a satisfaction and significance analysis. Interactive Learning Environments, 2015, vol. 23, is. 4, p. 515–527. DOI: 10.1080/10494820.2013.788033

8. Han I., Shin W. S. The use of a mobile learning management system and academic achievement of online students. Computers & Education, 2016, vol. 102, p. 79–89. DOI: 10.1016/j.compedu.2016.07.003

9. Farid S., Ahmad R., Alam M., Akbar A., Chang V. A sustainable quality assessment model for the information delivery in E-learning systems. Information Discovery and Delivery, 2018, vol. 46, is. 1, p. 1–25. DOI: 10.1108/IDD-11-2016-0047

10. Cerezo R., Bogarín A., Esteban M., Romero C. Process mining for self-regulated learning assessment in e-learning. Journal of Computing in Higher Education, 2020, vol. 32, is. 1, p. 74–88. DOI: 10.1007/s12528-019-09225-y

11. Henriksen D., Creely E., Henderson M. Folk pedagogies for teacher transitions: Approaches to synchronous online learning in the wake of COVID-19. Journal of Technology and Teacher Education, 2020, vol. 28, is. 2, p. 201–209.

12. Gillett-Swan J. The challenges of online learning: Supporting and engaging the isolated learner. Journal of Learning Design, 2017, vol. 10, no. 1, p. 20–30. DOI: 10.5204/jld.v9i3.293

13. Kara M., Erdoğdu F., Kokoç M., Cagiltay K. Challenges faced by adult learners in online distance education: A literature review. Open Praxis, 2019, vol. 11, no. 1, p. 5–22. DOI: 10.5944/openpraxis.11.1.929

14. Fresen J. W. Embracing distance education in a blended learning model: challenges and prospects. Distance Education, 2018, vol. 39, is. 2, p. 224–240. DOI: 10.1080/01587919.2018.1457949

15. Damary R., Markova T., Pryadilina N. Key challenges of on-line education in multi-cultural context. Procedia — Social and Behavioral Sciences, 2017, vol. 237, p. 83–89. DOI: 10.1016/j.sbspro.2017.02.034

16. Ouma R., Nkuyubwatsi B. Transforming university learner support in open and distance education: Staff and students perceived challenges and prospects. Cogent Education, 2019, vol. 6, is. 1. DOI: 10.1080/2331186X.2019.1658934

17. Nage-Sibande B., Morolong B. L. A trend analysis of opportunities and challenges of open and distance learning provision in dual-mode institutions. Distance Education, 2018, vol. 39, is. 4, p. 495–510. DOI: 10.1080/01587919.2018.1457951

18. Palvia S., Aeron P., Gupta P., Mahapatra D., Parida R., Rosner R., Sindhi S. Online education: Worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 2018, vol. 21, is. 4, p. 233–241. DOI: 10.1080/1097198X.2018.1542262

19. Kebritchi M., Lipschuetz A., Santiague L. Issues and challenges for teaching successful online courses in higher education: A literature review. Journal of Educational Technology Systems, 2017, vol. 46, is. 1, p. 4–29. DOI: 10.1177/0047239516661713

20. Chen S.-Y., Basma D., Ju J., Ng K.-M. Opportunities and challenges of multicultural and international online education. Professional Counselor, 2020, vol. 10, no. 1, p. 120–132. Available at: https://eric.ed.gov/?id=EJ1251004

21. Cloete A. L. Technology and education: Challenges and opportunities. Theological Studies, 2017, vol. 73, no. 3, p. 1–7. DOI: 10.4102/hts.v73i3.4589

22. Yusuf B. N. Are we prepared enough? A case study of challenges in online learning in a private higher learning institution during the COVID-19 outbreaks. Advances in Social Sciences Research Journal, 2020, vol. 7, no. 5, p. 205–212. DOI: 10.14738/assrj.75.8211

23. Kear K., Rosewell J., Williams K., Ossiannilsson E., Rodrigo C., Sánchez-Elvira Paniagua Á., Santamaría Lancho M., Vyt A., Mellar H. Quality assessment for E-learning: A benchmarking approach. Maastricht: EADTU, 2016. Available at: http://oro.open.ac.uk/47597/

24. Xu H., Mahenthiran S. Factors that influence online learning assessment and satisfaction: Using Moodle as a Learning Management System. International Business Research, 2016, vol. 9, no. 2, p. 1–18. DOI: 10.5539/ibr.v9n2p1

25. Andryushkova O. V., Gorbunov M. A., Kozlova A. V. Learning Management System kak neobkhodimyj ehlement Blended Learning [Learning Management System as a necessary element of Blended Learning]. Otkrytoe obrazovanie — Open Education, 2017, vol. 21, no. 3, p. 80–88. (In Russian.) DOI: 10.21686/1818-4243-2017-3-80-88

26. Stobart G. Testing times: The uses and abuses of assessment. London, Routledge, 2008. 224 p. DOI: 10.4324/9780203930502


Review

For citations:


Marinosyan A.Kh., Andryushkova O.V. Multi-criteria methodology for LMS quality assessment within the framework of emergent learning system. Informatics and education. 2021;(5):4-11. (In Russ.) https://doi.org/10.32517/0234-0453-2021-36-5-4-11

Views: 468


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.


ISSN 0234-0453 (Print)
ISSN 2658-7769 (Online)