|Computer Systems Science & Engineering |
Exploring Students Engagement Towards the Learning Management System (LMS) Using Learning Analytics
1Department of Information Systems, Faculty of Computer Science & Information Technology, University Malaya, Kuala Lumpur, 50603, Malaysia
2Department of Information systems, College of Computer and Information Sciences, Shaqra University, Saudi Arabia
3School of Computer Science and Engineering, SCE, Taylor’s University, Subang Jaya, 47500, Malaysia
*Corresponding Author: Suraya Hamid. Email: email@example.com
Received: 22 November 2020; Accepted: 17 December 2020
Abstract: Learning analytics is a rapidly evolving research discipline that uses the insights generated from data analysis to support learners as well as optimize both the learning process and environment. This paper studied students’ engagement level of the Learning Management System (LMS) via a learning analytics tool, student’s approach in managing their studies and possible learning analytic methods to analyze student data. Moreover, extensive systematic literature review (SLR) was employed for the selection, sorting and exclusion of articles from diverse renowned sources. The findings show that most of the engagement in LMS are driven by educators. Additionally, we have discussed the factors in LMS, causes of low engagement and ways of increasing engagement factors via the Learning Analytics approach. Nevertheless, apart from recognizing the Learning Analytics approach as being a successful method and technique for analyzing the LMS data, this research further highlighted the possibility of merging the learning analytics technique with the LMS engagement in every institution as being a direction for future research.
Keywords: Learning analytics; student engagement; learning management system; systematic literature review
In recent years, Learning Analytics has emerged as a promising area of research for tracing the digital footprints of learners as well as extracting useful knowledge of a student’s learning progress. Since the availability of the increased amount of data alongside the fact that potential benefits of learning analytics can be far-reaching to all stakeholders in the education field, such as students, teachers, leaders, and policymakers, many higher learning institutions have started using the new analytics models and the data stored in its Learning Management System (LMS) as a way of reaching their respective short- and long-term goals and objectives. However, the data produced from the LMS cannot only be transformed effectively into meaningful information but can be used to reflect on the learners’ interaction within a system. Moreover, this can improve the understanding of the user’s behaviour and characteristics that contributes to low LMS usage. The data mining and analysis applied in higher education is still in the early stages . LMS has been introduced as a technology to provide a better option for institutions to endorse more student participation and engagement . The LMS trace and log data or virtual learning environments (VLEs) that had evolved from traditional learning methods are now regarded as crucial learning analytics data sources . Although a study conducted by Nguyen et al. , revealed that most of the LMS interactions occurred between students, instructors and the institution’s resources. Moreover, Jovanovic et al.  discovered a low LMS interaction and engagement level among these learners.
Similarly, Zanjani et al.  and Emelyanova et al.  stated the use of LMS as a means of providing a flexible platform as well as a secure learning environment for students having different lifestyles. It is a tool that can help students and lecturers, improve learning and management processes. The LMS however, made it somewhat difficult for the educator to gauge the student’s behaviour and their study patterns as compared to the traditional face-to-face learning . Despite the above shortcoming, the user’s participation in LMS can still be used to gauge the student’s real learning behaviour since it may offer a different and more personal learning experience than those of conventional classroom learning. The examination on log information of student will radiate instinct into figuring out how to guarantee better decisions and be responsible to different partners. Assessment from the learning information will improve the learning condition to be more organized, as it diverges from customary to a web-based learning background. Consequently, learning investigation can recognize the relationship between log information and understudy conduct . Learning Analytics foresee understudy presentation, dropout rate, last stamps and grades, maintenance of the understudies in LMS, as well as coming up with an early cautioning framework for the understudies, all through their examination in college . Also, the execution of Learning Analytics to the college understudy helps with the college organization, assists the employees to eyewitness the development of the students and perceive how they advance in their study. Understudy commitment in LMS is an area that is yet under the disclosure of its status. In this investigation, the scientist utilizes commitment in LMS to foresee the presentation of the understudy. The issue of commitment concerns imparting and associating with others, that is, those that are still new impacts on the understudy’s conduct. In any case, the commitment and cooperation in LMS possibly occur if there is a push factor including the speakers or the teacher’s investment. By utilizing the learning examination, we can break down the status of the understudy’s commitment in the LMS stage. For this reason, this paper utilises the Systematic Literature Review of the Learning Analytics as a way of gauging the student’s level of engagement in the Learning Management System. Similar work has been conducted on the learning dashboard  the benefits and challenges of learning analytics  as well as the practice of learning analytics and educational data mining . This systematic literature review focused on student engagement factors with the application of learning analytics, which are the main objectives of this paper. The significance of this paper is to grab the understanding of student engagement factors classification using the learning analytics technique. Whereas the existing systematic literature review focused on the benefits and the challenges faced for learning success from the use of Learning Analytics.
Further, this paper elaborates on the factors that influence students’ engagement and their interaction levels in the LMS. The following section explains the methods applied in conducting this systematic literature review. The results and discussion of findings are the last sections of this systematic literature review. Finally, we summarize and highlight the study limitations and craft recommendations for future work.
Since Systematic Literature Review (SLR) utilises in-depth and absolute methodologies as well as guidelines in the evaluation of related literature of a research topic , this research, therefore, implemented the SLR methodology as suggested by Kitchenham  and Kitchenham et al.  in the investigation of factors that influences students’ engagement level in the Learning Management System. Composing of several isolated activities, the SLR is sorted into input, process and output phases. While the input process is a systematic review planning that was identified from the research question, the processing stage, on the other hand, is a phase that ensures the validity and reliability of the systematic review process as relates adhering to each of the prescribed step, which is stated in the evaluation, inclusion and exclusion criteria, as well as the quality assessment stages before concluding with a review report.
2.1 Planning Stages
Identifying the necessities - This study needs to identify the necessity of a comprehensive literature review on student’s engagement factors in the Learning Management System from the perspective of learning analytics, 3 research questions were thus generated for the literature review process.
Research questions - The research questions used in addressing the needs of this study are outlined as follows:
• How do students’ manage LMS engagement levels?
• What are the contributing factors to students’ engagement level in LMS as shown by the learning analytics?
• What are the learning analytics methods used for analysing the student’s engagement level?
2.2 Conducting Stages
In the second SLR stage of conducting the review, this article will not only examine the few primary scientific databases that were obtained from the selected keywords but will also highlight the inclusion and exclusion criteria that were used in this research.
2.3 Literature Search Procedure and Criteria Selection
2.3.1 Search Keywords
The keywords that were used in the search engine are shortlisted as follows:
• Student engagement AND learning management
• Learning Analytic AND learning management system
• Student Engagement model in higher education
• Student Engagement AND learning analytics
2.3.2 Online Databases
The online databases that were used in the Systematic Literature Review such as Web of Science, Science Direct, IEEE and Scopus database as described in Tab. 1, were selected based on their various article sources as well as their credibility levels.
2.3.3 Publication Selection
The inclusion criteria used for determining the relevant literature (journal papers, conference papers) are listed as follows: Studies that are related to the Learning Management System Engagement model, Studies that had examined the LMS log data through the use of Learning Analytics and Studies or journals that were published between 2005 to 2019. The exclusion criteria used are listed as follows: The omission of unrelated material, not pertinent to the research question and research field, only abstract paper and editorial paper. Selection of Primary Sources Tab. 2 shows the number of publications that appeared from using the keywords of the mentioned databases in Tab. 1, where those with the ‘Student Engagement AND Learning Management System’ keywords were found to have garnered a total of 850 articles, the ‘Learning Analytics AND Learning Management System’ with a total of 367 outcomes, while the ‘Student Engagement Model in Learning Management System’ and the ‘Student Engagement AND Learning Analytics’ obtained around 171 and 269 articles respectively.
Fig. 1 below shows the process of the systematic literature review from the four databases that are used in the structured keyword search.
From these four databases, a total of 892 papers that matched the earlier mentioned keywords of the previous section were found to have dwindled to 127 (100 journals and 27 conference papers) after undergoing the filtering of the inclusion and exclusion phases. Since the quality assessment step is important for ensuring the selection of higher impact journals, 58 relevant papers (45 journals and 13 conference papers) were therefore shortlisted to produce a better systematic literature review output. The selection on the quality attribute is thus defined as such:
• Adequate findings and results that have been displayed and discussed.
• The online learning platform was discussed and implemented in the research.
• Involvement of the learning analytic approach.
This section discusses the results obtained from the SLR process, as presented in sub-sections below as per the research question. The details of discussion from the findings are described in a means of answering each research question.
3.1 [RQ1]: How do Students’ Manage LMS Engagement Levels?
The student’s involvement level on the LMS was gauged from the diverse engagement, interaction and participation levels of the LMS as shown in Tab. 3. As depicted from the table, these LMS interactions had not only involved the students and their peers but also their instructors and the system. While the student to student interaction in the LMS platform might have occurred as a result of the students’ communication and participation in a forum, the student-lecturer interaction level, on the other hand, was based on the students’ motivation for communicating with their lecturers. For this reason, the student-lecturer interaction is seen as a two-way communication when both the student and lecturer strive to connect through messages or via online forums and discussions. As for the student-system interaction, this occurs when the student starts to log into the LMS and performs a download of a required file. Under this circumstance, the interaction between the system and resource is also regarded as the student-system interaction since it had involved the downloading of files and viewing of systems upon the login of the respective student’s account. According to Wang et al.  the learning design of the LMS can be divided into resource sharing and aggregation strategies that are depicted by not only the course forums but also the post and blog contents discussions as well as negotiations like those of Twitter and Facebook hence, facilitating a more thorough decision-making process. In another study, Mirriahi et al.  identified four clusters of students’ learning profiles from the behaviour logs of their video annotation tool. The categorization of the user profiles, namely the intensive users, selective users, limited users and non-users was based on the usage intensity of the various learning tools that are available within a learning management system. Pardo et al.  too had stated that information procured from learning logs and digital traces of learning analytics do not only provide the student with timely personalised feedback at scale but also enhances the student’s satisfaction; where the student’s inclination towards a task-oriented strategy rather than any other identified strategy is found to have been resulted from the formative feedback provided. Due to the high and regular usage among the strategies exhibited by most of the users, an intensive user is thus defined as one who spends more time in sourcing and assessing the additional required materials, while the regular users are seen as those spending the same amount of time on lesser significant materials. While a comprehensive learner is defined as possessing a high self-regulated learning profile that adheres to the course structure and completion of the course material via the online platform, the targeting learner on the other hand only aims at passing the minimum course requirement with a lower course engagement level. As for the sampling learner, this type of learner is defined as having a low self-regulated learning profile with a typical behaviour of not completing the required course .
3.2 [RQ2]: What are the Contributing Factors to the Student’s LMS Usage; As Shown by the Learning Analytics?
The factors that influence a student’s engagement level in the LMS can thus be divided into several themes as those shown in Tab. 4. As depicted in the table, these engagement factors are found to have been influenced by the motivation and rewards from using the LMS, the student’s satisfaction level on the system as well as the ease of usage. Most of the users agreed that an interactive and a user-friendly LMS interface would entice them on completing their assigned tasks, while a dull and unattractive design indicate otherwise, but also diminish their interest to engage in forum discussions . Other than the above, the other reasons that are cited for the low LMS engagement are higher workload, which is because of its cumbersome system . Since a low self-motivation is observed among the students, the motivating factor for using the LMS is seen as leaning towards the learning interface and the LMS design [6,28,29]; where those consisting of obsolete learning materials are hindering the student’s involvement in the LMS . For this reason, a higher LMS engagement is not only perceived as being influenced by the system’s design and its overall performance but is also stemmed from the reward system for using the LMS and the contributions provided by the lecturers . As such, the LMS should be improved with a more engaging system that offers rewards for a minimum level of system usage since this can be used to trigger a student’s motivation level  and consequently a higher LMS participation level . Next, the commitment in LMS is low because the understudy does not effectively take part in the LMS given the dawdling conduct. Hesitation is the demonstration of postponement and the delaying of the culmination of an assignment . The understudy’s exertion and time spent that should be yielded by the understudy in LMS are the motivation behind why the commitment in LMS is low because the understudy does not wish to invest more energy in LMS . Hesitation is likewise the learning procedure utilized by the understudy; it causes the understudy to interface however the commitment they produce are not dynamic . The activity of less than ideal and straightforward deferral of the entrance to the material in LMS is likewise a purpose behind the student dawdling . The inspirations are the understudy’s craving and enthusiasm for taking an interest in the action. The understudy inspiration relies upon three dependent measurements, intellectual is the self-guideline, nature of the learning methodologies, the feeling is the confidence, which alludes to the understudy’s exhibition and conduct is simply the understudy’s viability, and this is the place the inspiration originates from . The inspiration is impacted by the cooperation in the LMS . Student’s inspiration additionally impacts the social condition, the shared trust from the learning framework and the prizes they get when participating in the framework . Nguyen et al.  talked about the instructor association in the web-based learning structure and their revelations demonstrate that VLE responsibility isn’t simply considerably affected by the learning design, yet is particularly impacted by how educators inside and across over modules, balance their learning plan practices on seven days by-week premise. The appointment of steady re-evaluation into a module is found to have provoked extended understudy duty with that module, as assessed by joint efforts with the VLE. The evaluation in learning the executive’s framework makes greater commitment if the teacher structures the learning exercises well. A summary of the findings and the engagement factors of the LMS are thus shown in Tab. 4.
3.3 [RQ3]: What Are the Learning Analytics Methods Used for Analysing the Student’s Engagement Level?
The various learning analytic methods that were applied in the LMS trace and log data can be classified as qualitative and quantitative, where the former would involve the use of surveys, while the latter would focus on the analysis, validation and clarification of the obtained data. The learning analytic methods that are utilised by the researchers are depicted in Tab. 5. Apart from using the qualitative methods to determine if the LMS elements design is affecting the user’s engagement , the testing of past theories under a different context  and the gathering of student’s feedback  are other prediction methods that are used for predicting the student’s behaviour use of the LMS. A summary of the learning analytics methods that are practised by researchers, based on the review is shown in the table below.
The above tab. 5 shows the type of data that can be obtained from the log data of an online platform, like those of the LMS or through the use of a qualitative method. It implies that Learning Analytics is not being bound by specific methods. Since the results varied according to the research outcome, these methods are found to have consisted of the data mining technique; where the classification and clustering methods [8,17,22,27,28,67] are used to cluster the data and to form relationships between respective variables, while those of the Anova and Manova techniques [2,27,51,68] are used together with multiple regressions to establish a correlation between the subjects of study. Apart from the descriptive analysis that can be used to validate the results of the research , some researchers had also utilised the partial least square method in their data analysis and the testing of a developed model [32,47].
4 Challenge and Drawback of LMS
The major challenge of learning analytic research is the data. A more emphasized result requires more data. The data is very big, to the extent that specific methods and tools must be used to analyse them based on the Researcher’s need. Even though LMS brings some form of advancement into the education world, yet it has its disadvantages, where it may not record the student’s emotional or real situations faced during online learning. The complex function in LMS may lead the student and educator into confusion, as regarding how to use the function; thus, more systematic and simple functions can make the learning process better, thence producing a more realistic outcome. In terms of student usage in LMS, issues such as procrastination might occur, as the student may wait until the last minute to submit the given task.
5 Future Work
Although many achievements were obtained from studying the student’s LMS engagement level through the use of Learning Analytics, the systematic review of this study is still restricted by some of the future research direction.
With most of the students in this era being digitally literate, classroom engagement has now reached an all-time low level, with lecturers competing against countless diversions from phones, tablets and laptops. For this reason, countless resources should therefore be introduced to ensure enhancing the education material and making learning more fun and effective; where future researchers can look into the possibility of creating apps that support classroom ‘gamification,’ competitive scenarios as well as the distribution of points and rewards system.
5.2 Privacy and Security
Since privacy and security have become an increasing concern among the students, steps should therefore be taken to enhance the security system of the LMS. For this reason, the customization of system administration paths or more integrated learning environments for better security control of the LMS can thence be considered as part of the future research for this study.
5.3 Machine Learning
Despite the successful use of machine learning algorithms in other domain applications, this method is however not used in predicting the student’s behaviour on the LMS usage. Since the machine learning algorithm has the potential of detecting the learners’ study pattern and behaviour through the use of system automation, we, therefore, suggest that future researchers shift their attention to the application of machine learning in the LMS for a better assessment of the student’s engagement level.
The comprehensive SLR is performed on the engagement factors in the Learning Management System (LMS) to discover the factors for influencing students’ LMS engagement level and consequently the intensity of the student’s engagement level, but also the student’s engagement strategies. To support the learning environment in LMS to become more functioning, few approaches can be used like more active strategies in the educator approach towards the student. Also, the program requirement must be met, and the educator needs to be focused on student performance as well as a unique task. This is important to increase student engagement as the student’s motivation toward the LMS is the biggest thing to handle. The LMS design, awards and satisfaction from the student is a critical factor of the engagement. There are a lot of techniques that can be used as learning analytics tools. This method can help the institution to find what the students’ needs are, wherein the institution can map student requirement and institution objective. Since learning analytics and learning management system are regarded as an active research area with quite a number of publications, the engagement factors that are extracted from the use of SLR thus suggests the further need of generating a higher engagement level of the LMS. Apart from offering an overview of the existing LMS research and the engagement level of an LMS environment, this research also offers supplementary information on the learning analytic methods hence, prompting the possibility of a better learning analytics research.
Funding Statement: This research was supported by the University of Malaya, Bantuan Khas Penyelidikan under the research grant of BKS083-2017 and Fundamental Research Grant Scheme (FRGS) under Grant number FP112-2018A from the Ministry of Education Malaysia, Higher Education.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
|This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.|