Detecção de microações em vídeos faciais para análise de carga cognitiva em ambientes de aprendizado multimídia

Abstract

The technological evolution of recent years, accelerated by the Covid-19 pandemic, has been responsible for a rapid and continuous paradigm shift in education, with the adoption of multimedia learning environments. During the learning process in these environments, the absorption of content by students decreases as the volume of information transmitted to them increases. In other words, there is a cognitive overload in one or both visual and verbal channels. Currently, there is a scarcity of studies that use Computer Vision and Data Science resources for the analysis of Cognitive Load. Tools of this nature would enable automated analysis of large volumes of videos and, consequently, the evaluation and generation of multimedia content that optimize student learning. In this work, a pilot study was conducted with a sample of 13 students from the School of Medicine at the Universidad de Atacama, Chile. Thus, a methodology was developed to extract and investigate correlations between visual characteristics of the students’ faces and cognitive load. A database of facial videos of students watching multimedia-enhanced lectures on their computer screens was used. This video database was initially organized and preprocessed, followed by the application of Deep Learning models to extract visual points of interest from the face in each frame. Micro-actions were previously annotated by the researcher, and the resulting data were evaluated to identify relevant patterns related to cognitive load. In addition to addressing the main investigation of this research, the results of this study include a proof of concept for analyzing the correlation of facial expressions with individual exam scores, for further analysis of cognitive load in multimedia learning environments. The code for the experiment and the tools was made publicly available at the URL: https://github.com/cristovaor/CAST.