Design Framework for Multimodal Learning Analytics Leveraging Human Observations

Collecting and processing data from learning-teaching settings like classrooms is costly and time-consuming for human observers. Multimodal Learning Analytics (MMLA) is an avenue to approach in-depth data from multiple streams of data and information. MMLA researchers are working towards more theory-driven development of these systems, emphasizing transparent and explainable data and the availability of these systems. This article presents a design framework for leveraging human observations to integrate learning theory when designing an MMLA system. Supported by a pilot study using indicators of participation in group work, this study shows promise in human-understandable measures and analysis in MMLA to make connections between sensor data and human observations. However, it also shows challenges in the rigid nature of automatic analysis of data accessible by sensors.