,
Evaluating Multi-Knowledge Component Interpretability of Deep Knowledge Tracing Models in Programming
Proceedings of the 17th International Conference on Educational Data Mining, стр. 288--295. Atlanta, Georgia, USA, International Educational Data Mining Society, (июля 2024)
DOI: 10.5281/zenodo.12729818
Аннотация
Knowledge tracing (KT) models have been a commonly used tool for tracking students' knowledge status. Recent advances in deep knowledge tracing (DKT) have demonstrated increased performance for knowledge tracing tasks in many datasets. However, interpreting individual knowledge states from DKT models could be challenging when tracking multiple knowledge components (KCs) in one student submission attempt. In this paper, we evaluate the ability of DKT models to track students' knowledge using AUC scores. We further propose two possible solutions to improve multi-KC tracking performance: incorporating a KC layer and/or code features into the DKT models. In experiments, we compare the DKT to the proposed models and evaluate KC tracking performance in a CS1 dataset. Our results indicate that while all four models perform similarly on problem correctness predictions, incorporating KC layers may improve KC tracking performance, though limited. Our research shows that there's an important performance gap in the research of DKT when tracking multiple skills, especially on incorrect submissions.
