Deep Knowledge Tracing on Skills with Small Datasets

Deep Knowledge Tracing (DKT), as well as other machine learning approaches, is biased toward data used during the training step. Thus, for problems where we have few amounts of data for training, the generalization power will be low, and the models will tend to work well on classes containing many samples and poorly on those with few. This situation is quite common in educational data where some skills are very difficult to master while others are very easy. As a result, there will be less data on students who correctly answered questions related to difficult skills, but also on those who provided incorrect answers to questions related to easy skills. In those cases, the DKT is unable to correctly predict the student's answers to questions associated with these skills. To improve DKT performance under these conditions, we have developed a two-fold approach. Firstly, the loss function is modified so that some skills are masked to force the model's attention on those that are difficult to generalize. Secondly, to cope with the limited amount of data on some skills, we proposed a hybrid architecture that integrates a priori (expert) knowledge with DKT through an attentional mechanism. The resulting model accurately tracks student Knowledge in the Logic-Muse Intelligent Tutoring System (ITS), compared to the traditional Bayesian Knowledge Tracing (BKT) and the original DKT.