CoALA: Confidence-Aware Learning Assistant

Self-confidence plays an important role in improving the quality of knowledge. Undesirable situations such as confident incorrect and unconfident correct knowledge prevent learners from revising their knowledge because it is not always easy for them to perceive the situations. Therefore we propose a system that estimates self-confidence by sensors and gives feedback about which question should be reviewed carefully.

We report the results of three studies measuring its effectiveness. (1) On a well-controlled dataset with 10 participants, our approach detected confidence and unconfidence with 80% average precision. (2) With the help of 20 participants, we observed that correct answer rates of questions were increased by 14% and 17% by giving feedback about correct answers without confidence and incorrect answers with confidence, respectively. (3) We conducted a large-scale data recording in a private school (72 high school students solved 14,302 questions) to investigate effective features and the number of required training samples.

Related Publications

1. Shoya Ishimaru, Takanori Maruichi, Andreas Dengel and Koichi Kise. “Confidence-Aware Learning Assistant”. In arXiv preprint arXiv:2102.07312, 2021.
2. Shoya Ishimaru, Takanori Maruichi, Koichi Kise and Andreas Dengel. “Gaze-Based Self-Confidence Estimation on Multiple-Choice Questions and Its Feedback”. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20 Asian CHI Symposium 2020), 2020.
3. Takanori Maruichi, Shoya Ishimaru and Koichi Kise. “Self-confidence Estimation on Vocabulary Tests with Stroke-level Handwriting Logs”. In Proceedings of the 15th IAPR International Conference on Document Analysis and Recognition (ICDAR HDI'19), pp. 18–22, 2019.