HyperMind: Intelligent Digital Textbook

Although every learner has different preferences in reading, textbooks have traditionally been static. Reading experiences should become more immersive and interesting if textbooks behave differently for each learner. We propose such vivid interactions optimized for the context of learning by combining a digital document and affective state recognition (e.g. interest, mental workload, and self-confidence) using smart sensors.

Online Demo at Augmented Humans 2021

Related Publications

1. Pramod Vadiraja, Andreas Dengel and Shoya Ishimaru. “Text Summary Augmentation for Intelligent Reading Assistant”. In Proceedings of the 2nd Augmented Humans International Conference (AHs '21), pp. 319–321, 2021.
2. Shoya Ishimaru, Syed Saqib Bukhari, Carina Heisel, Nicolas Großmann, Pascal Klein, Jochen Kuhn and Andreas Dengel. “Augmented Learning on Anticipating Textbooks with Eye Tracking”. In Positive Learning in the Age of Information (PLATO), pp. 387–398, 2018.
3. Shoya Ishimaru, Ko Watanabe, Nicolas Großmann, Carina Heisel, Pascal Klein, Yutaka Arakawa, Jochen Kuhn and Andreas Dengel. “HyperMind Builder: Pervasive User Interface to Create Intelligent Interactive Documents”. In Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp '18 Adjunct), pp. 357–360, 2018.
4. Shoya Ishimaru, Soumy Jacob, Apurba Roy, Syed Saqib Bukhari, Carina Heisel, Nicolas Großmann, Michael Thees, Jochen Kuhn and Andreas Dengel. “Cognitive State Measurement on Learning Materials by Utilizing Eye Tracker and Thermal Camera”. In Proceedings of the 14th IAPR International Conference on Document Analysis and Recognition (ICDAR HDI '17), pp. 32–36, 2017.
5. Shoya Ishimaru, Syed Saqib Bukhari, Carina Heisel, Jochen Kuhn and Andreas Dengel. “Towards an Intelligent Textbook: Eye Gaze Based Attention Extraction on Materials for Learning and Instruction in Physics”. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp '16 Adjunct), pp. 1041–1045, 2016.