Joudeh, I. O.; Cretu, A. -M.; Bouchard, S. Predicting the Arousal and Valence Values of Emotional States Using Learned, Predesigned, and Deep Visual Features † Journal Article In: Sensors, vol. 24, no. 13, 2024, ISSN: 14248220 (ISSN), (Publisher: Multidisciplinary Digital Publishing Institute (MDPI)). @article{joudeh_predicting_2024,
title = {Predicting the Arousal and Valence Values of Emotional States Using Learned, Predesigned, and Deep Visual Features †},
author = {I. O. Joudeh and A. -M. Cretu and S. Bouchard},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85198382238&doi=10.3390%2fs24134398&partnerID=40&md5=cefa8b2e2c044d02f99662af350007db},
doi = {10.3390/s24134398},
issn = {14248220 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {Sensors},
volume = {24},
number = {13},
abstract = {The cognitive state of a person can be categorized using the circumplex model of emotional states, a continuous model of two dimensions: arousal and valence. The purpose of this research is to select a machine learning model(s) to be integrated into a virtual reality (VR) system that runs cognitive remediation exercises for people with mental health disorders. As such, the prediction of emotional states is essential to customize treatments for those individuals. We exploit the Remote Collaborative and Affective Interactions (RECOLA) database to predict arousal and valence values using machine learning techniques. RECOLA includes audio, video, and physiological recordings of interactions between human participants. To allow learners to focus on the most relevant data, features are extracted from raw data. Such features can be predesigned, learned, or extracted implicitly using deep learners. Our previous work on video recordings focused on predesigned and learned visual features. In this paper, we extend our work onto deep visual features. Our deep visual features are extracted using the MobileNet-v2 convolutional neural network (CNN) that we previously trained on RECOLA’s video frames of full/half faces. As the final purpose of our work is to integrate our solution into a practical VR application using head-mounted displays, we experimented with half faces as a proof of concept. The extracted deep features were then used to predict arousal and valence values via optimizable ensemble regression. We also fused the extracted visual features with the predesigned visual features and predicted arousal and valence values using the combined feature set. In an attempt to enhance our prediction performance, we further fused the predictions of the optimizable ensemble model with the predictions of the MobileNet-v2 model. After decision fusion, we achieved a root mean squared error (RMSE) of 0.1140, a Pearson’s correlation coefficient (PCC) of 0.8000, and a concordance correlation coefficient (CCC) of 0.7868 on arousal predictions. We achieved an RMSE of 0.0790, a PCC of 0.7904, and a CCC of 0.7645 on valence predictions. © 2024 by the authors.},
note = {Publisher: Multidisciplinary Digital Publishing Institute (MDPI)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The cognitive state of a person can be categorized using the circumplex model of emotional states, a continuous model of two dimensions: arousal and valence. The purpose of this research is to select a machine learning model(s) to be integrated into a virtual reality (VR) system that runs cognitive remediation exercises for people with mental health disorders. As such, the prediction of emotional states is essential to customize treatments for those individuals. We exploit the Remote Collaborative and Affective Interactions (RECOLA) database to predict arousal and valence values using machine learning techniques. RECOLA includes audio, video, and physiological recordings of interactions between human participants. To allow learners to focus on the most relevant data, features are extracted from raw data. Such features can be predesigned, learned, or extracted implicitly using deep learners. Our previous work on video recordings focused on predesigned and learned visual features. In this paper, we extend our work onto deep visual features. Our deep visual features are extracted using the MobileNet-v2 convolutional neural network (CNN) that we previously trained on RECOLA’s video frames of full/half faces. As the final purpose of our work is to integrate our solution into a practical VR application using head-mounted displays, we experimented with half faces as a proof of concept. The extracted deep features were then used to predict arousal and valence values via optimizable ensemble regression. We also fused the extracted visual features with the predesigned visual features and predicted arousal and valence values using the combined feature set. In an attempt to enhance our prediction performance, we further fused the predictions of the optimizable ensemble model with the predictions of the MobileNet-v2 model. After decision fusion, we achieved a root mean squared error (RMSE) of 0.1140, a Pearson’s correlation coefficient (PCC) of 0.8000, and a concordance correlation coefficient (CCC) of 0.7868 on arousal predictions. We achieved an RMSE of 0.0790, a PCC of 0.7904, and a CCC of 0.7645 on valence predictions. © 2024 by the authors. |
Sheehy, L.; Bouchard, S.; Kakkar, A.; Hakim, R. El; Lhoest, J.; Frank, A. Development and Initial Testing of an Artificial Intelligence-Based Virtual Reality Companion for People Living with Dementia in Long-Term Care Journal Article In: Journal of Clinical Medicine, vol. 13, no. 18, 2024, ISSN: 20770383 (ISSN), (Publisher: Multidisciplinary Digital Publishing Institute (MDPI)). @article{sheehy_development_2024,
title = {Development and Initial Testing of an Artificial Intelligence-Based Virtual Reality Companion for People Living with Dementia in Long-Term Care},
author = {L. Sheehy and S. Bouchard and A. Kakkar and R. El Hakim and J. Lhoest and A. Frank},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85205071099&doi=10.3390%2fjcm13185574&partnerID=40&md5=844732ff858a0d5feb0a95a54093ad4d},
doi = {10.3390/jcm13185574},
issn = {20770383 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {Journal of Clinical Medicine},
volume = {13},
number = {18},
abstract = {Background/Objectives: Feelings of loneliness are common in people living with dementia (PLWD) in long-term care (LTC). The goals of this study were to describe the development of a novel virtual companion for PLWD living in LTC and assess its feasibility and acceptability. Methods: The computer-generated virtual companion, presented using a head-mounted virtual reality display, was developed in two stages. In Stage 1, the virtual companion asked questions designed to encourage conversation and reminiscence. In Stage 2, more powerful artificial intelligence tools allowed the virtual companion to engage users in nuanced discussions on any topic. PLWD in LTC tested the application at each stage to assess feasibility and acceptability. Results: Ten PLWD living in LTC participated in Stage 1 (4 men and 6 women; average 82 years old) and Stage 2 (2 men and 8 women; average 87 years old). Session lengths ranged from 0:00 to 5:30 min in Stage 1 and 0:00 to 53:50 min in Stage 2. Speech recognition issues and a limited repertoire of questions limited acceptance in Stage 1. Enhanced conversational ability in Stage 2 led to intimate and meaningful conversations with many participants. Many users found the head-mounted display heavy. There were no complaints of simulator sickness. The virtual companion was best suited to PLWD who could engage in reciprocal conversation. After Stage 2, response latency was identified as an opportunity for improvement in future versions. Conclusions: Virtual reality and artificial intelligence can be used to create a virtual companion that is acceptable and enjoyable to some PLWD living in LTC. Ongoing innovations in hardware and software will allow future iterations to provide more natural conversational interaction and an enhanced social experience. © 2024 by the authors.},
note = {Publisher: Multidisciplinary Digital Publishing Institute (MDPI)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background/Objectives: Feelings of loneliness are common in people living with dementia (PLWD) in long-term care (LTC). The goals of this study were to describe the development of a novel virtual companion for PLWD living in LTC and assess its feasibility and acceptability. Methods: The computer-generated virtual companion, presented using a head-mounted virtual reality display, was developed in two stages. In Stage 1, the virtual companion asked questions designed to encourage conversation and reminiscence. In Stage 2, more powerful artificial intelligence tools allowed the virtual companion to engage users in nuanced discussions on any topic. PLWD in LTC tested the application at each stage to assess feasibility and acceptability. Results: Ten PLWD living in LTC participated in Stage 1 (4 men and 6 women; average 82 years old) and Stage 2 (2 men and 8 women; average 87 years old). Session lengths ranged from 0:00 to 5:30 min in Stage 1 and 0:00 to 53:50 min in Stage 2. Speech recognition issues and a limited repertoire of questions limited acceptance in Stage 1. Enhanced conversational ability in Stage 2 led to intimate and meaningful conversations with many participants. Many users found the head-mounted display heavy. There were no complaints of simulator sickness. The virtual companion was best suited to PLWD who could engage in reciprocal conversation. After Stage 2, response latency was identified as an opportunity for improvement in future versions. Conclusions: Virtual reality and artificial intelligence can be used to create a virtual companion that is acceptable and enjoyable to some PLWD living in LTC. Ongoing innovations in hardware and software will allow future iterations to provide more natural conversational interaction and an enhanced social experience. © 2024 by the authors. |
Linnaranta, O.; Cardona, L. G.; Seon, Q.; Tukkiapik, A.; Outerbridge, J.; Bouchard, S. Views on a Culturally Safe Psychotherapeutic Treatment by Inuit in Quebec: Co-Design of Cognitive Behavioral Therapy Manual and Virtual Exposure Environments Journal Article In: Cognitive and Behavioral Practice, 2024, ISSN: 10777229 (ISSN), (Publisher: Elsevier Inc.). @article{linnaranta_views_2024,
title = {Views on a Culturally Safe Psychotherapeutic Treatment by Inuit in Quebec: Co-Design of Cognitive Behavioral Therapy Manual and Virtual Exposure Environments},
author = {O. Linnaranta and L. G. Cardona and Q. Seon and A. Tukkiapik and J. Outerbridge and S. Bouchard},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85199212719&doi=10.1016%2fj.cbpra.2024.04.006&partnerID=40&md5=196f0e70c5d372d4384226d9452bc8f8},
doi = {10.1016/j.cbpra.2024.04.006},
issn = {10777229 (ISSN)},
year = {2024},
date = {2024-01-01},
journal = {Cognitive and Behavioral Practice},
abstract = {Cognitive-behavioral psychotherapy (CBT) can be combined with virtual reality (VR) to provide culturally safe and remotely delivered emotion regulation interventions. We conducted a co-design process of a CBT treatment manual and complementary VR environments for the Inuit populations from Nunavik. Here, we describe the knowledge gained during the adaptation process on the approach to mental well-being and psychotherapy. We followed qualitative, participatory, and research co-design methods. After an initial concept of VR-CBT, an advisory group made up of 7 adults identifying as or working with Inuit participated in 4 focus group meetings. A thematic analysis of the discussions was carried out. A non-symptom-focused approach with the therapist guiding the individual in empowerment and emotion management was accepted in the advisory group, replacing a symptom-focus. Several CBT in- and between-session techniques were seen critically or rejected, and time for working on a certain theme was increased. Some elements in the proposed landscape were rejected as unsafe, other elements added as culture-specific to increase safety. Future work should confirm broader acceptance and utility. Culturally specific factors play an essential role in acceptance of concepts and approaches used in psychotherapy. Accordingly, they can have an impact on acceptance and attendance in therapy. © 2024},
note = {Publisher: Elsevier Inc.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Cognitive-behavioral psychotherapy (CBT) can be combined with virtual reality (VR) to provide culturally safe and remotely delivered emotion regulation interventions. We conducted a co-design process of a CBT treatment manual and complementary VR environments for the Inuit populations from Nunavik. Here, we describe the knowledge gained during the adaptation process on the approach to mental well-being and psychotherapy. We followed qualitative, participatory, and research co-design methods. After an initial concept of VR-CBT, an advisory group made up of 7 adults identifying as or working with Inuit participated in 4 focus group meetings. A thematic analysis of the discussions was carried out. A non-symptom-focused approach with the therapist guiding the individual in empowerment and emotion management was accepted in the advisory group, replacing a symptom-focus. Several CBT in- and between-session techniques were seen critically or rejected, and time for working on a certain theme was increased. Some elements in the proposed landscape were rejected as unsafe, other elements added as culture-specific to increase safety. Future work should confirm broader acceptance and utility. Culturally specific factors play an essential role in acceptance of concepts and approaches used in psychotherapy. Accordingly, they can have an impact on acceptance and attendance in therapy. © 2024 |
