Slide
Centre Interdisciplinaire
de Recherche et d’Innovation
en Cybersécurité et Société
de Recherche et d’Innovation
en Cybersécurité et Société
1.
Tiwari, A.; Arrabito, R.; Davoust, A.; Falk, T. H.
Bias in Physiology-Based Cognitive State Detection for Human–Autonomy Teaming: A Comprehensive Survey Article d'actes
Dans: R.A., Sottilare; J., Schwarz (Ed.): Lect. Notes Comput. Sci., p. 139–158, Springer Science and Business Media Deutschland GmbH, 2026, ISBN: 03029743 (ISSN); 978-303230014-0 (ISBN), (Journal Abbreviation: Lect. Notes Comput. Sci.).
Résumé | Liens | BibTeX | Étiquettes: Behavioral research, Bias, Biases, Biomedical signal processing, Cognitive state, Cognitive systems, Human Autonomy Teaming, Instructional system, Intelligent systems, Learning pathway, Learning systems, Personalized learning, Physiological models, Physiological signals, Population statistics, Psychophysiology, State Detection, Student feedback, Surveying, Wearable devices, Wearable technology
@inproceedings{tiwariBiasPhysiologyBasedCognitive2026,
title = {Bias in Physiology-Based Cognitive State Detection for Human–Autonomy Teaming: A Comprehensive Survey},
author = {A. Tiwari and R. Arrabito and A. Davoust and T. H. Falk},
editor = {Sottilare R.A. and Schwarz J.},
url = {https://www.scopus.com/pages/publications/105044000677?origin=resultslist},
doi = {10.1007/978-3-032-30015-7_9},
isbn = {03029743 (ISSN); 978-303230014-0 (ISBN)},
year = {2026},
date = {2026-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {16737 LNCS},
pages = {139–158},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Advances in artificial intelligence (AI) are changing the role of adaptive instructional systems (AIS) from passive tools, that deliver personalized learning pathways by adapting to student feedback, to active team members and co-learners that dynamically adapt alongside human partners. In this emerging landscape of human–autonomy teaming (HAT), assessing the cognitive states of individuals interacting with highly intelligent systems has become a critical design consideration for improving team performance, collaboration, and learning outcomes. Physiological signals have emerged as a popular method for measurement of cognitive states in the past few decades. However, these signals can be strongly influenced by different user demographics, including age and biological sex. If unaccounted for, these differences could introduce systematic biases into AIS based learning pathways for different demographic groups. These biases can lead to discriminative performance of instructional systems and may also leave them vulnerable to adversarial exploitation. Aside from physiological signal-induced biases, there may also be major behavioural differences between different sex and/or age groups when interacting with and teaming with AIS, thus further confounding cognitive state monitoring. In this survey, we review emerging human-autonomy teaming studies that incorporate physiological data and analyze where biological sex- and age-related physiological or behavioral differences were documented and/or analyzed. We highlight evidence demonstrating that differences do exist during interaction with autonomous systems. We then discuss how biases can accumulate across study design and analysis modelling pipelines, and provide practical guidelines for mitigating these biases in future human-autonomy teaming research and applications. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.},
note = {Journal Abbreviation: Lect. Notes Comput. Sci.},
keywords = {Behavioral research, Bias, Biases, Biomedical signal processing, Cognitive state, Cognitive systems, Human Autonomy Teaming, Instructional system, Intelligent systems, Learning pathway, Learning systems, Personalized learning, Physiological models, Physiological signals, Population statistics, Psychophysiology, State Detection, Student feedback, Surveying, Wearable devices, Wearable technology},
pubstate = {published},
tppubtype = {inproceedings}
}
Advances in artificial intelligence (AI) are changing the role of adaptive instructional systems (AIS) from passive tools, that deliver personalized learning pathways by adapting to student feedback, to active team members and co-learners that dynamically adapt alongside human partners. In this emerging landscape of human–autonomy teaming (HAT), assessing the cognitive states of individuals interacting with highly intelligent systems has become a critical design consideration for improving team performance, collaboration, and learning outcomes. Physiological signals have emerged as a popular method for measurement of cognitive states in the past few decades. However, these signals can be strongly influenced by different user demographics, including age and biological sex. If unaccounted for, these differences could introduce systematic biases into AIS based learning pathways for different demographic groups. These biases can lead to discriminative performance of instructional systems and may also leave them vulnerable to adversarial exploitation. Aside from physiological signal-induced biases, there may also be major behavioural differences between different sex and/or age groups when interacting with and teaming with AIS, thus further confounding cognitive state monitoring. In this survey, we review emerging human-autonomy teaming studies that incorporate physiological data and analyze where biological sex- and age-related physiological or behavioral differences were documented and/or analyzed. We highlight evidence demonstrating that differences do exist during interaction with autonomous systems. We then discuss how biases can accumulate across study design and analysis modelling pipelines, and provide practical guidelines for mitigating these biases in future human-autonomy teaming research and applications. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2026.
2.
Tiwari, A.; Arrabito, R.; Davoust, A.; Falk, T. H.
Quantifying Biological Sex Leakage in Electroencephalography-Based Mental Workload Measurement and Its Impact on Model Performance Article d'actes
Dans: IEEE Int. Conf. Hum.-Mach. Syst., ICHMS, p. 392–397, Institute of Electrical and Electronics Engineers Inc., 2026, ISBN: 979-833154511-6 (ISBN), (Journal Abbreviation: IEEE Int. Conf. Hum.-Mach. Syst., ICHMS).
Résumé | Liens | BibTeX | Étiquettes: Biomedical signal processing, Demographic information, Designing systems, Economic and social effects, Electroencephalography, Electrophysiology, Human operator, Mental workload, Modeling performance, Optimal performance, Performance, Population statistics, Real- time, Well being, Workload measurements
@inproceedings{tiwariQuantifyingBiologicalSex2026,
title = {Quantifying Biological Sex Leakage in Electroencephalography-Based Mental Workload Measurement and Its Impact on Model Performance},
author = {A. Tiwari and R. Arrabito and A. Davoust and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105045575255?origin=resultslist},
doi = {10.1109/ICHMS69701.2026.11602273},
isbn = {979-833154511-6 (ISBN)},
year = {2026},
date = {2026-01-01},
booktitle = {IEEE Int. Conf. Hum.-Mach. Syst., ICHMS},
pages = {392–397},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Detection of mental workload is essential for designing systems that can monitor and adapt to the needs of human operators, thereby ensuring their optimal performance and well-being. Features derived from neurophysiological signals, particularly electroencephalograms (EEG), provide a realtime, unobtrusive, and objective indicator of mental workload. However, EEG signals are often correlated with demographic variables, including biological sex and age, which can introduce systematic performance differences across demographic groups, while unintentionally encoding demographic information into models trained on EEG data. In this work, we investigate and quantify the influence of user demographics on mental workload detection models using fairness analysis. Furthermore, we propose domain-adversarial training, in which an adversary penalizes the model for retaining sex-related information, to mitigate leakage of private demographic information (biological sex). The proposed approach simultaneously increases the odds ratio for sex from 0.83 to 0.90 (values closer to 1 indicate reduced demographic disparity) while reducing statistical significance. This approach, however, also leads to a reduction in mental workload detection performance, with a 12.7% decrease in balanced accuracy (BACC), suggesting an inherent trade-off between performance and privacy protection. © 2026 IEEE.},
note = {Journal Abbreviation: IEEE Int. Conf. Hum.-Mach. Syst., ICHMS},
keywords = {Biomedical signal processing, Demographic information, Designing systems, Economic and social effects, Electroencephalography, Electrophysiology, Human operator, Mental workload, Modeling performance, Optimal performance, Performance, Population statistics, Real- time, Well being, Workload measurements},
pubstate = {published},
tppubtype = {inproceedings}
}
Detection of mental workload is essential for designing systems that can monitor and adapt to the needs of human operators, thereby ensuring their optimal performance and well-being. Features derived from neurophysiological signals, particularly electroencephalograms (EEG), provide a realtime, unobtrusive, and objective indicator of mental workload. However, EEG signals are often correlated with demographic variables, including biological sex and age, which can introduce systematic performance differences across demographic groups, while unintentionally encoding demographic information into models trained on EEG data. In this work, we investigate and quantify the influence of user demographics on mental workload detection models using fairness analysis. Furthermore, we propose domain-adversarial training, in which an adversary penalizes the model for retaining sex-related information, to mitigate leakage of private demographic information (biological sex). The proposed approach simultaneously increases the odds ratio for sex from 0.83 to 0.90 (values closer to 1 indicate reduced demographic disparity) while reducing statistical significance. This approach, however, also leads to a reduction in mental workload detection performance, with a 12.7% decrease in balanced accuracy (BACC), suggesting an inherent trade-off between performance and privacy protection. © 2026 IEEE.
3.
Jaberi, M.; Bouchard, S.; Falk, T. H.
Quantifying the Risk of Private Information Leakage in the Metaverse with EEG-Instrumented Virtual Reality Headsets Article d'actes
Dans: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern., p. 4293–4298, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 1062922X (ISSN); 979-833153358-8 (ISBN), (Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.).
Résumé | Liens | BibTeX | Étiquettes: Biomedical signal processing, Electroencephalography, Electrophysiology, Immersive, Information leakage, Interactivity, Learning systems, Metaverses, Neurophysiology, Privacy risks, Private information, Real-time quality, User engagement, virtual reality, Virtual reality technology, Virtual-reality headsets
@inproceedings{jaberiQuantifyingRiskPrivate2025,
title = {Quantifying the Risk of Private Information Leakage in the Metaverse with EEG-Instrumented Virtual Reality Headsets},
author = {M. Jaberi and S. Bouchard and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105033158377?origin=resultslist},
doi = {10.1109/SMC58881.2025.11343511},
isbn = {1062922X (ISSN); 979-833153358-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
pages = {4293–4298},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {As virtual reality (VR) technologies become more immersive and metaverse applications burgeon, modern headsets are increasingly becoming equipped with sensors capable of capturing multiple neurophysiological signals. While these signals can be used to measure, in real-time, quality of experience metrics that can be used to enhance interactivity and user engagement, they may also introduce novel privacy risks by unintentionally leaking sensitive personal attributes. In this paper, we explore the extent in which electroencephalography (EEG) signals, recorded during an immersive VR memory task, can be used to infer users' private information, such as age, biological sex, and identity. We employ both classical machine learning models with hand-crafted features, as well as end-to-end deep learning approaches. Our findings demonstrate that EEG-based features can, indeed, leak information about biological sex, age, and user identity, with end-to-end models obtaining the best performance. Feature importance ranking and deep neural network saliency maps were used to provide explainability on the neural patterns used by the models. We conclude with recommendations on how these findings can also be used to help secure future metaverse applications. © 2025 IEEE.},
note = {Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
keywords = {Biomedical signal processing, Electroencephalography, Electrophysiology, Immersive, Information leakage, Interactivity, Learning systems, Metaverses, Neurophysiology, Privacy risks, Private information, Real-time quality, User engagement, virtual reality, Virtual reality technology, Virtual-reality headsets},
pubstate = {published},
tppubtype = {inproceedings}
}
As virtual reality (VR) technologies become more immersive and metaverse applications burgeon, modern headsets are increasingly becoming equipped with sensors capable of capturing multiple neurophysiological signals. While these signals can be used to measure, in real-time, quality of experience metrics that can be used to enhance interactivity and user engagement, they may also introduce novel privacy risks by unintentionally leaking sensitive personal attributes. In this paper, we explore the extent in which electroencephalography (EEG) signals, recorded during an immersive VR memory task, can be used to infer users' private information, such as age, biological sex, and identity. We employ both classical machine learning models with hand-crafted features, as well as end-to-end deep learning approaches. Our findings demonstrate that EEG-based features can, indeed, leak information about biological sex, age, and user identity, with end-to-end models obtaining the best performance. Feature importance ranking and deep neural network saliency maps were used to provide explainability on the neural patterns used by the models. We conclude with recommendations on how these findings can also be used to help secure future metaverse applications. © 2025 IEEE.
4.
Jesus, B. De; Lopes, M.; Perreault, L.; Roberge, M. -C.; Oliveira, A. A. De; Falk, T. H.
Dans: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern., p. 6033–6038, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 1062922X (ISSN); 979-833153358-8 (ISBN), (Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.).
Résumé | Liens | BibTeX | Étiquettes: Biomedical signal processing, Diseases, Electroencephalography, electroencephalography (EEG), Electrophysiology, Haptics, Immersive, Immersive virtual reality, Multisensory, Nature immersion, Odors, Olfactory, Patient monitoring, Patient treatment, Post-traumatic stress disorder, Post-traumatic stress disorder (PTSD), posttraumatic stress disorder, virtual reality, virtual reality (VR)
@inproceedings{dejesusElectroencephalographyNeuromarkersPredict2025,
title = {Electroencephalography Neuromarkers to Predict the Response of a Multisensory Virtual Reality Nature Immersion Intervention for Patients Diagnosed with Post-Traumatic Stress Disorder},
author = {B. De Jesus and M. Lopes and L. Perreault and M. -C. Roberge and A. A. De Oliveira and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105033154892?origin=resultslist},
doi = {10.1109/SMC58881.2025.11342787},
isbn = {1062922X (ISSN); 979-833153358-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
pages = {6033–6038},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Immersive virtual reality (VR) applications rapidly expand across domains, including training, gaming, and healthcare. More recently, multisensory immersive experiences, including olfactory and haptic stimulation, have emerged and shown great promise, especially for interventions in well-being and mental health management. Multisensory experiences, however, are very subjective (e.g., one subject may like certain smells, while others do not), and recent results have suggested that some participants may not respond positively to the treatment. As multisensory VR interventions can be costly and time-consuming for both patients and clinicians, being able to find neuromarkers that predict intervention outcomes would be invaluable. Here, we aim to take the first steps in the development of a neuromarker to predict the response to a multisensory nature immersion VR intervention. A pilot experiment was performed with twenty patients diagnosed with post-traumatic stress disorder. Potential neuromarkers are extracted from electroencephalography (EEG) signals measured from an instrumented VR headset. We show that some EEG patterns start to differ between responders and non-responders as early as the fourth session, i.e., one-third of the way into the entire intervention. This suggests that neuromarkers to predict the outcomes of a multisensory VR immersion intervention may exist. These markers could be used not only to save time and resources for clinicians and patients but also to promote precision treatment where interventions are adjusted to each patient, maximizing success rates. © 2025 IEEE.},
note = {Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
keywords = {Biomedical signal processing, Diseases, Electroencephalography, electroencephalography (EEG), Electrophysiology, Haptics, Immersive, Immersive virtual reality, Multisensory, Nature immersion, Odors, Olfactory, Patient monitoring, Patient treatment, Post-traumatic stress disorder, Post-traumatic stress disorder (PTSD), posttraumatic stress disorder, virtual reality, virtual reality (VR)},
pubstate = {published},
tppubtype = {inproceedings}
}
Immersive virtual reality (VR) applications rapidly expand across domains, including training, gaming, and healthcare. More recently, multisensory immersive experiences, including olfactory and haptic stimulation, have emerged and shown great promise, especially for interventions in well-being and mental health management. Multisensory experiences, however, are very subjective (e.g., one subject may like certain smells, while others do not), and recent results have suggested that some participants may not respond positively to the treatment. As multisensory VR interventions can be costly and time-consuming for both patients and clinicians, being able to find neuromarkers that predict intervention outcomes would be invaluable. Here, we aim to take the first steps in the development of a neuromarker to predict the response to a multisensory nature immersion VR intervention. A pilot experiment was performed with twenty patients diagnosed with post-traumatic stress disorder. Potential neuromarkers are extracted from electroencephalography (EEG) signals measured from an instrumented VR headset. We show that some EEG patterns start to differ between responders and non-responders as early as the fourth session, i.e., one-third of the way into the entire intervention. This suggests that neuromarkers to predict the outcomes of a multisensory VR immersion intervention may exist. These markers could be used not only to save time and resources for clinicians and patients but also to promote precision treatment where interventions are adjusted to each patient, maximizing success rates. © 2025 IEEE.