Slide
Centre Interdisciplinaire
de Recherche et d’Innovation
en Cybersécurité et Société
de Recherche et d’Innovation
en Cybersécurité et Société
1.
Pimentel, A.; Zhu, Y.; Falk, T. H.
Partial Audio Deepfake Detection: Are We Really Detecting Synthetic Media or Just Dataset Content Biases? Proceedings Article
In: IEEE Conf. Artif. Intell., CAI, pp. 1846–1851, Institute of Electrical and Electronics Engineers Inc., 2026, ISBN: 979-833156039-3 (ISBN), (Journal Abbreviation: IEEE Conf. Artif. Intell., CAI).
Abstract | Links | BibTeX | Tags: Audio signal processing, Computational linguistics, Condition, Detection models, Information integrity, Language model, Large datasets, Learning models, Optimistics, Performance, Self-supervised learning, Speech communication, Speech models, Speech recognition, Speech signals, Synthetic media, Transcription
@inproceedings{pimentelPartialAudioDeepfake2026,
title = {Partial Audio Deepfake Detection: Are We Really Detecting Synthetic Media or Just Dataset Content Biases?},
author = {A. Pimentel and Y. Zhu and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105042045476?origin=resultslist},
doi = {10.1109/CAI68641.2026.11536622},
isbn = {979-833156039-3 (ISBN)},
year = {2026},
date = {2026-01-01},
booktitle = {IEEE Conf. Artif. Intell., CAI},
pages = {1846–1851},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {In recent years, the generation of highly realistic audio deepfakes has raised significant concerns regarding privacy and information integrity. While most research has focused on fully bonafide or spoofed speech, partial deep-fakes, where only segments of an utterance are manipulated, remain less explored. Given the nature of the task, existing datasets are relying on large language models to manipulate bonafide speech signals into partial deepfakes by altering, deleting, or replacing segments with synthetic content. These manipulations may alter the semantics and sentiment of the generated content, creating biases that can be captured by deepfake detection models, leading to overly optimistic performance and poor generalizability. In this work, we investigate the presence of such biases in two popular partial deepfake datasets, namely AV-Deepfake1M and PartialEdit. We explore four self-supervised learning models, two relying only on text transcriptions (RoBERTa and RoBERTa-sentiment) and two relying on universal speech representations (WavLM Large and Wav2vec2-XLSR). Our experiments show that, while speech models achieve higher in-domain performance, they do not generalize to out-of-domain conditions. In turn, models trained on only transcribed speech can effectively distinguish manipulated content, achieving up to 97.8% AUC in-domain and nearly 70% AUC out-of-domain. These results suggest that linguistic patterns and dataset confounds may indeed be biasing partial deepfake detection models, leading to poor generalizability. © 2026 IEEE.},
note = {Journal Abbreviation: IEEE Conf. Artif. Intell., CAI},
keywords = {Audio signal processing, Computational linguistics, Condition, Detection models, Information integrity, Language model, Large datasets, Learning models, Optimistics, Performance, Self-supervised learning, Speech communication, Speech models, Speech recognition, Speech signals, Synthetic media, Transcription},
pubstate = {published},
tppubtype = {inproceedings}
}
In recent years, the generation of highly realistic audio deepfakes has raised significant concerns regarding privacy and information integrity. While most research has focused on fully bonafide or spoofed speech, partial deep-fakes, where only segments of an utterance are manipulated, remain less explored. Given the nature of the task, existing datasets are relying on large language models to manipulate bonafide speech signals into partial deepfakes by altering, deleting, or replacing segments with synthetic content. These manipulations may alter the semantics and sentiment of the generated content, creating biases that can be captured by deepfake detection models, leading to overly optimistic performance and poor generalizability. In this work, we investigate the presence of such biases in two popular partial deepfake datasets, namely AV-Deepfake1M and PartialEdit. We explore four self-supervised learning models, two relying only on text transcriptions (RoBERTa and RoBERTa-sentiment) and two relying on universal speech representations (WavLM Large and Wav2vec2-XLSR). Our experiments show that, while speech models achieve higher in-domain performance, they do not generalize to out-of-domain conditions. In turn, models trained on only transcribed speech can effectively distinguish manipulated content, achieving up to 97.8% AUC in-domain and nearly 70% AUC out-of-domain. These results suggest that linguistic patterns and dataset confounds may indeed be biasing partial deepfake detection models, leading to poor generalizability. © 2026 IEEE.
2.
Guimaraes, H. R.; Abdollahi, M.; Zhu, Y.; Maucourt, S.; Coallier, N.; Giovenazzo, P.; Falk, T. H.
Benchmarking Self-Supervised Audio Representations for IoT-Enabled Acoustic Beehive Monitoring Journal Article
In: IEEE Internet of Things Journal, vol. 12, no. 21, pp. 45000–45010, 2025, ISSN: 23274662 (ISSN).
Abstract | Links | BibTeX | Tags: Acoustics, Audio acoustics, Audio representation, Beehive monitoring, Benchmarking, Bioacoustics, Computer vision applications, Deep learning, Honeybee, honeybees, Internet of Things (IoT), IoT, Labeled data, Performance, Real time systems, Self-supervised learning, self-supervised learning (SSL), Societal benefits, Speech applications, Speech recognition, Supervised learning, Universal feature extractors
@article{guimaraesBenchmarkingSelfSupervisedAudio2025,
title = {Benchmarking Self-Supervised Audio Representations for IoT-Enabled Acoustic Beehive Monitoring},
author = {H. R. Guimaraes and M. Abdollahi and Y. Zhu and S. Maucourt and N. Coallier and P. Giovenazzo and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105013592730?origin=resultslist},
doi = {10.1109/JIOT.2025.3599483},
issn = {23274662 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Internet of Things Journal},
volume = {12},
number = {21},
pages = {45000–45010},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Self-supervised learning (SSL) has enabled the development of universal feature extractors that have redefined the performance envelope of computer vision and speech applications. Recent works have started to explore SSL in other domains, including bioacoustics, which could have significant societal benefits. Honeybees (Apis mellifera), for example, are crucial pollinators contributing to one-third of global food production. However, massive colony losses in recent years have raised concerns. Traditional hive monitoring methods rely on intrusive visual inspections by beekeepers, which can further disrupt colony dynamics. As such, Internet of Things (IoT)-based automated monitoring systems have emerged, integrating environmental and bioacoustic sensing to enable real-time, noninvasive hive assessment. In this work, we introduce a comprehensive evaluation and benchmarking of general-purpose and bioacoustic audio representations that generalize across various tasks in IoT-enabled acoustic beehive monitoring, even with limited labeled data. Herein, fourteen models are evaluated across four critical tasks: beehive state detection, beehive strength assessment, buzzing identification, and beekeeper voice activity detection. Reported results demonstrate the strong generalizability of existing representations, paving the way for advanced, scalable honeybee colony monitoring and preservation. © 2014 IEEE.},
keywords = {Acoustics, Audio acoustics, Audio representation, Beehive monitoring, Benchmarking, Bioacoustics, Computer vision applications, Deep learning, Honeybee, honeybees, Internet of Things (IoT), IoT, Labeled data, Performance, Real time systems, Self-supervised learning, self-supervised learning (SSL), Societal benefits, Speech applications, Speech recognition, Supervised learning, Universal feature extractors},
pubstate = {published},
tppubtype = {article}
}
Self-supervised learning (SSL) has enabled the development of universal feature extractors that have redefined the performance envelope of computer vision and speech applications. Recent works have started to explore SSL in other domains, including bioacoustics, which could have significant societal benefits. Honeybees (Apis mellifera), for example, are crucial pollinators contributing to one-third of global food production. However, massive colony losses in recent years have raised concerns. Traditional hive monitoring methods rely on intrusive visual inspections by beekeepers, which can further disrupt colony dynamics. As such, Internet of Things (IoT)-based automated monitoring systems have emerged, integrating environmental and bioacoustic sensing to enable real-time, noninvasive hive assessment. In this work, we introduce a comprehensive evaluation and benchmarking of general-purpose and bioacoustic audio representations that generalize across various tasks in IoT-enabled acoustic beehive monitoring, even with limited labeled data. Herein, fourteen models are evaluated across four critical tasks: beehive state detection, beehive strength assessment, buzzing identification, and beekeeper voice activity detection. Reported results demonstrate the strong generalizability of existing representations, paving the way for advanced, scalable honeybee colony monitoring and preservation. © 2014 IEEE.
3.
Zhu, Y.; Davoust, A.; Falk, T. H.
DeepSick: Deceiving Voice-Based Diagnostic Models with Synthetic Multilingual Pathological Speech Signals Proceedings Article
In: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern., pp. 69–74, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 1062922X (ISSN); 979-833153358-8 (ISBN), (Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.).
Abstract | Links | BibTeX | Tags: COVID-19, Detection models, Diagnosis, Diagnostic model, Diagnostic systems, Generative model, Health assessments, Pathological conditions, Pathological speech signals, Scalable solution, Speech communication, Speech recognition, Speech synthesis, State of the art, Voice model
@inproceedings{zhuDeepSickDeceivingVoiceBased2025,
title = {DeepSick: Deceiving Voice-Based Diagnostic Models with Synthetic Multilingual Pathological Speech Signals},
author = {Y. Zhu and A. Davoust and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105033143787?origin=resultslist},
doi = {10.1109/SMC58881.2025.11343240},
isbn = {1062922X (ISSN); 979-833153358-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
pages = {69–74},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Voice-based diagnostic systems offer a scalable solution for remote health assessment. However, recent advances in generative voice models may enable malicious manipulation of voice samples to simulate or conceal disease-related speech characteristics, which poses new risks to diagnostic systems. This paper investigates the vulnerability of diagnostic and detection models to such types of "deepfake"attacks. We show that it is possible to train a generative model to convert between healthy voices and pathological ones, which in turn, can successfully deceive existing diagnostic systems. Here, focus is placed on COVID-19 infection and respiratory abnormalities, but the method can be applied across different pathological conditions affecting vocal attributes. We also benchmark four state-of-the-art synthesized voice detection models on both real and generated pathological speech from three datasets. Our results show that current synthetic voice detectors, typically trained on healthy speech data, perform poorly on generated pathological samples. While fine-tuning with real pathological voices improves detection, a substantial performance gap remains. This work provides initial insights on an emerging threat to remote voice diagnostic systems that needs further work. © 2025 IEEE.},
note = {Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
keywords = {COVID-19, Detection models, Diagnosis, Diagnostic model, Diagnostic systems, Generative model, Health assessments, Pathological conditions, Pathological speech signals, Scalable solution, Speech communication, Speech recognition, Speech synthesis, State of the art, Voice model},
pubstate = {published},
tppubtype = {inproceedings}
}
Voice-based diagnostic systems offer a scalable solution for remote health assessment. However, recent advances in generative voice models may enable malicious manipulation of voice samples to simulate or conceal disease-related speech characteristics, which poses new risks to diagnostic systems. This paper investigates the vulnerability of diagnostic and detection models to such types of "deepfake"attacks. We show that it is possible to train a generative model to convert between healthy voices and pathological ones, which in turn, can successfully deceive existing diagnostic systems. Here, focus is placed on COVID-19 infection and respiratory abnormalities, but the method can be applied across different pathological conditions affecting vocal attributes. We also benchmark four state-of-the-art synthesized voice detection models on both real and generated pathological speech from three datasets. Our results show that current synthetic voice detectors, typically trained on healthy speech data, perform poorly on generated pathological samples. While fine-tuning with real pathological voices improves detection, a substantial performance gap remains. This work provides initial insights on an emerging threat to remote voice diagnostic systems that needs further work. © 2025 IEEE.
4.
Bacha, S.; Allili, M. S.; Benblidia, N.
Event recognition in photo albums using probabilistic graphical model and feature relevance Proceedings Article
In: Proceedings - International Conference on Pattern Recognition, pp. 2819–2823, Institute of Electrical and Electronics Engineers Inc., 2016, ISBN: 10514651 (ISSN); 978-150904847-2 (ISBN), (Journal Abbreviation: Proc. Int. Conf. Pattern Recognit.).
Abstract | Links | BibTeX | Tags: Discriminative power, Event recognition, Feature relevance, Graphic methods, Image features, Object detection, Photo album, Probabilistic graphical models, Probabilistic graphical models (PGM), Semantic event, Semantics, Speech recognition
@inproceedings{bacha_event_2016-1,
title = {Event recognition in photo albums using probabilistic graphical model and feature relevance},
author = {S. Bacha and M. S. Allili and N. Benblidia},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85019068722&doi=10.1109%2fICPR.2016.7900063&partnerID=40&md5=7aaf779f00590113afa31fde63016619},
doi = {10.1109/ICPR.2016.7900063},
isbn = {10514651 (ISSN); 978-150904847-2 (ISBN)},
year = {2016},
date = {2016-01-01},
booktitle = {Proceedings - International Conference on Pattern Recognition},
volume = {0},
pages = {2819–2823},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {The exponential use of digital cameras has raised a new problem: how to store/retrieve images/albums in very large photo databases that correspond to special events. In this paper, we propose a new probabilistic graphical model (PGM) to recognize events in photo albums stored by users. The PGM combines high-level image features consisting of scenes and objects detected in images. To consider the discriminative power of features, our model integrates the object/scene relevance for more precise prediction of semantic events in photo albums. Experimental results carried out on the challenging PEC dataset with 807 photo albums are presented. © 2016 IEEE.},
note = {Journal Abbreviation: Proc. Int. Conf. Pattern Recognit.},
keywords = {Discriminative power, Event recognition, Feature relevance, Graphic methods, Image features, Object detection, Photo album, Probabilistic graphical models, Probabilistic graphical models (PGM), Semantic event, Semantics, Speech recognition},
pubstate = {published},
tppubtype = {inproceedings}
}
The exponential use of digital cameras has raised a new problem: how to store/retrieve images/albums in very large photo databases that correspond to special events. In this paper, we propose a new probabilistic graphical model (PGM) to recognize events in photo albums stored by users. The PGM combines high-level image features consisting of scenes and objects detected in images. To consider the discriminative power of features, our model integrates the object/scene relevance for more precise prediction of semantic events in photo albums. Experimental results carried out on the challenging PEC dataset with 807 photo albums are presented. © 2016 IEEE.
5.
Bacha, S.; Allili, M. S.; Benblidia, N.
Event recognition in photo albums using probabilistic graphical models and feature relevance Journal Article
In: Journal of Visual Communication and Image Representation, vol. 40, no. Part B, pp. 546–558, 2016, ISSN: 10473203 (ISSN), (Publisher: Academic Press Inc.).
Abstract | Links | BibTeX | Tags: Event prediction, Event recognition, Feature relevance, Graphic methods, Object/scene relevance, Photo album, Photo albums, Probabilistic graphical models, Probabilistic graphical models (PGM), Speech recognition, Visual feature
@article{bacha_event_2016,
title = {Event recognition in photo albums using probabilistic graphical models and feature relevance},
author = {S. Bacha and M. S. Allili and N. Benblidia},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-84992111208&doi=10.1016%2fj.jvcir.2016.07.021&partnerID=40&md5=20ebb156819c8fcae6e28949046ceb6e},
doi = {10.1016/j.jvcir.2016.07.021},
issn = {10473203 (ISSN)},
year = {2016},
date = {2016-01-01},
booktitle = {Proceedings - International Conference on Pattern Recognition},
journal = {Journal of Visual Communication and Image Representation},
volume = {40},
number = {Part B},
pages = {546–558},
publisher = {Academic Press Inc.},
abstract = {This paper proposes a method for event recognition in photo albums which aims at predicting the event categories of groups of photos. We propose a probabilistic graphical model (PGM) for event prediction based on high-level visual features consisting of objects and scenes, which are extracted directly from images. For better discrimination between different event categories, we develop a scheme to integrate feature relevance in our model which yields a more powerful inference when album images exhibit a large number of objects and scenes. It allows also to mitigate the influence of non-informative images usually contained in the albums. The performance of the proposed method is validated using extensive experiments on the recently-proposed PEC dataset containing over 61 000 images. Our method obtained the highest accuracy which outperforms previous work. © 2016 Elsevier Inc.},
note = {Publisher: Academic Press Inc.},
keywords = {Event prediction, Event recognition, Feature relevance, Graphic methods, Object/scene relevance, Photo album, Photo albums, Probabilistic graphical models, Probabilistic graphical models (PGM), Speech recognition, Visual feature},
pubstate = {published},
tppubtype = {article}
}
This paper proposes a method for event recognition in photo albums which aims at predicting the event categories of groups of photos. We propose a probabilistic graphical model (PGM) for event prediction based on high-level visual features consisting of objects and scenes, which are extracted directly from images. For better discrimination between different event categories, we develop a scheme to integrate feature relevance in our model which yields a more powerful inference when album images exhibit a large number of objects and scenes. It allows also to mitigate the influence of non-informative images usually contained in the albums. The performance of the proposed method is validated using extensive experiments on the recently-proposed PEC dataset containing over 61 000 images. Our method obtained the highest accuracy which outperforms previous work. © 2016 Elsevier Inc.
6.
Bacha, S.; Allili, M. S.; Benblidia, N.
Event recognition in photo albums using probabilistic graphical models and feature relevance Journal Article
In: Journal of Visual Communication and Image Representation, vol. 40, no. Part B, pp. 546–558, 2016, ISSN: 10473203 (ISSN).
Abstract | Links | BibTeX | Tags: Event prediction, Event recognition, Feature relevance, Graphic methods, Object/scene relevance, Photo album, Photo albums, Probabilistic graphical models, Probabilistic graphical models (PGM), Speech recognition, Visual feature
@article{bachaEventRecognitionPhoto2016a,
title = {Event recognition in photo albums using probabilistic graphical models and feature relevance},
author = {S. Bacha and M. S. Allili and N. Benblidia},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-84992111208&doi=10.1016%2fj.jvcir.2016.07.021&partnerID=40&md5=20ebb156819c8fcae6e28949046ceb6e},
doi = {10.1016/j.jvcir.2016.07.021},
issn = {10473203 (ISSN)},
year = {2016},
date = {2016-01-01},
journal = {Journal of Visual Communication and Image Representation},
volume = {40},
number = {Part B},
pages = {546–558},
publisher = {Academic Press Inc.},
abstract = {This paper proposes a method for event recognition in photo albums which aims at predicting the event categories of groups of photos. We propose a probabilistic graphical model (PGM) for event prediction based on high-level visual features consisting of objects and scenes, which are extracted directly from images. For better discrimination between different event categories, we develop a scheme to integrate feature relevance in our model which yields a more powerful inference when album images exhibit a large number of objects and scenes. It allows also to mitigate the influence of non-informative images usually contained in the albums. The performance of the proposed method is validated using extensive experiments on the recently-proposed PEC dataset containing over 61 000 images. Our method obtained the highest accuracy which outperforms previous work. © 2016 Elsevier Inc.},
keywords = {Event prediction, Event recognition, Feature relevance, Graphic methods, Object/scene relevance, Photo album, Photo albums, Probabilistic graphical models, Probabilistic graphical models (PGM), Speech recognition, Visual feature},
pubstate = {published},
tppubtype = {article}
}
This paper proposes a method for event recognition in photo albums which aims at predicting the event categories of groups of photos. We propose a probabilistic graphical model (PGM) for event prediction based on high-level visual features consisting of objects and scenes, which are extracted directly from images. For better discrimination between different event categories, we develop a scheme to integrate feature relevance in our model which yields a more powerful inference when album images exhibit a large number of objects and scenes. It allows also to mitigate the influence of non-informative images usually contained in the albums. The performance of the proposed method is validated using extensive experiments on the recently-proposed PEC dataset containing over 61 000 images. Our method obtained the highest accuracy which outperforms previous work. © 2016 Elsevier Inc.
7.
Bacha, S.; Allili, M. S.; Benblidia, N.
Event recognition in photo albums using probabilistic graphical model and feature relevance Proceedings Article
In: Proceedings - International Conference on Pattern Recognition, pp. 2819–2823, Institute of Electrical and Electronics Engineers Inc., 2016, ISBN: 10514651 (ISSN); 978-150904847-2 (ISBN), (Journal Abbreviation: Proc. Int. Conf. Pattern Recognit.).
Abstract | Links | BibTeX | Tags: Discriminative power, Event recognition, Feature relevance, Graphic methods, Image features, Object detection, Photo album, Probabilistic graphical models, Probabilistic graphical models (PGM), Semantic event, Semantics, Speech recognition
@inproceedings{bachaEventRecognitionPhoto2016,
title = {Event recognition in photo albums using probabilistic graphical model and feature relevance},
author = {S. Bacha and M. S. Allili and N. Benblidia},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85019068722&doi=10.1109%2fICPR.2016.7900063&partnerID=40&md5=7aaf779f00590113afa31fde63016619},
doi = {10.1109/ICPR.2016.7900063},
isbn = {10514651 (ISSN); 978-150904847-2 (ISBN)},
year = {2016},
date = {2016-01-01},
booktitle = {Proceedings - International Conference on Pattern Recognition},
volume = {0},
pages = {2819–2823},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {The exponential use of digital cameras has raised a new problem: how to store/retrieve images/albums in very large photo databases that correspond to special events. In this paper, we propose a new probabilistic graphical model (PGM) to recognize events in photo albums stored by users. The PGM combines high-level image features consisting of scenes and objects detected in images. To consider the discriminative power of features, our model integrates the object/scene relevance for more precise prediction of semantic events in photo albums. Experimental results carried out on the challenging PEC dataset with 807 photo albums are presented. © 2016 IEEE.},
note = {Journal Abbreviation: Proc. Int. Conf. Pattern Recognit.},
keywords = {Discriminative power, Event recognition, Feature relevance, Graphic methods, Image features, Object detection, Photo album, Probabilistic graphical models, Probabilistic graphical models (PGM), Semantic event, Semantics, Speech recognition},
pubstate = {published},
tppubtype = {inproceedings}
}
The exponential use of digital cameras has raised a new problem: how to store/retrieve images/albums in very large photo databases that correspond to special events. In this paper, we propose a new probabilistic graphical model (PGM) to recognize events in photo albums stored by users. The PGM combines high-level image features consisting of scenes and objects detected in images. To consider the discriminative power of features, our model integrates the object/scene relevance for more precise prediction of semantic events in photo albums. Experimental results carried out on the challenging PEC dataset with 807 photo albums are presented. © 2016 IEEE.