Slide
Centre Interdisciplinaire
de Recherche et d’Innovation
en Cybersécurité et Société
de Recherche et d’Innovation
en Cybersécurité et Société
1.
Pimentel, A.; Zhu, Y.; Falk, T. H.
Partial Audio Deepfake Detection: Are We Really Detecting Synthetic Media or Just Dataset Content Biases? Proceedings Article
In: IEEE Conf. Artif. Intell., CAI, pp. 1846–1851, Institute of Electrical and Electronics Engineers Inc., 2026, ISBN: 979-833156039-3 (ISBN), (Journal Abbreviation: IEEE Conf. Artif. Intell., CAI).
Abstract | Links | BibTeX | Tags: Audio signal processing, Computational linguistics, Condition, Detection models, Information integrity, Language model, Large datasets, Learning models, Optimistics, Performance, Self-supervised learning, Speech communication, Speech models, Speech recognition, Speech signals, Synthetic media, Transcription
@inproceedings{pimentelPartialAudioDeepfake2026,
title = {Partial Audio Deepfake Detection: Are We Really Detecting Synthetic Media or Just Dataset Content Biases?},
author = {A. Pimentel and Y. Zhu and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105042045476?origin=resultslist},
doi = {10.1109/CAI68641.2026.11536622},
isbn = {979-833156039-3 (ISBN)},
year = {2026},
date = {2026-01-01},
booktitle = {IEEE Conf. Artif. Intell., CAI},
pages = {1846–1851},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {In recent years, the generation of highly realistic audio deepfakes has raised significant concerns regarding privacy and information integrity. While most research has focused on fully bonafide or spoofed speech, partial deep-fakes, where only segments of an utterance are manipulated, remain less explored. Given the nature of the task, existing datasets are relying on large language models to manipulate bonafide speech signals into partial deepfakes by altering, deleting, or replacing segments with synthetic content. These manipulations may alter the semantics and sentiment of the generated content, creating biases that can be captured by deepfake detection models, leading to overly optimistic performance and poor generalizability. In this work, we investigate the presence of such biases in two popular partial deepfake datasets, namely AV-Deepfake1M and PartialEdit. We explore four self-supervised learning models, two relying only on text transcriptions (RoBERTa and RoBERTa-sentiment) and two relying on universal speech representations (WavLM Large and Wav2vec2-XLSR). Our experiments show that, while speech models achieve higher in-domain performance, they do not generalize to out-of-domain conditions. In turn, models trained on only transcribed speech can effectively distinguish manipulated content, achieving up to 97.8% AUC in-domain and nearly 70% AUC out-of-domain. These results suggest that linguistic patterns and dataset confounds may indeed be biasing partial deepfake detection models, leading to poor generalizability. © 2026 IEEE.},
note = {Journal Abbreviation: IEEE Conf. Artif. Intell., CAI},
keywords = {Audio signal processing, Computational linguistics, Condition, Detection models, Information integrity, Language model, Large datasets, Learning models, Optimistics, Performance, Self-supervised learning, Speech communication, Speech models, Speech recognition, Speech signals, Synthetic media, Transcription},
pubstate = {published},
tppubtype = {inproceedings}
}
In recent years, the generation of highly realistic audio deepfakes has raised significant concerns regarding privacy and information integrity. While most research has focused on fully bonafide or spoofed speech, partial deep-fakes, where only segments of an utterance are manipulated, remain less explored. Given the nature of the task, existing datasets are relying on large language models to manipulate bonafide speech signals into partial deepfakes by altering, deleting, or replacing segments with synthetic content. These manipulations may alter the semantics and sentiment of the generated content, creating biases that can be captured by deepfake detection models, leading to overly optimistic performance and poor generalizability. In this work, we investigate the presence of such biases in two popular partial deepfake datasets, namely AV-Deepfake1M and PartialEdit. We explore four self-supervised learning models, two relying only on text transcriptions (RoBERTa and RoBERTa-sentiment) and two relying on universal speech representations (WavLM Large and Wav2vec2-XLSR). Our experiments show that, while speech models achieve higher in-domain performance, they do not generalize to out-of-domain conditions. In turn, models trained on only transcribed speech can effectively distinguish manipulated content, achieving up to 97.8% AUC in-domain and nearly 70% AUC out-of-domain. These results suggest that linguistic patterns and dataset confounds may indeed be biasing partial deepfake detection models, leading to poor generalizability. © 2026 IEEE.
2.
Valem, L. P.; Pedronette, D. C. G.; Allili, M. S.
Contrastive Loss Based on Contextual Similarity for Image Classification Proceedings Article
In: G., Bebis; V., Patel; J., Gu; J., Panetta; Y., Gingold; K., Johnsen; M.S., Arefin; S., Dutta; A., Biswas (Ed.): Lect. Notes Comput. Sci., pp. 58–69, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-303177391-4 (ISBN), (Journal Abbreviation: Lect. Notes Comput. Sci.).
Abstract | Links | BibTeX | Tags: Adversarial machine learning, Classification accuracy, Contrastive Learning, Cross entropy, Experimental evaluation, Federated learning, Image classification, Image comparison, Image embedding, Images classification, Model generalization, Model robustness, Neighborhood information, Self-supervised learning, Similarity measure
@inproceedings{valem_contrastive_2025,
title = {Contrastive Loss Based on Contextual Similarity for Image Classification},
author = {L. P. Valem and D. C. G. Pedronette and M. S. Allili},
editor = {Bebis G. and Patel V. and Gu J. and Panetta J. and Gingold Y. and Johnsen K. and Arefin M.S. and Dutta S. and Biswas A.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85218461565&doi=10.1007%2f978-3-031-77392-1_5&partnerID=40&md5=cf885303646c3b1a4f4eacb87d02a2b6},
doi = {10.1007/978-3-031-77392-1_5},
isbn = {03029743 (ISSN); 978-303177391-4 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15046 LNCS},
pages = {58–69},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Contrastive learning has been extensively exploited in self-supervised and supervised learning due to its effectiveness in learning representations that distinguish between similar and dissimilar images. It offers a robust alternative to cross-entropy by yielding more semantically meaningful image embeddings. However, most contrastive losses rely on pairwise measures to assess the similarity between elements, ignoring more general neighborhood information that can be leveraged to enhance model robustness and generalization. In this paper, we propose the Contextual Contrastive Loss (CCL) to replace pairwise image comparison by introducing a new contextual similarity measure using neighboring elements. The CCL yields a more semantically meaningful image embedding ensuring better separability of classes in the latent space. Experimental evaluation on three datasets (Food101, MiniImageNet, and CIFAR-100) has shown that CCL yields superior results by achieving up to 10.76% relative gains in classification accuracy, particularly for fewer training epochs and limited training data. This demonstrates the potential of our approach, especially in resource-constrained scenarios. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.},
note = {Journal Abbreviation: Lect. Notes Comput. Sci.},
keywords = {Adversarial machine learning, Classification accuracy, Contrastive Learning, Cross entropy, Experimental evaluation, Federated learning, Image classification, Image comparison, Image embedding, Images classification, Model generalization, Model robustness, Neighborhood information, Self-supervised learning, Similarity measure},
pubstate = {published},
tppubtype = {inproceedings}
}
Contrastive learning has been extensively exploited in self-supervised and supervised learning due to its effectiveness in learning representations that distinguish between similar and dissimilar images. It offers a robust alternative to cross-entropy by yielding more semantically meaningful image embeddings. However, most contrastive losses rely on pairwise measures to assess the similarity between elements, ignoring more general neighborhood information that can be leveraged to enhance model robustness and generalization. In this paper, we propose the Contextual Contrastive Loss (CCL) to replace pairwise image comparison by introducing a new contextual similarity measure using neighboring elements. The CCL yields a more semantically meaningful image embedding ensuring better separability of classes in the latent space. Experimental evaluation on three datasets (Food101, MiniImageNet, and CIFAR-100) has shown that CCL yields superior results by achieving up to 10.76% relative gains in classification accuracy, particularly for fewer training epochs and limited training data. This demonstrates the potential of our approach, especially in resource-constrained scenarios. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
3.
Guimaraes, H. R.; Abdollahi, M.; Zhu, Y.; Maucourt, S.; Coallier, N.; Giovenazzo, P.; Falk, T. H.
Benchmarking Self-Supervised Audio Representations for IoT-Enabled Acoustic Beehive Monitoring Journal Article
In: IEEE Internet of Things Journal, vol. 12, no. 21, pp. 45000–45010, 2025, ISSN: 23274662 (ISSN).
Abstract | Links | BibTeX | Tags: Acoustics, Audio acoustics, Audio representation, Beehive monitoring, Benchmarking, Bioacoustics, Computer vision applications, Deep learning, Honeybee, honeybees, Internet of Things (IoT), IoT, Labeled data, Performance, Real time systems, Self-supervised learning, self-supervised learning (SSL), Societal benefits, Speech applications, Speech recognition, Supervised learning, Universal feature extractors
@article{guimaraesBenchmarkingSelfSupervisedAudio2025,
title = {Benchmarking Self-Supervised Audio Representations for IoT-Enabled Acoustic Beehive Monitoring},
author = {H. R. Guimaraes and M. Abdollahi and Y. Zhu and S. Maucourt and N. Coallier and P. Giovenazzo and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105013592730?origin=resultslist},
doi = {10.1109/JIOT.2025.3599483},
issn = {23274662 (ISSN)},
year = {2025},
date = {2025-01-01},
journal = {IEEE Internet of Things Journal},
volume = {12},
number = {21},
pages = {45000–45010},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Self-supervised learning (SSL) has enabled the development of universal feature extractors that have redefined the performance envelope of computer vision and speech applications. Recent works have started to explore SSL in other domains, including bioacoustics, which could have significant societal benefits. Honeybees (Apis mellifera), for example, are crucial pollinators contributing to one-third of global food production. However, massive colony losses in recent years have raised concerns. Traditional hive monitoring methods rely on intrusive visual inspections by beekeepers, which can further disrupt colony dynamics. As such, Internet of Things (IoT)-based automated monitoring systems have emerged, integrating environmental and bioacoustic sensing to enable real-time, noninvasive hive assessment. In this work, we introduce a comprehensive evaluation and benchmarking of general-purpose and bioacoustic audio representations that generalize across various tasks in IoT-enabled acoustic beehive monitoring, even with limited labeled data. Herein, fourteen models are evaluated across four critical tasks: beehive state detection, beehive strength assessment, buzzing identification, and beekeeper voice activity detection. Reported results demonstrate the strong generalizability of existing representations, paving the way for advanced, scalable honeybee colony monitoring and preservation. © 2014 IEEE.},
keywords = {Acoustics, Audio acoustics, Audio representation, Beehive monitoring, Benchmarking, Bioacoustics, Computer vision applications, Deep learning, Honeybee, honeybees, Internet of Things (IoT), IoT, Labeled data, Performance, Real time systems, Self-supervised learning, self-supervised learning (SSL), Societal benefits, Speech applications, Speech recognition, Supervised learning, Universal feature extractors},
pubstate = {published},
tppubtype = {article}
}
Self-supervised learning (SSL) has enabled the development of universal feature extractors that have redefined the performance envelope of computer vision and speech applications. Recent works have started to explore SSL in other domains, including bioacoustics, which could have significant societal benefits. Honeybees (Apis mellifera), for example, are crucial pollinators contributing to one-third of global food production. However, massive colony losses in recent years have raised concerns. Traditional hive monitoring methods rely on intrusive visual inspections by beekeepers, which can further disrupt colony dynamics. As such, Internet of Things (IoT)-based automated monitoring systems have emerged, integrating environmental and bioacoustic sensing to enable real-time, noninvasive hive assessment. In this work, we introduce a comprehensive evaluation and benchmarking of general-purpose and bioacoustic audio representations that generalize across various tasks in IoT-enabled acoustic beehive monitoring, even with limited labeled data. Herein, fourteen models are evaluated across four critical tasks: beehive state detection, beehive strength assessment, buzzing identification, and beekeeper voice activity detection. Reported results demonstrate the strong generalizability of existing representations, paving the way for advanced, scalable honeybee colony monitoring and preservation. © 2014 IEEE.
4.
Valem, L. P.; Pedronette, D. C. G.; Allili, M. S.
Contrastive Loss Based on Contextual Similarity for Image Classification Proceedings Article
In: G., Bebis; V., Patel; J., Gu; J., Panetta; Y., Gingold; K., Johnsen; M.S., Arefin; S., Dutta; A., Biswas (Ed.): Lect. Notes Comput. Sci., pp. 58–69, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-303177391-4 (ISBN), (Journal Abbreviation: Lect. Notes Comput. Sci.).
Abstract | Links | BibTeX | Tags: Adversarial machine learning, Classification accuracy, Contrastive Learning, Cross entropy, Experimental evaluation, Federated learning, Image classification, Image comparison, Image embedding, Images classification, Model generalization, Model robustness, Neighborhood information, Self-supervised learning, Similarity measure
@inproceedings{valemContrastiveLossBased2025a,
title = {Contrastive Loss Based on Contextual Similarity for Image Classification},
author = {L. P. Valem and D. C. G. Pedronette and M. S. Allili},
editor = {Bebis G. and Patel V. and Gu J. and Panetta J. and Gingold Y. and Johnsen K. and Arefin M.S. and Dutta S. and Biswas A.},
url = {https://www.scopus.com/pages/publications/85218461565?origin=resultslist},
doi = {10.1007/978-3-031-77392-1_5},
isbn = {03029743 (ISSN); 978-303177391-4 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15046 LNCS},
pages = {58–69},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Contrastive learning has been extensively exploited in self-supervised and supervised learning due to its effectiveness in learning representations that distinguish between similar and dissimilar images. It offers a robust alternative to cross-entropy by yielding more semantically meaningful image embeddings. However, most contrastive losses rely on pairwise measures to assess the similarity between elements, ignoring more general neighborhood information that can be leveraged to enhance model robustness and generalization. In this paper, we propose the Contextual Contrastive Loss (CCL) to replace pairwise image comparison by introducing a new contextual similarity measure using neighboring elements. The CCL yields a more semantically meaningful image embedding ensuring better separability of classes in the latent space. Experimental evaluation on three datasets (Food101, MiniImageNet, and CIFAR-100) has shown that CCL yields superior results by achieving up to 10.76% relative gains in classification accuracy, particularly for fewer training epochs and limited training data. This demonstrates the potential of our approach, especially in resource-constrained scenarios. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.},
note = {Journal Abbreviation: Lect. Notes Comput. Sci.},
keywords = {Adversarial machine learning, Classification accuracy, Contrastive Learning, Cross entropy, Experimental evaluation, Federated learning, Image classification, Image comparison, Image embedding, Images classification, Model generalization, Model robustness, Neighborhood information, Self-supervised learning, Similarity measure},
pubstate = {published},
tppubtype = {inproceedings}
}
Contrastive learning has been extensively exploited in self-supervised and supervised learning due to its effectiveness in learning representations that distinguish between similar and dissimilar images. It offers a robust alternative to cross-entropy by yielding more semantically meaningful image embeddings. However, most contrastive losses rely on pairwise measures to assess the similarity between elements, ignoring more general neighborhood information that can be leveraged to enhance model robustness and generalization. In this paper, we propose the Contextual Contrastive Loss (CCL) to replace pairwise image comparison by introducing a new contextual similarity measure using neighboring elements. The CCL yields a more semantically meaningful image embedding ensuring better separability of classes in the latent space. Experimental evaluation on three datasets (Food101, MiniImageNet, and CIFAR-100) has shown that CCL yields superior results by achieving up to 10.76% relative gains in classification accuracy, particularly for fewer training epochs and limited training data. This demonstrates the potential of our approach, especially in resource-constrained scenarios. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
5.
Valem, L. P.; Pedronette, D. C. G.; Allili, M. S.
Contrastive Loss Based on Contextual Similarity for Image Classification Proceedings Article
In: G., Bebis; V., Patel; J., Gu; J., Panetta; Y., Gingold; K., Johnsen; M.S., Arefin; S., Dutta; A., Biswas (Ed.): Lect. Notes Comput. Sci., pp. 58–69, Springer Science and Business Media Deutschland GmbH, 2025, ISBN: 03029743 (ISSN); 978-303177391-4 (ISBN), (Journal Abbreviation: Lect. Notes Comput. Sci.).
Abstract | Links | BibTeX | Tags: Adversarial machine learning, Classification accuracy, Contrastive Learning, Cross entropy, Experimental evaluation, Federated learning, Image classification, Image comparison, Image embedding, Images classification, Model generalization, Model robustness, Neighborhood information, Self-supervised learning, Similarity measure
@inproceedings{valemContrastiveLossBased2025,
title = {Contrastive Loss Based on Contextual Similarity for Image Classification},
author = {L. P. Valem and D. C. G. Pedronette and M. S. Allili},
editor = {Bebis G. and Patel V. and Gu J. and Panetta J. and Gingold Y. and Johnsen K. and Arefin M.S. and Dutta S. and Biswas A.},
url = {https://www.scopus.com/inward/record.uri?eid=2-s2.0-85218461565&doi=10.1007%2f978-3-031-77392-1_5&partnerID=40&md5=cf885303646c3b1a4f4eacb87d02a2b6},
doi = {10.1007/978-3-031-77392-1_5},
isbn = {03029743 (ISSN); 978-303177391-4 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Lect. Notes Comput. Sci.},
volume = {15046 LNCS},
pages = {58–69},
publisher = {Springer Science and Business Media Deutschland GmbH},
abstract = {Contrastive learning has been extensively exploited in self-supervised and supervised learning due to its effectiveness in learning representations that distinguish between similar and dissimilar images. It offers a robust alternative to cross-entropy by yielding more semantically meaningful image embeddings. However, most contrastive losses rely on pairwise measures to assess the similarity between elements, ignoring more general neighborhood information that can be leveraged to enhance model robustness and generalization. In this paper, we propose the Contextual Contrastive Loss (CCL) to replace pairwise image comparison by introducing a new contextual similarity measure using neighboring elements. The CCL yields a more semantically meaningful image embedding ensuring better separability of classes in the latent space. Experimental evaluation on three datasets (Food101, MiniImageNet, and CIFAR-100) has shown that CCL yields superior results by achieving up to 10.76% relative gains in classification accuracy, particularly for fewer training epochs and limited training data. This demonstrates the potential of our approach, especially in resource-constrained scenarios. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.},
note = {Journal Abbreviation: Lect. Notes Comput. Sci.},
keywords = {Adversarial machine learning, Classification accuracy, Contrastive Learning, Cross entropy, Experimental evaluation, Federated learning, Image classification, Image comparison, Image embedding, Images classification, Model generalization, Model robustness, Neighborhood information, Self-supervised learning, Similarity measure},
pubstate = {published},
tppubtype = {inproceedings}
}
Contrastive learning has been extensively exploited in self-supervised and supervised learning due to its effectiveness in learning representations that distinguish between similar and dissimilar images. It offers a robust alternative to cross-entropy by yielding more semantically meaningful image embeddings. However, most contrastive losses rely on pairwise measures to assess the similarity between elements, ignoring more general neighborhood information that can be leveraged to enhance model robustness and generalization. In this paper, we propose the Contextual Contrastive Loss (CCL) to replace pairwise image comparison by introducing a new contextual similarity measure using neighboring elements. The CCL yields a more semantically meaningful image embedding ensuring better separability of classes in the latent space. Experimental evaluation on three datasets (Food101, MiniImageNet, and CIFAR-100) has shown that CCL yields superior results by achieving up to 10.76% relative gains in classification accuracy, particularly for fewer training epochs and limited training data. This demonstrates the potential of our approach, especially in resource-constrained scenarios. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.