Slide
Centre Interdisciplinaire
de Recherche et d’Innovation
en Cybersécurité et Société
de Recherche et d’Innovation
en Cybersécurité et Société
1.
Temmar, D. E.; Hamadene, A.; Nallaguntla, V.; Fursule, A.; Allili, M. S.; Kshirsagar, S.; Avila, A. R.
Phonetic Analysis of Real and Synthetic Speech Using HuBERT Embeddings: Perspectives for Deepfake Detection Article d'actes
Dans: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern., p. 86–91, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 1062922X (ISSN); 979-833153358-8 (ISBN), (Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.).
Résumé | Liens | BibTeX | Étiquettes: Artificial intelligence, Audio acoustics, Audio DeepFake Detection, Audio signal processing, Embeddings, Hu-BERT, KL-divergence, Linguistics, Phoneme and word Embedding, Phonetic analysis, Security systems, Self-Supervised Speech Representation, Speech analysis, Speech communication, Speech processing, Speech synthesis, Synthetic speech, Text to speech, Voice conversion
@inproceedings{temmarPhoneticAnalysisReal2025,
title = {Phonetic Analysis of Real and Synthetic Speech Using HuBERT Embeddings: Perspectives for Deepfake Detection},
author = {D. E. Temmar and A. Hamadene and V. Nallaguntla and A. Fursule and M. S. Allili and S. Kshirsagar and A. R. Avila},
url = {https://www.scopus.com/pages/publications/105033145913?origin=resultslist},
doi = {10.1109/SMC58881.2025.11343334},
isbn = {1062922X (ISSN); 979-833153358-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
pages = {86–91},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {The growing sophistication of speech generated by Artificial Intelligence (AI) has introduced new challenges in audio deepfake detection. Text-to-speech (TTS) and voice conversion (VC) technologies can now produce convincing synthetic speech with high quality and intelligibility. This poses a serious threat to voice biometric security systems, such as automatic speaker recognition. It also increases the risks associated to the spread of spoken disinformation, where synthetic voices can be used to disseminate malicious content. In this study, we conduct an analysis of real and synthetic speech at phonetic and word levels. For that, a parallel dataset comprising real and synthetic speech signals were developed based on a subset of the LibriSpeech ASR corpus. Synthetic speech samples were generated using two TTS and one VC systems: Coqui TTS, VITS TTS, and StarGANv2 VC. We adopted HuBERT, a self-supervised speech model, to extract speech embeddings. The motivation for using this model stems from its ability to recognize sound units corresponding to the so-called pseudo phonemes. Our analysis is based on the KL divergence (KLD) between the distributions of synthetic and real phonemes, which allowed us to rank synthetic phonemes based on their alignment with their real counterpart. We also trained several classifiers per phoneme to distinguish between real and synthetic samples. We then compute the correlations between KLD and accuracies per phoneme. Besides showing a list of phonemes that are more discriminative, our findings suggest that vowels correlate better with the classifiers' performance, suggesting that the KLD can be an indicator of the most distinguishable phonemes for deepfake detection. © 2025 IEEE.},
note = {Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
keywords = {Artificial intelligence, Audio acoustics, Audio DeepFake Detection, Audio signal processing, Embeddings, Hu-BERT, KL-divergence, Linguistics, Phoneme and word Embedding, Phonetic analysis, Security systems, Self-Supervised Speech Representation, Speech analysis, Speech communication, Speech processing, Speech synthesis, Synthetic speech, Text to speech, Voice conversion},
pubstate = {published},
tppubtype = {inproceedings}
}
The growing sophistication of speech generated by Artificial Intelligence (AI) has introduced new challenges in audio deepfake detection. Text-to-speech (TTS) and voice conversion (VC) technologies can now produce convincing synthetic speech with high quality and intelligibility. This poses a serious threat to voice biometric security systems, such as automatic speaker recognition. It also increases the risks associated to the spread of spoken disinformation, where synthetic voices can be used to disseminate malicious content. In this study, we conduct an analysis of real and synthetic speech at phonetic and word levels. For that, a parallel dataset comprising real and synthetic speech signals were developed based on a subset of the LibriSpeech ASR corpus. Synthetic speech samples were generated using two TTS and one VC systems: Coqui TTS, VITS TTS, and StarGANv2 VC. We adopted HuBERT, a self-supervised speech model, to extract speech embeddings. The motivation for using this model stems from its ability to recognize sound units corresponding to the so-called pseudo phonemes. Our analysis is based on the KL divergence (KLD) between the distributions of synthetic and real phonemes, which allowed us to rank synthetic phonemes based on their alignment with their real counterpart. We also trained several classifiers per phoneme to distinguish between real and synthetic samples. We then compute the correlations between KLD and accuracies per phoneme. Besides showing a list of phonemes that are more discriminative, our findings suggest that vowels correlate better with the classifiers' performance, suggesting that the KLD can be an indicator of the most distinguishable phonemes for deepfake detection. © 2025 IEEE.
2.
Zhu, Y.; Davoust, A.; Falk, T. H.
DeepSick: Deceiving Voice-Based Diagnostic Models with Synthetic Multilingual Pathological Speech Signals Article d'actes
Dans: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern., p. 69–74, Institute of Electrical and Electronics Engineers Inc., 2025, ISBN: 1062922X (ISSN); 979-833153358-8 (ISBN), (Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.).
Résumé | Liens | BibTeX | Étiquettes: COVID-19, Detection models, Diagnosis, Diagnostic model, Diagnostic systems, Generative model, Health assessments, Pathological conditions, Pathological speech signals, Scalable solution, Speech communication, Speech recognition, Speech synthesis, State of the art, Voice model
@inproceedings{zhuDeepSickDeceivingVoiceBased2025,
title = {DeepSick: Deceiving Voice-Based Diagnostic Models with Synthetic Multilingual Pathological Speech Signals},
author = {Y. Zhu and A. Davoust and T. H. Falk},
url = {https://www.scopus.com/pages/publications/105033143787?origin=resultslist},
doi = {10.1109/SMC58881.2025.11343240},
isbn = {1062922X (ISSN); 979-833153358-8 (ISBN)},
year = {2025},
date = {2025-01-01},
booktitle = {Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
pages = {69–74},
publisher = {Institute of Electrical and Electronics Engineers Inc.},
abstract = {Voice-based diagnostic systems offer a scalable solution for remote health assessment. However, recent advances in generative voice models may enable malicious manipulation of voice samples to simulate or conceal disease-related speech characteristics, which poses new risks to diagnostic systems. This paper investigates the vulnerability of diagnostic and detection models to such types of "deepfake"attacks. We show that it is possible to train a generative model to convert between healthy voices and pathological ones, which in turn, can successfully deceive existing diagnostic systems. Here, focus is placed on COVID-19 infection and respiratory abnormalities, but the method can be applied across different pathological conditions affecting vocal attributes. We also benchmark four state-of-the-art synthesized voice detection models on both real and generated pathological speech from three datasets. Our results show that current synthetic voice detectors, typically trained on healthy speech data, perform poorly on generated pathological samples. While fine-tuning with real pathological voices improves detection, a substantial performance gap remains. This work provides initial insights on an emerging threat to remote voice diagnostic systems that needs further work. © 2025 IEEE.},
note = {Journal Abbreviation: Conf. Proc. IEEE Int. Conf. Syst. Man Cybern.},
keywords = {COVID-19, Detection models, Diagnosis, Diagnostic model, Diagnostic systems, Generative model, Health assessments, Pathological conditions, Pathological speech signals, Scalable solution, Speech communication, Speech recognition, Speech synthesis, State of the art, Voice model},
pubstate = {published},
tppubtype = {inproceedings}
}
Voice-based diagnostic systems offer a scalable solution for remote health assessment. However, recent advances in generative voice models may enable malicious manipulation of voice samples to simulate or conceal disease-related speech characteristics, which poses new risks to diagnostic systems. This paper investigates the vulnerability of diagnostic and detection models to such types of "deepfake"attacks. We show that it is possible to train a generative model to convert between healthy voices and pathological ones, which in turn, can successfully deceive existing diagnostic systems. Here, focus is placed on COVID-19 infection and respiratory abnormalities, but the method can be applied across different pathological conditions affecting vocal attributes. We also benchmark four state-of-the-art synthesized voice detection models on both real and generated pathological speech from three datasets. Our results show that current synthetic voice detectors, typically trained on healthy speech data, perform poorly on generated pathological samples. While fine-tuning with real pathological voices improves detection, a substantial performance gap remains. This work provides initial insights on an emerging threat to remote voice diagnostic systems that needs further work. © 2025 IEEE.