Tehnici de analiză și prelucrare a semnalului vocal pentru recunoașterea automată a elementelor paralingvistice, cu aplicații în expertiza criminalistică a vorbirii / Speech signal analysis and processing techniques for automatic recognition of paralinguistic elements, with applications in forensic speech

The scope and objectives of this thesis cover the development of machine learning and deep learning systems for automatic recognition of paralinguistic elements using only speech data, with a focus on manifestations of negative emotions, high psychological stress levels, and engagement in deceptive behavior, the main applications areas being forensic and law enforcement tasks and operations. This led to developments for 3 paralinguistic recognition tasks: speech under stress detection (SSD), deceptive speech detection (DSD), and speech emotion recognition (SER). For each of the 3 tasks, neural network-based deep learning systems and techniques, including novel architectures, were proposed, developed, implemented, and validated on the most important publicly available datasets (SUSAS for SSD; RLDD and RODeCAR for DSD; EMODB, CREMAD, and IEMOCAP for SER). Novel experimental methodologies (e.g., recognition of groups of negative emotions, utterance-level deceptive speech detection) were also developed and employed as alternatives to more common approaches, justified by the focus on forensic speech applications. Additional original contributions were made for various subtasks (e.g., voice activity detection, short- and long term emotion evolution pattern recognition) required within the processing pipelines of the proposed system architectures for the main tasks; in particular, for the DSD task, the novel, reliable, realistic, and consistent RODeCAR (Romanian Deva Criminal Investigation Audio Recordings) dataset was developed end-to-end, one of the very few such datasets available for DSD and the only one for Romanian. For all 3 tasks (i.e., SSD, DSD, and SER), performance improvements over the previous state-of-the-art results were obtained for the great majority of the concerned experimental methodologies.