Tehnici de învățare automată pentru analiza vizuală a interacțiunii umane / Machine learning-based visual analysis of human-to-human interaction

In contemporary discourse within academia and technology-focused media, the subject of public safety has come to the forefront. This is particularly pertinent as the digital transformation landscape evolves and the proliferation of big data affects diverse sectors ranging from retail environments to public transportation and open public spaces. Within this context, computer vision has become increasingly influential. It facilitates the automatic ingestion, preprocessing, and interpretation of data, transforming massive amounts of data into actionable insights with ease. However, the challenge of processing trainable data in-the-wild complicates the task for artificial intelligence. To address this, advanced multi-tiered data processing systems are essential, particularly for visual sensing mechanisms, including video surveillance cameras and imagery systems. In this regard, this thesis introduces the first lip reading system for under-resourced Romanian language and an end-to-end system for aerial surveillance systems designed for managing emergency situations. It also explores the deep learning models to interpret human-to-human interaction, providing an algorithm to extract unique crowd behavior scenes necessary to automatically highlight areas of coherent crowd groups by a supervised deep learning algorithm in the semantic segmentation task. This thesis also extends beyond human interactions to human-to-media product interaction, presenting a captivating understanding not only for the generation of synthetic content like deepfakes but also for encoding media with features that enhance its memorability. Focusing on improving public safety through practical machine learning techniques, this work includes methodologies for creating AI-training datasets and outlines system design considerations, tested against real perturbation to verify their generalization efficacy