Revolutionizing crowd surveillance through voice-driven face recognition empowering rapid identification: towards development of sustainable smart cities

Recent global efforts to create sustainable smart cities have significantly transformed society and improved the lives of people. Nowadays, crowd surveillance (CS) has become essential in sustainable smart cities and society to protect public safety and security. In this regard, the face-based human detection system has received considerable attention because it is recognized as an emerging method in crowd surveillance applications. Thus, in this work, a new method for real-time identification of people for a crowd surveillance system (CSS) that uses facial and speech recognition technology has been introduced. In traditional CS systems, human operators are frequently used by crowd surveillance systems to watch and evaluate video feeds. Human error and operator weariness may result in lost opportunities or slow replies, which reduce the system’s efficacy. Certain procedures, including the initial identification and monitoring of people in video feeds, can be automated using a voice-activated system. To address the issues with the present CSS, a new framework Voice-Activated Face Recognition (VAFR) is proposed in this work. The proposed framework combines the speech and face recognition models for crowd surveillance. Experimental and simulation studies have been performed to analyze the performance of the proposed VAFR framework. The proposed framework uses the Viola-Jones algorithm for face identification and the Conformer architecture for speech analysis, reaching a noteworthy 99.8% accuracy rate in live video feeds. In addition, the ethical and safety aspect of the proposed VAFR system is presented.