Retrieving visual microphone sound using the PSO-CNN hybrid technique

The goal of the presented work is to create a novel technique to extract sound from silent video content by utilizing a concept known as a "virtual microphone". Using sound waves, this technique allows sound to be recovered by creating micro-vibrations on the body in the video. It extracts moving frames and generates the audio that goes with them using swarm optimization methods, a virtual microphone, and deep learning algorithms. In order to enhance the performance of the CNN architecture, the paper suggests a hybrid PSO-CNN technique which combines particle swarm optimization (PSO) and convolutional neural networks (CNN). The hybrid method had faced an issue that is related to the conversion of the visual microphones to auditory noise through using the ability of the CNNs for the extraction of complicated characteristics and spatial relations from the video frames. Usually, the PSO performs dynamic modification of mesh resolution for the purpose of enhancing the performances of the virtual microphone systems. Good results have been noted as the proposed method beats the conventional procedures in terms of the sound restoration quality and accuracy. Robustness of the system and convergence have been improved when the CNN and PSO were joined. Visual analysis technique advancement in security and surveillance, amongst other types of domain, has been facilitated by the present research.