Systems for video surveillance are widely used and ubiquitous in various settings. bank, airports, and the prison has seen substantialimprovements by video monitoring. Nowadays, corporations, government organizations, and evenschools have started to use video surveillance to improve public safety. In this paper, Convolution Neural Network (CNN) algorithms are used in conjunction with advanced deep learning techniques to monitor video data from numerous cameras and identify anomalous activities in real-time by analyzing using UCFCrime trained datasets. The Smart Video Surveillance systemexamines behavioral patterns and a number of other factors, such as body posture andmovement, in order to spot potential risks and foretell anomalous activity. It notifies specifiedpeople in real-time when an unexpected behavior is discovered and records strange activities in a database for further examination. A proactive approach to security is provided by the smart video surveillance system, which foresees potentially dangerousscenarioslike (Abuse, Arrest, Burglary, Fighting, Etc.) and takes actionto prevent them. Its sophisticated algorithms and real-time alert feature using SMTP server gives anextra layer of security by enabling people to reactswiftly to any potential dangers. With the CNN deep learning algorithm, we were able to effectively predict unusual activities for both prerecorded video files and real time detection. Byutilizing this technology, public safety and securityare tremendously boosted, and crime rates are simultaneously reduced.