Assessment of urban structural damage is crucial in the management of disasters like earthquakes or any structural damages, such as complete collapse, collapsed roofs, rubble piles, and sloping facets. To gather data for this assignment, high-resolution camera is installed on an unmanned aerial vehicle (UAV). Then utilize image based analysis and classification to detect and quantify any structural alterations or damage. Segmenting the photos into distinct objects based on their features, such as shape, size, and dimensions, is the process in image-based analysis and classification. Semantic reasoning involves inferring relationships between items by leveraging contextual information from the images, such as the position and orientation of the objects. In comparison to conventional visual inspection methods, image-based analysis and classification are used to identify and classify structural damage with greater accuracy and efficiency. This method uses Convolutional Neural Network for classifying images into damaged and non-damaged. Further, the identified damaged structures are classified according to the EMS-98 scale. Further, it is used to track structural changes over time and determine which maintenance and repair tasks should be prioritized.