Fruits and vegetables have always had a significant economic impact on human survival, providing food security and boosting output with minimal input. This review focuses on an in-depth analysis of the grading criteria and the identification of exterior quality characteristics of major vegetables and fruits through various noninvasive spectroscopic and imaging methods, along with a brief discussion of their key components, schematic operations, potential for application in place of conventional approaches, and highlights the potential research gaps. In this review, the attention was focused on preprocessing, data analysis techniques, and the specific and overall values of performance accuracy by using a specific performance metric in relation to fruits and vegetables. Several machine learning (ML), as well as deep learning (DL) techniques, such as K-nearest neighbor (KNN), artificial neural networks (ANN), support vector machines (SVM), convolutional neural networks (CNN) with transfer learning (TL), generative adversarial networks (GAN) and recurrent neural network (RNN), have recently been used for inspection along with the processing of spectral data. ML and DL techniques have been proposed in recent publications for the external quality inspection of fruits and vegetables.