In this paper, a new automated orange sorting technique is presented. It sorts orange fruit based on size and external defects. We use image processing techniques for segmentation of oranges and then based on our algorithm categorize them according to the presence and type of defect. The defects considered are anthracnose, unripe and stem-end injury. Non-defected oranges are further categorized into large, medium or small based upon their size. We created an image dataset containing images of 189 oranges. We achieve high accuracy with success rates of 97.4% and 100% for defect-based and size-based sorting, respectively.