Improvements in trait phenotyping are needed to increase the quantity and quality of data available for genetic improvement of crops. In this study, we used moderate throughput image analysis and machine learning as a pipeline for phenotyping a key wheat spike characteristic: spikelet number per spike. A population of 594 soft red winter wheat inbred lines was evaluated in the field for 2 years and images of wheat spikes were taken and used to train deep‐learning algorithms to predict spikelet number. A total of 12,717 images were used to train, test, and validate a basic regression convolutional neural network (CNN), a visual geometry group application regression model, VGG16, the ResNet152V2 model, and the EfficientNetV2L model. The EfficientNetV2L model was the most accurate, having the lowest mean absolute error, second lowest root mean square error, and highest coefficient of determination (mean absolute error [MAE] = 0.60, root mean square error [RMSE] = 0.79, and R² = 0.90). The ResNet152V2 model was slightly less accurate with a slightly better fit (MAE = 0.61,m RMSE = 0.78, and R² = 0.87), followed by the basic CNN (MAE = 0.75, RMSE = 1.00, and R² = 0.74) and finally by the VGG16 (MAE = 1.51, RMSE = 1.29, and R² = 0.076). With an average error of just above one half of a spikelet, utilizing image analysis and machine learning counting methods could be used for multiple breeding applications, including direct selection of spikelet number, to provide data to identify quantitative trait loci, or for training whole genome selection models.