Compressive strength is the most important metric of concrete quality. Various nondestructive and semi-destructive tests can be used to evaluate the compressive strength of concrete. In the present study, a new image-based machine learning method is used to predict concrete compressive strength, including evaluation of six different models. These include support-vector machine model and various deep convolutional neural network models, namely AlexNet, GoogleNet, VGG19, ResNet, and Inception-ResNet-V2. In the present investigation, cement mortar samples were prepared using each of the cement:sand ratios of 1:3, 1:4, and 1:5, and using the water:cement ratios of 0.35 and 0.55. Cement concrete was prepared using the cement:sand:coarse aggregate ratios of 1:5:10, 1:3:6, 1:2:4, 1:1.5:3 and 1:1:2, using the water:cement ratio of 0.5 for all samples. The samples were cut, and several images of the cut surfaces were captured at various zoom levels using a digital microscope. All samples were then tested destructively for compressive strength. The images and corresponding compressive strength were then used to train machine learning models to allow them to predict compressive strength based upon the image data. The Inception-ResNet-V2 models exhibited the best predictions of compressive strength among the models tested. Overall, the present findings validated the use of machine learning models as an efficient means of estimating cement mortar and concrete compressive strengths based on digital microscopic images, as an alternative nondestructive/semi-destructive test method that could be applied at relatively less expense.