Stress field prediction is typically provided by means of Finite Element Analysis (FEA) which can become computationally prohibitive considering complex material behavior. Such a limitation is especially important within the context of structure–property exploration and inverse design for materials discovery where a larger number of FEA model evaluations is required. Recently, the advent of data-driven methods has directed attention towards machine learning (ML) techniques as cost effective FEA surrogates in materials applications. Despite the large body of work, the majority of ML approaches are either limited to low-dimensional (vector valued) problems and/or do not provide uncertainty estimates in the predictions. In this regard, building upon previous results at the intersection between solid mechanics and data-driven materials modeling, this work proposes a framework for stress field prediction and uncertainty quantification for diverse materials microstructures. First, a modified U-Net neural network (NN) is employed to provide a data-driven image-to-image mapping together with an uncertainty heatmap. To achieve this, the NN parameters are treated as random variables in a probabilistic setting. Next, the posterior of the parameters with respect to the data is estimated in order to quantify the degree of confidence in the predictions. Specifically, three state-of-the-art algorithms which are relevant to UQ in deep learning are considered: the Monte-Carlo Dropout technique, the Bayes by Backprop algorithm and the Hamiltonian Monte Carlo method. A systematic comparison of the efficacy of these methods is performed by considering a fiber reinforced composite system as well a polycrystalline material application. It is shown that the proposed methods yield predictions of comparable accuracy to FEA but also offer interpretable uncertainty estimates in the predictions.