Quantum chemical calculations using density functional theory (DFT) are now commonplace in materials science. There are several publicly available materials databases (Materials Project, OQMD, AFLOW, JARVIS, etc.) that house DFT calculation results for millions of inorganic crystals. One of the tabulated properties for each calculation (structure) is the DFT-calculated formation energy. Machine learning (ML) models are then often tasked with learning from these databases to predict compound formation energies. These formation energies can be used as inputs to construct phase diagrams and determine the stability of materials. Recent efforts have shown that ML models trained on DFT-calculated formation energies for solid-state materials achieve accuracies comparable to DFT itself (relative to experiment). In principle, these ML models should therefore be capable of predicting the stability of materials with comparable accuracy to DFT at a miniscule fraction of the computational expense. However, a systematic evaluation of phase diagrams resulting from DFT and ML reveals that this is not necessarily the case. This talk will introduce the role of thermodynamic stability calculations for novel materials discovery, compare the application of DFT and ML for stability predictions, and discuss the limitations of thermodynamic stability in the context of novel materials discovery.