Interatomic potentials are invaluable tools in the fields of computational materials science, chemistry, and biology for their ability to accelerate atomic-scale simulations beyond the length- and time-scales that are accessible using first-principles techniques. With the advent of machine learning interatomic potentials (MLIPs), an increasingly overlooked aspect of interatomic potentials is in using them to help researchers identify and categorize trends in atomic interactions within and across chemical systems. While this functionality is critical for learning structure-property relationships, MLIP developers typically design models with an emphasis on accuracy at the expense of interpretability. In this work we introduce a new MLIP framework which uses spline-based filters for mapping atomic environments into neural network inputs. We show that this framework, which we call the spline-based neural network potential (s-NNP), is a simplified version of the traditional neural network potential that can be used to describe complex datasets in a computationally efficient manner while also providing straightforward techniques for identifying key physical trends learned by the model. Further, we demonstrate that the spline filters transfer well to different chemical systems, thus producing a convenient reference point from which to begin performing cross-system analyses.