We describe a new adaptive algorithm for training a shallow neural network based on random Fourier features. We apply the algorithm to learn and approximate dynamics defined by autonomous differential equations. Furthermore, we demonstrate, in computational examples, that the method decreases training time. We show that given sufficient training data, the discovered dynamics are robust against extrapolation in time and initial conditions. We compare the developed algorithm to more standard off-the-shelf neural network approaches.