The complex physics and numerous failure modes of structural impact create challenges when designing for impact resistance. While simple geometries of layered material are conventional, advances in 3D printing and additive manufacturing techniques have made tailored geometries or integrated multi-material structures achievable. Here, we apply gradient-based topology optimization to the design of such structures. We start by constructing a variational model of an elastic-plastic material enriched with a gradient phase-field damage model and develop a novel method to efficiently compute the transient dynamic time-evolution. We consider a finite element discretization with explicit updates for the displacements. The damage field is solved through an augmented Lagrangian formulation, splitting the operator coupling between the nonlinearity and non-locality. Sensitivities over this trajectory are computed through the adjoint method, resulting in an adjoint problem we solve in a similar method to the forward dynamics. We demonstrate this formulation by studying the optimal design of a 2D solid-void structure undergoing dynamic loading. Then, we explore the trade-offs between strength and toughness to design a spall-resistant structure composed of two materials of differing properties.