When deformed beyond their elastic limits, crystalline solids flow plastically via particle rearrangements localized around structural defects. Disordered solids also flow, but without obvious structural defects. We link structure to plasticity in disordered solids via a microscopic structural quantity, “softness,” designed by machine learning to be maximally predictive of rearrangements. Experiments and computations measure spatial correlations and strain response of softness, as well as two measures of plasticity—the size of rearrangements and the yield strain. All four quantities maintain remarkable commonality in their values for disordered packings of objects ranging from atoms to grains, spanning over 7 to 13 orders of magnitude in diameter and elastic modulus. These commonalities link softness spatial correlations to rearrangement size, and softness strain response to yield strain.