Robust stability and stochastic stability have separately seen intense study in control theory since its inception. In this work we establish relations between these properties for discrete-time systems. Specifically, we examine a robustness framework which models the inherent uncertainty and variation in the system dynamics which arise in model-based learning control methods such as adaptive control and reinforcement learning. We provide results which guarantee mean-square stability margins in terms of multiplicative noises which affect the nominal dynamics, as well as connections to prior work which together imply that robust stability and mean-square stability are, in a certain sense, equivalent.