The formation of stellar-mass black holes (BHs) is still very uncertain. Two main uncertainties are the amount of mass ejected in the supernova (SN) event (if any) and the magnitude of the natal kick (NK) the BH receives at birth (if any). Repetto et al., studying the position of Galactic X-ray binaries containing BHs, found evidence for BHs receiving high NKs at birth. In this paper, we extend that study, taking into account the previous binary evolution of the sources as well. The seven short-period BH X-ray binaries that we use are compact binaries consisting of a low-mass star orbiting a BH in a period less than 1 d. We trace their binary evolution backwards in time, from the current observed state of mass transfer, to the moment the BH was formed, and we add the extra information on the kinematics of the binaries. We find that several systems could be explained by no NK, just mass ejection, while for two systems (and possibly more) a high kick is required. So unless the latter have an alternative formation, such as within a globular cluster, we conclude that at least some BHs get high kicks. This challenges the standard picture that BH kicks would be scaled down from neutron star kicks. Furthermore, we find that five systems could have formed with a non-zero NK but zero mass ejected (i.e. no SN) at formation, as predicted by neutrino-driven NKs.