This manuscript introduces a cutting depth and angle control strategy based on vibration signal to improve robotic milling performance on easily displaced curved bone. The influence of the cutting depth and angle on the vibration signal of the cutter handle is discussed by analyzing its dynamic model. The handle vibrations are acquired via a three-axis accelerometer and processed by the fast Fourier transform (FFT) to extract their first harmonic amplitudes during the robotic bone milling process. Before milling the curved bone, a cutting depth and angle estimation model is first to fit based on the first harmonic amplitudes via calibration experiments on artificial bone plates. In consideration of the model estimation accuracy and its influence on actual milling performance, the robotic actions on adjusting the cutting depth and angle are controlled with different frequencies. Experimental results on femur models with curved surfaces demonstrate that the proposed control strategy has a better milling performance (0.48 ± 0.09 mm) than the cutting depth control without angle adjustment (0.44 ± 0.13 mm). The proposed method can improve the cutting safety for robot-assisted curved bone milling.