Recently, official statistics reported that the Chinese population aged 60 and above has been 26.402 million, which accounts for 18.70% of total population. It is urgent to develop fall detection technologies for alleviating the risk causing by falling of elder person. In this paper, we propose a real-time, high-precision, and deep learning-based fall detection method with automatic person detection and tracking. Specifically, the proposed method first improves the YOLOv3 network to more efficiently detect person and extract feature maps of the object. Then, it inputs the extracted feature maps from the YOLOv3 into a multi-target tracking network for cascade matching and IOU matching in a Deep SORT algorithm, respectively. Next, it improves YOLOv5 network to detect posture anomalies. Finally, it refines the detected posture anomalies for obtaining the final fall detection result. Experimental results show that the proposed method simultaneously improves accuracy and efficiency of the fall detection.