More and more ships are coming to the economic end of life nowadays, it is a sustainable way to send end-of-life ships for recycling considering the material circulation by the ship owners. Due to the hazardous materials and dangerous working process, it is important for the ship recycling companies to ensure the workers' safety. Herein, the objective of this study is to find out the factors affecting personal safety during ship recycling activities, and to evaluate their interrelationships. By exploring the main steps, hazardous materials and potential risks during ship recycling, customized factors affecting workers’ safety were proposed. A questionnaire survey about the proposed factors was designed and conducted by scholars, managers and workers in ship recycling industry. Meanwhile, a theoretical model with hypotheses was developed and evaluated based on structural equation modeling by using the survey data. The findings showed that all the proposed factors, i.e., disposal of hazardous materials, dismantling operation safety, dismantling operation management, dismantling operation equipment, and safety awareness, directly or indirectly affect personal safety during ship dismantling, especially the dismantling operation management factor. The implications and limitations of this study were also discussed, which could be beneficial to ship recycling companies and policy makers.