The robotic assembly of screws is the basic task for the automation assembly of complex equipment. However, a complete robotic assembly framework is difficult to be designed due to the integration of multiple technologies to achieve efficient and stable operations. In this paper, a robotic assembly workflow is proposed, which mainly consists of a feature extraction stage, a grasping stage, and an installation stage. In the feature extraction stage, a feature extraction algorithm consisting of a semantic segmentation network and an object classification module is designed. The semantic segmentation network segments the areas of multiple categories’ objects, and the object classification module selects an appropriate target object. The grasping stage and installation stage involve the position alignment of the objects. A position alignment method is developed based on image-based visual servoing using the point features extracted from the segmented areas. The experiments are conducted on a real robot. The alignment errors in grasping stage are less 0.53 mm. The assemblies for a M6-sized screw in ten experiments are successful. The experiment results verify the effectiveness of the proposed method.