The concept of smart factory is being applied into traditional manufacturing system. Since factory automation is applied gradationally, Human-robot collaboration (HRC) system is becoming an important issue. In order to construct an effective HRC system, clear communication with the human workers and the robots has to be considered. This research proposed a conceptual framework of process model based HRC system for efficient human-robot collaboration in a semi automation process to produce electric motors. We applied a process modeling methodology for capturing collaborative features, activity and resource flow in the manufacturing process. The model defined by the proposed methodology is the data storage to contain process information and interface between the human workers and robots to provide accurate information in the appropriate context. Furthermore, machine vision technology is implemented to recognize specifications of work in process (WIP) parts. The recognized parts are mapped with the correct work order and work instruction manual defined by the proposed model. In order to reduce human worker’s errors, the extracted work information is transmitted to the worker through the augmented reality device. The proposed HRC system is expected to be able to support the construction of a semi automation system that can reduce errors of human workers and ensure production flexibility.