The integration of robotic technologies into industrial production has transformed manufacturing processes, but traditional HRI systems often involve complex interfaces and require specialized training. Our project aims to develop a more intuitive and efficient HRI system, aligning with Industry 5.0 principles, to simplify interactions, reduce cognitive load, and enhance productivity.The novelty of our system lies in its combination of advanced technologies: Mixed Reality (MR) with HoloLens 2, digital twin systems, computer vision algorithms (OVE6D-Pose), and robotic arm path planning (Inverse Kinematics). The user interface, central controller, object detection, and path planning subsystems were designed and integrated to provide real-time feedback and adaptability in dynamic environments.Our design problem focused on creating a user-friendly, efficient, and adaptable HRI system. After evaluating several concepts, the chosen solutions were MR-based control using HoloLens 2, Unity for digital twin creation, OVE6D-Pose for object detection, and Inverse Kinematics for path planning. These choices were based on their balance of efficiency, accuracy, robustness, simplicity, and cost.The detailed design includes a user-friendly interface developed with HoloLens 2 and Unity, robust object detection using OVE6D-Pose, and precise robotic arm path planning through Inverse Kinematics. We employed Unity's Mixed Reality Toolkit (MRTK) to build the virtual environment, integrated a digital twin system for real-time simulation, and utilized the OVE6D-Pose algorithm for accurate object detection and pose estimation. Validation was conducted through rigorous testing of the system's accuracy, efficiency, and response times in real-world manufacturing scenarios.In conclusion, this project successfully developed a novel MR-based HRI system that enhances user experience and operational efficiency in smart manufacturing. By leveraging cutting-edge technologies, our system addresses the limitations of traditional HRI methods, setting a new standard for human-robot collaboration in industrial automation.
