Research on coal mine XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots
WANG Xuewen;LIU Shuguang;WANG Xuesong;XIE Jiacheng;LIU Jingquan;WANG Xilong
随着煤矿智能化的发展与煤矿机器人的研发应用,煤矿操作员与煤矿机器人之间的高效协作对于井下复杂任务起到至关重要的作用。为优化多工种煤矿操作员与多机器人协作的复杂运行关系,基于数字孪生理念与在XR领域的长期实践,开展面向多人−多机复杂协作任务的煤矿XR智能运维系统设计与关键技术研究。首先,针对复杂任务中2类煤矿操作员(包括集控操作员与就地操作员)与2类煤矿机器人(包括探测机器人与作业机器人)协作的典型场景,设计了系统总体架构,将系统划分为物理子系统、VR运维子系统与AR运维子系统3部分,并对各部分的内容、功能以及3部分之间的协同运行关系进行介绍;然后,对系统涉及的VR运维子系统构建、AR运维子系统构建以及通讯网络构建等关键技术进行剖析,对各关键技术对应的解决方案进行了探讨,并实现了2类煤矿操作员、2类煤矿机器人与VR/AR运维子系统的集成运行;最后,在实验室环境下模拟井下复杂环境搭建了试验场地,在试验场地中设定了任务点与具体任务,对系统的可行性与有效性进行测试验证。试验结果表明,煤矿XR智能运维系统能够在不同复杂任务对应的多人−多机协作情形中发挥作用。通过VR运维子系统与AR运维子系统的协同运行,可实现虚拟空间和物理空间的协同感知、决策与控制,能够在虚拟空间中对物理空间的复杂任务进行迭代、优化和验算,形成了人−人、人−机、机−机交互协作的智能运行模式。
With the development of coal mine intelligence and the application of coal mine robots, an efficient collaboration between coal mine operators and coal mine robots plays a crucial role in the execution of complex underground tasks. To optimize the complex operational relationship of multiple coal mine operators and multiple robots, based on the concept of digital twin and extensive experience in the XR field, the research is conducted on the design and key technologies of XR intelligent operation and maintenance system for complex collaborative tasks involving multiple humans and multiple robots in coal mines. Firstly, for a typical scenario of collaboration between two types of coal mine operators (i.e central control operators and field control operators) and two types of coal mine robots (i.e. detection robots and operating robots) in complex tasks, the overall system architecture is designed. The system is divided into three parts: the physical subsystem, VR operation and maintenance subsystem, and AR operation and maintenance subsystem. The content, functions, and collaborative operation relationships among these three parts are introduced. Then, an analysis of key technologies related to the VR operation and maintenance subsystem, AR operation and maintenance subsystem, and communication networking is carried out. The solutions corresponding to each key technology are discussed, and the integration and operation of the two types of coal mine operators, two types of coal mine robots, and VR/AR operation and maintenance subsystem are implemented. Finally, a laboratory environment simulating complex underground conditions is set up to create a test site, where the task points and specific tasks are defined. The feasibility and effectiveness of the system are tested and verified. The experimental results show that the coal mine XR intelligent operation and maintenance system is able to function in collaborative situations between multiple humans and multiple robots corresponding to different complex tasks. Through the collaborative operation of the VR operation and maintenance subsystem and the AR operation and maintenance subsystem, the collaborative perception, decision-making, and control between virtual space and physical space can be achieved. This allows for the iterative optimization and verification of complex tasks in a physical space from a virtual space, forming an intelligent operational mode of human-human, human-robot, and robot-robot interactive collaboration.
coal mine intelligence;coal mine robots;coal mine operators;human-robot collaboration;XR intelligent operation and maintenance;digital twin
主办单位:煤炭科学研究总院有限公司 中国煤炭学会学术期刊工作委员会