为突破综采工作面数据驱动实时监控技术瓶颈，进一步满足对综采工作面协同运行状态的深度感知，以虚拟现实(Virtual Reality, VR)监控技术及成果为基础，以HoloLens2增强现实(Augmented Reality, AR)头显为主要开发设备，结合工业互联网、云技术、数字孪生(Digital Twin，DT)等先进手段，进行AR/VR融合驱动的综采工作面智能监控关键技术研究与实践，充分利用AR技术的特性弥补VR监控与真实场景有割裂感、监测数据来源较为单一的缺陷，使综采工作面智能监控体系得到完善与扩充。提出三种关键技术，并对各关键技术的具体解决方案进行剖析，具体包括：(1)构建了通用高效可扩展端云协同异步网络框架，基于该框架研究了多终端协同工作关键技术，实现了空间虚实融合全息辅助标注功能。该技术可为综采工作面智能监控全过程中的多人协作提供便利；(2)将基于稀疏匹配的识别跟踪方法与基于AR标识的识别跟踪方法相结合，实现了对综采工作面全综采装备的识别跟踪，为综采工作面的AR监测与控制提供了先决条件；(3)在由物理综采工作面、云服务器、数字综采工作面三者构成的数字孪生闭环上，进行多传感器与视觉融合的综采装备位姿监测与运行状态识别，并通过AR交互手段完成了对物理综采工作面的反向控制。在实验室环境下展开测试验证，试验表明，系统数据传输压力小，延迟低，操作流畅无卡顿，在保留VR监控已有优势的基础上进一步提升了沉浸感、便捷性与虚实融合程度，具备单人或多人协作监测综采工作面运行状态并进行反向控制的能力。可为综采工作面智能监控提供一种以VR技术为基础、AR技术为突破的AR/VR技术融合新思路。

In order to break through the bottleneck of data driven real time monitoring and control technology and satisfy the in depth perception of the collaborative operation for a fully mechanized coal mining face，a Virtual Reality (VR) monitoring and control technology and its research achievements are taken as the technical foundation. Also，the HoloLens2 Augmented Reality (AR) headset is taken as a main development device. Advanced methods such as industrial Internet，cloud technology，and Digital Twin (DT) are combined to carry out the research and practice of intelligent monitoring and control technology for a fully mechanized coal mining face driven by AR/VR. The characteristics of AR technology are fully utilized to make up for the sense of separation between VR monitoring and control and real scenes，and to compensate for the shortcomings of a relatively single source of monitoring data，so that the intelligent monitoring and control system for fully mechanized coal mining faces can be improved and expanded. Three key technologies are proposed and the specific solutions of each technology are analyzed. The details are as follows:① A universal，efficient，scalable，and asynchronous network framework for client cloud collaboration is constructed. Based on this framework，the key technologies of multi terminal collaborative work are studied，and the function of spatial holographic mark of the fusion of virtual and reality is realized. This technology can provide convenience for multi person collaboration in the whole process of intelligent monitoring and control of fully mechanized coal mining face. ② The recognition and tracking method based on sparse matching is combined with the recognition and tracking method based on AR identification，so that the recognition and tracking of fully mechanized coal mining equipment in fully mechanized coal mining face is realized，which provides a prerequisite for the AR monitoring and control of fully mechanized coal mining face. ③ In the DT closed loop composed of physical fully mechanized coal mining face，cloud server and digital fully mechanized coal mining face，the position and posture monitoring and the operating status recognition of fully mechanized coal mining equipment are carried out by means of multi sensor and vision fusion. The reverse control of physical fully mechanized coal mining face is completed by means of AR interaction. Test verification is carried out in a laboratory environment. The test shows that the system has low data transmission pressure，low delay，smooth operation and no jamming. On the basis of retaining the existing advantages of VR monitoring and control，the degree of immersion，convenience and the integration of virtual and reality are further improved. It has the ability to monitor the operating status of fully mechanized coal mining equipment by one or more persons and control it in reverse.

1 总体架构

   1.1 综采工作面AR/VR监控技术融合原理框架

   1.2 AR/VR融合驱动的综采工作面智能监控技术架构

2 综采工作面多监控终端协同工作技术

   2.1 通用高效可扩展端云协同异步网络框架

   2.2 跨终端空间虚实融合全息辅助标注

   2.3 多终端AR同地无缝协作

   2.4 多终端远程AR视频会议

3 基于计算机视觉的综采工作面AR识别跟踪技术

   3.1 基于稀疏匹配的综采装备识别跟踪

   3.2 基于AR标识的液压支架群识别跟踪

4 数字孪生综采工作面AR/VR融合闭环监控技术

   4.1 视觉与多传感器融合的数字孪生综采工作面虚拟监测

   4.2 数字孪生综采工作面AR反向控制

5 试验验证

   5.1 同地AR无缝协作与远程AR视频会议功能测试

   5.2 综采工作面AR识别跟踪与虚实映射智能监控测试

6 应用分析

   6.1 应用现状

   6.2 应用展望

7 结论