College of Electronic and Information Engineering, Shandong University of Science and Technology
College of Computer Science and Engineering, Shandong University of Science and Technology

矿井作业人员因疲劳引发误操作是导致煤矿事故发生的重要原因。为解决采集的矿井作业图像质量低、疲劳特征单一以及个体差异等问题,本研究提出改进的RetinaFace-PFLD轻量级自适应面部疲劳检测算法(RP-LA)。具体地,使用中值滤波和伽马校正对实时视频数据进行预处理以提高图像质量;在RetinaFace模型的基础上改进MobileNetv3网络提取特征,简化特征金字塔网络,降低识别算法复杂度;通过PFLD框架获取人脸关键点和疲劳特征,利用自适应疲劳检测方法检测疲劳。在人脸数据集、自采集矿工数据集和驾驶数据集上进行测试,疲劳检测准确率达到97.73%。进一步将算法移植到JetsonNano上,每秒检测帧数为16.13,大于采样速度,表明本算法适用于移动终端设备进行实时监测预警。

Misoperation caused by fatigue of mine workers is an important reason for coal mine accidents. To address the problems of the low quality of collected mine operation images, the singleness of fatigue characteristics, and individual differences, this study proposed an improved RetinaFace-PFLD lightweight adaptive facial fatigue detection algorithm (RPLA). Specifically, median filtering and gamma correction were used to preprocess real-time video data to improve image quality. Based on the RetinaFace model, MobileNetv3 network extracted features were improved, and feature pyramid network was simplified to reduce the complexity of the recognition algorithm. Facial key points and fatigue characteristics were obtained through the practical facial landmark detector PFLD framework, and fatigue was detected by using an adaptive fatigue detection method. Tested on the face data set, self-collected miner data set and driving data set, the fatigue detection accuracy reached 97.73%. The algorithm is transplanted to Jetson Nano, and the frame rate per second is 16.13, faster than the sampling speed, which shows that this algorithm is suitable for real-time monitoring and early warning in mobile terminal devices.

科技部新一代人工智能国家科技重大专项(2022ZD0119501);国家自然科学基金项目(52374221);山东省自然科学基金项目(ZR2022MF288,ZR2023MF097);山东省泰山学者特聘专家支持项目(ts20190936)