A mine image reconstruction method based on residual neural network
ZHANG Fan1,2 ,XU Zhichao1
针对煤矿井下噪声对可视化作业环境扰动影响,面向智能开采对井下作业目标的图像清晰化需求,研究基于机器视觉的矿井视频图像重构理论与方法,对提高矿井智能监控与安全开采技术具有重要意义。传统的视频监控系统采用经典的Nyquist采样定理来解决视频图像的信号采集、压缩和编解码问题,但矿井视频图像数据庞大,采用传统的编解码方法不仅浪费大量采样资源及增大系统开销,而且难以解决矿井视频图像重构时出现的信号保真度低、图像边缘模糊和视频传输时延等问题,其直接影响矿井智能监控系统性能与视频传输质量。针对矿井视频监控图像重构中存在的信号保真度不足及图像边缘模糊等问题,提出一种基于残差网络的图像压缩与重构方法。该方法通过建立一种新的残差神经网络结构,采用下采样矩阵将矿井图像进行压缩,再通过多次上采样将特征图变换为与原始图像相同大小的特征图,并使用残差网络块对其优化,最后利用优化后的重构网络将特征图聚合成重构图像。提出融合离散小波结构相似度损失与均方误差损失的损失函数方法,并据此训练网络参数。为评价本文所提出方法的有效性,实验选取了基于压缩感知的D-AMP,TVAL3算法和基于深度学习的ReconNet算法与之进行对比。结果表明,较小压缩比条件下对矿井图像重构,本文方法在结构相似度和峰值信噪比性能方面均优于其他算法;在噪声环境下,本文方法相较于其他方法,图像重构的峰值信噪比与结构相似度受噪声强度扰动较小,对噪声具有较强鲁棒性,较显著增强矿井重构图像的保真度和清晰度;在图像重构的时间复杂度方面,本文方法用时最短,有助于改善矿井视频监控系统的实时性。
In order to address the problems of the disturbance from environmental noise on visual operations in under- ground mines,and to satisfy the demands of intelligent mining that call for imaging clarity of the operating objects,it is of significance to investigate into the theories and the methods of the reconstruction of mine video images based on computer vision. This is of vital importance of the advancement in intelligent mine monitoring and safe mining technol- ogies. Conventional video monitoring system adopts classical Nyquist sampling theorem to address the points of video acquisition,compression,and encoding and decoding. However,the conventional compression methods would waste val- uable acquisition resources and increase the system overheads when dealing with data as large as videos. It is more difficult for conventional methods to address the problems of fidelity loss,edge blurring,and transmission latency in re- gards of the reconstruction of the image,which downgrades the performance of the mine surveillance system and the quality of video transmission. In response to these challenges,an image compression and reconstruction method based on residual network is proposed. The proposed method establishes a novel network featuring skipping connections. Down-sampling matrix is applied to compress the original mine image. Then multiple up-sampling layers are inserted to scale up the feature maps to the size of the original image. The feature maps are learnt by residual network blocks. Fi- nally the feature maps are aggregated to the reconstructed image by the optimized network. In addition,a new loss function called discrete wavelet structural similarity ( DW-SSIM) loss is proposed. The DW-SSIM loss and mean square error loss are added up together as the total loss when training the network. The experiments are carried out to validate the effectiveness of the proposed method. The proposed method is compared to the algorithms of the com- pressed sensing based D-AMP,TVAL3,and deep learning based ReconNet. The experiments show that the proposed method surpasses other algorithms at low compression ratios in regards of PSNR ( Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity). As for reconstruction at the present of noise,there are less fluctuations in PSNR and SSIM for the proposed method as the noise level varies. It is concluded that the proposed method features a strong noise robustness and is able to significantly improve the fidelity and clarity of the reconstructed images. Compared to other algorithms,the proposed method consumes the least time,which can improve the real time performance of the mine video surveillance systems.
mine intelligent surveillance;residual neural network;DW-SSIM;compressed sensing;image restoration
主办单位:煤炭科学研究总院有限公司 中国煤炭学会学术期刊工作委员会