A Simple Augmentation Method Using Cutout for Ground Penetrating Radar Image in Deep Learning

Jun SONODA, Kazusa NAKAMICHI

Summary :

Ground penetrating radar (GPR) has the advantage of non-destructively and quickly inspecting internal structures such as voids and buried pipes under roads. However, it is necessary to estimate the internal structures from the GPR images. Recently, recognition and detection methods for GPR images using deep learning have been studied. This paper examines a data augmentation method using a cutout method necessary to estimate GPR images with deep learning accurately. We find that the cutout augmentation exhibits higher detection rates for all objects used in this study than a commonly used horizontal shift augmentation.

• Publication: IEICE TRANSACTIONS on Electronics Vol.E107-C No.11 pp.497-500
• Publication Date: 2024/11/01
• Publicized: 2024/04/26
• Online ISSN: 1745-1353
• DOI: 10.1587/transele.2023ESS0005
• Type of Manuscript: BRIEF PAPER

1.  Introduction

Ground Penetrating Radar (GPR) is a non-destructive and fast measurement method of internal structures by radio waves in several hundred MHz bands [1]. Currently, it has been widely used to detect such as voids and buried pipes under roads. Since GPR images obtained by radar surveys cannot directly observe objects’ material, size, and location, we need to estimate inner structures from GPR images. Nowadays, skilled engineers determine internal conditions by image reading, but automatic estimation is necessary due to significant variations and lack of precision.

In recent years, many researches have been conducted using convolutional neural network (CNN) and other deep learning techniques to estimate structures and objects from GPR images automatically [2], [10]. Deep learning can accurately and automatically estimate objects in images, but it is necessary to prepare a large number of images for training. In the case of GPR, it is challenging to prepare many training images for deep learning. For this problem, augmentation methods such as rotation, scaling, and contrast adjustment are commonly used in images of natural objects such as animals and plants. Note that we cannot apply rotation, vertical shift, or random brightness often used augmentation methods in natural images because amplitude and pattern of reflected waves from inner objects are observed according to material, size, and depth in GPR images. Several augmentation methods using finite-difference time-domain (FDTD) method [10], [11] and generative adversarial network (GAN) based style transformation method [12], [13] have been proposed for GPR images. Because of the high computational cost of FDTD and GAN methods, we require uncomplicated and inexpensive augmentation methods for GPR images.

We investigate simple and effective augmentation for GPR images using a cutout method [14]. In this paper, GPR images of four types of buried objects by a sandbox experiment are augmented by the cutout method. We further evaluate detection rates using YOLOv3 [16] and compare them to a commonly used horizontal shift method to demonstrate the effectiveness of the cutout method.

2.  Augmentation Method Using Cutout for GPR Images in Deep Learning

This study examines simple and effective augmentation methods in deep learning that can apply to GPR images. Deep learning, such as CNN, requires a large number of labeled images for training to achieve high accuracy. For example, the standard CIFAR-10 dataset consists of 60000 images with 10 classes of objects. In the case of GPR, it is not easy to experimentally acquire many GPR images labeled with different materials, sizes, and depths. Data augmentation is generally used to expand initially obtained data sets if many training images cannot be prepared for deep learning.

In this study, GPR images of four types of buried objects were augmented by experiments in a sandbox shown in Fig. 1. Figure 1 (a) presents buried objects with styrene foam, concrete, wood, and aluminum. GPR was used for the GSSI SIR-EZ 2600 MHz shown in Fig. 1 (b). Relative permittivities of sand, styrene foam, wood, and concrete by a time domain reflectometry (TDR) were 4.5, 1.1, 2.4, and 3.3, respectively. The buried objects were made in various sizes by combining rectangular blocks of approximately 5 cm per side shown in Fig. 1 (c). In the experiment, the sizes of the buried objects were set to 5 cm \(\times\) 5 cm \(\times\) 5 cm, 5 cm \(\times\) 5 cm \(\times\) 10 cm, 10 cm \(\times\) 10 cm \(\times\) 5 cm, and 10 cm \(\times\) 10 cm \(\times\) 10 cm. Here, the size of the object was written in \(x\) cm \(\times\) \(y\) cm \(\times\) \(z\) cm with \(x\) and \(y\) in the horizontal direction and \(z\) in the depth direction. Also, the objects were buried at depths of 5, 10, 15, and 20 cm. Thus, 64 labeled GPR images were obtained according to the materials, sizes, and depths.

Fig. 1  Experiments of GPR image generation for learning in a sandbox.

The total 64 GPR images obtained in the sandbox experiments were augmented using the cutout method. The cutout augmentation is as simple as randomly placed rectangular boxes in images. Figure 2 demonstrates augmentations of the GPR images using the horizontal shift and the cutout method. Here, the size of the GPR images shown in Fig. 2 is 2132 \(\times\) 1240 pixels. The horizontal shift shown in Fig. 2 (a) represents the reflected wave from the buried object moved to the left and right. This method moves the image horizontally within a given frame. Moving the image horizontally allows more images to be obtained, thus reducing overfitting and improving the robustness of the model. Horizontal translation is particularly beneficial in object recognition and classification tasks, where the position of objects in the image is not fixed, and they may appear in different locations. The cutout method in Fig. 2 (b) illustrates a box with 50 \(\times\) 50 pixels randomly placed around the reflected wave from the buried object in the GPR images. This method masks square regions at random locations to avoid excessive reliance on one part of the image, allowing the system to cope with cases where certain features are not visible. In the case of GPR, the area around the hyperbola is masked to identify underground objects from the hyperbolic reflection image in the GPR images. A similar method is the Random Erase method [15], which requires changing the size and color of the mask region. Consequently, the 64 GPR images generated in the sandbox experiments were augmented to 192 images by each method.

Fig. 2  Examples of augmentation for GPR images.

You only look once (YOLO) was used to detect the buried object in the GPR images in this study. It is a widely used real-time object detection algorithm based on CNN. In YOLO training, objects are enclosed by bounding boxes and labeled with position coordinates and types of objects in images. Similarly, in the case of GPR images, reflected waves from buried objects are enclosed by bounding boxes and labeled for each type of object. After training, YOLO can detect the types and positions of buried objects in GPR images using bounding boxes.

3.  Results and Discussions

Detection rates of the GPR images of the four different objects with the cutout augmentation were evaluated using YOLOv3, which was used as YOLO implementation. The detection rate was defined as whether the bounding box could detect the material and position of buried objects. Of all the 192 images generated by the sandbox experiments and the cutout augmentation, 160 were used for training. The remaining 32 and 16 additional images were used for validation for 48 images.

Table 1 shows confusion matrices of the detection rates using the horizontal shift and the cutout compared to without augmentation. The average values of 10 cross-validations are shown in Table 1. Table 1 (a) indicates that the detection rates were high in the styrene foam and the aluminum and were low in the concrete and the wood due to differences in electric permittivity. The concrete with relative permittivity of 3.3, which had a slight dielectric constant difference between the sand, the detection rate was as low as approximately 40%, and the non-detection rate was high at approximately 50%. Similarly, for the wood with relative permittivity of 2.4, the detection rate was low, approximately 40%, and the false positive rates for the concrete and styrene foam were high, approximately 10% and 30%, respectively. On the other hand, for the styrene foam with relative permittivity of 1.1, which had a significant difference in dielectric constant, the detection rate was as high as approximately 80%, and the non-detection and false detection rates were less than approximately 5%. For the concrete augmented by the horizontal shift and the cutout, the detection rate doubly improved to approximately 70%, and the non-detection rate halved to approximately 25% shown in Tables 1 (b) and (c). As in the concrete case, the detection and non-detection rates were improved for the wood. Table 1 (d) compares various methods for the detection rates of four types of buried objects. The detection rates were 63.2%, 82.9%, and 86.5% without augmentation, with the horizontal shift and the cutout, respectively. Furthermore, the detection rates with the cutout were higher than the horizontal shift for all objects. The reason for the high accuracy of the cutout method is that it is a method that can handle cases where the entire reflection image is not observed due to clutter caused by underground inhomogeneity. We found that the cutout augmentation for GPR images improved the detection rates. This paper indicated that the cutout only requires the placement of a box, which could easily augment and improve the detection rates. However, since the cutout method is not highly accurate, it must be combined with other data expansion methods.

Table 1  Confusion matrices of the detection rates with the augmentations.

4.  Conclusions

This paper investigated the augmentation method by the cutout for GPR images in deep learning. We evaluated the detection rates using YOLOv3 for the GPR images of the four types of buried objects. As a result, it is shown that the average detection by the cutout method was 3.6% higher than the horizontal shift. We found that the cutout augmentation for GPR images was a simple and effective method.

In the future, we will study the detection rates depending on the size and number of boxes used in the cutout method. Moreover, we will apply it to real-time detection on sites such as voids and pipes under roads.

Acknowledgments

This work was supported by the JSPS KAKENHI Grant Number 23H01650 and the research grant from the JKA Foundation.

References

[1] D.J. Daniels, Surface-Penetrating Radar, Inst of Engineering & Technology, London, 1996.

[2] K. Dinh, N. Gucunski, and T.H. Duong, “An algorithm for automatic localization and detection of rebars from GPR data of concrete bridge decks,” Automation in Construction, vol.89, pp.292-298, 2018.

[3] T. Yamaguchi, T. Mizutani, and T. Nagayama, “Mapping Subsurface Utility Pipes by 3-D Convolutional Neural Network and Kirchhoff Migration Using GPR Images,” IEEE Trans. Geosci. and Remote Sens., vol.59, no.8, pp.6525-6536, Aug. 2021.

[4] J. Wang, H. Liu, P. Jiang, Z. Wang, Q. Sui, and F. Zhang, “GPRI2Net: A Deep-Neural-Network-Based Ground Penetrating Radar Data Inversion and Object Identification Framework for Consecutive and Long Survey Lines,” IEEE Trans. Geosci. and Remote Sens., vol.60, pp.1-20, 2022.

[5] B.P.A. Rohman, M. Nishimoto, and K. Ogata, “Reconstruction of Missing Ground-Penetrating Radar Traces Using Simplified U-Net,” IEEE Geosci. and Remote Sens. Lett., vol.19, pp.1-5, 2022.

[6] J. Wang, K. Chen, H. Liu, J. Zhang, W. Kang, S. Li, P. Jiang, Q. Sui, and Z. Wang, “Deep Learning-Based Rebar Clutters Removal and Defect Echoes Enhancement in GPR Images,” IEEE Access, vol.9, pp.87207-87218, 2021.

[7] Q. Dai, Y.H. Lee, H.-H. Sun, G. Ow, M.L.M. Yusof, and A.C. Yucel, “3DInvNet: A Deep Learning-Based 3D Ground-Penetrating Radar Data Inversion,” IEEE Trans. Geosci. and Remote Sens., vol.61, pp.1-16, 2023.

[8] H. Hu, H. Fang, N. Wang, H. Liu, J. Lei, D. Ma, and J. Dong, “A Study of Automatic Recognition and Localization of Pipeline for Ground Penetrating Radar Based on Deep Learning,” IEEE Geosci. and Remote Sens. Lett., vol.19, pp.1-5, 2022.

[9] T. Yamaguchi, T. Mizutani, K. Meguro, and T. Hirano, “Detecting Subsurface Voids From GPR Images by 3-D Convolutional Neural Network Using 2-D Finite Difference Time Domain Method,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol.15, pp.3061-3073, 2022.

[10] J. Sonoda and T. Kimoto, “Object identification form GPR images by deep learning,” Proc. Asia-Pac. Microw. Conf., pp.1298-1300, Kyoto, Japan, Nov. 2018.

[11] S. Li, X. Cui, L. Guo, L. Zhang, X. Chen, and X. Cao, “Enhanced Automatic Root Recognition and Localization in GPR Images Through a YOLOv4-Based Deep Learning Approach,” IEEE Trans. Geosci. and Remote Sens., vol.60, 2022.

[12] A. Fazeel, J. Rottmayer, R. Mehta, and N. Bajcinca, “GPR-GANs: Generation of Synthetic Ground Penetrating Radargrams Using Generative Adversarial Networks,” 2021 11th International Workshop on Advanced Ground Penetrating Radar, Valletta, Malta, Dec. 2021.

[13] G. Chen, X. Bai, G. Wang, L. Wang, X. Luo, M. Ji, P. Feng, and Y. Zhang, “Subsurface Voids Detection from Limited Ground Penetrating Radar Data Using Generative Adversarial Network and YOLOV5,” 2021 IEEE International Geoscience and Remote Sensing Symposium, Brussels, Belgium, July 2021.

[14] T. DeVries and G.W. Taylor, “Improved Regularization of Convolutional Neural Networks with Cutout,” arXiv preprint arXiv: 1708. 04552, 2017.

[15] Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random Erasing Data Augmentation,” arXiv preprint arXiv:1708.04896, 2017.

[16] J. Redmon and A. Farhadi, “YOLOv3: An Incremental Improvement,” arXiv preprint arXiv:1804.02767, 2018.

Authors

Jun SONODA
  Sendai College
Kazusa NAKAMICHI
  Sendai College

Keyword

ground penetrating radar,  deep learning,  data augmentation,  cutout