The first robot which I used to search for land mines / metal in the ground was build 2015 by myself in my workshop. In 2023 I started with the training of neural nets to detect anti-personal land mines like the PFM-1. Then I paused the project and restarted it in April 2024 after beeing asked by the https://www.deminefoundation.com NGO to join the team to combine forces and efforts.
The threat posed by surface landmines remains significant in numerous parts of the world, necessitating efficient methods for detection. Recent advancements in robotics, object detection algorithms like YOLO (You Only Look Once), and imaging technologies present an opportunity to address this problem with real-time applicability. This blog post aims to explore how integrating FLIR camera modules and RGB cameras with the YOLO neural network can lead to effective PFM-1 anti-personal landmine detection by robotic systems.
danger mines
Robot Configuration: The Idea in General
In my different “Demining Robot” projects, I have designed different robots equipped with advanced sensors for various roles related to landmine detection and demining. Once knowing the best solution a fleet of robots needs to be build to test the idea. One such robot idea I am currently working on is the so called “Long leg robot”, which plays a crucial role in detecting surface landmines by leveraging an array of sophisticated sensors such as thermal modules, RGB camera modules, accelerometers and high-precision GPS with remarkable 5 cm positional accuracy. The deployed software inside the robot will not only detect surface mines like the PFM-1 mines but be able to detect trip-wire systems to handle specific landmine types like “starfish,” or EODs which are connected to nearly invisible tripwires that can trigger detonation and harm the deminer and robot.
The robot will be build out of cheap automotive parts which are available around the globe.
long leg demining robot
Image Sensing: FLIR Camera Module and RGB Camera
For real-time detection of surface landmines, the Long leg robot will be equipped with both a simple RGB camera module array and an infrared (FLIR) camera module array. The integration of these two imaging technologies provides the system with comprehensive visual data crucial for efficient object detection. With the usage of OpenCV those images will be overlayed and stiched for the analysis. While RGB cameras capture high-resolution color images, FLIR cameras specialize in detecting temperature differences within a scene—a feature particularly useful when identifying landmines that may blend into their environment or be difficult to spot under certain conditions. Just think of the growing vegitation which will grow over the land mines in a few days depending on the season.
Why a robot and not a drone: Currently depending on the sensors and energy consumtion a land base solution maybe the best idea. But the fast developing of drones during the last two years for military usage drones maybe be available in high quantities with a lot of sensros already build in the drones. Maybe from the cost perspective a robot moving on the ground is the best solution.
The Role of YOLO Neural Network for Landmine Detection
To process the images captured by the FLIR and RGB camera modules, I am utilizing a neural network like YOLO (You Only Look Once), renowned for its accuracy and speed in object detection. The strengths of the YOLO algorithm make it an ideal choice for real-time applications as it can identify objects within milliseconds by dividing the image into a grid and predicting bounding boxes and class probabilities simultaneously. The YOLO frameworks is available for many platforms and a lot of best praticies and projects are avialble in the internet.
YOLO Training:
Currently the YOLO V8.2 M model was trained with approxemately 230.000 synthetic generated pictures of PFM-1 anti-personal land mines. Each picture is showing 10 PFM-1 anti-personal landmines. The training took round about 35 hours on my single NVIDIA A6000 GPU with 30 EPOCHs and a BATCH-SIZE of 16.
Real Time Landmine Detection
Integrating FLIR Camera Module, RGB Camera, and YOLO Neural Network: The Approach
The integration of our chosen imaging technologies (FLIR camera module and RGB camera) with the YOLO neural network provides a multi-faceted approach to PFM-1 anti-personal landmine detection. Here’s how this integrated system can work in real-time:
Step 1: Data Acquisition
The FLIR camera captures thermal images of the ground, highlighting temperature differences caused by buried objects like landmines. Concurrently, the RGB camera acquires high-resolution color imagery for better visual context and clarity. This combination provides a more comprehensive understanding of the robot’s surroundings. Currently the idea is that the processing of the images will be done remote in near realtime to protect the expensive NVIDIA hardware like a NVIDIA JETSON AGX ORIN DEVELOPMENT KIT 64GB for getting blown up together with the robot. But a decision has to be made which approch is the best.
Step 2: Preprocessing and Sensor Fusion
The captured thermal and color images are preprocessed to enhance image quality, remove noise, and standardize dimensions before being fed into our system. The two datasets from FLIR camera module and RGB camera are then fused, creating an enriched dataset that combines the strengths of both imaging technologies. With the help of OpenCV this should work straight foreward.
Step 3: YOLO Neural Network Implementation
The integrated thermal-visual data is processed using a neural network like YOLO. By training this algorithm on landmine datasets (specifically PFM-1 anti-personal landmines), it learns to identify and locate these dangerous objects with high accuracy in real time. The speed of the YOLO algorithm allows for swift responses, enabling timely reactions to potential threats by deactivating or marking detected landmines for further investigation. The detection also helps the demining personal to understand the dimension of a mine filed even if not all mines are detected. From a statistic perspective and the knowlodge how those mines ware layed out a better understanding of the ground situtaion is made with the help of the automatic detection of landmines.
Example of an early test setup
The video shows a Jetson Nano from 2019 live inferencing with the YOLO v8.2m model which is too heavy for the small Jetson Nano but for testing and performance optimization a good solution.
Conclusion: A Promising Approach Towards Real-Time Landmine Detection
By integrating FLIR camera modules and RGB cameras with the YOLO neural network, we can address the persistent problem of surface landmine detection in real time. The strengths of these imaging technologies combined with object detection algorithms like YOLO present a compelling approach to solving this global issue. Working with ground penetrating radar in addtion to the RGB and FLIR camera modules whould also be an option like geologits are unsing it. Our “Demining Robots” project is one such initiative aimed at harnessing the power of robotics, advanced sensors, and machine learning techniques to protect lives and restore safety in mine-infested areas worldwide.
