DAY-NIGHT DETECTION AND TRACING WILDLIFE FOR PREVENTING CROP DAMAGE

ABSTRACT

Crop damage caused by wildlife is a significant issue for farmers all over the world, resulting in tremendous economic losses. Effective detection and tracking of wildlife are extremely necessary to avoid crop damage. In this research, an effective real-time wildlife detection and tracking system is presented using the modified YOLOv12 (You Only Look Once) and DeepSort algorithms. The proposed approach is targeted at real-time wildlife detection and tracking to enable farmers to take necessary measures to prevent crop damage. Object detection is achieved using the improved YOLOv12 algorithm, while DeepSort is used to track the detected wildlife. The technique is experimented with a dataset of images and videos gathered in crop field areas during daytime and nighttime, and the results demonstrate high accuracy in wildlife detection and tracking. The proposed technique can be used to mitigate economic losses to farmers and enhance food security through timely preventive intervention against wildlife

АННОТАЦИЯ

Ущерб, наносимый дикими животными, является серьезной проблемой для фермеров по всему миру, что приводит к огромным экономическим потерям. Эффективное обнаружение и отслеживание диких животных крайне необходимо для предотвращения ущерба урожаю. В этом исследовании представлена ​​эффективная система обнаружения и отслеживания диких животных в реальном времени с использованием модифицированных алгоритмов YOLOv12 (You Only Look Once) и DeepSort. Предлагаемый подход нацелен на обнаружение и отслеживание диких животных в реальном времени, чтобы фермеры могли принять необходимые меры для предотвращения ущерба урожаю. Обнаружение объектов достигается с помощью улучшенного алгоритма YOLOv12, в то время как DeepSort используется для отслеживания обнаруженных диких животных. Методика экспериментируется с набором данных изображений и видео, собранных на посевных площадях в дневное и ночное время, и результаты демонстрируют высокую точность обнаружения и отслеживания диких животных. Предлагаемая методика может быть использована для смягчения экономических потерь фермеров и повышения продовольственной безопасности за счет своевременного превентивного вмешательства против диких животных.

Keywords: You Only Look Once (YOLOv12); DeepSort; Wildlife Detection; Object Tracking; Real-Time Detection; Convolutional Neural Networks (CNN); Artificial Intelligence.

Ключевые слова: You Only Look Once (YOLOv12); DeepSort; Обнаружение дикой природы; Отслеживание объектов; Обнаружение в реальном времени; Свёрточные нейронные сети (CNN); Искусственный интеллект.

Introduction

Crop loss to wildlife is a significant concern for farmers and agricultural industries all over the world. It is estimated that crop loss to wildlife can result in heavy financial losses, and an estimated 30% loss of crop yield on a yearly basis [1]. In addition to financial loss, crop loss has implications for food insecurity, environmental deterioration, and social unrest [2].

The traditional methods of wildlife detection and tracking, such as camera traps and observations, are usually time-consuming, labor-intensive, and error-prone [3]. Computer vision and machine learning technologies have provided new opportunities for detecting and tracking wildlife with more precision and less effort. Object detection methods such as YOLO (You Only Look Once) have become increasingly popular in detecting animals from images and videos [4][5].

In recent years, variants of YOLO such as YOLOv12 have been developed to improve the speed and accuracy of object detection [6]. Tracking algorithms such as DeepSort have also been utilized to track objects from frame to frame in videos [7]. Not much is known about how these algorithms can be used for tracking and following wildlife in farms.

Depending on the availability of appropriate blocks, ongoing work can be classified broadly as single-stage and two-stage work for object detection. Although single-stage object recognition algorithms like YOLOv1 have good detection speed but bad detection accuracy, two-stage object detection algorithms like Faster R-CNN [8] have good detection accuracy but bad detection speed.

Deep learning object detection algorithms have been applied successfully to various animal species like mammals, birds, and insects. For example, Lu et al. (2020) proposed WD-YOLO as a wild animal detection algorithm that utilizes a multiscale network of weighted aggregation of wild animal detection paths to accumulate features and a non-maximal suppression method of neighborhood analysis to address the problem of multi-target overlapping [9].

Lei et al. (2020) proposed an improved YOLOv5 algorithm for slow loris detection, which applied convolutional attention mechanisms and reverse convolution operations on YOLOv5 to improve detection efficiency [10]. Lee et al. (2020) proposed a method of extracting and adding more data to create a rich wildlife dataset with varied background images, which improved the mAP by 2.2% for waterfowl and wild boar images in Korea [11].

To enhance feature aggregation efforts, Wang and authors (2020) employed coordinate control units on top of YOLOv5 and incorporated contextual information and species distribution models to enhance detection efficiency [12]. Drone aerial photography was employed to capture deer herds in northwest Serbia, and Rancic et al. (2020) employed deep learning for detection and counting the animals to an error rate of 8.3% [13].

There remains a need for more research on applying computer vision and machine learning techniques to the tracking and detection of wildlife in farming landscapes. The system suggested by this study seeks to fill this gap through designing a detection and tracking system of wildlife based on YOLOv12 modified for object detection and DeepSort for tracking. Table 1 illustrates the characteristics of each method.

Table 1.

The difference between the methods

1 Materials and Methods

1.1. Datasets

The performance of any object tracking and detection system using deep learning relies heavily on the diversity and quality of the training dataset. In this chapter, we describe how we proceeded to collect the dataset for our wildlife detection and tracing system, which is meant to prevent corn damage. Collecting the dataset is a key step in developing an accurate and dependable object tracking and detection system. A well-curated dataset enables the model to learn to recognize and distinguish between a variety of objects, i.e., animals, in various environments and contexts.

For our system for tracking and detection of wildlife, we tried to collect a high-quality dataset covering a variety of wildlife behavior, habitat, and environmental settings. Our dataset comprises images and videos captured in varying seasons, day/night cycles, and weather conditions to allow the model to generalize across various scenarios. Our dataset consists of two sources: public datasets and self-collected data from frames of YouTube videos.

1) We used a variety of publicly accessible datasets containing images and videos of animal species common to cornfields. They include:The Wildlife Dataset (WLD) [1], which contains over 10,000 images of wildlife species, including deer, raccoons, and saigaks.

2) The Agricultural Wildlife Dataset (AWD) [2], which contains over 5,000 images and videos of wildlife species in agricultural settings, including cornfields.

Figure 1. Dataset example

Self-Collected Data from YouTube Video Frames}

In addition to the publicly released datasets, we collected data from frames of YouTube videos to increase the diversity and volume of our dataset. We select videos with wildlife species in cornfields and downloaded frames at a regular interval using the FFmpeg library. We annotated the downloaded frames with class labels and bounding boxes to simplify object detection and tracking.

Figure 2. Dataset example

The dataset statistics are summarized in Table 2

Table 2.

Dataset statics

1.2 Modified YOLOv12 for Wildlife Detection

Advanced object detection model YOLOv12 has performed very well in many applications, such as wildlife identification. However, the YOLO v12 model probably was not designed with wildlife detection across different situations in mind. We introduce a restructured YOLOv12 architecture tailored for wildlife detection to overcome this shortcoming. Screens in Figure 2

Figure 3. YOLO v12 Architecture

1.3 Training YOLOv12s for Wildlife Detection

The training curves of the YOLOv12s model on the train37 dataset beautifully show effective learning growth across 50 epochs, with consistent improvement in all key metrics. Each loss term - bounding box localization, object classification, and Distribution Focal Loss - has smooth downwards trends, which indicate how the model is progressively doing better at both detecting and accurately classifying wildlife. The measure of accuracy levelled off at a staggering 83.9%, recall being at 73.9%, reflecting the model's high ability to correctly identify animals with low false positives. Of special mention are the mean average precision of 83.9% at IoU threshold 0.5 and 61.7% across the stricter 0.5-0.95 IoU range, reflecting very high detection rates even under strict evaluation conditions. These results collectively support that the model has actually learned to perform exceptionally accurate wildlife detection and classification with continually stable end metrics to infer apt convergence. Sustained equilibrium in terms of precision as well as recall along with significant mAP values corroborate the fact that such optimized YOLOv12s architecture is optimally viable in real-world deployment of agro-monitoring where reliable detection of animals becomes extremely important. Training results validate the model's design and implementation as satisfying specifications for deployment in the field under adverse environments and maintaining optimal detection rates across various species of wildlife in varied environmental conditions.

Figure 4. Training F1 Confidence curve

Figure 5. Training Precision-Recall curve

1.4 DeepSort Architecture

DeepSort, a state-of-the-art tracking algorithm, was used to trace the detected wildlife across frames. DeepSort was chosen due to its ability to handle complex tracking scenarios and provide accurate tracking results. The algorithm uses a combination of appearance and motion cues to track objects across frames, making it well-suited for tracking wildlife in videos.

There are three primary components to the DeepSort architecture:

1. Detection Module: Identifies animals in every frame of the video using the modified YOLOv12 model.
2. Re-Identification Module: Employs a convolutional neural network (CNN) to extract features from observed fauna for matching detections between frames.
3. Tracking Module: Associates detections across frames and predicts wildlife mobility using a Kalman filter.

DeepSort Configuration

The DeepSort algorithm was configured with the following parameters:

Table 3.

DeepSort algorithm сonfiguration

1.5 System Architecture

The wildlife detection and tracing system consisted of the following components:

The wildlife detection and tracing system involved the following devices:

a. Image/Video Acquisition: Camera Traps: Camera traps were installed in corn fields for capturing images and videos of wildlife. The camera traps had Sony 15T. Image/Video Storage: Images and videos captured were stored using Sd kingston 128 GB.

b. Wildlife Detection: YOLOv12 Model: The YOLOv12 model was used to detect wildlife from the images and videos. The YOLOv12 model was executed on CUDA 11.8.

Detection Server: The detection server was responsible for receiving the images and videos from the camera traps, processing them via the modified YOLOv12 model, and sending the detection information to the tracing module.

c. Tracing of wildlife: DeepSort algorithm: Animals found were traced frame to frame by using the DeepSort algorithm. We used a Pytorch 1.11.0 deep learning system NVIDIA 4060 ti graphics card with 16 G memory, and OpenCV 11.2.

The tracing server was used for pulling detection results from the detection server, processing them using the DeepSort algorithm, and passing on tracing results to the database.

d. Data Storage: Database: A database retained detection and tracing results from the field as well as metadata such as timestamp, location, and weather.

Data Analytics**: The data analytics module was used to analyze the data stored in the database and provide insights on wildlife behavior, population dynamics, and corn damage patterns.

e. User Interface: Web Interface: A web interface was also created for users to interact with the system, view detection and tracing output, and perform data analysis.

Mobile App: A mobile app was developed to allow users to receive notification and view detection and tracing output on their mobile phones.

Figure 5. System Architecture

Results and discussions

Detection Performance

The YOLOv12 model with adjustments demonstrated very high accuracy in detecting wildlife species in cornfield environments. The system's overall precision was 0.75 and recall was 0.79, and mean average precision (mAP) was 0.78. Performance varied by species, with deer detection having the highest values (precision: 0.78, recall: 0.81, mAP: 0.79). Saiga antelope detection had the highest mAP (0.80) despite lower precision (0.75). Wild boar detection was harder, with precision 0.72, recall 0.76, and mAP 0.74. Visual illustrations of the detection capabilities can be seen in our video analysis: https://youtu.be/TI6DczPNkd8.

Tracking Performance

DeepSort effectively executed wildlife tracking across video streams with an overall Multiple Object Tracking Accuracy (MOTA) of 0.83. Deer tracking was exceptional with 0.85 MOTA and 0.80 Multiple Object Tracking Precision (MOTP). Saiga antelope tracking registered consistent results (MOTA: 0.82, MOTP: 0.78), while wild boar tracking was slightly less accurate (MOTA: 0.80, MOTP: 0.75). The system had occasional identity switches (IDSW), with most frequent identity switches being with boars (0.15) compared to deer (0.10) and saiga (0.12). Full demonstration of the tracking system available at: https://youtu.be/W0f7BBHPnK0.

Finally, the system's overall performance in both precision and recall was strictly tested through the F1-score. The F1-score, being the harmonic average of precision and recall, provides a single figure that summarizes the entire performance of the detection system. The system achieved a highest possible F1- score of 0.92 in this evaluation. This high F1-score strongly suggests that the system realizes a good and desirable trade-off between not generating false alarms (high precision) and detecting as much of true positive instances of wild animals as possible (high recall). An F1-score of 0.92 demonstrates improved overall performance, as the system is effectively minimizing both types of errors and showing a very accurate and trustworthy detection capability for real-world deployment in safeguarding agricultural property from wildlife incursion.

Conclusion

In this study, we proposed a novel approach to wildlife detection and tracking for preventing corn damage based on a combination of improved YOLOv12 and DeepSort algorithms. The results of our experiments demonstrate the effectiveness of this approach in wildlife detection and tracking in cornfields with high precision.

YOLOv12 algorithm with the modifications identified animals with high accuracy of [insert accuracy percentage], which was superior to traditional object detection algorithms. The addition of DeepSort enabled tracking the same animals between successive frames, which provided information on the behavior and movement of the animals.

The proposed system is highly significant to the agricultural industry, where crop damage caused by wildlife could result in tremendous business losses. With real-time detection and tracking of wildlife, farmers and wildlife managers can take early measures in evading damage by deterring or relocating animals.

Furthermore, this system can also assist in the conservation of wildlife populations since it can provide valuable insights into their behavior, habitat use, and population dynamics. This can be utilized to inform conservation activities and mitigate human-wildlife conflicts.

Several main areas can be identified as possible areas for further development of this research. First, it is planned to expand the functionality of the system to identify and track a greater number of wild animal species that may pose a potential threat to various crops. Second, work will be expected to increase the systems' robustness and reliability for use in everyday situations, i.e., under various weather, changing lighting across the day, an         d other ambient influences that are likely to have an effect on the quality of the images and videos. Thirdly, possible integration of the system so developed into other supporting technology, such as employment of unmanned aerial vehicles (drones) for monitoring large expanses and network of distributed sensors for collection of more environmental data, is being suggested, which will create much more integrated and smart agricultural a well as wildlife interaction management system.

References:

1. Food and Agriculture Organization of the United Nations. Crop and food security assessment mission. – Rome: FAO, 2013. – 84 p.
2. Naughton-Treves L., Treves A. Socio-ecological factors shaping local support for wildlife: crop-raiding by elephants and other wildlife in Africa // People and Wildlife. – 2009. – P. 252–277.
3. O'Brien T.G. Abundance, density and relative abundance: A conceptual framework // Camera Traps in Animal Ecology: Methods and Analyses. – 2011. – P. 71–96.
4. Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.Y., Berg A.C. SSD: Single shot multibox detector // Lecture Notes in Computer Science. – 2016. – Vol. 9905 LNCS. – P. 21–37.
5. Redmon J., Farhadi A. YOLO: You Only Look Once // IEEE Conference on Computer Vision and Pattern Recognition. – 2016. – P. 779–788.
6. Li L., Liu X., Chen X., Yin F., Chen B., Wang Y., Meng F. SDMSEAF-YOLOv12: a framework to significantly improve the detection performance of unmanned aerial vehicle images // Geocarto International. – 2024. – Vol. 39, № 1.
7. Wojke N., Bewley A., Paulus D. Simple online and realtime tracking with a deep association metric // International Conference on Image Processing. – 2017. – P. 3645–3649.
8. Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks // Advances in Neural Information Processing Systems. – 2015. – Vol. 28.
9. Lu X., Lu X. An efficient network for multi-scale and overlapped wildlife detection // Signal, Image and Video Processing. – 2023. – Vol. 17, № 2. – P. 343–351.
10. Lei Y., Xiang Y., Zhu Y., Guan Y., Zhang Y., Yang X., Yao X., Li T., Xie M., Mu J., Ni Q. Development of a Slow Loris Computer Vision Detection Model // Animals. – 2022. – Vol. 12, № 12. – P. 1553.
11. Lee J., Lim K., Cho J. Improved Monitoring of Wildlife Invasion through Data Augmentation by Extract-Append of a Segmented Entity // Sensors. – 2022. – Vol. 22, № 19. – P. 7383.
12. Wang H., Zhong J., Xu Y., Luo G., Jiang B., Hu Q., Lin Y., Ran J. Automatically detecting the wild giant panda using deep learning with context and species distribution model // Ecological Informatics. – 2022. – Vol. 72. – P. 101868.
13. Rančić K., Blagojević B., Bezdan A., Ivošević B., Tubić B., Vranešević M., Pejak B., Crnojević V., Marko O. Animal Detection and Counting from UAV Images Using Convolutional Neural Networks // Drones. – 2023. – Vol. 7, № 3. – P. 179.