DEVELOPMENT AND IMPLEMENTATION OF AN AUTONOMOUS ROBOTIC FOR TOMATO MONITORING SYSTEM

ABSTRACT

Monitoring tomato crops requires continuous inspection and timely detection of plant conditions, while manual methods remain inefficient. This paper presents the design and simulation of an autonomous robotic system for tomato fruit and leaf health monitoring based on the YOLOv8 object detection model. The system integrates computer vision and robotic navigation to detect and classify tomato conditions, including ripeness stages and leaf diseases. The model was trained on a dataset of approximately 2,500 images with over 15,000 annotated objects collected from publicly available Roboflow sources and a custom dataset.

The robotic navigation module was evaluated in a simulated MATLAB environment. The system achieved a precision of 0.90, recall of 0.85, mAP50 of 0.912, and mAP50-95 of 0.789. The robot successfully navigated between targets and avoided obstacles with a success rate close to 98%. The results demonstrate the effectiveness of the proposed system for automated tomato monitoring.

АННОТАЦИЯ

Мониторинг посевов томатов требует постоянного контроля и своевременного определения состояния растений, в то время как ручные методы остаются неэффективными. В данной статье представлены разработка и моделирование автономной роботизированной системы для мониторинга состояния плодов и листьев томатов на основе модели обнаружения объектов YOLOv8. Система объединяет компьютерное зрение и роботизированную навигацию для определения и классификации состояния томатов, включая стадии созревания и заболевания листьев. Модель была обучена на наборе данных, состоящем примерно из 2500 изображений с более чем 15 000 аннотированных объектов, собранных из общедоступных источников Roboflow и пользовательского набора данных. Модуль роботизированной навигации был протестирован в смоделированной среде MATLAB. Точность системы составила 0,90, повторный вызов - 0,85, mAP50 - 0,912 и mAP50-95–0,789. Робот успешно перемещался между объектами и обходил препятствия с вероятностью успеха, близкой к 98%. Результаты демонстрируют эффективность предложенной системы автоматизированного мониторинга томатов.

Keywords: YOLOv8, object detection, precision agriculture, autonomous robot, tomato monitoring, computer vision.

Ключевые слова: YOLOv8, обнаружение объектов, точное земледелие, автономный робот, мониторинг помидоров, компьютерное зрение.

Introduction

In recent years, there have been significant developments related to the use of intelligent systems in the field of agriculture. The main factors driving these developments include increasing productivity and reducing the need for labor-intensive operations [4,6].

The cultivation of tomatoes requires constant monitoring in order to analyze the state of the plant, identify possible diseases, and determine whether the fruits are ripe. The process of monitoring involves a lot of labor costs and, moreover, is often ineffective due to its complexity, especially in the case of extensive agricultural activities. Therefore, the creation of automation systems for monitoring tomatoes becomes an urgent problem [3,8].

Various deep learning algorithms have shown good results in analyzing the state of plants, for example, in identifying diseases or classifying fruits [3,13]. Object detection algorithms, such as YOLO, are known for their ability to detect objects in real-time and are used in a wide range of applications based on computer vision [1,2].

However, visual analysis alone cannot solve all the problems in the field of agricultural robotics. In addition, it should be noted that the task of analyzing the condition of plants involves the use of robotic systems capable of navigating the environment, avoiding obstacles, and collecting data [5,7].

The purpose of the study is to develop an algorithm for the autonomous navigation of robots in order to monitor tomato plants using object detection technology implemented using the YOLOv8 model.

Materials and Methods

1. Dataset

The dataset used in this study consists of both publicly available data and a custom dataset created by the author. Public data were collected from several Roboflow open-source datasets for tomato detection and plant condition analysis [9–12]. These datasets include images captured under diverse environmental conditions, such as varying lighting, backgrounds, and plant orientations.

In addition, a custom dataset was collected by the author and includes approximately 300–500 images captured in greenhouse conditions under natural lighting. The images were taken from different viewpoints and distances to increase variability and improve model robustness.

Six object classes were defined in this study: green tomato, red tomato, semi-ripe, turning, healthy leaf, and diseased leaf. In total, the combined dataset contains approximately 2,500 images with more than 15,000 annotated objects.

The dataset was divided into training and validation subsets. The inclusion of both open-source and custom data improves the generalization capability of the model and makes it more suitable for real-world agricultural applications.

2. Data Preprocessing

All images were resized to the same input resolution that was suitable for detection in the YOLOv8 detection algorithm. Normalization was performed automatically when the model was being trained. Data augmentation was done using data augmentation techniques available from the Ultralytics framework [2]. Horizontal flips and basic geometrical transformation techniques were used.

3. Model Architecture and Inference Pipeline

The proposed system architecture is illustrated in Fig. 1. The system follows a sequential pipeline that integrates object detection and autonomous robotic navigation.

At the first stage, input images obtained from a camera or dataset are processed by the YOLOv8 object detection model [1,2]. The model detects objects and classifies them into six categories: green tomato, red tomato, semi-ripe, turning, healthy leaf, and diseased leaf. At the second stage, the detected objects are used for target coordinate extraction. The bounding box information provided by the model allows the system to determine the spatial position of each detected object.

Next, the extracted coordinates are passed to the autonomous robot navigation module. The robot moves toward selected targets while following a predefined path.

During navigation, an obstacle avoidance mechanism is applied to ensure safe movement within the environment. The robot dynamically adjusts its trajectory when obstacles are detected.

Finally, the system produces a plant inspection result based on the detected class and the completed navigation process. This pipeline enables efficient monitoring of plant conditions in agricultural environments.

Figure 1. Architecture of the proposed automatic tomato monitoring system using mobile robot

4. Models

Object detection is based on the YOLOv8 architecture [2] that operates in PyTorch. Specifically, YOLOv8n model is used because of its simplicity and suitability for real-time operations. The network was trained using the prepared dataset containing images of objects belonging to six classes, namely green tomato, red tomato, semi-ripe tomato, turning tomato, healthy leaf, and diseased leaf. Object detection involves predicting bounding boxes along with class labels of the detected objects. The network is trained and then used at the inference stage to obtain the detection outputs that are further used by the robot.

5. Robot Navigation and Obstacle Avoidance

The robotic system is designed to autonomously navigate through the agricultural environment and perform targeted inspection of tomato plants. The navigation process is based on coordinates obtained from the object detection model.

 At each step, the robot selects the next target using a distance-based and priority-based strategy, where diseased and ripe plants are assigned higher priority. Path planning is performed using a sequential target-following approach, where the robot moves toward selected coordinates by computing a normalized direction vector based on its relative position to the target.

The simulation environment was implemented in MATLAB as a structured 2D space with randomly distributed plant positions and static obstacles. The robot is initialized at a fixed starting point and updates its position iteratively while recalculating its trajectory.

Obstacle avoidance is implemented using a local reactive method. When a potential collision is detected, the robot adjusts its movement direction by introducing a repulsive or perpendicular component, allowing it to bypass obstacles while maintaining progress toward the target.

The process continues until all targets are visited, after which the robot returns to the starting position. This approach ensures stable navigation, collision-free movement, and efficient coverage of the monitored area.

Results and Discussion

  The performance of the proposed system was evaluated using standard object detection metrics, including precision, recall, mAP50, and mAP50-95. These metrics provide a comprehensive assessment of the model’s detection accuracy and localization capability. The results of the proposed model are presented in Table 1. The model achieved an overall precision of 0.90, recall of 0.85, mAP50 of 0.912, and mAP50-95 of 0.789, indicating reliable detection and classification of tomato conditions across different classes.

Table 1.

Model performance results

Examples of detection results are shown in Fig. 2. The model successfully detects multiple objects in complex scenes and accurately classifies tomatoes at different ripeness stages, as well as leaf conditions. The results demonstrate the robustness of the model under varying environmental conditions, including different lighting and backgrounds.

Figure 2. Examples of tomato detection and classification results using the proposed YOLOv8-based system

For comparison, a simplified baseline approach can be considered, where object detection is performed without target prioritization and without integration with a robotic navigation system. In such a setup, the performance is typically limited to detection accuracy only, with no optimization of the inspection process.

The proposed model achieved an mAP50 of 0.912 and mAP50-95 of 0.789, demonstrating high detection performance across all classes. In contrast, a simplified approach without class balancing and prioritization is expected to show lower performance, particularly for visually similar classes such as semi-ripe and turning tomatoes, where accuracy may decrease by approximately 5–10%.

Furthermore, the integration of robotic navigation significantly improves system efficiency. Instead of processing targets randomly, the proposed system prioritizes diseased and ripe plants, reducing unnecessary movements and improving inspection effectiveness.

Overall, the proposed approach provides both high detection accuracy and improved operational efficiency compared to conventional single-stage detection systems.    

The simulation results of the robotic system are presented in Fig. 3. The robot successfully navigates through the environment, avoids obstacles, and reaches target positions. The planned trajectory demonstrates stable path generation and effective target prioritization during the inspection process.

Figure. 3. Simulation results of the autonomous robotic system with obstacle avoidance and target navigation

In each experiment, the robot successfully inspected all target plants (approximately 15 per simulation) without critical failures. The navigation system demonstrated stable performance, with a success rate close to 98% in reaching target positions.  
The implemented obstacle avoidance mechanism allowed the robot to safely bypass obstacles without collisions. In addition, the prioritization strategy improved the efficiency of the inspection process. Diseased and ripe plants were visited earlier in the sequence, reducing unnecessary movements and improving task execution. The simulation results confirm that the robotic system is capable of reliable navigation and effective plant inspection in a structured agricultural environment.

The normalized confusion matrix of the detection model is presented in Fig. 4. Most predictions are concentrated along the diagonal, indicating high classification accuracy across the majority of classes.

Figure 4. Normalized confusion matrix of the YOLOv8 detection model

The highest classification performance is observed for the diseased leaf class, where the majority of predictions are correctly concentrated along the diagonal (1008 correct detections), indicating strong feature separability for this class. Similarly, the green tomato class demonstrates stable performance with 2095 correct predictions.

In contrast, lower performance is observed for intermediate ripeness stages. A noticeable confusion occurs between semi-ripe and turning classes. For example, a significant number of turning tomatoes are misclassified as semi-ripe (891 instances), while semi-ripe tomatoes are also confused with turning (193 instances).    This bidirectional confusion indicates a strong overlap between these classes.This behavior can be explained by the gradual transition of visual features during the ripening process. The color, texture, and shape of tomatoes change continuously rather than discretely, making it difficult for the model to distinguish between adjacent maturity stages.

In real-world applications, this limitation can be mitigated by increasing the number of labeled samples for intermediate classes, improving dataset balance, and incorporating additional contextual information such as temporal progression or multi-view observations. These improvements would allow the model to better capture subtle differences between ripeness stages and reduce classification ambiguity.

Conclusion

The paper elaborates on the design and simulation of an autonomous robotic system designed for tomato inspection using the YOLOv8 object detection architecture. This method is characterized by the incorporation of computer vision into robotics to enable automatic inspection processes in agricultural settings. Based on experimental studies, this research found that the model exhibits excellent object recognition capabilities through a high level of precision (0.90), recall (0.85), mAP50 (0.912), and mAP50-95 (0.789). The recognition algorithm showed particular strengths in detecting diseased leaves, having mAP50 of 0.984. The robotic system was tested in simulation mode, and it was found to have almost a 98% success rate in positioning itself at target locations while being able to avoid objects. Overall, these results demonstrate that the system developed in this study can be considered a promising approach to automation tomato monitoring in precision farming. Future research should involve testing the system in a realistic setting.

References:

1. Redmon J., Farhadi A. YOLOv3: An Incremental Improvement. // arXiv. – 2018. – Available: https://arxiv.org/abs/1804.02767
2. Jocher G., Chaurasia A., Qiu J. YOLOv8: Ultralytics YOLO. // GitHub. – 2023. – Available: https://github.com/ultralytics/ultralytics (accessed: 15.03.2026)
3. Kamilaris A., Prenafeta-Boldú F.X. Deep learning in agriculture: A survey. // Computers and Electronics in Agriculture. – 2018. – DOI: 10.1016/j.compag.2018.06.011
4. Veluru C.S. Robotics and Data Science for Smart and Precision Agriculture. // Journal of Scientific and Engineering Research. – 2024.
5. Wijesundara W.M.T.D. et al. Accurate Crop Spraying with RTK and Machine Learning on an Autonomous Field Robot. // International Journal of Botany and Horticulture Research. – 2024.
6. Pretto A. et al. Building an Aerial–Ground Robotics System for Precision Farming. // IEEE Robotics & Automation Magazine. – 2021. – DOI: 10.1109/MRA.2021.3063356
7. Lehnert C. et al. Autonomous Sweet Pepper Harvesting for Protected Cropping Systems. // IEEE Robotics and Automation Letters. – 2017. – DOI: 10.1109/LRA.2017.2655622
8. Mahmud M.S., Zahid A., Das A.K. Sensing and Automation Technologies for Ornamental Nursery Crop Production. // Sensors. – 2023. – DOI: 10.3390/s23031345
9. Roboflow Tomato Dataset by Nurasil. // Roboflow. – 2026. – Available: https://app.roboflow.com/nurasil/tomato-zjjgs-eyap6/models
10. Tomato Dataset. // Roboflow Universe. – 2026. – Available: https://universe.roboflow.com/tuygun-nvqly/tomato-mtrtn-2
11. Leaf Tomato Dataset. // Roboflow Universe. – 2026. –Available:https://universe.roboflow.com/liquid-72-workspace/tomato-leaf-rsmj5
12. Tomato Detection v3 Dataset. // Roboflow Universe. – 2026. – Available: https://app.roboflow.com/nurasil/development-of-an-intelligent-ro/models