Detection of pests and diseases for vegetable and fruit plants using machine vision: A review

Research
Shafik Kiraga1Ali Mohammad2Reza Md Nasim1,2Chowdhury Milon1,2Chung Sun-Ok1Hong Soon Jung3*

Abstract

Early detection of pests and diseases can significantly increase crop yield in agricultural fields. Farmers normally use visible characteristics to identify and characterize different pests and diseases. In some cases, human experts are hired to identify the resulting anomalies. All these measures are time-consuming, expensive, and impractical in some cases. The objective of this paper was to review the previous research related to pests and diseases detection in fruit and vegetable plants using machine vision techniques, focusing on visible RGB images. . Based on captured images of infected leaves or pests, machine vision techniques were applied to give a general idea about disease/pest infestation. Machine vision applications in agriculture have been encouraging for several decades, and the steady development of useful vision algorithms has been well matched by modern computers' ability to implement them at sufficiently high speeds to make them viable. The development of these vision systems has involved moving from academic research to agricultural application, which presents more problems to solve to ensure mature application in the real natural environment.

Keyword



Introduction

Precision agriculture is a concept of farm management based on measuring, monitoring, and responding to crop variability (Mavridou et al., 2019). It defines systems that optimize inputs while preserving resources. Since the introduction of automation, machine vision has been widely used to support precision agriculture by automating tasks usually done by human workers. The various crop yield affecting factors like pests and diseases have made machine vision attractive in agriculture. The traditional crop inspection methods are slow, and error-prone due to human mistakes and some parts of the field may not be easily reached by humans in cases where plants are cultivated on a large scale. Changing weather patterns have led to the emergency of new pests and diseases, more so, in areas where they had not been seen before (Johannes et al., 2017). It is further complicated by the fact that they are transferred more easily than ever before (Sladojevic et al., 2016). Excess use and misuse of pesticides have led to the development of long-term resistant pathogens, economic loss and environmental contamination (He et al., 2019). To prevent losses in crop production, accurate and timely diagnosis of crop pests and diseases is a necessity. Traditionally, crop technicians and experienced farmers are often relied on by agricultural workers for plant protection, which is a delayed, inefficient, and subjective method (Bai et al., 2018).

Some plants do not have visible symptoms of infection and the effect becomes noticeable too late for any action to be taken. Although some diseases produce some manifestation in the visible spectrum for a trained pathologist to detect, variation in symptoms could lead to false identification. In other cases, signs can only be visible in some parts of the electromagnetic spectrum which are not visible to humans.

In response to these challenges, machine vision systems have been deployed for automatic identification of pests and diseases which may increase the work speedand decrease human errors. The main components of vision systems are image acquisition and processing. For image acquisition, the systems are normally constituted of a holding platform, a camera, and a light source while image processing includes the classical image processing techniques, machine learning methods, and deep learning architectures, which solve the difficult problems that could not be solved by traditional methods.

In this paper, research trends for pests and disease detection of fruits and vegetables using machine vision were reviewed. The scope of the review was limited to the processing of visible images in the RGB space using machine vision for images from smartphones, and single or multiple cameras. First, a brief introduction to machine vision was provided. Then, reviews on pests and disease detection for fruit and vegetable plants as applied on different plant parts, including fruits, and leaves were explored. Alternative methods of pests and disease detection from various researchers were also presented with concluding future remarks.

Major pests and diseases in fruit and vegetable plants with visual symptoms

Majority of diseases in plants are caused by nematodes, bacteria, and fungi. For fruit crops, fungal disease symptoms include Powdery mildew, Downey mildew, and Anthracnose while vegetable crops include additional symptoms such as early blight, late blight, and rust(Pujari et al., 2015).

On the other hand, the increase in trade of agricultural products between countries has led to an increase in the risk of pest invasions in regions where they had never existed. For example, the destructive nature of Asian long-horned beetle has shown how alien pests can affect agriculture(Poland, 1998). For fruits and vegetables, pest infestations have different visual symptoms and for diagnosis, visual inspection and further examination is normally carried out. For instance, Moradi et al. (2011) determined pest infestation of apple fruits through examining defects on its skin color. In other cases, some pests such as fruit fly cause injuries to fruits by creating tunnels in infested fruit(Yang et al., 2006). The created tunnel has a different shape to the internal structure of the fruit.

For tomato plants, variety of pests and diseases affect its plants as shown in Figure 1 (Fuentes et al., 2017). These include Gray mold, Canker, leaf mold, Plague, Leaf miner, Whitefly, and powdery mildew. Visible symptoms vary from light or gray spots for gray mold, occurrence of cankers on the leaf surface with small brownish-black lesions for cankers, to visible yellow spots on the inside of leaves for plague.

http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Figure_PASTJ_22-004_F1.png

Fig. 1. Common diseases and pest infestation to tomato plants. a) Leaf mold b) Gray mold c) Canker d)Canker e) Miners f) Whitefly (Modified from Fuentes et al., 2017).

Selvaraj et al.(2019) classified five banana diseases and pest into two classes; dried-age leaves and banana corm weevil. The five major diseases included Xanthomonas wilt, Bunchy top disease, Black sigatoka, Yellow sigatoka, and Fusarium wilt and the corm weevil as the major pest as shown in Figure 2. He stated that various visual symptoms occur in different parts of the plant and the disease occurrence depend on many factors including environment, humidity, temperature, and season. The symptoms can be observed at the leaf, pseudostem, fruit bunch, cut fruit, and corm.

http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Figure_PASTJ_22-004_F2.png

Fig. 2. Diseases and pest infestation for banana fruit. a) Bunchy top virus b) black sigatoka c) Xanthomonas wilt d) xanthomonas wilt e) xanthomonas wilt f) corm weevil (modified from Selvaraj et al.(2019))

http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Figure_PASTJ_22-004_F3.png

Fig. 3. Grapevine leaves infected with spider mite (a) and downy mildew (b)

Downy mildew and spider mite are known to infect grapevine leaves (Gutiérrez et al., 2021) as illustrated in Figure 3. Three sets of RGB grapevine canopy leaf images were taken in a commercial vineyard with downy mildew symptoms, spider mite symptoms, and without symptoms. Accuracy of 94% was obtained for classifying the leaves at the same time, demonstrating the effectiveness of machine vision techniques for the classification of grapevine leaf images taken under field conditions. Similar yellowing was observed for both diseases but the patches had different shapes and sizes.

Detection of diseases for fruit and vegetable plants using machine vision

Visible characteristics, plant type, pathogen and the infected plant part are the factors used to classify plant diseases as shown in Figure 4. The diseases may be infectious or non-infectious (Yialouris & Sideridis, 1996).

http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Figure_PASTJ_22-004_F4.png

Fig. 4. Classification of plant diseases (modified from Yialouris & Sideridis, 1996)

In most cases, disease detection is visual as shown in Figure 5 and requires intuitive judgement as well as scientific methods. Pydipati et al.(2014) used a colour co-occurrence method (CCM) for textual analysis to identify whether diseased and normal citrus leaves can be identified using classification algorithms. Forty diseased and normal citrus leaves with greasy spots, melanoses, and scabs were collected from the field and captured using an analogue CCD camera interfaced with a frame grabber as shown in Figure 6. For classification analysis, four feature models were developed based on a Mahalanobis minimum distance classifier, using the nearest neighbour principle, as well as neural network classifiers based on the back-propagation algorithm and radial basis functions. Using the Mahalanobis statistical classifier and the CCM textural analysis, classification accuracies of over 95% for all classes (99% mean accuracy) when using hue and saturation texture features. Likewise, a back-propagation neural network algorithm achieved accuracies of over 90% for all classes (95% mean accuracy).

http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Figure_PASTJ_22-004_F5.png

Fig. 5. Common diseases for vegetables (modified from Khadabadi et al., 2015)

http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Figure_PASTJ_22-004_F6.png

Fig. 6. Setup of a machine vision system with frame grabbers (Pydipati et al., 2014)

Table 1. Detection of diseases and pests in fruit and vegetable plants using RGB images http://dam.zipot.com:8080/sites/pastj/images/PASTJ_22-004_image/Table_PASTJ_22-004_T1.png

An automatic online machine vision-based agro-medical expert system was developed (Habib et al., 2020) for the detection and classification of papaya diseases. The diseases included Black spot, powdery mildew, brown spot, phytophthora blight and anthracnose. The disease processes the captured image through mobile or handheld devices and determines the disease. To segment out the disease-attacked region, a K-means clustering algorithm was used and the diseases are classified with an SVM classifier. They achieved an accuracy of over 90%.

To increase scouting efficiency through strawberry fields and increase the monitoring of strawberry powdery mildew disease, an automatic system to detect powdery mildew disease in strawberry fields by using a real-time machine vision system was developed (Mahmud et al., 2020). The disease detection system is composed of a global positioning system, two µEye 1240 LE/C cameras (IDS Imaging Development System Inc., Woburn, MA, USA), a laptop computer (Toshiba Corporation, Minato, Tokyo, Japan) and a custom image processing program. A colour co-occurrence matrix (CCM) and an Artificial Neural Network (ANN) were used to process and classify the continuously acquired images, respectively. The system was tested with a total of 36 strawberry rows within three fields. They achieved a precision of 87.65%, 87.14% and 84.71% in the field site-I, field site-II, and field site-III, respectively. Table 1 summarizes the research reviewed about disease detection using machine vision for fruits and vegetable plants.

Detection of diseases for fruit and vegetable plants using machine vision

Visible characteristics, plant type, pathogen and the infected plant part are the factors used to classify plant diseases as shown in Figure 4. The diseases may be infectious or non-infectious (Yialouris & Sideridis, 1996).

In most cases, disease detection is visual as shown in Figure 5 and requires intuitive judgement as well as scientific methods. Pydipati et al.(2014) used a colour co-occurrence method (CCM) for textual analysis to identify whether diseased and normal citrus leaves can be identified using classification algorithms. Forty diseased and normal citrus leaves with greasy spots, melanoses, and scabs were collected from the field and captured using an analogue CCD camera interfaced with a frame grabber as shown in Figure 6. For classification analysis, four feature models were developed based on a Mahalanobis minimum distance classifier, using the nearest neighbour principle, as well as neural network classifiers based on the back-propagation algorithm and radial basis functions. Using the Mahalanobis statistical classifier and the CCM textural analysis, classification accuracies of over 95% for all classes (99% mean accuracy) when using hue and saturation texture features. Likewise, a back-propagation neural network algorithm achieved accuracies of over 90% for all classes (95% mean accuracy).

An automatic online machine vision-based agro-medical expert system was developed (Habib et al., 2020) for the detection and classification of papaya diseases. The diseases included Black spot, powdery mildew, brown spot, phytophthora blight and anthracnose. The disease processes the captured image through mobile or handheld devices and determines the disease. To segment out the disease-attacked region, a K-means clustering algorithm was used and the diseases are classified with an SVM classifier. They achieved an accuracy of over 90%.

To increase scouting efficiency through strawberry fields and increase the monitoring of strawberry powdery mildew disease, an automatic system to detect powdery mildew disease in strawberry fields by using a real-time machine vision system was developed (Mahmud et al., 2020). The disease detection system is composed of a global positioning system, two µEye 1240 LE/C cameras (IDS Imaging Development System Inc., Woburn, MA, USA), a laptop computer (Toshiba Corporation, Minato, Tokyo, Japan) and a custom image processing program. A colour co-occurrence matrix (CCM) and an Artificial Neural Network (ANN) were used to process and classify the continuously acquired images, respectively. The system was tested with a total of 36 strawberry rows within three fields. They achieved a precision of 87.65%, 87.14% and 84.71% in the field site-I, field site-II, and field site-III, respectively. Table 1 summarizes the research reviewed about disease detection using machine vision for fruits and vegetable plants.

Detection of pests in fruits and vegetables plant using machine vision

A system to identify beet armyworms was developed for the vegetable, field, and flower crop pests (Asefpour Vakilian and Massah, 2013). Armyworms were acquired from a sugar beet farm and placed in a dark chamber, images were then acquired using a Canon CCD digital camera with an LDR lighting module. An array of 200 LEDs were used as a light source with 100 armyworm images and 100 images of other pest species. Four morphological and three textural features were extracted for each image. The former included area, perimeter, eccentricity, and sphericity while the latter included local homogeneity, entropy, and energy. These were used to prepare a dataset. A total of 150 images were used to train an ANN classifier and the rest for evaluation. ANN classifier was able to classify armyworms with an accuracy of 90%.

An automatic pest identification system was developed to detect whiteflies, aphids, and cabbage moths (Rajan et al., 2017). They used a Digital camera (VIS) for image acquisition of crop leaves that may have pests. A database was created and used to store the histograms of captured Images. An SVM classifier was used for training and the threshold values with the slack variables stored in the database. Then, the threshold values were used to distinguish the object from the background. Classification of pests was done using slack variables. Detection accuracy of 95% was achieved. Table 1 summarizes the pests for both fruit and vegetable plants as reviewed in this paper.

Doitsidis et al. (2017) developed a web-based pest detection system for detecting and counting olive fruit flies in olive orchards. An automatic McPhail trap with a 2MP digital camera was used. To attract olive fruit flies, 200ml of ammonium sulfate were filled in the glass trap. Images of trapped fruit flies were captured and sent to a web server through a GSM module. For any newly uploaded image, an image analyzer and system monitoring components are activated using a pre-programmed listener module. A One-way ANOVA was used for image analysis online. Fruit flies were indicated with black areas (black pixels) in the images. Detection accuracy of 75% was achieved.

An imaging system in the strawberry greenhouse for the detection of thrips and classifying parasites was developed by Ebrahimi et al. (2017). They used LabVIEW to control a horizontal mobile agricultural robot with an 18MP Canon EOS M digital camera to capture the strawberry flower images. An SVM classier was used and an identification accuracy of greater than 97.5% was achieved.

Partel et al. (2019) developed an automatic pest detection and counting system for Asian citrus psyllid citrus crops. They used six cameras to capture psyllid pest’s images falling on a board fixed to a mobile vehicle which has a tapping unit to shake citrus tree branches. CNN was used to identify psyllids from captured images. Precision and recall of 80% and 95%, respectively were achieved.

Alternative methods of pests and disease detection for fruits and vegetables plant

Potamitis et al. (2017) developed a system for monitoring the entrance of pests to improve the operation of a low-cost McPhail trap. It was based on identifying the species of incoming pests from the optoacoustic spectrum analysis of their wingbeat. They achieved an accuracy of 91% for distinguishing fruit flies from other insects. However, it couldn’t distinguish between fruit fly species. To improve the system, Potamitis et al. (2018) developed a bimodal optoelectronic sensor that records the wingbeats of insects in flight with a Fresnel lens. This system had an accuracy of 98.99%.

Sounds of Red palm weevil were collected inside palm trees in different conditions (Dosunmu et al., 2014). They created a model which was based on the pattern and frequency of the emitted sounds. They concluded that the weevil could be differentiated based on burst duration, and detection of larvae was easier in offshoots than inside the trunk.

Rach et al. (2013) proposed a method to detect Red Palm Weevils (RPW) acoustically in the field. The proposed system was able to monitor and record acoustic emissions of the adult RPW using an audio probe with an acoustic sensor. The sound is then amplified and processed through an algorithm with a wireless interface to send detected signals remotely. Audio signals are first digitized and subsequent execution reveals the presence or absence of RPW in real-time.

Future projections and tasks

Machine vision is deployed in agriculture for several tasks. The choice of an appropriate machine vision system solely depends on the task at hand. In this paper, several studies from previous research about pests and disease detection have been reviewed, with alternative detection methods also presented. More machine vision algorithms will be required since, in different growth cycles, the appearance of diseases and pests will be different. For example, the manifestation of some pests and diseases symptoms is not obvious, limiting early diagnosis. Besides, even high-resolution images are difficult to analyze in the early stages of the pests and disease occurrence.. Moreover, occlusion of leaves, branches and light in the field is also a limitation that could lead to false/missed detection due to the lack of features and noise overlap.

Implementation of digital signal processing techniques and the use of acoustic sensors can supplement machine vision algorithms, specifically, for pests detection. This technique has been applied by various researchers and it has shown good potential over a wide range of pests. Also, it is necessary to combine meteorological data such as temperature and rainfall with plant protection data to recognize and predict pests and diseases efficiently. In addition, automatic recognition algorithms can improve the recognition rate and speed up real-time automation systems in the future. The algorithms need to be suitable for the automation of semi-structured environments, such as an agricultural environment. With more research focused on this domain given the advancement in image processing methods and development of low-cost hardware systems, there will be a greater benefit of machine vision systems in precision agriculture.

Acknowledgement

This work was supported by the Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) through the Agriculture, Food and Rural Affairs Convergence Technologies Program for Educating Creative Global Leaders, funded by Ministry of Agriculture, Food and Rural Affairs (MAFRA) (Project No.: 320001-4), Republic of Korea.

References

1 Asefpour VK, Massah J. 2013. Performance evaluation of a machine vision system for insect pests identification of field crops using artificial neural networks. Archives of Phytopathology and Plant Protection 46: 1262–1269.  

2 Bai X, Cao Z, Zhao L, Zhang J, Lv C, Li C, Xie J. 2018. Rice heading stage automatic observation by multi-classifier cascade based rice spike detection method. Agricultural and Forest Meteorology 259: 260–270.  

3 Doitsidis L, Fouskitakis G N, Varikou K N, Rigakis I I, Chatzichristofis S A, Papafilippaki A K, Birouraki A E. 2017. Remote monitoring of the Bactrocera oleae (Gmelin) (Diptera: Tephritidae) population using an automated McPhail trap. Computers and Electronics in Agriculture 137:69–78.  

4 Dosunmu OG, Herrick NJ, Haseeb M, Hix RL, Mankin RW. 2014. Acoustic detectability of rhynchophorus cruentatus (Coleoptera: Dryophthoridae). Florida Entomologist, 97: 431–438.  

5 Ebrahimi MA, Khoshtaghaza MH, Minaei S, Jamshidi B. 2017. Vision-based pest detection based on SVM classification method. Computers and Electronics in Agriculture 137: 52–58.  

6 Fuentes A, Yoon S, Kim SC, Park DS. 2017. A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors 17:9.  

7 Gutiérrez S, Hernández I, Ceballos S, Barrio I, Díez-Navajas AM, Tardaguila J. 2021. Deep learning for the differentiation of downy mildew and spider mite in grapevine under field conditions. Computers and Electronics in Agriculture, 182:1–9.  

8 Habib MT, Majumder A, Jakaria AZM, Akter M, Uddin MS, Ahmed F. 2020. Machine vision based papaya disease recognition. Journal of King Saud University - Computer and Information Sciences 32: 300–309.  

9 He Y, Zeng H, Fan Y, Ji S, Wu J. 2019. Application of Deep Learning in Integrated Pest Management: A Real-Time System for Detection and Diagnosis of Oilseed Rape Pests. Mobile Information Systems, 2019.  

10 Johannes A, Picon A, Alvarez-Gila A, Echazarra J, Rodriguez-Vaamonde S, Navajas A D, Ortiz-Barredo A. 2017. Automatic plant disease diagnosis using mobile capture devices, applied on a wheat use case. Computers and Electronics in Agriculture 138: 200–209.  

11 Khadabadi GC, Rajpurohit VS, Kumar A, Nargund VBN. 2015. Disease Detection in Vegetables Using Image Processing Techniques: A Review. International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE) 14: 954–960.  

12 Mahmud Sultan M, Zaman QU, Esau TJ, Chang YK, Price GW, Prithiviraj B. 2020. Real‐time detection of strawberry powdery mildew disease using a mobile machine vision system. Agronomy, 10:7.  

13 Mavridou E, Vrochidou E, Papakostas GA, Pachidis T, Kaburlasos VG. 2019. Machine vision systems in precision agriculture for crop farming. Journal of Imaging, 5:12.      

14 Moradi G, Shamsi M, Sedaaghi MH, Moradi S. 2011. Apple defect detection using statistical histogram based fuzzy c-means algorithm. 2011 7th Iranian Conference on Machine Vision and Image Processing, MVIP 2011 - Proceedings, 1–6.  

15 Partel V, Nunes L, Stansly P, Ampatzidis Y. 2019. Automated vision-based system for monitoring Asian citrus psyllid in orchards utilizing artificial intelligence. Computers and Electronics in Agriculture, 162:328–336.  

16 Poland T. 1998. Chicago joins New York in battle with the Asian longhorned beetle. Newsl. Mich. Entomol. 43:15–17.  

17 Potamitis I, Rigakis I, Tatlas NA. 2017. Automated surveillance of fruit flies. Sensors 17:1.      

18 Potamitis I, Rigakis I, Vidakis N, Petousis M,Weber M. 2018. Affordable Bimodal Optical Sensors to Spread the Use of Automated Insect Monitoring. Journal of Sensors, 2018.  

19 Pujari JD, Yakkundimath R, Byadgi AS. 2015. Image processing Based Detection of Fungal Diseases in Plants. Procedia Computer Science 46: 1802–1808.  

20 Pydipati R, Burks TF, Lee WS. 2014. S n n c c d d u m v. 48:2007–2014.  

21 Rach MM, Gomis HM, Granado OL, Malumbres MP, Campoy AM, Martín JJS. 2013. On the design of a bioacoustic sensor for the early detection of the red palm weevil. Sensors, 13: 1706–1729.      

22 Rajan P, Radhakrishnan B, Padma SL. 2017. Detection and classification of pests from crop images using Support Vector Machine. Proceedings of IEEE International Conference on Emerging Technological Trends in Computing, Communications and Electrical Engineering, ICETT.  

23 Selvaraj MG, Vergara A, Ruiz H, Safari N, Elayabalan S, Ocimati W, Blomme G. 2019. AI-powered banana diseases and pest detection. Plant Methods 15: 1–11.  

24 Sladojevic S, Arsenovic M, Anderla A, Culibrk D, Stefanovic D. 2016. Deep Neural Networks Based Recognition of Plant Diseases by Leaf Image Classification. Computational Intelligence and Neuroscience, 2016.      

25 Yang EC, Yang MM, Liao LH, Wu WY, Chen TW, Chen TM, Lin T, Jiang JA. 2006. Non-Destructive Quarantine Technique- Potential Application of Using X-ray Images to Detect Early Infestations Caused by Oriental Fruit Fly (Bactrocera dorsalis) (Diptera: Tephritidae) in Fruit. Formosan Entomologist 26: 171–186.  

26 Yialouris CP, Sideridis AB. 1996. An expert system for tomato diseases. Computers and Electronics in Agriculture, 14: 61–76.