Introduction
Precision agriculture is a concept of farm management based on measuring, monitoring, and responding to crop variability (Mavridou et al., 2019). It defines systems that optimize inputs while preserving resources. Since the introduction of automation, machine vision has been widely used to support precision agriculture by automating tasks usually done by human workers. The various crop yield affecting factors like pests and diseases have made machine vision attractive in agriculture. The traditional crop inspection methods are slow, and error-prone due to human mistakes and some parts of the field may not be easily reached by humans in cases where plants are cultivated on a large scale. Changing weather patterns have led to the emergency of new pests and diseases, more so, in areas where they had not been seen before (Johannes et al., 2017). It is further complicated by the fact that they are transferred more easily than ever before (Sladojevic et al., 2016). Excess use and misuse of pesticides have led to the development of long-term resistant pathogens, economic loss and environmental contamination (He et al., 2019). To prevent losses in crop production, accurate and timely diagnosis of crop pests and diseases is a necessity. Traditionally, crop technicians and experienced farmers are often relied on by agricultural workers for plant protection, which is a delayed, inefficient, and subjective method (Bai et al., 2018).
Some plants do not have visible symptoms of infection and the effect becomes noticeable too late for any action to be taken. Although some diseases produce some manifestation in the visible spectrum for a trained pathologist to detect, variation in symptoms could lead to false identification. In other cases, signs can only be visible in some parts of the electromagnetic spectrum which are not visible to humans.
In response to these challenges, machine vision systems have been deployed for automatic identification of pests and diseases which may increase the work speedand decrease human errors. The main components of vision systems are image acquisition and processing. For image acquisition, the systems are normally constituted of a holding platform, a camera, and a light source while image processing includes the classical image processing techniques, machine learning methods, and deep learning architectures, which solve the difficult problems that could not be solved by traditional methods.
In this paper, research trends for pests and disease detection of fruits and vegetables using machine vision were reviewed. The scope of the review was limited to the processing of visible images in the RGB space using machine vision for images from smartphones, and single or multiple cameras. First, a brief introduction to machine vision was provided. Then, reviews on pests and disease detection for fruit and vegetable plants as applied on different plant parts, including fruits, and leaves were explored. Alternative methods of pests and disease detection from various researchers were also presented with concluding future remarks.
Major pests and diseases in fruit and vegetable plants with visual symptoms
Majority of diseases in plants are caused by nematodes, bacteria, and fungi. For fruit crops, fungal disease symptoms include Powdery mildew, Downey mildew, and Anthracnose while vegetable crops include additional symptoms such as early blight, late blight, and rust(Pujari et al., 2015).
On the other hand, the increase in trade of agricultural products between countries has led to an increase in the risk of pest invasions in regions where they had never existed. For example, the destructive nature of Asian long-horned beetle has shown how alien pests can affect agriculture(Poland, 1998). For fruits and vegetables, pest infestations have different visual symptoms and for diagnosis, visual inspection and further examination is normally carried out. For instance, Moradi et al. (2011) determined pest infestation of apple fruits through examining defects on its skin color. In other cases, some pests such as fruit fly cause injuries to fruits by creating tunnels in infested fruit(Yang et al., 2006). The created tunnel has a different shape to the internal structure of the fruit.
For tomato plants, variety of pests and diseases affect its plants as shown in Figure 1 (Fuentes et al., 2017). These include Gray mold, Canker, leaf mold, Plague, Leaf miner, Whitefly, and powdery mildew. Visible symptoms vary from light or gray spots for gray mold, occurrence of cankers on the leaf surface with small brownish-black lesions for cankers, to visible yellow spots on the inside of leaves for plague.
Selvaraj et al.(2019) classified five banana diseases and pest into two classes; dried-age leaves and banana corm weevil. The five major diseases included Xanthomonas wilt, Bunchy top disease, Black sigatoka, Yellow sigatoka, and Fusarium wilt and the corm weevil as the major pest as shown in Figure 2. He stated that various visual symptoms occur in different parts of the plant and the disease occurrence depend on many factors including environment, humidity, temperature, and season. The symptoms can be observed at the leaf, pseudostem, fruit bunch, cut fruit, and corm.
Downy mildew and spider mite are known to infect grapevine leaves (Gutiérrez et al., 2021) as illustrated in Figure 3. Three sets of RGB grapevine canopy leaf images were taken in a commercial vineyard with downy mildew symptoms, spider mite symptoms, and without symptoms. Accuracy of 94% was obtained for classifying the leaves at the same time, demonstrating the effectiveness of machine vision techniques for the classification of grapevine leaf images taken under field conditions. Similar yellowing was observed for both diseases but the patches had different shapes and sizes.
Detection of diseases for fruit and vegetable plants using machine vision
Visible characteristics, plant type, pathogen and the infected plant part are the factors used to classify plant diseases as shown in Figure 4. The diseases may be infectious or non-infectious (Yialouris & Sideridis, 1996).
In most cases, disease detection is visual as shown in Figure 5 and requires intuitive judgement as well as scientific methods. Pydipati et al.(2014) used a colour co-occurrence method (CCM) for textual analysis to identify whether diseased and normal citrus leaves can be identified using classification algorithms. Forty diseased and normal citrus leaves with greasy spots, melanoses, and scabs were collected from the field and captured using an analogue CCD camera interfaced with a frame grabber as shown in Figure 6. For classification analysis, four feature models were developed based on a Mahalanobis minimum distance classifier, using the nearest neighbour principle, as well as neural network classifiers based on the back-propagation algorithm and radial basis functions. Using the Mahalanobis statistical classifier and the CCM textural analysis, classification accuracies of over 95% for all classes (99% mean accuracy) when using hue and saturation texture features. Likewise, a back-propagation neural network algorithm achieved accuracies of over 90% for all classes (95% mean accuracy).
An automatic online machine vision-based agro-medical expert system was developed (Habib et al., 2020) for the detection and classification of papaya diseases. The diseases included Black spot, powdery mildew, brown spot, phytophthora blight and anthracnose. The disease processes the captured image through mobile or handheld devices and determines the disease. To segment out the disease-attacked region, a K-means clustering algorithm was used and the diseases are classified with an SVM classifier. They achieved an accuracy of over 90%.
To increase scouting efficiency through strawberry fields and increase the monitoring of strawberry powdery mildew disease, an automatic system to detect powdery mildew disease in strawberry fields by using a real-time machine vision system was developed (Mahmud et al., 2020). The disease detection system is composed of a global positioning system, two µEye 1240 LE/C cameras (IDS Imaging Development System Inc., Woburn, MA, USA), a laptop computer (Toshiba Corporation, Minato, Tokyo, Japan) and a custom image processing program. A colour co-occurrence matrix (CCM) and an Artificial Neural Network (ANN) were used to process and classify the continuously acquired images, respectively. The system was tested with a total of 36 strawberry rows within three fields. They achieved a precision of 87.65%, 87.14% and 84.71% in the field site-I, field site-II, and field site-III, respectively. Table 1 summarizes the research reviewed about disease detection using machine vision for fruits and vegetable plants.
Detection of diseases for fruit and vegetable plants using machine vision
Visible characteristics, plant type, pathogen and the infected plant part are the factors used to classify plant diseases as shown in Figure 4. The diseases may be infectious or non-infectious (Yialouris & Sideridis, 1996).
In most cases, disease detection is visual as shown in Figure 5 and requires intuitive judgement as well as scientific methods. Pydipati et al.(2014) used a colour co-occurrence method (CCM) for textual analysis to identify whether diseased and normal citrus leaves can be identified using classification algorithms. Forty diseased and normal citrus leaves with greasy spots, melanoses, and scabs were collected from the field and captured using an analogue CCD camera interfaced with a frame grabber as shown in Figure 6. For classification analysis, four feature models were developed based on a Mahalanobis minimum distance classifier, using the nearest neighbour principle, as well as neural network classifiers based on the back-propagation algorithm and radial basis functions. Using the Mahalanobis statistical classifier and the CCM textural analysis, classification accuracies of over 95% for all classes (99% mean accuracy) when using hue and saturation texture features. Likewise, a back-propagation neural network algorithm achieved accuracies of over 90% for all classes (95% mean accuracy).
An automatic online machine vision-based agro-medical expert system was developed (Habib et al., 2020) for the detection and classification of papaya diseases. The diseases included Black spot, powdery mildew, brown spot, phytophthora blight and anthracnose. The disease processes the captured image through mobile or handheld devices and determines the disease. To segment out the disease-attacked region, a K-means clustering algorithm was used and the diseases are classified with an SVM classifier. They achieved an accuracy of over 90%.
To increase scouting efficiency through strawberry fields and increase the monitoring of strawberry powdery mildew disease, an automatic system to detect powdery mildew disease in strawberry fields by using a real-time machine vision system was developed (Mahmud et al., 2020). The disease detection system is composed of a global positioning system, two µEye 1240 LE/C cameras (IDS Imaging Development System Inc., Woburn, MA, USA), a laptop computer (Toshiba Corporation, Minato, Tokyo, Japan) and a custom image processing program. A colour co-occurrence matrix (CCM) and an Artificial Neural Network (ANN) were used to process and classify the continuously acquired images, respectively. The system was tested with a total of 36 strawberry rows within three fields. They achieved a precision of 87.65%, 87.14% and 84.71% in the field site-I, field site-II, and field site-III, respectively. Table 1 summarizes the research reviewed about disease detection using machine vision for fruits and vegetable plants.
Detection of pests in fruits and vegetables plant using machine vision
A system to identify beet armyworms was developed for the vegetable, field, and flower crop pests (Asefpour Vakilian and Massah, 2013). Armyworms were acquired from a sugar beet farm and placed in a dark chamber, images were then acquired using a Canon CCD digital camera with an LDR lighting module. An array of 200 LEDs were used as a light source with 100 armyworm images and 100 images of other pest species. Four morphological and three textural features were extracted for each image. The former included area, perimeter, eccentricity, and sphericity while the latter included local homogeneity, entropy, and energy. These were used to prepare a dataset. A total of 150 images were used to train an ANN classifier and the rest for evaluation. ANN classifier was able to classify armyworms with an accuracy of 90%.
An automatic pest identification system was developed to detect whiteflies, aphids, and cabbage moths (Rajan et al., 2017). They used a Digital camera (VIS) for image acquisition of crop leaves that may have pests. A database was created and used to store the histograms of captured Images. An SVM classifier was used for training and the threshold values with the slack variables stored in the database. Then, the threshold values were used to distinguish the object from the background. Classification of pests was done using slack variables. Detection accuracy of 95% was achieved. Table 1 summarizes the pests for both fruit and vegetable plants as reviewed in this paper.
Doitsidis et al. (2017) developed a web-based pest detection system for detecting and counting olive fruit flies in olive orchards. An automatic McPhail trap with a 2MP digital camera was used. To attract olive fruit flies, 200ml of ammonium sulfate were filled in the glass trap. Images of trapped fruit flies were captured and sent to a web server through a GSM module. For any newly uploaded image, an image analyzer and system monitoring components are activated using a pre-programmed listener module. A One-way ANOVA was used for image analysis online. Fruit flies were indicated with black areas (black pixels) in the images. Detection accuracy of 75% was achieved.
An imaging system in the strawberry greenhouse for the detection of thrips and classifying parasites was developed by Ebrahimi et al. (2017). They used LabVIEW to control a horizontal mobile agricultural robot with an 18MP Canon EOS M digital camera to capture the strawberry flower images. An SVM classier was used and an identification accuracy of greater than 97.5% was achieved.
Partel et al. (2019) developed an automatic pest detection and counting system for Asian citrus psyllid citrus crops. They used six cameras to capture psyllid pest’s images falling on a board fixed to a mobile vehicle which has a tapping unit to shake citrus tree branches. CNN was used to identify psyllids from captured images. Precision and recall of 80% and 95%, respectively were achieved.
Alternative methods of pests and disease detection for fruits and vegetables plant
Potamitis et al. (2017) developed a system for monitoring the entrance of pests to improve the operation of a low-cost McPhail trap. It was based on identifying the species of incoming pests from the optoacoustic spectrum analysis of their wingbeat. They achieved an accuracy of 91% for distinguishing fruit flies from other insects. However, it couldn’t distinguish between fruit fly species. To improve the system, Potamitis et al. (2018) developed a bimodal optoelectronic sensor that records the wingbeats of insects in flight with a Fresnel lens. This system had an accuracy of 98.99%.
Sounds of Red palm weevil were collected inside palm trees in different conditions (Dosunmu et al., 2014). They created a model which was based on the pattern and frequency of the emitted sounds. They concluded that the weevil could be differentiated based on burst duration, and detection of larvae was easier in offshoots than inside the trunk.
Rach et al. (2013) proposed a method to detect Red Palm Weevils (RPW) acoustically in the field. The proposed system was able to monitor and record acoustic emissions of the adult RPW using an audio probe with an acoustic sensor. The sound is then amplified and processed through an algorithm with a wireless interface to send detected signals remotely. Audio signals are first digitized and subsequent execution reveals the presence or absence of RPW in real-time.
Future projections and tasks
Machine vision is deployed in agriculture for several tasks. The choice of an appropriate machine vision system solely depends on the task at hand. In this paper, several studies from previous research about pests and disease detection have been reviewed, with alternative detection methods also presented. More machine vision algorithms will be required since, in different growth cycles, the appearance of diseases and pests will be different. For example, the manifestation of some pests and diseases symptoms is not obvious, limiting early diagnosis. Besides, even high-resolution images are difficult to analyze in the early stages of the pests and disease occurrence.. Moreover, occlusion of leaves, branches and light in the field is also a limitation that could lead to false/missed detection due to the lack of features and noise overlap.
Implementation of digital signal processing techniques and the use of acoustic sensors can supplement machine vision algorithms, specifically, for pests detection. This technique has been applied by various researchers and it has shown good potential over a wide range of pests. Also, it is necessary to combine meteorological data such as temperature and rainfall with plant protection data to recognize and predict pests and diseases efficiently. In addition, automatic recognition algorithms can improve the recognition rate and speed up real-time automation systems in the future. The algorithms need to be suitable for the automation of semi-structured environments, such as an agricultural environment. With more research focused on this domain given the advancement in image processing methods and development of low-cost hardware systems, there will be a greater benefit of machine vision systems in precision agriculture.
Acknowledgement
This work was supported by the Korea Institute of Planning and Evaluation for Technology in Food, Agriculture and Forestry (IPET) through the Agriculture, Food and Rural Affairs Convergence Technologies Program for Educating Creative Global Leaders, funded by Ministry of Agriculture, Food and Rural Affairs (MAFRA) (Project No.: 320001-4), Republic of Korea.