Overview of Recent Deep Learning Methods Applied in Fruit Counting for Yield Estimation
Abstract
This paper is an overview of the latest advancements of image recognition for fruit counting and yield estimation. Considering this domain is developing rapidly, we have considered the cutting-edge literature in the field, for the last 5 years, focused on the task of yield estimation by detecting and counting fruit in the tree canopy. This is a much more complex task than the classification of fruit post-harvesting, which has been more widely reviewed. Moreover, we identify the major challenges and propose the next steps for advancing this research field.
References
[2] Chen, S. W., Shivakumar, S. S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C. J., and Kumar, V. Counting apples and oranges with deep learning: A data-driven approach. IEEE Robotics and Automation Letters 2, 2 (2017), 781–788.
[3] Fourie, J., Hsaio, J., and Werner, A. Crop yield estimation using deep learning. In 7th Asian-Australasian Conference on Precision Agriculture (2017), pp. 1–10.
[4] Kamilaris, A., and Prenafeta-Bold´u, F. X. Deep learning in agriculture: A survey. Computers and electronics in agriculture 147 (2018), 70–90.
[5] Kang, H., and Chen, C. Fast implementation of real-time fruit detection in apple orchards using deep learning. Computers and Electronics in Agriculture 168 (2020), 105108.
[6] Koirala, A., Walsh, K. B., Wang, Z., and McCarthy, C. Deep learning–method overview and review of use for fruit detection and yield estimation. Computers and Electronics in Agriculture 162 (2019), 219–234.
[7] Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (May 2017), 84–90.
[8] Lavania, S., and Matey, P. S. Novel method for weed classification in maize field using otsu and pca implementation. In 2015 IEEE International Conference on Computational Intelligence Communication Technology (2 2015), pp. 534–537.
[9] Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (6 1998), 2278–2324.
[10] Lin, T., Doll´ar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (7 2017), pp. 936–944.
[11] Mao, S., Li, Y., Ma, Y., Zhang, B., Zhou, J., and Wang, K. Automatic cucumber recognition algorithm for harvesting robots in the natural environment using deep learning and multi-feature fusion. Computers and Electronics in Agriculture 170 (2020), 105254.
[12] Nist´er, D., and Stew´enius, H. Linear time maximally stable extremal regions. In Computer Vision – ECCV 2008 (Berlin, Heidelberg, 2008), D. Forsyth, P. Torr, and A. Zisserman, Eds., Springer Berlin Heidelberg, pp. 183–196.
[13] Rahnemoonfar, M., and Sheppard, C. Real-time yield estimation based on deep learning. In Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping II (2017), J. A. Thomasson, M. McKee, and R. J. Moorhead, Eds., vol. 10218, International Society for Optics and Photonics, SPIE, pp. 59 – 65.
[14] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition challenge. International journal of computer vision 115, 3 (2015), 211–252.
[15] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (2018), pp. 4510–4520.
[16] Simonyan, K., and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014).
[17] Sun, Y. Iterative relief for feature weighting: Algorithms, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 6 (6 2007), 1035–1051.
[18] Szegedy, C., Ioffe, S., and Vanhoucke, V. Inception-v4, inception-resnet and the impact of residual connections on learning. CoRR abs/1602.07261 (2016).
[19] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 2818–2826.
[20] Wang, L., and Duan, H.-c. Application of otsu’method in multi-threshold image segmentation [j]. Computer Engineering and Design 11 (2008), 2844–2845.
[21] Xiang, Q., Wang, X., Li, R., Zhang, G., Lai, J., and Hu, Q. Fruit image classification based on mobilenetv2 with transfer learning technique. In Proceedings of the 3rd International Conference on Computer Science and Application Engineering (2019), pp. 1–7.
[22] Yang, R., Wu, M., Bao, Z., and Zhang, P. Cherry recognition based on color channel transform. In Proceedings of the 2019 International Conference on Artificial Intelligence and Computer Science (2019), pp. 292–296.
[23] Yu, H., Song, S., Ma, S., and Sinnott, R. O. Estimating fruit crop yield through deep learning. In Proceedings of the 6th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (2019), pp. 145–148.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
When the article is accepted for publication, I, as the author and representative of the coauthors, hereby agree to transfer to Studia Universitatis Babes-Bolyai, Series Informatica, all rights, including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author specifically retain: the right to make further copies of all or part of the published article for my use in classroom teaching; the right to reuse all or part of this material in a review or in a textbook of which I am the author; the right to make copies of the published work for internal distribution within the institution that employs me.
