[1]李颀,郭梦媛.基于深度学习的休眠期苹果树点云语义分割[J].江苏农业学报,2023,(05):1189-1198.[doi:doi:10.3969/j.issn.1000-4440.2023.05.011]
　LI Qi,GUO Meng-yuan.Semantic segmentation of apple tree point cloud in dormant period based on deep learning[J].,2023,(05):1189-1198.[doi:doi:10.3969/j.issn.1000-4440.2023.05.011]
点击复制

基于深度学习的休眠期苹果树点云语义分割()
分享到：

江苏农业学报[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2023年05期
页码:
1189-1198
栏目:
农业信息工程
出版日期:
2023-08-31

文章信息/Info

Title:
Semantic segmentation of apple tree point cloud in dormant period based on deep learning
作者:
李颀1; 郭梦媛2
(1.陕西科技大学电子信息与人工智能学院，陕西西安710021；2.陕西科技大学电气与控制工程学院，陕西西安710021)
Author(s):
LI Qi1; GUO Meng-yuan2
(1.School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an 710021, China;2.School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi’an 710021, China)
关键词:
多视角果树三维重建; 超点图; 深度学习; 果树点云语义分割
Keywords:
multi-view fruit tree three-dimensional reconstruction; super-point graph; deep learning; fruit tree point cloud semantic segmentation
分类号:
S661.1
DOI:
doi:10.3969/j.issn.1000-4440.2023.05.011
文献标志码:
A
摘要:
针对苹果树结构复杂、树干之间相互遮挡、导致国内外大规模机械设备自动剪枝误剪率高等问题，提出1种基于深度学习的休眠期苹果树点云的语义分割。以陕西省礼泉苹果种植基地的休眠期苹果树为研究对象，为了解决双视角点云配准之间非重叠点对距离过大导致配准误差大的问题，用 Kinect V2 传感器获取休眠期苹果树点云，对每株果树采用改进迭代最近点算法(Iterative closest point, ICP)进行多视角三维重建，对于大规模的果树点云，构建基于超点图的果树分割网络(Super point graphs network, SPGNet)，对果树点云进行语义分割，保留果树点云的复杂几何信息。结果表明，当果树双视角点云的配准误差小于1 mm时，可成功分割休眠期苹果树的树干与分枝，对分类精度、预测值与真实值的交并比(IoU)进行评估，其中树干的分类精度、IoU分别为94.0%、0.85，分枝的分类精度、IoU分别为83.1%、0.75。由此可见，研究结果可解决机械设备自动剪枝误剪率高的问题，能在自然光线条件、大规模休眠期苹果树场景下实现对休眠期苹果树树干与分枝的分割，为大规模自动剪枝提供依据。
Abstract:
Aiming at the problems of complex structure of apple trees, mutual occlusion between the trunks, and false pruning of large-scale mechanical equipment at home and abroad, a semantic segmentation of apple tree point cloud in dormant period based on deep learning was proposed. Taking the dormant apple trees in Liquan apple planting base in Shaanxi province as the research object, in order to solve the problem of large registration error caused by the large distance between non-overlapping point pairs in dual-view point cloud registration, the Kinect V2 sensor was used to obtain the point cloud of dormant apple trees, and the improved iterative closest point algorithm (ICP) was used for multi-view three-dimensional reconstruction of each fruit tree. For large-scale fruit tree point cloud data, a fruit tree segmentation network based on super point graphs (SPGNet) was constructed to perform semantic segmentation on fruit tree point cloud, and the complex geometric information of fruit tree point cloud was retained. The results showed that when the registration error of the dual-view point cloud of the fruit tree was less than 1 mm, the trunk and branches of the apple tree in the dormant period could be successfully segmented. The classification accuracy and intersection over union of predicted and true values (IoU) were evaluated. The classification accuracy and IoU of the trunk were 94.0% and 0.85, respectively, and the classification accuracy and IoU of the branches were 83.1% and 0.75, respectively. In a word, the research results could solve the problem of high mis-cutting rate in the process of automatic pruning of mechanical equipment, and could realize the segmentation of trunks and branches of dormant apple trees under natural light conditions and large-scale dormant apple tree scenes, and could provide a basis for large-scale automatic pruning.

参考文献/References:

[1]KARKEE M, ADHIKARI B, AMATYA S, et al. Identification of pruning branches in tall spindle apple trees for automated pruning[J]. Computers & Electronics in Agriculture, 2014, 103:127-135.
[2]MIKA A, BULER Z, TREDER W. Mechanical pruning of apple trees as an alternative to manual pruning[J]. Ogrodnictwo,2016,15(1):113-121.
[3]HE L, SCHUPP J. Sensing and automation in pruning of apple trees: a review[J]. Agronomy, 2018, 8(10):211.
[4]AMATYA S, KARKEE M, GONGAL A, et al. Detection of cherry tree branches with full foliage in planar architecture for automated sweet-cherry harvesting[J]. Biosystems Engineering, 2016, 146(4):3-15.
[5]HAN W K, CHEN C. Fruit detection and segmentation for apple harvesting using visual sensor in orchards[J]. Sensors, 2019, 19(20): 4599.
[6]MAJEED Y, ZHANG J, ZHANG X, et al. Deep learning based segmentation for automated training of apple trees on trellis wires[J]. Computers and Electronics in Agriculture, 2020, 170: 105277.
[7]ELFIKY N M, AKBAR S A, SUN J, et al. Automation of dormant pruning in specialty crop production: an adaptive framework for automatic reconstruction and modeling of apple trees[C]. Piscataway, NJ：IEEE Computer Society，2015: 65-73.
[8]MEDEIROS H, KIM D, SUN J, et al. Modeling dormant fruit trees for agricultural automation[J]. Journal of Field Robotics, 2017, 34(7):1203-1224.
[9]徐思雨,祝继华,田智强,等. 逐步求精的多视角点云配准方法[J].自动化学报,2019,45(8):1486-1494.
[10]张蕊,孟晓曼,曾志远,等. 图卷积神经网络在点云语义分割中的研究综述[J].计算机工程与应用, 2022, 58(24):29-46.
[11]LANDRIEU L, SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018：4558-4567.
[12]GUINARD S，LANDRIEU L. Weakly supervised segmentation-aided classification of urban scenes from 3d LiDAR point clouds[C]//International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2017：151-157.
[13]LI Y, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[C]. Ithaca, NY:Openreview. net,2016.
[14]CHO K, VAN MERRINBOER B, BOUGARES F, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]. Stroudsburg:ACL,2014.
[15]LI Y, BU R, SUN M, et al. Pointcnn: convolution on x-trans formed points[C]. Montreal Canada: Neural Information Processing System. Foundation, 2018.
[16]WANG Y, SUN Y B, LIU Z W, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2018, 38(5):1-12.

备注/Memo

备注/Memo:
收稿日期：2022-09-08 基金项目：陕西省农业科技创新工程项目
[201806117YF05NC13(1)]；陕西省科技厅农业科技攻关项目(2015NY028)；陕西科技大学博士科研启动基金项目(BJ13-15) 作者简介：李颀(1973-)，女，陕西西安人，博士，教授，主要研究方向为农业智能化、信息化、深度学习。(E-mail)liqidq@sust.edu.cn 通讯作者：郭梦媛，(E-mail)940516893@qq.com
更新日期/Last Update: 2023-09-13