[1]许鑫,耿庆,郑凯,等.基于纹理特征与深度学习的小麦图像中的穗粒分割与计数[J].江苏农业学报,2024,(04):661-674.[doi:doi:10.3969/j.issn.1000-4440.2024.04.010]
　XU Xin,GENG Qing,ZHENG Kai,et al.Segmentation and counting of wheat spikes and grains based on texture features and deep learning[J].,2024,(04):661-674.[doi:doi:10.3969/j.issn.1000-4440.2024.04.010]
点击复制

基于纹理特征与深度学习的小麦图像中的穗粒分割与计数()
分享到：

江苏农业学报[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2024年04期
页码:
661-674
栏目:
农业信息工程
出版日期:
2024-04-30

文章信息/Info

Title:
Segmentation and counting of wheat spikes and grains based on texture features and deep learning
作者:
许鑫1; 耿庆1; 郑凯2; 石磊2; 马新明1; 3
(1.河南农业大学信息与管理科学学院,河南郑州450002;2.国家统计局河南调查总队,河南郑州450002；3.河南农业大学农学院,河南郑州450002)
Author(s):
XU Xin1; GENG Qing1; ZHENG Kai2; SHI Lei2; MA Xin-ming1; 3
(1.College of Information and Management Science, Henan Agricultural University, Zhengzhou 450002, China；2.National Bureau of Statistics Henan Survey Corps, Zhengzhou 450002, China;3.College of Agronomy, Henan Agricultural University, Zhengzhou 450002, China)
关键词:
麦穗; 籽粒数; 图像处理; HRNet; 深度学习; 图像分割
Keywords:
wheat ear; grain number; image processing; HRNet; deep learning; image segmentation
分类号:
TP391；S512.1
DOI:
doi:10.3969/j.issn.1000-4440.2024.04.010
摘要:
穗粒数是小麦产量构成的重要因素和估测产量的参数之一,传统的人工计数方法耗时费力,人为因素影响大。为了实现对小麦穗粒数的智能、快速监测,以百农307、新麦26、稷麦336这3个小麦品种为试验材料,利用智能手机于小麦灌浆后期拍摄麦穗图像,随后对麦穗图像进行预处理并归一化为480×480像素大小,结合深度学习和迁移学习机制,构建基于冻结-解冻机制的HRNet模型的小麦小穗图像分割计数深度学习模型,利用图像处理算法、小麦小穗图像纹理特征确定小穗像素数与穗粒数之间的关系阈值,构建小穗粒数预测模型,实现对小麦穗粒的预测计数。结果表明,对比同样采用冻结-解冻机制的PSPNet模型、DeeplabV3+分割模型、U-Net模型及无冻结解冻机制的HRNet模型,采用基于冻结-解冻机制的HRNet模型对小麦小穗的分割效果更优,且具有更好的鲁棒性,分割精确度为0.959 4,平均交并比(mIoU)为0.911 9,类别平均像素准确率(mPA)为0.941 9,召回率为0.941 9；通过3个不同品种小麦的麦穗图像对小穗进行计数,所得决定系数(R2)为0.92,平均绝对误差为0.73,平均相对误差为2.89%；籽粒计数的R2为0.92,平均绝对误差为0.43,平均相对误差为5.51%。由研究结果可知,基于冻结-解冻机制得出的HRNet模型的小麦小穗图像分割算法能够有效分割小麦图像中的小穗,并获得更加丰富的语义信息,可用于解决小目标图像分割困难及训练欠拟合问题,通过粒数预测模型可以快速、精确地对小麦的籽粒数进行预测,从而为小麦高效、智能化估产提供算法支撑。
Abstract:
The number of grains per ear is an important factor in the composition of wheat yield and one of the parameters for estimating wheat yield. The traditional manual counting method is time-consuming and labor-intensive, and human factors have a great influence. In order to realize the intelligent and rapid monitoring of the number of grains per ear, three varieties of Bainong 307, Xinmai 26 and Jimai 336 were used as test materials, and the wheat ear images were taken with a smart phone at the late stage of wheat grain filling. Based on the image processing technology, the wheat ear images were preprocessed and normalized to 480×480 pixels. Combining deep learning and transfer learning mechanisms, a HRNet wheat spikelet segmentation and counting deep learning model based on the freeze-thaw mechanism was constructed. Image processing algorithms and wheat spikelet texture features were used to determine the threshold of the relationship between the number of spikelet pixels and the number of grains per spike. The spikelet-grain number prediction model was constructed to realize the prediction and counting of wheat spikes. The results showed that compared with PSPNet, DeeplabV3 + segmentation model, U-Net which also used freeze-thaw mechanism and HRNet without freeze-thaw mechanism, the HRNet model based on the freeze-thaw mechanism had a better segmentation effect on wheat spikelets, and had better robustness. The segmentation accuracy was 0.959 4, the mean intersection over union (mIoU) was 0.911 9, the mean pixel accuracy (mPA) was 0.941 9, and the recall rate was 0.941 9. The spikelets were counted by the images of three different wheat varieties. The determination coefficient (R2) was 0.92, the average absolute error was 0.73, and the average relative error was 2.89%. The R2 of grain counting was 0.92, the average absolute error was 0.43, and the average relative error was 5.51%. It shows that the HRNet wheat spikelet image segmentation algorithm through the freeze-thaw mechanism can effectively segment wheat spikelets and obtain richer semantic information, which can be used to solve the problems of difficult segmentation of small target images and training underfitting. The model can quickly and accurately predict the number of wheat grains, so as to provide algorithm support for efficient and intelligent yield estimation of wheat.

参考文献/References:

[1]张领先,陈运强,李云霞,等. 基于卷积神经网络的冬小麦麦穗检测计数系统[J]. 农业机械学报,2019,50(3):144-150.
[2]LI Q Y, CAI J, BERGER B, et al. Detecting spikes of wheat plants using neural networks with laws texture energy[J]. Plant Methods,2017,13(1):1-13.
[3]HU W J, FAN J, DU Y X, et al. MDFC-ResNet: an agricultural IoT system to accurately recognize crop diseases[J]. IEEE Access,2020,8:115287-115298.
[4]MISRA T, ARORA A, MARWAHA S, et al. SpikeSegNet-a deep learning approach utilizing encoder-decoder network with hourglass for spike segmentation and counting in wheat plant from visual imaging[J]. Plant Methods,2020,16(1):1-20.
[5]赵三琴,李毅念,丁为民,等. 稻穗结构图像特征与籽粒数相关关系分析[J]. 农业机械学报,2014,45(12):323-328.
[6]王栋,陈佳玮,沈利言,等. 基于图像的水稻穗粒相关性状智能检测算法研究[J]. 植物生理学报,2022,58(5):957-971.
[7]杜世伟,李毅念,姚敏,等. 基于小麦穗部小穗图像分割的籽粒计数方法[J]. 南京农业大学学报,2018,41(4):742-751.
[8]QIU R, HE Y, ZHANG M. Automatic detection and counting of wheat spikelet using semi-automatic labeling and deep learning[J]. Frontiers in Plant Science,2022,13:872555.
[9]FERNANDEZ-GALLEGO J A, KEFAUVER S C, GUTIRREZ N A, et al. Wheat ear counting in-field conditions: high throughput and low-cost approach using RGB images[J]. Plant Methods,2018,14(1):1-12.
[10]刘哲,黄文准,王利平. 基于改进K-means聚类算法的大田麦穗自动计数[J]. 农业工程学报,2019,35(3):174-181.
[11]XU X, LI H Y, YIN F, et al. Wheat ear counting using K-means clustering segmentation and convolutional neural network[J]. Plant Methods,2020,16:106.
[12]李毅念,杜世伟,姚敏,等. 基于小麦群体图像的田间麦穗计数及产量预测方法[J]. 农业工程学报,2018,34(21):185-194.
[13]WEI W, YANG T L, RUI L, et al. Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales[J]. Journal of Integrative Agriculture,2020,19(8):1998-2008.
[14]HU G, QIAN L, LIANG D, et al. Self-adversarial training and attention for multi-task wheat phenotyping[J]. Applied Engineering in Agriculture,2019,35(6):1009-1014.
[15]DANDRIFOSSE S, ENNADIFI E, CARLIER A, et al. Deep learning for wheat ear segmentation and ear density measurement: from heading to maturity[J]. Computers and Electronics in Agriculture,2022,199:107161.
[16]ZHAO J, ZHANG X, YAN J, et al. A wheat spike detection method in UAV images based on improved YOLOv5[J]. Remote Sensing,2021,13(16):3095.
[17]MISRA T, ARORA A, MARWAHA S, et al. Web-SpikeSegNet: deep learning framework for recognition and counting of spikes from visual images of wheat plants[J]. IEEE Access,2021,9:76235-76247.
[18]LI J, LI C, FEI S, et al. Wheat ear recognition based on RetinaNet and transfer learning[J]. Sensors,2021,21(14):4845. DOI:10.3390/s21144845.
[19]段凌凤,熊雄,刘谦,等. 基于深度全卷积神经网络的大田稻穗分割[J]. 农业工程学报,2018,34(12):202-209.
[20]WANG J D, SUN K, CHENG T H, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(10):3349-3364.
[21]ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]. Honolulu, HI, USA:IEEE,2017:2881-2890.
[22]CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. Berlin:Springer,2018:801-818.
[23]邱云飞,温金燕. 基于DeepLabV3+与注意力机制相结合的图像语义分割[J]. 激光与光电子学进展,2022,59(4):130-139.
[24]马冬梅,李鹏辉,黄欣悦,等. 改进DeepLabV3+的高效语义分割[J]. 计算机工程与科学,2022,44(4):737-745.
[25]RONNEBERGER O, FISCHER P, BROX T. U-net:convolutional networks for biomedical image segmentation[C]. Berlin:Springer International Publishing,2015:234-241.
[26]梁波,卢军,曹阳. 基于改进U-Net卷积神经网络的钢轨表面损伤检测方法[J]. 激光与光电子学进展,2021,58(2):334-340.
[27]MA J C, LI Y X, DU K M, et al. Segmenting ears of winter wheat at flowering stage using digital images and deep learning[J]. Computers and Electronics in Agriculture,2020,168:105159.
[28]ALHARBI N, ZHOU J, WANG W. Automatic counting of wheat spikes from wheat growth images[C]. Funchal, Madeira, Portugal:SciTePress,2018:346-355.
[29]FERNANDEZ-GALLEGO J A, KEFAUVER S C, GUTIRREZ N A, et al. Wheat ear counting in-field conditions: high throughput and low-cost approach using RGB images[J]. Plant Methods,2018,14(1):22.
[30]KHAKI S, SAFAEI N, PHAM H, et al. Wheatnet: a lightweight convolutional neural network for high-throughput image-based wheat head detection and counting[J]. Neurocomputing,2022,489:78-89.
[31]WANG Y, QIN Y, CUI J. Occlusion robust wheat ear counting algorithm based on deep learning[J]. Frontiers in Plant Science,2021,12:645899. DOI:10.3389/fpls.2021.645899.
[32]MAJI A K, MARWAHA S, KUMAR S, et al. SlypNet:spikelet-based yield prediction of wheat using advanced plant phenotyping and computer vision techniques[J]. Frontiers in Plant Science,2022,13:889853.
[33]ALKHUDAYDI T, ZHOU J, DE LA LGLESIA B. Counting spikelets from infield wheat crop images using fully convolutional networks[J]. Neural Computing and Applications,2022,34(20):17539-17560.
[34]XU C, JIANG H Y, YUEN P, et al. MHW-PD: a robust rice panicles counting algorithm based on deep learning and multi-scale hybrid window[J]. Computers and Electronics in Agriculture,2020,173:105375.
[35]HASAN M M, CHOPIN J P, LAGA H, et al. Detection and analysis of wheat spikes using convolutional neural networks[J]. Plant Methods,2018,14(1):1-13.
[36]SADEGHI-TEHRAN P, VIRLET N, AMPE E M, et al. DeepCount: in-field automatic quantification of wheat spikes using simple linear iterative clustering and deep convolutional neural networks[J]. Frontiers in Plant Science,2019,10:1176.

备注/Memo

备注/Memo:
收稿日期：2023-02-27基金项目：国家统计局重大统计专项(2022ZX11)；河南省科技创新杰出人才项目(184200510008)；河南省现代农业产业技术体系项目(S2010-01-G04)作者简介：许鑫(1984-),男,河南邓州人,博士,副教授,主要从事智慧农业与大数据技术方面的研究。(E-mail)xuxin468@163.com通讯作者：马新明,(E-mail) wheatdoctor@163.com
更新日期/Last Update: 2024-05-22