[1]王皓,曹静,胡佳楠,等.基于SfM-MVS的作物三维重建：挑战与创新[J].江苏农业学报,2024,(09):1768-1776.[doi:doi:10.3969/j.issn.1000-4440.2024.09.021]
　WANG Hao,CAO Jing,HU Jianan,et al.Three-dimensional crop reconstruction based on structure from motion-multiple view stereo (SfM-MVS): challenges and innovations[J].,2024,(09):1768-1776.[doi:doi:10.3969/j.issn.1000-4440.2024.09.021]
点击复制

基于SfM-MVS的作物三维重建：挑战与创新()
分享到：

江苏农业学报[ISSN:1006-6977/CN:61-1281/TN]

卷:
期数:
2024年09期
页码:
1768-1776
栏目:
综述
出版日期:
2024-09-30

文章信息/Info

Title:
Three-dimensional crop reconstruction based on structure from motion-multiple view stereo (SfM-MVS): challenges and innovations
作者:
王皓1; 2; 曹静2; 胡佳楠2; 潘嫄嫄2; 毕然2; 孙挺2; 张文宇1; 2
(1.江苏大学农业工程学院,江苏镇江212013；2.江苏省农业科学院无锡分院,江苏无锡214174)
Author(s):
WANG Hao1; 2; CAO Jing2; HU Jianan2; PAN Yuanyuan2; BI Ran2; SUN Ting2; ZHANG Wenyu1; 2
(1.College of Agricultural Engineering, Jiangsu University, Zhenjiang 212013, China;2.Jiangsu Academy of Agricultural Sciences Wuxi Branch, Wuxi 214174, China)
关键词:
三维重建; 运动恢复结构; 多视角立体视觉; 作物; 表型
Keywords:
three-dimensional reconstruction; structure from motion; multiple view stereo; crop; phenotype
分类号:
S126
DOI:
doi:10.3969/j.issn.1000-4440.2024.09.021
文献标志码:
A
摘要:
三维重建技术可提供更全面、准确的作物表型信息,已成为智慧农业和智慧育种领域的研究热点。基于多目视觉法的三维重建技术因其低成本、高精度的特性而被广泛应用于作物三维重建,其中运动恢复结构-多视角立体视觉(SfM-MVS)流程是其主流实现方法。本文对不断发展的SfM-MVS流程进行了技术体系梳理,并从当前作物三维重建的角度出发,总结了SfM-MVS所面临的挑战和已有创新之处。本研究可为基于SfM-MVS的作物三维重建技术集成提供有力支撑,并为关键技术突破提供重要参考。
Abstract:
Three-dimensional (3D) reconstruction technology can provide more comprehensive and accurate crop phenotypic information, which is a research hotspot in the field of intelligent farming and intelligent breeding. The 3D reconstruction technology based on multi-view vision is widely adopted for its low cost and high accuracy. The structure from motion-multiple view stereo (SfM-MVS) pipeline is the primary method used for this technology. In this paper, the technical system of the evolving SfM-MVS process was sorted out, and the challenges and innovations of SfM-MVS were summarized from the perspective of current crop three-dimensional reconstruction. This study can provide robust support for the integration of crop 3D reconstruction technology based on SfM-MVS, and provide an important reference for key technology breakthroughs.

参考文献/References:

[1]张建华,姚琼,周国民,等. 作物农艺性状与形态结构表型智能识别技术综述[J]. 智慧农业(中英文),2024,6(2):14-27.
[2]周静静,郭新宇,吴升,等. 基于多视角图像的植物三维重建研究进展[J]. 中国农业科技导报,2019,21(2):9-18.
[3]EBRAHIMNEZHAD H, GHASSEMIAN H. Robust motion from space curves and 3D reconstruction from multiviews using perpendicular double stereo rigs[J]. Image and Vision Computing,2008,26(10):1397-1420.
[4]ISGRO F, ODONE F, VERRI A. An open system for 3D data acquisition from multiple sensor[C]//Institute of Electrical and Electronics Engineers. Proceedings of the Seventh International Workshop on Computer Architectures for Machine Perception,July 4-6,2005. Palermo,Italy：IEEE,2005:120-125.
[5]GOEBEL W, KAMPA B M, HELMCHEN F. Imaging cellular network dynamics in three dimensions using fast 3D laser scanning[J]. Nature Methods,2007,4(1):73-79.
[6]ROCCHINI C, CIGNONI P, MONTANI C, et al. A low cost 3D scanner based on structured light[J]. Computer Graphics Forum,2001,20(3):299-308.
[7]PARK J, KIM H, TAI Y W, et al. High quality depth map upsampling for 3D-TOF cameras[C]//Institute of Electrical and Electronics Engineers. Proceedings of the IEEE International Conference on Computer Vision,November 06-13,2011. Barcelona,Spain：IEEE,2011:1623-1630.
[8]KHOSHELHAM K, ELBERINK S O. Accuracy and resolution of kinect depth data for indoor mapping applications[J]. Sensors,2012,12(2):1437-1454.
[9]张文明,刘彬,李海滨. 基于双目视觉的三维重建中特征点提取及匹配算法的研究[J]. 光学技术,2008,34(2):181-185.
[10]BAILLARD C, ZISSERMAN A. A plane-sweep strategy for the 3D reconstruction of buildings from multiple images[C]//International Society for Photogrammetry and Remote Sensing. Proceedings of the International Archives of Photogrammetry and Remote Sensing,1999. Amsterdam,Netherlands：ISPRS,2000:56-62.
[11]PARK H, LEE H, SULL S. Efficient viewer-centric depth adjustment based on virtual fronto-parallel planar projection in stereo 3D images[J]. Ieee Transactions on Multimedia,2014,16(2):326-336.
[12]WU S, WEN W L, WANG Y J, et al. MVS-Pheno:a portable and low-cost phenotyping platform for maize shoots using multiview stereo 3D reconstruction[J]. Plant Phenomics,2020,2020:1848437.
[13]WU S, WEN W, GOU W, et al. A miniaturized phenotyping platform for individual plants using multi-view stereo 3D reconstruction[J]. Frontiers in Plant Science,2022,13:897746.
[14]DAS CHOUDHURY S, MATURU S, SAMAL A, et al. Leveraging image analysis to compute 3D plant phenotypes based on voxel-grid plant reconstruction[J]. Frontiers in Plant Science,2020,11:521431.
[15]LI Y L, WEN W L, MIAO T, et al. Automatic organ-level point cloud segmentation of maize shoots by integrating high-throughput data acquisition and deep learning[J]. Computers and Electronics in Agriculture,2022,193:106702.
[16]LI Y C, LIU J Y, ZHANG B, et al. Three-dimensional reconstruction and phenotype measurement of maize seedlings based on multi-view image sequences[J]. Frontiers in Plant Science,2022,13:974339.
[17]LIU H, XIN C, LAI M Z, et al. RepC-MVSNet:a reparameterized self-supervised 3D reconstruction algorithm for wheat 3D reconstruction[J]. Agronomy-Basel,2023,13(8):13081975.
[18]SANDHU J, ZHU F Y, PAUL P, et al. PI-Plat:a high-resolution image-based 3D reconstruction method to estimate growth dynamics of rice inflorescence traits[J]. Plant Methods,2019,15(1):162.
[19]TROSS M C, GAILLARD M, ZWEINER M, et al. 3D reconstruction identifies loci linked to variation in angle of individual sorghum leaves[J]. PeerJ,2021,9:12628.
[20]GAILLARD M, MIAO C Y, SCHNABLE J C, et al. Voxel carving-based 3D reconstruction of sorghum identifies genetic determinants of light interception efficiency[J]. Plant Direct,2020,4(10). DOI:10.1002/pld3.255.
[21]SUN Y Z, ZHANG Z X, SUN K, et al. Soybean-MVS:annotated three-dimensional model dataset of whole growth period soybeans for 3D plant organ segmentation[J]. Agriculture-Basel,2023,13(7):13071321.
[22]HE W, YE Z H, LI M S, et al. Extraction of soybean plant trait parameters based on SfM-MVS algorithm combined with GRNN[J]. Frontiers in Plant Science,2023,14:1181322.
[23]ZHU R S, SUN K, YAN Z Z, et al. Analysing the phenotype development of soybean plants using low-cost 3D reconstruction[J]. Scientific Reports,2020,10(1):7055.
[24]SALTER W T, SHRESTHA A, BARBOUR M M. Open source 3D phenotyping of chickpea plant architecture across plant development[J]. Plant Methods,2021,17(1):95.
[25]WANG L Y, MIAO Y L, HAN Y X, et al. Extraction of 3D distribution of potato plant CWSI based on thermal infrared image and binocular stereovision system[J]. Frontiers in Plant Science,2023,13:1104390.
[26]XIAO S F, CHAI H H, WANG Q, et al. Estimating economic benefit of sugar beet based on three-dimensional computer vision:a case study in Inner Mongolia, China[J]. Eur J Agron, 2021,130:126378.
[27]WANG Y H, HU S T, REN H, et al. 3DPhenoMVS:a low-cost 3D tomato phenotyping pipeline using 3D reconstruction point cloud based on multiview images[J]. Agronomy-Basel,2022,12(8):12081865.
[28]YU L, SUSSMAN H, KHMELNITSKY O, et al. Development of a mobile,high-throughput,and low-cost image-based plant growth phenotyping system[J]. Plant Physiology,2024. DOI：10.1093/plphys/kiae237.
[29]SZELISKI R. Computer vision:algorithms and applications[M]. Cham:Springer,2011.
[30]LOU L, LIU Y, HAN J, et al. Accurate multi-view stereo 3D reconstruction for cost-effective plant phenotyping[C]//International Association for Pattern Recognition. Proceedings of the 11th International Conference on Image Analysis and Recognition,October 22-24,2014. Porto:Springer,2014:467-474.
[31]SCHNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Institute of Electrical and Electronics Engineers. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,June 27-30,2016. Las Vegas：IEEE,2016:4104-4113.
[32]WU C. Towards linear-time incremental structure from motion[C]//Institute of Electrical and Electronics Engineers. Proceedings of the International Conference on 3D Vision. Seattle：IEEE,2013:127-134.
[33]GHERARDI R, FARENZENA M, FUSIELLO A. Improving the efficiency of hierarchical structure-and-motion[C]//Institute of Electrical and Electronics Engineers. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition,June 13-18,2010. San Francisco：IEEE,2010:1594-1600.
[34]SWEENEY C, SATTLER T, HLLERER T, et al. Optimizing the viewing graph for structure-from-motion[C]//Institute of Electrical and Electronics Engineers. Proceedings of the IEEE International Conference on Computer Vision,December 7-13,2015. Santiago,Chile：IEEE,2015:801-809.
[35]LHUILLIER M, QUAN L. A quasi-dense approach to surface reconstruction from uncalibrated images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2005,27(3):418-433.
[36]FURUKAWA Y, PONCE J. Accurate,dense,and robust multiview stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(8):1362-1376.
[37]YANG D N, YANG H J, LIU D F, et al. Research on automatic 3D reconstruction of plant phenotype based on multi-view images[J]. Computers and Electronics in Agriculture,2024,220. DOI:10.1016/j.compag.2024.108866.
[38]TIAN Y J, ZHANG J N, ZHANG Z J, et al. Research on super-resolution enhancement technology using improved transformer network and 3D reconstruction of wheat grains[J]. IEEE Access,2024,12:62882-62898.
[39]DETONE D, MALISIEWICZ T, RABINOVICH A, et al. Superpoint:self-supervised interest point detection and description[C]//Institute of Electrical and Electronics Engineers. CVF Conference on Computer Vision and Pattern Recognition,June 18-22,2018. Salt Lake City：IEEE,2018:224-233.
[40]SARLIN P E, DETONE D, MALISIEWICZ T, et al. SuperGlue:learning feature matching with graph neural networks[C]//Institute of Electrical and Electronics Engineers. CVF Conference on Computer Vision and Pattern Recognition,June 14-19,2020. Seattle：IEEE,2020:4938-4947.
[41]POTENA C, KHANNA R, NIETO J, et al. AgriColMap:aerial-ground collaborative 3D mapping for precision farming[J]. IEEE Robotics and Automation Letters,2019,4(2):1085-1092.
[42]GAO J, LIU J, JI S P. A general deep learning based framework for 3D reconstruction from multi-view stereo satellite images[J]. Isprs Journal of Photogrammetry and Remote Sensing,2023,195:446-461.
[43]LIU S, BONELLI W P, PIETRZYK P, et al. Comparison of open-source three-dimensional reconstruction pipelines for maize-root phenotyping[J]. The Plant Phenome Journal,2023,6(1):20068.
[44] XU B, WAN X Y, YANG H, et al. TIPS:a three-dimensional phenotypic measurement system for individual maize tassel based on TreeQSM[J]. Computers and Electronics in Agriculture,2023,212. DOI:10.1016/j.compag.2023.108150.
[45]ZIA A, LIANG J, ZHOU J, et al. 3D reconstruction from hyperspectral images[C]//Institute of Electrical and Electronics Engineers. Proceedings of the IEEE Winter Conference on Applications of Computer Vision,December 7-13,2015. Santiago,Chile：IEEE,2015:318-325.

备注/Memo

备注/Memo:
收稿日期：2024-07-09基金项目：国家重点研发计划子课题(2022YFD2001001、2022YFD2001005)；国家自然科学基金项目(32201664、31871522)；江苏省重点研发计划项目(BE2023323)；江苏省农业自主创新基金项目[CX(21)2042]；无锡市财政项目(33212303)作者简介：王皓(2000-),男,江苏徐州人,硕士研究生,研究方向为作物三维重建。(E-mail)454618365@qq.com通讯作者：孙挺,(E-mail)drsunting@163.com
更新日期/Last Update: 2024-11-17