Enhancing AI Binocular Vision for Camellia oleifera Fruit Localization Using Zero-Shot Lightweight Segmentation Model and 3D Surface Optimization

Shouxiang Jin; Lei Zhou; Linyun Xu; Minghong Shi; Hongping Zhou

doi:10.1016/j.eng.2026.06.020

Engineering ›› :202606020 DOI: 10.1016/j.eng.2026.06.020

research-article

Enhancing AI Binocular Vision for Camellia oleifera Fruit Localization Using Zero-Shot Lightweight Segmentation Model and 3D Surface Optimization

Shouxiang Jin ^a
, Lei Zhou ^a^,^b^,^*
, Linyun Xu ^a^,^b
, Minghong Shi ^a^,^b
, Hongping Zhou ^a^,^b^,^*

Author information +

History +

PDF (12337KB)

Abstract

The segmentation and three-dimensional (3D) localization accuracy of Camellia oleifera fruits by harvesting robots are critical factors influencing their success rate. To address the challenges of high annotation costs and significant localization errors in fruit segmentation and localization, an artificial intelligence (AI)-based binocular vision method for Camellia oleifera fruit localization is proposed using zero-shot lightweight segmentation and 3D surface optimization models. First, the large vision model Grounding DINO generates the bounding boxes of fruit objects using the text prompt “fruit”. These bounding boxes were provided as prompts to the segment anything model (SAM) to obtain segmentation masks, eliminating the need for manual annotation. To meet the real-time application requirements of embedded devices, a knowledge transfer method is proposed. The segmentation results of the large models were used as labels to train YOLO11n-seg, a considerably small model without

Keywords

Zero-shot learning / Large vision model / Instance segmentation / Camellia oleifera fruit / Fruit location

Cite this article

Download citation ▾

Shouxiang Jin, Lei Zhou, Linyun Xu, Minghong Shi, Hongping Zhou. Enhancing AI Binocular Vision for Camellia oleifera Fruit Localization Using Zero-Shot Lightweight Segmentation Model and 3D Surface Optimization. Engineering 202606020 DOI:10.1016/j.eng.2026.06.020

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	Zhang F, Zhu F, Chen B, Su E, Chen Y, Cao F . Composition, bioactive substances, extraction technologies and the influences on characteristics of Camellia oleifera oil: a review. Food Res Int 2022; 156:111159.

[2]	Qin P, Shen J, Wei J, Chen Y . A critical review of the bioactive ingredients and biological functions of Camellia oleifera oil. Curr Res Food Sci 2024; 8:100753.

[3]	Wang R, Fang D, Wu C, Wang B, Zhu H, Hu T, et al. Dynamic response of Camellia oleifera fruit—branch based on mathematical model and high—speed photography. Biosyst Eng 2024; 237:232—41.

[4]	Qu H, Du H, Tang X, Zhai S . Citrus fruit diameter estimation in the field using monocular camera. Biosyst Eng 2025; 252:47-60.

[5]	Yi W, Xia S, Kuzmin S, Gerasimov I, Cheng X . RTFVE—YOLOv9: real—time fruit volume estimation model integrating YOLOv9 and binocular stereo vision. Comput Electron Agric 2025; 236:110401.

[6]	Lin P, Yang H, Cheng S, Guo F, Wang L, Lin Y . An improved YOLOv5s method based bruises detection on apples using cold excitation thermal images. Postharvest Biol Technol 2023; 199:112280.

[7]	Zheng C, Abd—Elrahman A, Whitaker VM, Dalid C . Deep learning for strawberry canopy delineation and biomass prediction from high—resolution images. Plant Phenomics 2022; 2022:9850486.

[8]	Tsoulias N, Saha KK, Zude—Sasse M . In—situ fruit analysis by means of LiDAR 3D point cloud of normalized difference vegetation index (NDVI). Comput Electron Agric 2023; 205:107611.

[9]	Gao X, Zan X, Yang S, Zhang R, Chen S, Zhang X, et al. Maize seedling information extraction from UAV images based on semi—automatic sample generation and Mask R—CNN model. Eur J Agron 2023; 147:126845.

[10]	Wang D, He D . Channel pruned YOLO V5s—based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning. Biosyst Eng 2021; 210:271-81.

[11]	Hou G, Chen H, Niu R, Li T, Ma Y, Zhang Y . Research on multi—layer model attitude recognition and picking strategy of small tomato picking robot. Comput Electron Agric 2025; 232:110125.

[12]	Liang Y, Jiang W, Liu Y, Wu Z, Zheng R . Picking—point localization algorithm for citrus fruits based on improved YOLOv8 model. Agriculture 2025; 15(3):237.

[13]	Bilal H, Kumar Singh S, Alnajim AM, Islam M, Almusharraf A, Cascone L, et al. Optimized attention assisted network for forensic—aware malware detection in consumer electronics. IEEE Trans Consum Electron 2025; 71(4):11113-20.

[14]	Ahmed F, Zhou T, Bilal H, Ul Islam F, Ullah R, Vasilakos AV . Enhancing healthcare data integrity and access control using blockchain and Industry 5.0. IEEE Internet Things J 2025; 12(20):43630-43.

[15]	Torres—Lomas E, Lado—Bega J, Garcia—Zamora G, Diaz—Garcia L . Segment anything for comprehensive analysis of grapevine cluster architecture and berry properties. Plant Phenomics 2024; 6:0202.

[16]

Giménez—Gallego J, Martinez—del—Rincon J, González—Teruel JD, Navarro—Hellín H, Navarro PJ, Torres—Sánchez R . On—tree fruit image segmentation comparing Mask R—CNN and vision transformer models. Application in a novel algorithm for pixel—based fruit size estimation. Comput Electron Agric 2024; 222:109077.

[17]	Pan J, Xia L, Wu Q, Guo Y, Chen Y, Tian X . Automatic strawberry leaf scorch severity estimation via faster R—CNN and few—shot learning. Ecol Inform 2022; 70:101706.

[18]	Zhou W, Cui Y, Huang H, Huang H, Wang C . A fast and data—efficient deep learning framework for multi—class fruit blossom detection. Comput Electron Agric 2024; 217:108592.

[19]	Denarda AR, Crocetti F, Costante G, Valigi P, Fravolini ML . MangoDetNet: a novel label—efficient weakly supervised fruit detection framework. Precis Agric 2024; 25(6):3167-88.

[20]	Devanna RP, Reina G, Cheein FA, Milella A . Boosting grape bunch detection in RGB—D images using zero—shot annotation with Segment Anything and GroundingDINO. Comput Electron Agric 2025; 229:109611.

[21]	Zhang P, Sun X, Zhang D, Yang Y, Wang Z . Lightweight deep learning models for high—precision rice seedling segmentation from UAV—based multispectral images. Plant Phenomics 2023; 5:0123.

[22]	Bilal H, Muhammad Y, Ullah I, Garg S, Choi BJ, Hassan MM . Identification and diagnosis of chronic heart disease: a deep learning—based hybrid approach. Alex Eng J 2025; 124:470-83.

[23]	Bilal H, Obaidat MS, Shamrooz Aslam M, Zhang J, Yin B, Mahmood K . Online fault diagnosis of industrial robot using IoRT and hybrid deep learning techniques: an experimental approach. IEEE Internet Things J 2024; 11(19):31422—37.

[24]	Gu Z, He D, Huang J, Chen J, Wu X, Huang B, et al. Simultaneous detection of fruits and fruiting stems in mango using improved YOLOv8 model deployed by edge device. Comput Electron Agric 2024; 227:109512.

[25]	Huang Z, Zhang X, Wang H, Wei H, Zhang Y, Zhou G . Pear fruit detection model in natural environment based on lightweight transformer architecture. Agriculture 2024; 15(1):24.

[26]	Zhu K, Li J, Zhang K, Arunachalam C, Bhattacharya S, Lu R, et al. Foundation model—based apple ripeness and size estimation for selective harvesting. Comput Electron Agric 2025; 236:110407.

[27]	Ven J, Sharma C, Syed A . MangoSense: a time—series vision sensing dataset for mango tree segmentation and detection toward yield prediction. Comput Electron Agric 2025; 237:110524.

[28]	Niu J, Bi M, Yu Q . Apple pose estimation based on SCH—YOLO11s segmentation. Agronomy 2025; 15(4):900.

[29]	Li J, Liu Y, Li C, Luo Q, Lu J . Pineapple detection with YOLOv7—Tiny network model improved via pruning and a lightweight backbone sub—network. Remote Sens 2024; 16(15):2805.

[30]	Shaikh AM, Zhao Y, Kumar A, Ali M, Kang Y . Efficient Bayesian CNN model compression using bayes by backprop and L1—norm regularization. Neural Process Lett 2024; 56(2):140.

[31]	Kumar A, Shaikh AM, Li Y, Bilal H, Yin B . Pruning filters with L1—norm and capped L1—norm for CNN compression. Appl Intell 2021; 51(2):1152-60.

[32]	Cao Q, Zhang K, He X, Shen J . Be an excellent student: review, preview, and correction. IEEE Signal Process Lett 2023; 30:1722-6.

[33]	Huang Q, Wu X, Wang Q, Dong X, Qin Y, Wu X, et al. Knowledge distillation facilitates the lightweight and efficient plant diseases detection model. Plant Phenomics 2023; 5:0062.

[34]	Gong Z, Gao C, Feng Z, Dong P, Qiao H, Zhang H, et al. Integrating masked generative distillation and network compression to identify the severity of wheat fusarium head blight. Comput Electron Agric 2024; 227:109647.

[35]	Zhang G, Geng L, Xie F, He C, Liu Y . Acoustic imaging method based on local—nonlocal attention and masked generative knowledge distillation. Mech Syst Signal Process 2026; 242:113597.

[36]	Liu X, Jing X, Jiang H, Younas S, Wei R, Dang H, et al. Performance evaluation of newly released cameras for fruit detection and localization in complex kiwifruit orchard environments. J Field Robot 2024; 41(4):881-94.

[37]	Zhou L, Jin S, Wang J, Zhang H, Shi M, Zhou H . 3D positioning of Camellia oleifera fruit—grabbing points for robotic harvesting. Biosyst Eng 2024; 246:110-21.

[38]	Duan J, Lai L, Yang Z, Luo Z, Yuan H . Multi—feature language—image model for fruit quality image classification. Comput Electron Agric 2024; 227:109462.

[39]	Li Y, Chao X . Semi—supervised few—shot learning approach for plant diseases recognition. Plant Methods 2021; 17(1):68.

[40]	Wang S, Chen W, Xie SM, Azzari G, Lobell DB . Weakly supervised deep learning for segmentation of remote sensing imagery. Remote Sens 2020; 12(2):207.

[41]	Cheng Y, Lan S, Fan X, Tjahjadi T, Jin S, Cao L . A dual—branch weakly supervised learning based network for accurate mapping of woody vegetation from remote sensing images. Int J Appl Earth Obs Geoinf 2023; 124:103499.

[42]	Bao QZ, Yang YX, Li Q, Yang HC . Zero—shot instance segmentation for plant phenotyping in vertical farming with foundation models and VC—NMS. Front Plant Sci 2025; 16:1536226.

[43]	Jin S, Zhou L, Zhou H . CO—YOLO: a lightweight and efficient model forCamellia oleifera fruit object detection and posture determination. Comput Electron Agric 2025; 235:110394.

[44]	Li Z, Deng X, Lan Y, Liu C, Qing J . Fruit tree canopy segmentation from UAV orthophoto maps based on a lightweight improved U—Net. Comput Electron Agric 2024; 217:108538.

[45]	Shen Q, Zhang X, Shen M, Xu D . Multi—scale adaptive YOLO for instance segmentation of grape pedicels. Comput Electron Agric 2025; 229:109712.

[46]	Wang J, Lin X, Luo L, Chen M, Wei H, Xu L, et al. Cognition of grape cluster picking point based on visual knowledge distillation in complex vineyard environment. Comput Electron Agric 2024; 225:109216.