面向增强现实的虚实遮挡技术综述

吴宇晖; 李晓娟; 刘越

doi:10.11834/jig.240045

混合现实 | 浏览量 : 0 下载量: 18 CSCD: 0

PDF
导出
分享
收藏
专辑

面向增强现实的虚实遮挡技术综述
Virtual-real occlusion handling technologies in augmented reality
2024年29卷第10期页码：2859-2879
纸质出版日期： 2024-10-16 ，
DOI： 10.11834/jig.240045
稿件说明：

移动端阅览

吴宇晖，李晓娟，刘越. 2024. 面向增强现实的虚实遮挡技术综述. 中国图象图形学报， 29(10):2859-2879

Wu Yuhui， Li Xiaojuan， Liu Yue. 2024. Virtual-real occlusion handling technologies in augmented reality. Journal of Image and Graphics， 29(10):2859-2879
吴宇晖，李晓娟，刘越. 2024. 面向增强现实的虚实遮挡技术综述. 中国图象图形学报， 29(10):2859-2879 DOI： 10.11834/jig.240045.

Wu Yuhui， Li Xiaojuan， Liu Yue. 2024. Virtual-real occlusion handling technologies in augmented reality. Journal of Image and Graphics， 29(10):2859-2879 DOI： 10.11834/jig.240045.

摘要

随着软件技术的快速发展以及硬件设备的不断更新，增强现实技术已逐步成熟并广泛应用于各个领域。在增强现实中，虚实遮挡处理是实现虚拟世界和真实世界无缝融合的前提，对提升用户的沉浸感和真实感具有重要的研究意义。该技术通过建立尽可能精确的虚实物体遮挡关系以呈现逼真的虚实融合效果，使得用户能够正确地感知虚拟物体和真实物体的空间位置关系，从而提升交互体验。本文首先介绍了虚实遮挡的相关背景、概念和总体处理流程。然后针对刚性物体和非刚性物体的不同特点，总结了现有的基于深度、基于图像分析和基于模型3类虚实遮挡处理方法的具体原理、代表性研究工作以及它们对刚性物体和非刚性物体的适用性。在此基础上，从实时性、自动化程度、是否支持动态场景及适用范围等多个角度对现有的虚实遮挡方法进行了对比分析，并归纳了3类虚实遮挡处理方法的具体流程、难点以及局限性。最后针对相关工作中存在的问题，提出了目前虚实遮挡技术所面临的挑战以及未来可能的研究方向，希望能为后续的研究工作提供参考。

Abstract

With the rapid development of software technology and the continuous updating of hardware devices， augmented reality technology has gradually matured and been widely used in various fields， such as military， medical， gaming， industry， and education. Accurate depth perception is crucial in augmented reality， and simply overlaying virtual objects onto video sequences no longer meets user demands. In many augmented reality scenarios， users need to interact with virtual objects constantly， and without accurate depth perception， augmented reality can hardly provide a seamless interactive experience. Virtual-real occlusion handling is one of the key factors to achieve this goal. It presents a realistic virtual-real fusion effect by establishing accurate occlusion relationship， so that the fusion scene can correctly reflect the spatial position relationship between virtual and real objects， thereby enhancing the user’s sense of immersion and realism. This paper first introduces the related background， concepts， and overall processing flow of virtual-real occlusion handling. Existing occlusion handling methods can be divided into three categories： depth based， image analysis based， and model based. By analyzing the distinct characteristics of rigid and nonrigid objects， we summarize the specific principles， representative research works， and the applicability to rigid and nonrigid objects of these three virtual-real occlusion handling methods. The shape and size of rigid objects remain unchanged after motion or force， and they mainly use two types virtual-real occlusion handling methods： depth based and model based. The depth-based methods have evolved from the early use of stereo vision algorithms to the use of depth sensors for indoor depth image acquisition and further to the prediction of moving objects’ depth by using outdoor map data， as well as the densification of sparse simultaneous localization and mapping depth in monocular mobile augmented reality. Further research should focus on the depth image restoration algorithms and the balance between real-time performance and accuracy of scene-dense depth computation algorithms in mobile augmented reality. The model-based methods have developed from constructing partial 3D models by segmenting object contours in video key frames or directly using modeling software to achieving dense reconstruction of indoor static scenes using depth images and constructing approximate 3D models of outdoor scenes by incorporating geographic spatial information. Model-based methods already have a relative well-established processing flow， but further exploration is still needed on how to enhance real-time performance while ensuring tracking and occlusion accuracy. In contrast to rigid objects， nonrigid objects are prone to irregular deformations during movement. Typical nonrigid objects in augmented reality are user’s hands or the bodies of other users. For nonrigid objects， related research has been conducted on all three types virtual-real occlusion handling methods. Depth-based methods focus on the depth image restoration algorithms. These algorithms aim to repair depth image noise while ensuring precise alignment between depth and RGB image， especially in extreme scenarios， such as when foreground and background have similar colors. Image analysis-based methods focus on foreground segmentation algorithms and occlusion relationship judgment means. Foreground segmentation algorithms have evolved from the early color models and background subtraction techniques to the deep learning-based segmentation networks. Moreover， the occlusion relationship judgment means have transitioned from user-specified to incorporating depth information to assist judgment. The key challenge in image analysis-based methods lies in overcoming the irregular deformations of nonrigid objects， obtaining accurate foreground segmentation masks and tracking continuously. Model-based methods initially used LeapMotion combined with customized hand parameters to fit hand model， but now using deep learning networks to reconstruct hand models has become mainstream. Model-based methods should improve the speed and accuracy of hand reconstruction. On the basis of summarizing the virtual-real occlusion handling methods for rigid and nonrigid objects， we also conduct a comparative analysis of existing methods from various perspectives including real-time performance， automation level， whether to support perspective or scene changes， and application scope. In addition， we summarize the specific workflows， difficulties and limitations of the three virtual-real occlusion handling methods. Finally， aiming at the problems existing in related research， we explore the challenges faced by current virtual-real occlusion technology and propose potential future research directions： 1） Occlusion handling for moving nonrigid objects. Obtaining accurate depth or 3D models of nonrigid objects is the key to solving this problem. The accuracy and robustness of hand segmentation must be further improved. Additionally， the use of simpler monocular depth estimation and rapid reconstruction of nonrigid objects other than user’s hands need to be further explored. 2） Occlusion handling for outdoor dynamic scenes. Existing depth cameras have limited working range， which makes them ineffective in outdoor scenes. Sparse 3D models obtained from geographic information systems have low precision and cannot be applied to dynamic objects， such as automobiles. Therefore， further research on dynamic objects’ virtual-real occlusion handling in large outdoor scenes is needed. 3） Registration algorithms for depth and RGB images. The accuracy of edge alignment between depth and color images must be improved without consuming too much computing resources.

关键词

增强现实虚实遮挡刚体及非刚体深度图修复前景提取

Keywords

augmented realityvirtual-real occlusion handlingrigid and non-rigid bodiesdepth image restorationforeground extraction

references

Abate A F， Narducci F and Ricciardi S. 2014. An image based approach to hand occlusions in mixed reality environments//Proceedings of the 6th International Conference on Virtual， Augmented and Mixed Reality. Heraklion， Greece： Springer： 319-328 ［DOI： 10.1007/978-3-319-07458-0_30http://dx.doi.org/10.1007/978-3-319-07458-0_30］

Azuma R T. 1997. A survey of augmented reality. Presence： Teleoperators and Virtual Environments， 6（4）： 355-385 ［DOI： 10.1162/pres.1997.6.4.355http://dx.doi.org/10.1162/pres.1997.6.4.355］

Barron J T and Poole B. 2016. The fast bilateral solver//Proceedings of the 14th European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 617-632 ［DOI： 10.1007/978-3-319-46487-9_38http://dx.doi.org/10.1007/978-3-319-46487-9_38］

Bartczak B， Schiller I， Beder C and Koch R. 2008. Integration of a time-of-flight camera into a mixed reality system for handling dynamic scenes， moving viewpoints and occlusions in real-time//Proceedings of the 4th International Symposium on 3D Data Processing， Visualization and Transmission. Atlanta， USA： IEEE： 155-162

Battisti C， Messelodi S and Poiesi F. 2018. Seamless bare-hand interaction in mixed reality//Proceedings of 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct. Munich， Germany： IEEE： 198-203 ［DOI： 10.1109/ISMAR-Adjunct.2018.00066http://dx.doi.org/10.1109/ISMAR-Adjunct.2018.00066］

Behzadan A H and Kamat V R. 2010. Scalable algorithm for resolving incorrect occlusion in dynamic augmented reality engineering environments. Computer-Aided Civil and Infrastructure Engineering， 25（1）： 3-19 ［DOI： 10.1111/j.1467-8667.2009.00601.xhttp://dx.doi.org/10.1111/j.1467-8667.2009.00601.x］

Berger M O. 1997. Resolving occlusion in augmented reality： a contour based approach without 3D reconstruction//Proceedings of 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Juan， USA： IEEE： 91-96 ［DOI： 10.1109/CVPR.1997.609304http://dx.doi.org/10.1109/CVPR.1997.609304］

Breen D E， Whitaker R T， Rose E and Tuceryan M. 1996. Interactive occlusion and automatic object placement for augmented reality. Computer Graphics Forum， 15（3）： 11-22 ［DOI： 10.1111/1467-8659.1530011http://dx.doi.org/10.1111/1467-8659.1530011］

Chen X Y， Liu Y F， Dong Y J， Zhang X， Ma C Y， Xiong Y M， Zhang Y and Guo X Y. 2022. MobRecon： mobile-friendly hand mesh reconstruction from monocular image//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans， USA： IEEE： 20544-20554 ［DOI： 10.1109/CVPR52688.2022.01989http://dx.doi.org/10.1109/CVPR52688.2022.01989］

Cruz L， Lucio D and Velho L. 2012. Kinect and RGBD images： challenges and applications//Proceedings of the 25th SIBGRAPI Conference on Graphics， Patterns and Images Tutorials. Ouro Preto， Brazil： IEEE： 36-49 ［DOI： 10.1109/SIBGRAPI-T.2012.13http://dx.doi.org/10.1109/SIBGRAPI-T.2012.13］

Du C， Chen Y L， Ye M and Ren L. 2016. Edge snapping-based depth enhancement for dynamic occlusion handling in augmented reality//Proceedings of 2016 IEEE International Symposium on Mixed and Augmented Reality. Merida， Mexico： IEEE： 54-62 ［DOI： 10.1109/ISMAR.2016.17http://dx.doi.org/10.1109/ISMAR.2016.17］

Duchesne C and Herve J Y. 2000. A point-based approach to the interposition problem in augmented reality//Proceedings of the 15th International Conference on Pattern Recognition. Barcelona， Spain： IEEE： 261-265 ［DOI： 10.1109/ICPR.2000.905315http://dx.doi.org/10.1109/ICPR.2000.905315］

Engel J， Koltun V and Cremers D. 2018. Direct sparse odometry. IEEE Transactions on Pattern Analysis and Machine Intelligence， 40（3）： 611-625 ［DOI： 10.1109/TPAMI.2017.2658577http://dx.doi.org/10.1109/TPAMI.2017.2658577］

Feng Q， Shum H P H and Morishima S. 2018. Resolving occlusion for 3D object manipulation with hands in mixed reality//Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology. Tokyo， Japan： ACM： #119 ［DOI： 10.1145/3281505.3283390http://dx.doi.org/10.1145/3281505.3283390］

Feng Q， Shum H P H and Morishima S. 2020. Resolving hand-object occlusion for mixed reality with joint deep learning and model optimization. Computer Animation and Virtual Worlds， 31（4-5）： #e1956 ［DOI： 10.1002/cav.1956http://dx.doi.org/10.1002/cav.1956］

Figueiredo L， Dos Anjos R， Lindoso J， Neto E， Roberto R， Silva M and Teichrieb V. 2013. Bare hand natural interaction with augmented objects//Proceedings of 2013 IEEE International Symposium on Mixed and Augmented Reality. Adelaide， Australia： IEEE： 1-6 ［DOI： 10.1109/ISMAR.2013.6671836http://dx.doi.org/10.1109/ISMAR.2013.6671836］

Fischer J， Bartz D and Straßer W. 2004. Occlusion handling for medical augmented reality using a volumetric phantom model//Proceedings of the 11th ACM Symposium on Virtual Reality Software and Technology. Hong Kong， China： ACM： 174-177 ［DOI： 10.1145/1077534.1077570http://dx.doi.org/10.1145/1077534.1077570］

Fischer J， Huhle B and Schilling A. 2007. Using time-of-flight range data for occlusion handling in augmented reality//Proceedings of the 13th Eurographics conference on Virtual Environments. Weimar， Germany： Eurographics Association： 109-116 ［DOI： 10.2312/EGVE/IPT_EGVE2007/109-116http://dx.doi.org/10.2312/EGVE/IPT_EGVE2007/109-116］

Gimeno J， Morillo P， Orduña J M and Fernandez M. 2012. An occlusion-aware AR authoring tool for assembly and repair tasks//Proceedings of 2012 International Conference on Computer Graphics Theory and Applications and International Conference on Information Visualization Theory and Applications. Rome， Italy： SciTePress： 377-386 ［DOI： 10.5220/0003843303770386http://dx.doi.org/10.5220/0003843303770386］

Hafiz A M and Bhat G M. 2020. A survey on instance segmentation： state of the art. International Journal of Multimedia Information Retrieval， 9（3）： 171-189 ［DOI： 10.1007/s13735-020-00195-xhttp://dx.doi.org/10.1007/s13735-020-00195-x］

Hayashi K， Kato H and Nishida S. 2005. Occlusion detection of real objects using contour based stereo matching//Proceedings of 2005 International Conference on Augmented Tele-existence. New York， USA： ACM： 180-186 ［DOI： 10.1145/1152399.1152432http://dx.doi.org/10.1145/1152399.1152432］

He K M， Sun J and Tang X O. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence， 35（6）： 1397-1409 ［DOI： 10.1109/TPAMI.2012.213http://dx.doi.org/10.1109/TPAMI.2012.213］

Hebborn A K， Höhner N and Müller S. 2017. Occlusion matting： realistic occlusion handling for augmented reality applications//Proceedings of 2017 IEEE International Symposium on Mixed and Augmented Reality. Nantes， France： IEEE： 62-71 ［DOI： 10.1109/ISMAR.2017.23http://dx.doi.org/10.1109/ISMAR.2017.23］

Holynski A and Kopf J. 2018. Fast depth densification for occlusion-aware augmented reality. ACM Transactions on Graphics， 37（6）： #194 ［DOI： 10.1145/3272127.3275083http://dx.doi.org/10.1145/3272127.3275083］

Hosni A， Rhemann C， Bleyer M， Rother C and Gelautz M. 2013. Fast cost-volume filtering for visual correspondence and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence， 35（2）： 504-511 ［DOI： 10.1109/TPAMI.2012.156http://dx.doi.org/10.1109/TPAMI.2012.156］

Izadi S， Kim D， Hilliges O， Molyneaux D， Newcombe R A， Kohli P， Shotton J， Hodges S， Freeman D， Davison A J and Fitzgibbon A W. 2011. KinectFusion： real-time 3D reconstruction and interaction using a moving depth camera//Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. Santa Barbara， USA： ACM： 559-568 ［DOI： 10.1145/2047196.2047270http://dx.doi.org/10.1145/2047196.2047270］

Kanbara M， Okuma T， Takemura H and Yokoya N. 1999. Real-time composition of stereo images for video see-through augmented reality//Proceedings of 1999 IEEE International Conference on Multimedia Computing and Systems. Florence， Italy： IEEE： 213-219 ［DOI： 10.1109/MMCS.1999.779195http://dx.doi.org/10.1109/MMCS.1999.779195］

Kanbara M， Okuma T， Takemura H and Yokoya N. 2000. A stereoscopic video see-through augmented reality system based on real-time vision-based registration//Proceedings of 2000 IEEEVirtual RealityCat. No.00CB37048. New Brunswick， USA： IEEE： 255-262 ［DOI： 10.1109/VR.2000.840506http://dx.doi.org/10.1109/VR.2000.840506］

Kasapakis V and Gavalas D. 2015. Determining field of view in outdoors augmented reality applications//Proceedings of the 12th European Conference on Ambient Intelligence. Athens， Greece： Springer： 344-348 ［DOI： 10.1007/978-3-319-26005-1_23http://dx.doi.org/10.1007/978-3-319-26005-1_23］

Kasapakis V and Gavalas D. 2017. Occlusion handling in outdoors augmented reality games. Multimedia Tools and Applications， 76（7）： 9829-9854 ［DOI： 10.1007/s11042-016-3581-1http://dx.doi.org/10.1007/s11042-016-3581-1］

Kasperi J， Edwardsson M P and Romero M. 2017. Occlusion in outdoor augmented reality using geospatial building data//Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. Gothenburg， Sweden： ACM： #30 ［DOI： 10.1145/3139131.3139159http://dx.doi.org/10.1145/3139131.3139159］

Kilimann J E， Heitkamp D and Lensing P. 2019. An augmented reality application for mobile visualization of GIS-referenced landscape planning projects//Proceedings of the 17th International Conference on Virtual-Reality Continuum and Its Applications in Industry. Brisbane， Australia： ACM： #23 ［DOI： 10.1145/3359997.3365712http://dx.doi.org/10.1145/3359997.3365712］

Kim H， Yang S J and Sohn K. 2003. 3D reconstruction of stereo images for interaction between real and virtual worlds//Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality， 2003. Proceedings. Tokyo， Japan： IEEE： 169-176 ［DOI： 10.1109/ISMAR.2003.1240700http://dx.doi.org/10.1109/ISMAR.2003.1240700］

Kirillov A， Wu Y X， He K M and Girshick R. 2020. PointRend： image segmentation as rendering//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle， USA： IEEE： 9799-9808 ［DOI： 10.1109/CVPR42600.2020.00982http://dx.doi.org/10.1109/CVPR42600.2020.00982］

Klein G and Drummond T. 2004. Sensor fusion and occlusion refinement for tablet-based AR//Proceedings of the 3rd IEEE and ACM International Symposium on Mixed and Augmented Reality. Arlington， USA： IEEE： 38-47 ［DOI： 10.1109/ISMAR.2004.54http://dx.doi.org/10.1109/ISMAR.2004.54］

Kolb A， Barth E， Koch R and Larsen R. 2010. Time-of-flight cameras in computer graphics. Computer Graphics Forum， 29（1）： 141-159 ［DOI： 10.1111/j.1467-8659.2009.01583.xhttp://dx.doi.org/10.1111/j.1467-8659.2009.01583.x］

Kroeger T， Timofte R， Dai D X and Gool L V. 2016. Fast optical flow using dense inverse search//Proceedings of the 14th European Conference on Computer Vision. Amsterdam， the Netherlands： Springer： 471-488 ［DOI： 10.1007/978-3-319-46493-0_29http://dx.doi.org/10.1007/978-3-319-46493-0_29］

Leal-Meléndrez J A， Altamirano-Robles L and Gonzalez J A. 2013. Occlusion handling in video-based augmented reality using the kinect sensor for indoor registration//Proceedings of the 18th Iberoamerican Congress on Pattern Recognition. Havana， Cuba： Springer： 447-454 ［DOI： 10.1007/978-3-642-41827-3_56http://dx.doi.org/10.1007/978-3-642-41827-3_56］

Lepetit V and Berger M O. 2000. Handling occlusion in augmented reality systems： a semi-automatic method// Proceedings of 2000 IEEE and ACM International Symposium on Augmented Reality （ISAR 2000）. Munich， Germany： IEEE： 137-146 ［DOI： 10.1109/ISAR.2000.880937http://dx.doi.org/10.1109/ISAR.2000.880937］

Li F， Zheng J B， Zhang Y F， Liu N and Jia W J. 2021. AMDFNet： adaptive multi-level deformable fusion network for RGB-D saliency detection. Neurocomputing， 465： 141-156 ［DOI： 10.1016/j.neucom.2021.08.116http://dx.doi.org/10.1016/j.neucom.2021.08.116］

Long J， Shelhamer E and Darrell T. 2015. Fully convolutional networks for semantic segmentation//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston， USA： IEEE： 3431-3440 ［DOI： 10.1109/CVPR.2015.7298965http://dx.doi.org/10.1109/CVPR.2015.7298965］

Luo T R， Liu Z H， Pan Z G and Zhang M M. 2019. A virtual-real occlusion method based on GPU acceleration for MR//Proceedings of 2019 IEEE Conference on Virtual Reality and 3D User Interfaces. Osaka， Japan： IEEE： 1068-1069 ［DOI： 10.1109/VR.2019.8797811http://dx.doi.org/10.1109/VR.2019.8797811］

Luo T R， Zhang M M， Pan Z G， Li Z， Cai N， Miao J D， Chen Y B and Xu M X. 2020. Dream-experiment： a MR user interface with natural multi-channel interaction for virtual experiments. IEEE Transactions on Visualization and Computer Graphics， 26（12）： 3524-3534 ［DOI： 10.1109/TVCG.2020.3023602http://dx.doi.org/10.1109/TVCG.2020.3023602］

Macedo M C F and Apolinario A L. 2023. Occlusion handling in augmented reality： past， present and future. IEEE Transactions on Visualization and Computer Graphics， 29（2）： 1590-1609 ［DOI： 10.1109/TVCG.2021.3117866http://dx.doi.org/10.1109/TVCG.2021.3117866］

Nie P. 2013. Survey on occlusion handling in augmented reality. Digital Communication， 40（5）： 34-37

聂平. 2013. 增强现实中虚实遮挡技术的研究现状. 数字通信， 40（5）： 34-37 ［DOI： 10.3969/j.issn.1001-3824.2013.05.009http://dx.doi.org/10.3969/j.issn.1001-3824.2013.05.009］

Nirkin Y， Wolf L and Hassner T. 2021. HyperSeg： patch-wise hypernetwork for real-time semantic segmentation//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville， USA： IEEE： 4061-4070 ［DOI： 10.1109/CVPR46437.2021.00405http://dx.doi.org/10.1109/CVPR46437.2021.00405］

Ogawa T and Mashita T. 2021. Occlusion handling in outdoor augmented reality using a combination of map data and instance segmentation//Proceedings of 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct. Bari， Italy： IEEE： 246-250 ［DOI： 10.1109/ISMAR-Adjunct54149.2021.00057http://dx.doi.org/10.1109/ISMAR-Adjunct54149.2021.00057］

Olshevsky V， Bondarets I， Trunov O and Shcherbina A. 2021. Realistic occlusion of virtual objects using three-dimensional hand model//Proceedings of the 23rd International Conference on Human-Computer Interaction. Virtual Event： Springer： 295-301 ［DOI： 10.1007/978-3-030-78642-7_40http://dx.doi.org/10.1007/978-3-030-78642-7_40］

Ong K C， Teh H C and Tan T S. 1998. Resolving occlusion in image sequence made easy. The Visual Computer， 14（4）： 153-165 ［DOI： 10.1007/s003710050131http://dx.doi.org/10.1007/s003710050131］

Sanches S R R， Tokunaga D M， Silva V F， Sementille A C and Tori R. 2012. Mutual occlusion between real and virtual elements in augmented reality based on fiducial markers//Proceedings of 2012 IEEE Workshop on the Applications of Computer Vision. Breckenridge， USA： IEEE： 49-54 ［DOI： 10.1109/WACV.2012.6163037http://dx.doi.org/10.1109/WACV.2012.6163037］

Schmidt J， Niemann H and Vogt S. 2002. Dense disparity maps in real-time with an application to augmented reality//Proceedings of the 6th IEEE Workshop on Applications of Computer Vision， 2002. （WACV 2002）. Proceedings. Orlando， USA： IEEE： 225-230 ［DOI： 10.1109/ACV.2002.1182186http://dx.doi.org/10.1109/ACV.2002.1182186］

Shen Y， Ong S K and Nee A Y C. 2011. Vision-based hand interaction in augmented reality environment. International Journal of Human-Computer Interaction， 27（6）： 523-544 ［DOI： 10.1080/10447318.2011.555297http://dx.doi.org/10.1080/10447318.2011.555297］

Sizintsev M， Mithun N C， Chiu H P， Samarasekera S and Kumar R. 2021. Long-range augmented reality with dynamic occlusion rendering. IEEE Transactions on Visualization and Computer Graphics， 27（11）： 4236-4244 ［DOI： 10.1109/TVCG.2021.3106434http://dx.doi.org/10.1109/TVCG.2021.3106434］

Song W， Zhu M F， Zhang M H， Zhao D F and He Q. 2022. A review of monocular depth estimation techniques based on deep learning. Journal of Image and Graphics， 27（2）： 292-328

宋巍，朱孟飞，张明华，赵丹枫，贺琪. 2022. 基于深度学习的单目深度估计技术综述. 中国图象图形学报， 27（2）： 292-328 ［DOI： 10.11834/jig.210554http://dx.doi.org/10.11834/jig.210554］

Sun J， Zheng N N and Shum H Y. 2003. Stereo matching using belief propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence， 25（7）： 787-800 ［DOI： 10.1109/TPAMI.2003.1206509http://dx.doi.org/10.1109/TPAMI.2003.1206509］

Tang X， Hu X W， Fu C W and Cohen-Or D. 2020. GrabAR： occlusion-aware grabbing virtual objects in AR//Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. Virtual Event， USA： ACM： 697-708 ［DOI： 10.1145/3379337.3415835http://dx.doi.org/10.1145/3379337.3415835］

Tang X， Wang T Y and Fu C W. 2021. Towards accurate alignment in real-time 3D hand-mesh reconstruction//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal， Canada： IEEE： 11698-11707 ［DOI： 10.1109/ICCV48922.2021.01149http://dx.doi.org/10.1109/ICCV48922.2021.01149］

Tian Y， Guan T and Wang C. 2010. Real-time occlusion handling in augmented reality based on an object tracking approach. Sensors， 10（4）： 2885-2900 ［DOI： 10.3390/s100402885http://dx.doi.org/10.3390/s100402885］

Tian Y， Long Y， Xia D， Yao H and Zhang J C. 2015. Handling occlusions in augmented reality based on 3D reconstruction method. Neurocomputing， 156： 96-104 ［DOI： 10.1016/j.neucom.2014.12.081http://dx.doi.org/10.1016/j.neucom.2014.12.081］

Tian Y， Zhou X L， Wang X F， Wang Z F and Yao H. 2021. Registration and occlusion handling based on the FAST ICP-ORB method for augmented reality systems. Multimedia Tools and Applications， 80（14）： 21041-21058 ［DOI： 10.1007/s11042-020-10342-5http://dx.doi.org/10.1007/s11042-020-10342-5］

Ullman S. 1979. The interpretation of structure from motion. Proceedings of the Royal Society B： Biological Sciences， 203（1153）： 405-426 ［DOI： 10.1098/rspb.1979.0006http://dx.doi.org/10.1098/rspb.1979.0006］

Valentin J， Kowdle A， Barron J T， Wadhwa Neal， Dzitsiuk M， Schoenberg M， Verma V， Csaszar A， Turner E， Dryanovski I， Afonso J， Pascoal J， Tsotsos K， Leung M， Schmidt M， Guleryuz O， Khamis S， Tankovitch V， Fanello S R， Izadi S and Rhemann C. 2018. Depth from motion for smartphone AR. ACM Transactions on Graphics， 37（6）： #194 ［DOI： 10.1145/3272127.3275041http://dx.doi.org/10.1145/3272127.3275041］

Walton D R and Steed A. 2017. Accurate real-time occlusion for mixed reality//Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. Gothenburg， Sweden： ACM： #11 ［DOI： 10.1145/3139131.3139153http://dx.doi.org/10.1145/3139131.3139153］

Wang H， Zhou Y， Ma J T and Liu X P. 2008. Research on the real-virtual occlusion in an optical see-through AR system. Journal of Image and Graphics， 13（8）： 1566-1569

王红，周雅，马晋涛，刘宪鹏. 2008. 光学透视式增强现实显示系统虚实遮挡问题研究. 中国图象图形学报， 13（8）： 1566-1569 ［DOI： 10.11834/jig.20080830http://dx.doi.org/10.11834/jig.20080830］

Wang H L， Sengupta K， Kumar P and Sharma R. 2005. Occlusion handling in augmented reality using background-foreground segmentation and projective geometry. Presence： Teleoperators and Virtual Environments， 14（3）： 264-277 ［DOI： 10.1162/105474605323384636http://dx.doi.org/10.1162/105474605323384636］

Wloka M M and Anderson B G. 1995. Resolving occlusion in augmented reality//Proceedings of 1995 Symposium on Interactive 3D Graphics. Monterey， USA： ACM： 5-12 ［DOI： 10.1145/199404.199405http://dx.doi.org/10.1145/199404.199405］

Woo S， Park J， Lee J Y and Kweon I S. 2018. CBAM： convolutional block attention module//Proceedings of the 15th European Conference on Computer Vision. Munich， Germany： Springer： 3-19 ［DOI： 10.1007/978-3-030-01234-2_1http://dx.doi.org/10.1007/978-3-030-01234-2_1］

Wu Y H， Liu Y and Wang J J. 2023. Real-time hand-object occlusion for augmented reality using hand segmentation and depth correction//Proceedings of 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops. Shanghai， China： IEEE： 631-632 ［DOI： 10.1109/VRW58643.2023.00158http://dx.doi.org/10.1109/VRW58643.2023.00158］

Xu W P， Wang Y T， Liu Y and Weng D D. 2013. Survey on occlusion handling in augmented reality. Journal of Computer-Aided Design and Computer Graphics， 25（11）： 1635-1642

徐维鹏，王涌天，刘越，翁冬冬. 2013. 增强现实中的虚实遮挡处理综述. 计算机辅助设计与图形学学报， 25（11）： 1635-1642 ［DOI： 10.3969/j.issn.1003-9775.2013.11.005http://dx.doi.org/10.3969/j.issn.1003-9775.2013.11.005］

Zhang Z. 2000. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence， 22（11）： 1330-1334 ［DOI： 10.1109/34.888718http://dx.doi.org/10.1109/34.888718］

Zheng Y. 2014. Review and prospects of occlusion handling among virtual and real objects in augmented reality. Journal of System Simulation， 26（1）： 1-10

郑毅. 2014. 增强现实虚实遮挡方法评述与展望. 系统仿真学报， 26（1）： 1-10 ［DOI： 10.16182/j.cnki.joss.2014.01.025http://dx.doi.org/10.16182/j.cnki.joss.2014.01.025］

Zhou T， Dong Y L， Huo B Q， Liu S and Ma Z J. 2021. U-Net and its applications in medical image segmentation： a review. Journal of Image and Graphics， 26（9）： 2058-2077

周涛，董雅丽，霍兵强，刘珊，马宗军. 2021. U-Net网络医学图像分割应用综述. 中国图象图形学报， 26（9）： 2058-2077 ［DOI： 10.11834/jig.200704http://dx.doi.org/10.11834/jig.200704］

Zhu J J and Pan Z G. 2008. Occlusion registration in video-based augmented reality//Proceedings of the 7th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. Singapore， Singapore： ACM： #10 ［DOI： 10.1145/1477862.1477875http://dx.doi.org/10.1145/1477862.1477875］

Zhu J J， Pan Z G， Sun C and Chen W Z. 2010. Handling occlusions in video-based augmented reality using depth information. Computer Animation and Virtual Worlds， 21（5）： 509-521 ［DOI： 10.1002/cav.326http://dx.doi.org/10.1002/cav.326］

Zollmann S and Reitmayr G. 2012. Dense depth maps from sparse models and image coherence for augmented reality//Proceedings of the 18th ACM Symposium on Virtual Reality Software and Technology. Toronto， Canada： ACM： 53-60 ［DOI： 10.1145/2407336.2407347http://dx.doi.org/10.1145/2407336.2407347］

文章被引用时，请邮件提醒。

提交