First experience of using augmented reality technology in liver and pancreas laparoscopy

Aim. To describe the experience of using augmented reality system in abdominal surgery at one clinical center.

Materials and methods. In 2021–2022, five patients underwent laparoscopy with augmented reality technology. The interventions included echinococcectomy with resection of IV, V, VI liver segments, pancreaticoduodenal resection for pancreatic head cancer, excision of mesostenium cyst, resection of pancreas body and tail for neuroendocrine tumor.

Results. Application of 3D models requires putting on glasses, scaling and setting a model on the screen image, which sometimes prolonged surgery time to 25 minutes. In a number of operations the use of augmented reality navigated the surgeon when working near vascular structures. After looking through the AR model, a surgeon felt more confident in terms of individual anatomy.

Conclusion. Augmented reality can become a reliable and promising tool in abdominal surgery. However, further technological development in augmented reality systems is needed to increase their performance.

1. Wake N., Wysock J.S., Bjurlin M.A., et al. Pin the tumor on the kidney: an evaluation of how surgeons translate CT and MRI data to 3D models. Urology. 2019; 131: 255–61

2. Bernhard J.C., Isotani S., Matsugasumi T., et al. Personalized 3D printed model of kidney and tumor anatomy: a useful tool for patient education. World J Urol. 2016; 34: 337–45.

3. Silberstein J.L., Maddox M.M., Dorsey P., et al. Physical models of renal malignancies using standard cross-sectional imaging and 3-dimensional printers: a pilot study. Urology. 2014; 84: 268–72

4. Pensieri C, Pennacchini M, Sivan Y, Gefen D, Boulos MK, Claudio P, Maddalena P: Overview: Virtual reality in medicine. J Virtual Worlds Res 7:1–34, 2014

5. Healthcare Augmented & Virtual Reality Market Worth $5.1 Billion By 2025. https://www.grandviewresearch.com/press-release/global-augmented-reality-ar-virtual-reality-vrin-healthcare-market. Accessed January 2018

6. Rosen JM, Soltanian H, Redett RJ, Laub DR: Evolution of virtual reality. IEEE Eng Med Biol Mag 15:16–22, 1996

7. Heilig ML: Stereoscopic-television apparatus for individual use. US Patent 2:955–156, 1960

8. Tara J. Brigham (2017) Reality Check: Basics of Augmented, Virtual, and Mixed Reality, Medical Reference Services Quarterly, 36:2, 171-178, DOI:10.1080/02763869.2017.1293987

9. Microsoft. About HoloLens 2, 2021, https://docs.microsoft.com/en-us/hololens/hololens2-hardware (accessed September 23 2021)

10. Case Report: Looking through the Lens: The Reality of Telesurgical Support with Interactive Technology Using Microsoft’s HoloLens 2 K. van der Putten, M. B. Anderson, R. C. van Geenen, Case Reports in Orthopedics Volume 2022, Article ID 5766340, 5 pages https://doi.org/10.1155/2022/5766340 Received 6 December 2021; Revised 14 February 2022; Accepted 16 February 2022; Published 1 March 2022

11. Vavra P, Roman J, Zonca P, Ihnat P, Nemec M, Kumar J, Habib N, El-Gendi A (2017) Recent development of augmented reality in surgery: a review. J Healthc Eng. https://doi.org/10.1155/2017/4574172

12. Cutolo F, Parchi PD, Ferrari V (2014) Video see through AR head-mounted display for medical procedures. In: 2014 IEEE international symposium on mixed and augmented reality (ISMAR), 10–12 Sept 2014, pp 393–396. https://doi.org/10.1109/ismar.2014.6948504

13. Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C (2007) Action- and workfow-driven augmented reality for computeraided medical procedures. IEEE Comput Graph 27(5):10–14. https://doi.org/10.1109/Mcg.2007.117

14. Cutolo F Augmented reality in image-guided surgery. In: Lee N. Encyclopedia of computer graphics and games. Springer, Cham, pp 1–11. https://doi.org/10.1007/978-3-319-08234-9_78-1

15. Haouchine N, Dequidt J, Berger MO, Cotin S (2013) Deformationbased augmented reality for hepatic surgery. Stud Health Technol Inf 184:182–188

16. Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative usecases. Ann Surg. 2017;266:706–712.

17. Huber T, Hadzijusufovic E, Hansen C, et al. Head-mounted mixed-reality technology during robotic-assisted transanal total mesorectal excision. Dis Colon Rectum. 2019;62:258– 261.

18. Lang H, Radtke A, Hindennach M, et al. Impact of virtual tumor resection and computerassisted risk analysis on operation planning and intraoperative strategy in major hepatic resection. Arch Surg. 2005;140:629–638 [discussion 38

19. Monch J, Muhler K, Hansen C, et al. The LiverSurgeryTrainer: training of computer-based planning in liver resection surgery. Int J Comput Assist Radiol Surg. 2013;8:809–818.

20. Peterhans M, Oliveira T, Banz V, et al. Computerassisted liver surgery: clinical applications and technological trends. Crit Rev Biomed Eng. 2012;40:199–220

21. Hauke LangY Tobias Huber Department of General, Visceral and Transplant Surgery, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany. YHauke.Lang@unimedizin-mainz.de

22. Okamoto T, Onda S, Yasuda J, Yanaga K, Suzuki N, Hattori A (2015) Navigation surgery using an augmented reality for pancreatectomy. Dig Surg 32(2):117–123. https://doi.org/10.1159/000371860

23. Onda S, Okamoto T, Kanehira M, Fujioka S, Suzuki N, Hattori A, Yanaga K (2013) Short rigid scope and stereo-scope designed specifically for open abdominal navigation surgery: clinical

24. application for hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 20(4):448–453. https://doi.org/10.1007/s00534-012-0582-y

25. Ntourakis D, Memeo R, Soler L, Marescaux J, Mutter D, Pessaux P (2016) Augmented reality guidance for the resection of missing colorectal liver metastases: an initial experience. World J Surg 40(2):419–426. https://doi.org/10.1007/s00268-015-3229-8

26. Okamoto T, Onda S, Matsumoto M, Gocho T, Futagawa Y, Fujioka S, Yanaga K, Suzuki N, Hattori A (2013) Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 20(2):249–253. https://doi.org/10.1007/s00534-012-0504-z

27. Tang R, Ma L, Xiang C, Wang X, Li A, Liao H, Dong J (2017)Augmented reality navigation in open surgery for hilar cholangiocarcinoma resection with hemihepatectomy using video-based in situ three-dimensional anatomical modeling: a case report. Medicine (Baltimore) 96(37):e8083. ttps://doi.org/10.1097/MD.0000000000008083].