Landmark-free automatic digital twin registration in robot-assisted partial nephrectomy using a generic end-to-end model

Zhong F-P, Zhang Y-J, Liu Y, Zou S-B (2017) Prognostic impact of surgical margin in patients with hepatocellular carcinoma: a meta-analysis. Medicine 96(37):8043

Google Scholar 

Khojasteh SB, Fuentes-Jimenez D, Pizarro D, Espinel Y, Bartoli A: Mis-nerf: Neural radiance fields in minimally-invasive surgery. International Journal of Computer Assisted Radiology and Surgery (2025)

Desai P, Patel P (2011) Fibroids, infertility and laparoscopic myomectomy. J Gynecol Endoscopy Surg 2(1):36

Google Scholar 

Bex A, Albiges L, Bedke J, Capitanio U, Dabestani S, Hora M, Klatte T, Kuusk T, Lund L, Marconi L, et al.: Renal cell carcinoma. In: European Association of Urology Guidelines (2025)

Collins T, Pizarro D, Gasparini S, Bourdel N, Chauvet P, Canis M, Calvet L, Bartoli A (2020) Augmented reality guided laparoscopic surgery of the uterus. IEEE Trans Med Imag 40(1):371–380

Google Scholar 

Espinel Y, Calvet L, Botros K, Buc E, Tilmant C, Bartoli A.: Using multiple images and contours for deformable 3d-2d registration of a preoperative ct in laparoscopic liver surgery. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, pp. 657–666 (2021). Springer

Haouchine N, Roy F, Untereiner L, Cotin S.: Using contours as boundary conditions for elastic registration during minimally invasive hepatic surgery. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 495–500 (2016). IEEE

Robu MR, Ramalhinho J, Thompson S, Gurusamy K, Davidson B, Hawkes D, Stoyanov D, Clarkson MJ (2018) Global rigid registration of ct to video in laparoscopic liver surgery. Int J Comput Assist Radiol Surg 13:947–956

PubMed  PubMed Central  Google Scholar 

Modrzejewski R, Collins T, Seeliger B, Bartoli A, Hostettler A, Marescaux J (2019) An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions. Int J Comput Assist Radiol Surg 14:1237–1245

PubMed  Google Scholar 

Labrunie M, Pizarro D, Tilmant C, Bartoli A: Automatic 3d/2d deformable registration in minimally invasive liver resection using a mesh recovery network. In: MIDL, pp. 1104–1123 (2023)

Okamoto T, Onda S, Yasuda J, Yanaga K, Suzuki N, Hattori A (2015) Navigation surgery using an augmented reality for pancreatectomy. Dig Surg 32(2):117–123

PubMed  Google Scholar 

Tang R, Yang W, Hou Y, Yu L, Wu G, Tong X, Yan J, Lu Q (2021) Augmented reality-assisted pancreaticoduodenectomy with superior mesenteric vein resection and reconstruction. Gastroenterol Res Practice 2021(1):9621323

Google Scholar 

Checcucci E, Piana A, Volpi G, Piazzolla P, Amparore D, De Cillis S, Piramide F, Gatti C, Stura I, Bollito E, Massa F, Di Dio M, Fiori C, Porpiglia F (2023) Three-dimensional automatic artificial intelligence driven augmented-reality selective biopsy during nerve-sparing robot-assisted radical prostatectomy: A feasibility and accuracy study. Asian J Urol 10(4):407–415

PubMed  PubMed Central  Google Scholar 

Khaddad A, Bernhard J-C, Margue G, Michiels C, Ricard S, Chandelon K, Bladou F, Bourdel N, Bartoli A (2023) A survey of augmented reality methods to guide minimally invasive partial nephrectomy. World J Urol 41(2):335–343

PubMed  Google Scholar 

Della Corte M, Quarà A, De Cillis S, Volpi G, Amparore D, Piramide F, Piana A, Sica M, Di Dio M, Alba S, Porpiglia F, Checcucci E, Fiori C (2024) 3d virtual models and augmented reality for radical prostatectomy: a narrative review. Chin Clin Oncol 13(4):56–56

PubMed  Google Scholar 

Su L-M, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3d-ct to stereoscopic video registration. Urology 73(4):896–900

PubMed  Google Scholar 

Nakamura K, Naya Y, Zenbutsu S, Araki K, Cho S, Ohta S, Nihei N, Suzuki H, Ichikawa T, Igarashi T (2010) Surgical navigation using three-dimensional computed tomography images fused intraoperatively with live video. J Endourol 24(4):521–524

PubMed  Google Scholar 

Pratt P, Mayer E, Vale J, Cohen D, Edwards E, Darzi A, Yang G-Z (2012) An effective visualisation and registration system for image-guided robotic partial nephrectomy. J Robot Surg 6:23–31

PubMed  Google Scholar 

Chen Y, Li H, Wu D, Bi K, Liu C (2014) Surgical planning and manual image fusion based on 3d model facilitate laparoscopic partial nephrectomy for intrarenal tumors. World J Urol 32:1493–1499

PubMed  Google Scholar 

Wang D, Zhang B, Yuan X, Zhang X, Liu C (2015) Preoperative planning and real-time assisted navigation by three-dimensional individual digital model in partial nephrectomy with three-dimensional laparoscopic system. Int J Comput Assist Radiol Surg 10:1461–1468

PubMed  Google Scholar 

Porpiglia F, Fiori C, Checcucci E, Amparore D, Bertolo R (2018) Hyperaccuracy three-dimensional reconstruction is able to maximize the efficacy of selective clamping during robot-assisted partial nephrectomy for complex renal masses. Eur Urol 74(5):651–660

PubMed  Google Scholar 

Porpiglia F, Checcucci E, Amparore D, Piramide F, Volpi G, Granato S, Verri P, Manfredi M, Bellin A, Piazzolla P, Autorino R, Morra I, Fiori C, Mottrie A (2020) Three-dimensional augmented reality robot-assisted partial nephrectomy in case of complex tumours: a new intraoperative tool overcoming the ultrasound guidance. Eur Urol 78(2):229–238

PubMed  Google Scholar 

Schiavina R, Bianchi L, Chessa F, Barbaresi U, Cercenelli L, Lodi S, Gaudiano C, Bortolani B, Angiolini A, Bianchi FM, Ercolino A, Casablanca C, Molinaroli E, Porreca A, Golfieri R, Diciotti S, Marcelli E, Brunocilla E (2021) Augmented reality to guide selective clamping and tumor dissection during robot-assisted partial nephrectomy: a preliminary experience. Clin Genitourin Cancer 19(3):149–155

Google Scholar 

Amparore D, Piramide F, Pecoraro A, Verri P, Checcucci E, De Cillis S, Piana A, Busacca G, Manfredi M, Fiori C, Porpiglia F (2022) Identification of recurrent anatomical clusters using three-dimensional virtual models for complex renal tumors with an imperative indication for nephron-sparing surgery: new technological tools for driving decision-making. European Urol Open Sci 38:60–66

Google Scholar 

Amparore D, Checcucci E, Piramide F, Busacca G, Volpi G, De Cillis S, Sica M, Verri P, Piana A, Di Dio M, Fiori C, Porpiglia F (2024) Robotic vena cava thrombectomy with three-dimensional augmented reality guidance. European Urol Open Sci 62:43–46

Google Scholar 

Altamar HO, Ong RE, Glisson CL, Viprakasit DP, Miga MI, Herrell SD, Galloway RL (2011) Kidney deformation and intraprocedural registration: a study of elements of image-guided kidney surgery. J Endourol 25(3):511–517

PubMed  Google Scholar 

Amparore D, Checcucci E, Piazzolla P, Piramide F, De Cillis S, Piana A, Verri P, Manfredi M, Fiori C, Vezzetti E, Porpiglia F (2022) Indocyanine green drives computer vision based 3d augmented reality robot assisted partial nephrectomy: the beginning of “automatic’’ overlapping era. Urology 164:312–316

Google Scholar 

Zhang X, Wang J, Wang T, Ji X, Shen Y, Sun Z, Zhang X (2019) A markerless automatic deformable registration framework for augmented reality navigation of laparoscopy partial nephrectomy. Int J Comput Assist Radiol Surg 14:1285–1294

PubMed  Google Scholar 

Teluob G, Calvet L, Espinel Y, Savareux L, Guandalino M, Ravel A, Bourdel N, Chauvet P, Guy L, Chabrot P, Bartoli A: Preliminary trial of augmented reality performed on a regular and a robot-assisted laparoscopic partial nephrectomies. Videourology 33(3) (2019)

Piana A, Amparore D, Sica M, Volpi G, Checcucci E, Piramide F, De Cillis S, Busacca G, Scarpelli G, Sidoti F, Alba S, Piazzolla P, Fiori C, Porpiglia F, Di Dio M (2024) Automatic 3d augmented-reality robot-assisted partial nephrectomy using machine learning: our pioneer experience. Cancers 16(5):1047

PubMed  PubMed Central  Google Scholar 

Anguelov D, Srinivasan P, Koller D, Thrun S, Rodgers J, Davis J: Scape: shape completion and animation of people. ACM SIGGRAPH 2005 Papers, 408–416 (2005)

Loper M, Mahmood N, Romero J, Pons-Moll G, Black MJ (2023) Smpl: a skinned multi-person linear model. Seminal Graph Papers Push Boundaries 2:851–866

Google Scholar 

Tian Y, Zhang H, Liu Y, Wang L: Recovering 3d human mesh from monocular images: a survey. IEEE transactions on pattern analysis and machine intelligence (2023)

Guan P, Weiss A, Balan AO, Black MJ: Estimating human shape and pose from a single image. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1381–1388 (2009). IEEE

Kanazawa A, Black MJ, Jacobs DW, Malik J: End-to-end recovery of human shape and pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7122–7131 (2018)

Kolotouros N, Pavlakos G, Black MJ, Daniilidis K: Learning to reconstruct 3d human pose and shape via model-fitting in the loop. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2252–2261 (2019)

Zhang H, Tian Y, Zhou X, Ouyang W, Liu Y, Wang L, Sun Z: Pymaf: 3d human pose and shape regression with pyramidal mesh alignment feedback loop. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11446–11456 (2021)

Zhang H, Tian Y, Zhang Y, Li M, An L, Sun Z, Liu Y (2023) Pymaf-x: towards well-aligned full-body model regression from monocular images. IEEE Trans Pattern Anal Mach Intell 45(10):12287–12303

PubMed  Google Scholar 

Farinha R, Breda A, Porter J, Mottrie A, Van Cleynenbreugel B, Vander Sloten J, Mottaran A, Gallagher AG (2023) RAPN-Delphi Surgeons Group: international expert consensus on metric-based characterization of robot-assisted partial nephrectomy. Eur Urol Focus 9(2):388–395

PubMed  Google Scholar 

De Backer P, Peraire Lores M, Demuynck M, Piramide F, Simoens J, Oosterlinck T, Bogaert W, Shan CV, Van Regemorter K, Wastyn A, Checcucci E, Debbaut C, Van Praet C, Farinha R, De Groote R, Gallagher A, Decaestecker K, Motttrie A (2023) Surgical phase duration in robot-assisted partial nephrectomy: a surgical data science exploration for clinical relevance. Diagnostics 13(21):3386

PubMed  PubMed Central  Google Scholar 

Chandelon K, Bartoli A (2022) Tracking better, tracking longer: automatic keyframe selection in model-based laparoscopic augmented reality. Int J Comput Assist Radiol Surg 17(8):1507–1511

PubMed  Google Scholar 

Geist, A.R., Frey, J., Zobro, M., Levina, A., Martius, G.: Learning with 3d rotations, a hitchhiker’s guide to so (3). arXiv preprint arXiv:2404.11735 (2024)

Fuentes-Jimenez D, Pizarro D, Casillas-Pérez D, Collins T, Bartoli A (2022) Deep shape-from-template: Single-image quasi-isometric deformable registration and reconstruction. Image Vis Comput 127:104531

Google Scholar 

Labrunie, M., Pizarro, D., Tilmant, C., Bartoli, A.: Generic liver modelling with application to mini-invasive surgery guidance. In: International Conference on Medical Imaging and Computer-Aided Diagnosis, pp. 219–229 (2024). Springer

Comments (0)

No login
gif