Abstract
PURPOSE: Automatic identification of interventional devices in X-ray (XR) fluoroscopy offers the potential of improved navigation during transcatheter endovascular procedures. This paper presents a prototype implementation of fully automatic 3D reconstruction of a cryo-balloon catheter during pulmonary vein isolation (PVI) procedures by deep learning approaches. METHODS: We employ convolutional neural networks (CNN) to automatically identify the cryo-balloon XR marker and catheter shaft in 2D fluoroscopy during PVI. Training data are generated exploiting established semiautomatic techniques, including template-matching and analytical graph building. A first network of U-net architecture uses a single grayscale XR image as input and yields the mask of the XR marker. A second network of the similar architecture is trained using the mask of the XR marker as additional input to the grayscale XR image for the segmentation of the cryo-balloon catheter shaft mask. The structures automatically identified in two 2D images with different angulations are then used to reconstruct the cryo-balloon in 3D. RESULTS: Automatic identification of the XR marker was successful in 78% of test cases and in 100% for the catheter shaft. Training of the model for prediction of the XR marker mask was successful with 3426 training samples. Incorporation of the XR marker mask as additional input for the model predicting the catheter shaft allowed to achieve good training result with only 805 training samples. The average prediction time per frame was 14.47 ms for the XR marker and 78.22 ms for the catheter shaft. Localization accuracy for the XR marker yielded on average 1.52 pixels or 0.56 mm. CONCLUSIONS: In this paper, we report a novel method for automatic detection and 3D reconstruction of the cryo-balloon catheter shaft and marker from 2D fluoroscopic images. Initial evaluation yields promising results thus indicating the high potential of CNNs as alternatives to the current state-of-the-art solutions.