Abstract
Deep learning-based tractography implicitly learns anatomical prior knowledge that is required to resolve ambiguities inherent in traditional streamline tractography. TractSeg is a particularly widely used example of such an approach. Even though it has exclusively been trained on healthy subjects, a certain level of generalization to different pathologies has been demonstrated, and TractSeg is now increasingly used for clinical cases. We explore the limits of TractSeg by evaluating it on a unique dataset of 25 patients with epilepsy who underwent hemispherotomy, a type of surgery in which the two hemispheres are surgically separated. We compare results to those on 25 healthy controls who have been imaged with the same setup. We find that TractSeg generalizes remarkably well, given the severity of the abnormalities. However, to our knowledge, we are the first to document cases in which TractSeg erroneously reconstructs ("hallucinates") tracts that are known to have been surgically disconnected, and we found cases in which it implausibly continues tracts through obvious lesions. At the same time, TractSeg failed to reconstruct or undersegmented some tracts that are known to be preserved. We subsequently propose a refinement of TractSeg which aims to improve its applicability to data with pathologies, by using its Tract Orientation Maps as an anatomical prior in low-rank tensor approximation based tractography such that tracking is guaranteed to continue only where presence of the tract is directly supported by the data ("data fidelity"). We demonstrate that our extension not only eliminates hallucinated tracts and reconstructions within lesions, but that it also increases the ability to reconstruct the preserved tracts, and leads to more complete reconstructions even in healthy controls. Despite these advances, we recommend caution and manual quality control when applying deep learning based tractography to patient data.