Abstract
This paper focuses on designing and developing novel architectures termed Hybrid Vision UNet-Encoder Decoder (HVU-ED) segmenter and Hybrid Vision UNet-Encoder (HVU-E) classifier for brain tumor segmentation and classification, respectively. The proposed model integrates the powerful feature extraction capabilities of hybrid methods like ResNet50, VGG16, Dense121 and Xception with Vision Transformer(ViT). These extracted hybrid features are fused with UNet features in the bottleneck and are passed to the HVU-ED decoder path for the segmentation task. In HVU-E, same features fed as input to the classification layer and machine learning algorithms such as SVM, RF, DT, Logistic Regression and AdaBoost. The proposed DenseVU-ED model obtained the highest segmentation accuracy of 98.91% with the BraTS2020 dataset. The highest dice score of 0.902 for the enhanced tumor, 0.954 for the core tumor, and 0.966 for the whole tumor were obtained. The DenseVU-E classifier achieved the highest accuracy of 99.18% with neural network classification and 92.21% accuracy with SVM on Figshare dataset. Grad-CAM, SHAP, and LIME techniques provide model interpretability, highlighting the models' focus on significant brain areas and decision-making transparency. The proposed models outperform existing methods in segmentation and classification tasks.