Abstract
To achieve real-time monitoring and intelligent maintenance of transformers, a framework based on deep vision and digital twin has been developed. An enhanced visual detection model, DETR + X, is proposed, implementing multidimensional sample data augmentation through Swin2SR and GAN networks. This model converts one-dimensional DGA data into three-dimensional feature images based on Gram angle fields, facilitating the transformation and fusion of heterogeneous modal information. The Pyramid Vision Transformer (PVT) is innovatively adopted as the backbone for image feature extraction, replacing the traditional ResNet structure. A Deformable Attention mechanism is employed to handle the complex spatial structure of multi-scale features. Testing results indicate that the improved DETR + X model performs well in transformer state recognition tasks, achieving a classification accuracy of 100% for DGA feature maps. In object detection tasks, it surpasses advanced models such as Faster R-CNN, RetinaNet, YOLOv8, and Deformable DETR in terms of overall mAP50 scores, particularly demonstrating significant enhancements in small object detection. Furthermore, the Llava-7b model, fine-tuned based on domain expertise, serves as an expert decision-making tool for transformer maintenance, providing accurate operational recommendations based on visual detection results. Finally, based on digital twin and inference models, a comprehensive platform has been developed to achieve real-time monitoring and intelligent maintenance of transformers.