Abstract
This paper investigates Electrocardiogram (ECG) rhythm classification using a progressive deep learning framework that combines time-frequency representations with complementary hand-crafted features. In the first stage, ECG signals from the PhysioNet Challenge 2017 dataset are transformed into scalograms and input to diverse architectures, including Simple Convolutional Neural Network (SimpleCNN), Residual Network with 18 Layers (ResNet-18), Convolutional Neural Network-Transformer (CNNTransformer), and Vision Transformer (ViT). ViT achieved the highest accuracy (0.8590) and F1-score (0.8524), demonstrating the feasibility of pure image-based ECG analysis, although scalograms alone showed variability across folds. In the second stage, scalograms were fused with scattering and statistical features, enhancing robustness and interpretability. FusionViT without dimensionality reduction achieved the best performance (accuracy = 0.8623, F1-score = 0.8528), while Fusion ResNet-18 offered a favorable trade-off between accuracy (0.8321) and inference efficiency (0.016 s per sample). The application of Principal Component Analysis (PCA) reduced the dimensionality of the feature from 509 to 27, reducing the computational cost while maintaining competitive performance (FusionViT precision = 0.8590). The results highlight a trade-off between efficiency and fine-grained temporal resolution. Training-time augmentations mitigated class imbalance, enabling lightweight inference (0.006-0.043 s per sample). For real-world use, the framework can run on wearable ECG devices or mobile health apps. Scalogram transformation and feature extraction occur on-device or at the edge, with efficient models like ResNet-18 enabling near real-time monitoring. Abnormal rhythm alerts can be sent instantly to users or clinicians. By combining visual and statistical signal features, optionally reduced with PCA, the framework achieves high accuracy, robustness, and efficiency for practical deployment.