July 2018
Digital pathology involves the use of high-resolution scans, or images, of biopsy specimens. Managing and utilizing the information generated from these digital pathology scans can be done through computerized methods. Whole-Slide Imaging (WSI), which has been recently introduced, has made digital pathology a promising field in diagnostic medicine. Digitized tissue pathology offers various applications of machine learning and image analysis techniques. When physicians examine medical images, retrieving similar images from a large archive can provide valuable information to support diagnosis. However, conventional search methods fail when dealing with archives of medical images, especially in digital pathology, due to the prohibitively large number of gigapixel files. These scans contain gigabytes of information, with a typical resolution of one pixel per 0.5 micron. It would be useful to tag each image, or its regions/patches, with a “barcode” for fast retrieval, reducing storage requirements and enabling efficient indexing and reference. In this paper, we propose novel compact descriptors for medical image retrieval using deep barcodes. We conduct experiments on a publicly available dataset called Kimia Path24, which consists of 7,055 training/indexing images and 1,325 testing images. We perform experiments using three pre-trained networks: AlexNet, VGG-16, and VGG-19. The paper includes a literature review, a description of deep barcodes, details of the experiments, results, and discussions. The paper concludes with the findings and their implications.
Computer algorithms are commonly used in medical imaging to assist physicians in decision-making. Pathology slides provide a comprehensive view of diseases and their effects on tissues, as the preparation process preserves the essential tissue architecture. Pathology images are considered the “gold standard” in diagnosing many diseases, including various types of cancer. The exploration of spatial structure in pathology images has gained attention in recent years, with studies focusing on tissue micro-texture classification, segmentation of glands and nuclei, and visual semantic analysis of tumor-prone regions. Digital pathology, which involves the digitization of pathology glass slides, offers advantages such as long-term preservation, remote access for multiple specialists, and easy retrieval if efficient image processing algorithms are used. However, the large size of digital pathology images makes storage, processing, and real-time transfer challenging. Deep neural networks have emerged as powerful tools for image recognition and analysis, outperforming conventional methods. Convolutional neural networks (CNNs), a type of deep neural network, are particularly effective for image data. Deep learning approaches, trained on large-scale non-medical image datasets like ImageNet, have shown promise in pathology image analysis. Pre-trained networks, such as AlexNet, VGG-16, and VGG-19, have learned rich feature representations and can be applied to medical images without the need for domain-specific training. Binary descriptors, such as barcodes, have proven to be powerful and concise for image retrieval. In the medical imaging domain, barcodes have been used for annotation and retrieval of different regions of interest. Deep features combined with barcodes offer the opportunity to address resource challenges and achieve efficient retrieval.
barcodes (bottom) for two sample images.
Given the large dimensions of pathology scans, they need to be split into smaller images, or patches, for manageability. Deep barcodes are computed to represent each image or patch in a binary format. Two methods are used to calculate the deep barcodes: (a) Min-max algorithm, which outputs a binary value of 1 if there is an increase in consecutive feature values, and 0 otherwise, resulting in a barcode of dimension 1×4095; (b) Zero-thresholding, where features above zero are assigned 1, and features below or equal to zero are assigned 0, resulting in a barcode of dimension 1×4096. These processes are repeated for all images in the training and testing datasets, generating a barcode of 4096 digits for each image. The barcodes are then used to compute the distance between each test barcode and each training barcode using logical XOR metric, allowing for fast and efficient retrieval. This two-stage search yields top N matches in the first stage and identifies the best match in the second stage based on deep features and distance metrics.
Several experiments were conducted using different approaches and metrics. Deep features extracted from pre-trained networks were compared using L1 and L2 distance metrics. Principal component analysis (PCA) was applied to reduce dimensionality and obtain compact features. Deep barcodes were generated using binary search techniques and compared with deep features and reduced deep features. The accuracy of classification and retrieval was computed for all experiments using the Kimia Path24 dataset, which consists of 7,055 training/indexing images and 1,325 testing images. The results were analyzed and compared, and the impact of different factors, such as the number of principal components, distance metrics, and network architectures, was examined.
The results of the experiments showed the performance of the proposed methods in classification and retrieval tasks. Deep features achieved high accuracy, with VGG-16 performing the best among the pre-trained networks. PCA with deep features demonstrated comparable accuracies while reducing dimensionality. Deep barcodes, generated using binary search techniques, showed comparable or higher accuracies compared to deep features and reduced deep features. The retrieval accuracy reached 71.62%, surpassing the accuracies obtained using real-valued deep features. The results were compared with previous methods, such as Local Binary Patterns (LBP) and Bag-of-words (BOW), and showed superior performance while offering the advantage of reduced dimensionality and computational expense.
The discussions highlighted the effectiveness of pre-trained networks, especially VGG-16, in the classification and retrieval of pathology images. The experiments demonstrated that deep features, reduced deep features, and deep barcodes achieved high accuracies, with deep barcodes showing promising results for efficient retrieval. The findings suggest that deep learning trained on non-medical image datasets can be applied to medical image recognition tasks, even with the limited availability of labeled pathology image databases. The advantages of deep barcodes, including compactness and binary representation, make them suitable for resource-constrained environments and fast search operations. The results also emphasized the potential for further exploration and development of deep learning approaches for pathology image analysis.
In conclusion, this paper presented a novel approach for medical image retrieval using deep barcodes. The proposed methods achieved high accuracies in classification and retrieval tasks, outperforming previous methods while reducing dimensionality and computational expense. The experiments demonstrated the effectiveness of pre-trained networks, particularly VGG-16, in pathology image analysis. Deep barcodes, generated from deep features, offered a compact and binary representation for efficient retrieval using binary search techniques. The findings suggest that deep learning trained on non-medical image datasets can be valuable in pathology image recognition tasks, despite challenges such as limited labeled data and differences in domain-specific architectures. The results open avenues for future research and development in this field, with the potential to address resource limitations and improve efficiency in medical image analysis.
Additional details: Deep Barcodes for Fast Retrieval of Histopathology Scans