Multimodal Image Analysis for Tissue Diagnostics
The tissue diagnostics of skin melanoma was achieved by multimodal imaging combined with an automatic image analysis. Normal epithelial, melanoma, and other skin tissues were successfully distinguished by a hierarchical classifier based on a first-order histogram and GLCM textural features calculated in a local model. Textural features based on a first-order histogram have meanwhile been proven superior to those based on GLCM according to the classification results.
By: Shuxia Guo // Susanne Pfeifenbring // Tobias Meyer // Jürgen Popp // Thomas Bocklitz
Most diseases change tissue structures and functions; therefore, information on the biological alterations can be deduced from tissue measurements. This information can be utilized for tissue diagnostics and disease detections. The ‘gold standard’ diagnosis method for many diseases is still histopathological examination . In this method, the tissue is removed, sectioned, stained, and imaged via microscopy. The microscopic images are inspected by a pathologist who conducts the diagnosis according to the visual characteristics of the tissue. The requirement of tissue biopsy and staining makes histopathological examination unsuitable for in vivo diagnosis and also leads to a long time delay between resection and diagnosis. More importantly, the diagnosis depends on the pathologist, and different pathologists may conduct controversial diagnostics for the same tissue section. These issues motivated us to apply multimodal imaging for tissue diagnosis, which refers to the combination of two-photon excited fluorescence (TPEF) and second harmonic generation (SHG). Multimodal imaging is label free, noninvasive, and provides a molecular contrast; hence the technique is ideally suited for in vivo diagnostics . One illustrative example of our tissue-based studies is the diagnostics of skin melanoma, which is a common cancer in the western world and the annual occurrence has increased by 4.6% from 1975 to 1985 and 2.7% from 1986 to 2007 . To achieve clinical application, however, it is required to translate the optical signals into high-level diagnostically relevant information.
Therefore, we established an image analysis pipeline and benchmarked its performance by multimodal images measured from thirteen human skin tissues with melanocytic lesions . After multimodal imaging, these tissues were HE (hematoxylin and eosin) stained, imaged via a microscope, and inspected by an experienced pathologist to annotate the (normal) epithelial and tumor regions. The image analysis starts with the removal of uneven illumination artefacts through the combination of wavelet and Fourier transformation. Thereafter, the background estimation and intensity standardization follow. In the next step, textural features including first-order histogram features and the local gray-level co-occurrence matrix (GLCM) features were calculated using moving windows with different window sizes. On the basis of these textural features, a hierarchical statistical model, which is composed of Fisher’s discriminant ratio (FDR)-based feature selection and support vector machine (SVM), was developed to distinguish three tissue types (normal epithelial, melanoma, and other tissues). The model was constructed in local mode (i.e., pixel wise) using the pathologist’s annotation as the ground truth and verified by a leave-one-image-out cross validation. The three tissue types were differentiated with satisfactory accuracy, demonstrating the great potential of multimodal imaging combined with image analysis for tissue diagnostics. Results from one of the images are shown in Figure 1 as an example. In addition, we conducted global-mode (i.e., image-wise) classification to relate the whole image to the clinical diagnosis. The prediction from leave-one-image-out cross validation fairly agreed with the clinical diagnosis. Finally, we compared the performance of the first-order histogram and GLCM-based textural features according to the FDR, as well as the results of the classification. In summary, the first-order histogram features were superior to the GLCM-based textural features for the above-described diagnostic task.
Funded by: EU, BMBF, DFG, FCI, Carl-Zeiss Foundation, China Scholarship Council, Italian Ministry for Education, University and Research