Deep fusion of contextual and object-based representations for delineation of multiple nuclear phenotypes

Abstract Motivation Nuclear delineation and phenotypic profiling are important steps in the automated analysis of histology sections. However, these are challenging problems due to (i) technical variations (e.g. fixation, staining) that originate as a result of sample preparation; (ii) biological heterogeneity (e.g. vesicular versus high chromatin phenotypes, nuclear atypia) and (iii) overlapping nuclei. This Application-Note couples contextual information about the cellular organization with the individual signature of nuclei to improve performance. As a result, routine delineation of nuclei in H&E stained histology sections is enabled for either computer-aided pathology or integration with genome-wide molecular data. Results The method has been evaluated on two independent datasets. One dataset originates from our lab and includes H&E stained sections of brain and breast samples. The second dataset is publicly available through IEEE with a focus on gland-based tissue architecture. We report an approximate AJI of 0.592 and an F1-score 0.93 on both datasets. Availability and implementation The code-base, modified dataset and results are publicly available. Supplementary information Supplementary data are available at Bioinformatics online.