1311 H&E 2.0: deep learning-enabled identification of tumor-specific CD39+CD8+ T cells in marker-free images for predicting immunotherapy response

BackgroundSeveral groups, including ours, have shown CD39 to be a tumor-specific CD8+ T cell marker. In non-small cell lung cancer (NSCLC) and colorectal carcinoma (CRC), CD8+ T cells lacking CD39 expression are bystander tumor infiltrating lymphocytes1; while CD39+CD8+ T cells are tumor antigen-specific in treatment-naïve NSCLC2 and triple-negative breast cancer (TNBC).3 Thus, combining CD39+CD8+ T cell abundance and spatial localization is a potential predictor of patient response to PD-1/PD-L1 blockade immunotherapy for numerous cancer types.3–6 To redirect resources from repeatedly conducting laborious and costly multi-marker assays for immunotherapy patient stratification, we developed deep learning (DL) models trained on multiplex immunofluorescence (mIF) and fluorescence imaging data to identify CD39+CD8+ T cells by morphology in hematoxylin and eosin (H&E)-stained tissue images and brightfield images of immune cells from blood samples.MethodsSeparate convolutional neural network models were developed to identify CD39+CD8+ T cells in human CRC samples and peripheral blood mononuclear cells (PBMCs) from CT26 tumor-bearing mice (CRC mouse tumor models).CD39+CD8+ T cells in the CRC samples were first visualized with mIF and subsequently stained with H&E. The DL pipeline stages are: (1) alignment of fluorescence and H&E images, (2) cell segmentation, (3) manual annotation of CD39+CD8+ cells as ground truth labels, (4) extracting each cell as a small image patch, and (5) training a DL model (θH&E) for CD39+CD8+ prediction using 2,426 positive examples and 101,084 negative examples (figure 1A).The mouse PBMCs were immunostained with fluorescent antibodies and visualized with imaging flow cytometry. The DL pipeline stages are: (1) gating CD8+ and CD39+ positivity based on fluorescence intensity, and (2) training a DL model (θblood) for CD39+CD8+ prediction using 1,985 positive examples and 4,639 negative examples (figure 1B).The models’ performance was evaluated with F1 scores.ResultsThe current version of θH&E has a test F1-score of 0.83; θblood has a test F1-score of 0.80.ConclusionsThe F1-scores indicate that both DL models can identify CD39+CD8+ T cells from marker-free H&E images and brightfield images, respectively. Ongoing improvements to the models include validating them across independent cohorts with different cancer types and evaluating their predictive capabilities for checkpoint immunotherapy response on pre-treatment patient samples. By impl