Human induced pluripotent stem cell formation and morphology prediction during reprogramming with time-lapse bright-field microscopy images using deep learning methods

•Deep learning methods detect cells forming human iPS cell in early reprogramming stage.•CNN classifies a brightfield microscopy image as probability images of cell classes with specific features.•U-net segments cells on a microscopy image and classifies to count the cells by its probability images for the early detection.•RNN by training the probability images predicts morphology of future human iPS cell colonies.•Test for 150 sets of time-lapse images from reprogramming CD34+ cells achieved high detection accuracy. Human induced pluripotent stem cells (hiPSCs) represent an ideal source for patient specific cell-based regenerative medicine; however, efficiency of hiPSC formation from reprogramming cells is low. We use several deep-learning results from time-lapse brightfield microscopy images during culture, to early detect the cells potentially reprogramming into hiPSCs and predict the colony morphology of these cells for improving efficiency of culturing a new hiPSC line. Sets of time-lapse bright-field images are taken to track reprogramming process of CD34+ cells biologically identified as just beginning reprogramming. Prior the experiment, 9 classes of templates with distinct cell features clipped from microscopy images at various reprogramming stages are used to train a CNN model. The CNN is then used to classify a microscopy image as probability images of these classes. Probability images of some class are used to train a densely connected convolutional network for extracting regions of this class on a microscopy image. A U-net is trained to segment cells on the time-lapse images in early reprogramming stage during culture. The segmented cells are classified by the extracted regions to count various types of cells appearing in the early reprogramming stage for predicting the identified cells potentially forming hiPSCs. The probability images of hiPSC classes are also used to train a spatiotemporal RNN for predicting the future hiPSC colony morphology of the potential cells. Experimental results show the prediction (before 7 days after of beginning of the reprogramming) achieved 0.8 accuracy, and 66% of the identified cells under different culture conditions, predicted as forming, finally formed hiPSCs. The predicted hiPSC images and extracted colonies on the images show the prediction for future 1.5 days achieved high accuracy of hiPSC colony areas and image similarity. Our study proposes a method using several deep learning models to efficiently s