CaDIS: Cataract dataset for surgical RGB-image segmentation

•A dataset for semantic segmentation for cataract surgery.•4670 images sampled from the CATARACTS dataset with semantic pixel-level annotations.•Includes 36 classes (4 anatomy classes, 29 surgical instruments and 3 other objects).•Different experimental setups to tackle various applications.•Strong baseline results using state-of-the-art segmentation models. [Display omitted] Video feedback provides a wealth of information about surgical procedures and is the main sensory cue for surgeons. Scene understanding is crucial to computer assisted interventions (CAI) and to post-operative analysis of the surgical procedure. A fundamental building block of such capabilities is the identification and localization of surgical instruments and anatomical structures through semantic segmentation. Deep learning has advanced semantic segmentation techniques in the recent years but is inherently reliant on the availability of labelled datasets for model training. This paper introduces a dataset for semantic segmentation of cataract surgery videos complementing the publicly available CATARACTS challenge dataset. In addition, we benchmark the performance of several state-of-the-art deep learning models for semantic segmentation on the presented dataset. The dataset is publicly available at https://cataracts-semantic-segmentation2020.grand-challenge.org/.