Diagnosis report generation method and system oriented to multi-modal medical data and medium

The invention provides a diagnosis report generation method and system for multi-modal medical data and a medium, and the method comprises the steps: obtaining a multi-modal training data set which comprises a medical image and a text report; splitting the text report according to preset keywords, and determining a negative description text and a positive description text; inputting the medical image, the negative description text and the positive description text into a pre-trained diagnosis report generation model, and outputting a loss function; performing optimization processing on a pre-trained diagnosis report generation model according to the loss function, and determining a multi-modal medical diagnosis report generation model; and inputting a to-be-detected medical image into the multi-modal medical diagnosis report generation model, and generating a diagnosis report corresponding to the to-be-detected medical image. According to the invention, the problem of unbalanced training samples in the model