TRCT-GAN: CT reconstruction from biplane X-rays using transformer and generative adversarial networks

Computed tomography (CT) provides a three-dimensional view of a patient's internal organs. Compared to CT volumes, X-ray imaging can significantly reduce the patient's exposure to ionizing radiation. Moreover, X-ray images are more economical and widely applied in surgical procedures. However, X-ray images can only provide two-dimensional information. In this paper, an end-to-end GAN network, named TRCT-GAN, is proposed to reconstruct chest CT volumes from biplane X-ray images. In the GAN network, the Transformer network module is employed to enhance the feature representation of X-ray images. Moreover, a dynamic attention module is added to exploit some 2D feature maps and 3D feature maps to enhance the contextual association. The experimental results demonstrate that the proposed network can effectively produce high-quality CT reconstructions from X-ray images. •Add Transformer module to the feature conversion of 2D X-ray images to 3D CT volumes.•Our proposed 2D dynamic attention module can encode complex 2D X-ray image features very well.•3D dynamic attention helps reconstruct more accurate CT images.