Comparison of different treatment planning optimization methods for vaginal HDR brachytherapy with multichannel applicators: A reduction of the high doses to the vaginal mucosa is possible

•Dosimetric performances of forward and inverse optimization methods were compared.•Two inverse optimization methods achieved the best dose homogeneity to the target.•One of the two was also most effective in reducing hot spots to the vaginal mucosa. A direct planning approach with multi-channel vaginal cylinders (MVCs) used for HDR brachytherapy of vaginal cancers is particularly challenging. Purpose of this study was to compare the dosimetric performances of different forward and inverse methods used for the optimization of MVC-based vaginal treatments for endometrial cancer, with a particular attention to the definition of strategies useful to limit the high doses to the vaginal mucosa. Twelve postoperative vaginal HDR brachytherapy treatments performed with MVCs were considered. Plans were retrospectively optimized with three different methods: Dose Point Optimization followed by Graphical Optimization (DPO + GrO), Inverse Planning Simulated Annealing with two different class solutions as starting conditions (surflPSA and homogIPSA) and Hybrid Inverse Planning Optimization (HIPO). Several dosimetric parameters related to target coverage, hot spot extensions and sparing of organs at risk were analyzed to evaluate the quality of the achieved treatment plans. Dose homogeneity index (DHI), conformal index (COIN) and a further parameter quantifying the proportion of the central catheter loading with respect to the overall loading (i.e., the central catheter loading index: CCLI) were also quantified. The achieved PTV coverage parameters were highly correlated with each other but uncorrelated with the hot spot quantifiers. HomogIPSA and HIPO achieved higher DHIs and CCLIs and lower volumes of high doses than DPO + GrO and surflPSA. Within the investigated optimization methods, HIPO and homoglPSA showed the highest dose homogeneity to the target. In particular, homogIPSA resulted also the most effective in reducing hot spots to the vaginal mucosa.