Turning effect of nonlinear optical properties depends on their phase transition behaviors of Sb2Te2Se thin film

•Sb2Te2Se enables a unique phase transition, promising novel all-optical devices.•Lower crystallization temperature in Sb2Te2Se aids energy-efficient devices.•Annealing tailors Sb2Te2Se's microstructure, fine-tuning NOA coefficient.•Structural and elemental composition crucial in determining NOA in Sb-based films. Tetradymites have extensive application in nonlinear optical devices due to their significant nonlinear optical absorption (NOA). In this work, the NOA properties of tetradymite Sb2Te2Se was investigated. It is found that a switch from reverse saturable absorption (RSA) to saturable absorption (SA) can be achieved with relatively low annealing treatment, and the modulation depth can be finely tuned through variation of crystallinity. In addition, Influence of microstructure and composition on the NOA effect is explored by comparing Sb2Te2Se, Sb2Te3 and Sb2Se3, revealing a competitive mechanism. The study reveals the NOA properties intrinsic to Sb2Te2Se, which offers greater understanding to the improvement of novel NOA materials and nonlinear optical devices.