Enhanced optical nonlinearity of epsilon-near-zero metasurface by quasi-bound state in the continuum

Bound states in the continuum (BICs) provide a powerful way to enhance the nonlinear properties of materials, epsilon-near-zero (ENZ) materials are considered as promising candidates with strong nonlinearities. However, the realization of BIC based on ENZ materials in the near-infrared (NIR) is very challenging due to the large loss in the NIR. Here, a high-quality quasi-BIC based on the ENZ metasurface is proposed for the first time, which is composed of patterned ENZ films embedded in a dielectric-metal hybrid structure, and realizes destructive interference between the Berreman mode and photonic mode to form the Friedrich-Wintergen BIC (FW-BIC). The electric field is strongly confined in the ENZ film, resulting in considerable field enhancement, and the nonlinear refractive index coefficient is 1.63 × 10−12 m2/W, which is three orders of magnitude larger than that of the ITO film. The instantaneous response time is 600 fs and extremely high modulation speed up to the THz level. Benefiting from the perfect absorption and narrow linewidth of quasi-BIC and the change in refractive index of the metasurface induced by Kerr nonlinearity, the absolute modulation is from near-zero to 92% with an extinction ratio of 23.2 dB. It provides a promising platform for the development of integrated ultrafast high-speed photonics. The large loss of epsilon-near-zero (ENZ) materials leads to short propagation lengths and low Q-factors. High Q-factor quasi-BIC based on ENZ metasurface is formed for the first time by embedding ITO thin films into metal-dielectric hybrid structures to eliminate radiation losses. The electric field is strongly confined in the ENZ film, resulting in a dramatic increase in the nonlinearity. [Display omitted] •We realize quasi-BIC based on ENZ materials in the near-infrared for the first time.•The electric field of the quasi-BIC metasurface is strongly confined in the ENZ film.•Quasi-BIC improves nonlinearity is three orders of magnitude larger than ENZ films.•The metasurface exhibits 23.2 dB high extinction ratio and THz modulation speed.