Semiconductor device and method of fabricating the same

A method of fabricating a semiconductor device includes the following steps. At first, a semiconductor substrate is provided. A gate stack layer is formed on the semiconductor substrate, and the gate stack layer further includes a cap layer disposed thereon. Furthermore, two first spacers surrounding opposite sidewalls of the gate stack layer are further formed. Subsequently, the cap layer is removed, and two second spacers are formed on a part of the gate stack layer. Afterwards, a part of the first spacer and the gate stack layer not overlapped with the two second spacers are removed to form two gate stack structures.