Method of fabricating semiconductor device having semiconductor elements

According to an aspect of the invention, there is provided a semiconductor device including a first semiconductor element formed on a semiconductor substrate and using electrons as carriers, and a second semiconductor element formed on the semiconductor substrate and using holes as carriers, a first insulating film and a second insulating film formed on source/drain regions and gate electrodes of the first element and the second element, the first insulating film having tensile stress with respect to the first element, and the second insulating film having compression stress with respect to the second element, and sidewall spacers of the gate electrodes of the first element and the second element, at least portions of the sidewall spacers being removed, wherein at least one of the first insulating film and the second insulating film does not close a spacing between the gate electrodes of the first element and the second element.