Semiconductor device and forming method thereof

A semiconductor device and forming method thereof is provided. A stacked layer is formed on a substrate, wherein the stacked layer comprises first layers and second layers stacked alternatively. A trench is formed in the stacked layer, and the trench has a first sidewall and a second sidewall opposite to the first sidewall. Trench materials are formed in the trench, wherein the trench materials are disposed along the first sidewall and the second sidewall respectively. Parts of the trench materials are replaced by a first dielectric structure, so that units are carried out. Remaining parts of the trench materials along the first sidewall and the second sidewall constitute first trench structures of first transistor arrays and second trench structures of second transistor arrays, and each of the first transistor arrays and each of the second transistor arrays are vertically disposed on the substrate.