Effect of buffer layer on the characteristics of the AlN thin films deposited on SiNx/Si substrate was systematically examined. Among the buffer layers examined, both Mo and Ti buffer layers can not only greatly enhance the (002) preferred orientation of the films, but also improve the smoothness of the AlN films, whereas the Al thin films contain large grains microstructure and resulting in rough surface and wide distribution of (002) preferred orientation of the films. AlN thin films with smooth surface with (r.m.s. < 6 nm) and narrow distribution of grains' orientation (rocking curve < 3.8°), which is suitable for fabricating the devices. A thin film bulk acoustic wave resonator with resonance frequency around 1.7 GHz was fabricated from thus obtained AlN thin films.