We study the interferometric observation of intensity and polarization anisotropies of a stochastic gravitational wave background. We show that the observed correlated data is defined in the group manifold of the three-dimensional rotation. Explicit correlation between two detectors in the interferometry experiments such as LIGO-Virgo and KAGRA is constructed in terms of the Wigner-D functions. Our results may provide a tool for constructing data pipelines to estimate the power spectra of the stochastic gravitational wave background anisotropies.