In this paper we study gauge symmetry breaking and restoration in an SU(N) gauge theory defined on space-time manifold R1,d−2 × S1 when fermions obey non-trivial boundary conditions. We derive the one-loop effective potentials for the non-integrable phases of Wilson lines at both zero and finite temperature, using the zeta-function technique. It is shown that gauge symmetry can be broken with fermions in the adjoint representation, but is restored at high temperature. Detailed analyses are given for the groups SU(2), SU(3) and SU(5) on R1,2 × S1.