Deep-learning-based methods have achieved significant success in the Visual Place Recognition (VPR) task, which is important for autonomous driving and robotics systems. Recent advancements primarily focus on the sophisticated feature aggregation module. This paper argues for a shift in emphasis toward the backbone features. Through an in-depth analysis of GeM, one of the simplest pooling aggregator based VPR method, we identify a prevalent issue, termed ’Channel vanishing’. The issue manifests as a substantial proportion of channels in both the final GeM descriptor and the backbone output local features turning zero-valued and inactive during training, thereby drastically diminishing the representational capacity of the model and undermining its VPR performance. In order to solve this problem, we propose a regularization branch with a fully connected layer for the GeM pipeline. This branch successfully mitigates Channel vanishing and further enriches the diversity and representation of the backbone output features. During inference, our streamlined model, using only the GeM aggregator, achieves state-of-the-art performance among backbones that are not transformerbased. Notably, when utilizing the DINOv2-B backbone, our method derives 99.1% recall@1 and 100% recall@5 VPR scores on the Tokyo24/7 dataset. This result suggests that strengthening backbone features can substantially narrow the gap between simple GeM pooling and more complex aggregators; assessing how broadly this observation transfers to other aggregators is an interesting direction.