In Taiwan, groundwater commonly becomes important water resources in dry periods, and/or areas lack of water storage facility due to its low cost, steady water supply and good water quality. However, improper groundwater development brings about serious decreases in groundwater levels and land subsidence which causes disasters, such as seawater intrusion or soil salination, accompanied with environmental and economic losses. It is critical to develop strategies for water resources conservation in mountainous areas. The complex heterogeneity of mountainous physiographic environment makes it challenging in the forecasts of groundwater level variations, particularly in mountainous areas. Artificial neural networks (ANNs) have been recognized as an effective modeling tool for complex nonlinear systems in the last two decades. This study aims to investigate the interactive mechanisms of groundwater at the mountainous areas of the Jhuoshuei river basin in central Taiwan through analyzing and modeling the groundwater level variations. Several issues are discussed in this study, which includes the correlation between groundwater level variation and rainfall as well as streamflow, the identification of groundwater recharge patterns and effective rainfall thresholds for estimating groundwater level variations. The results indicate: (1) the daily variation of groundwater level is closely correlated with river flow and one-day antecedent rainfall based on correlation analyses; (2) effective rainfall thresholds can be identified successfully; (3) groundwater level variations can be classified into four types for monitoring wells; and (4) the daily variations of groundwater level can be well estimated by constructed ANNs. The identified interactive mechanisms between surface water and groundwater can facilitate the mountainous water resource conservation strategy for better water management, especially irrigation water supply and for alleviating land subsidence in downstream areas in the future.