Recent evidence showed that bacteria and fungi appear to have different latitudinal diversity gradients at the global scale. However, the ecological drivers explaining these decoupled ecological and evolutionary patterns remain poorly understood. We identified the ecological predictors of such a decoupled pattern between bacterial and fungal diversity across a 4100 km latitudinal transect, from tropical to temperate forests in eastern China. Bacterial diversity showed a hump-shaped trend with latitude, while the diversity of fungi, and especially fungal saprobes and pathogens decreased with increasing latitude. In addition, our results provided evidence that while temperature and primary productivity, which decreased with increasing latitude, were the best predictors of fungal diversity, soil properties such as pH and N:P ratio were the best predictors for the latitudinal pattern in bacterial diversity. Further statistical results showed that sampling bias should be carefully considered in disentangling the underlying mechanisms of microbial geographical distribution. Our findings suggest that temperature is more likely to associate with fungal diversity than with bacterial diversity in eastern China, with important implications for the prediction of soil biodiversity and functions under climate change scenarios.