The slender top-mouth gudgeon Pseudorasbora elongata is a rare fish belonging to the family Cyprinidae. This species is endemic to China and distributed in restricted regions around the tributaries of the middle and lower Yangtze and Xijiang Rivers. P. elongata has suffered an increasing decline and has been vulnerable to extinction in the wild due to illegal and overfishing, water project construction, and other anthropogenic factors that destroyed their natural habitat and spawning grounds over the last few decades. Therefore, it is vital to assess the genetic diversity and structure of P. elongata for conserving and utilizing this fish effectively. In this study, we analyzed three P. elongata populations from Shanli (SL), Likou (LK), and Shitai (ST) in the Anhui section along the Yangtze River based on mitochondrial Cyt b gene and D-loop region sequences. The results showed that the total haplotypes defined by Cyt b gene and D-loop region in all populations were 18 and 27, respectively; the corresponding overall haplotype diversity (H_d) and nucleotide diversity (π) were 0.792 and 0.01332 and 0.777 and 0.01140, respectively. The data based on Cyt b gene and D-loop region showed that the genetic diversity of the ST population was lowest among those of the three populations. The genetic distances among the population were 0.00173–0.03615 (Cyt b) and 0.00193–0.02639 (D-loop). Obvious genetic differentiation and limited gene exchange were observed between the ST and SL and the ST and LK populations based on fixation indexes (F_st) and the gene flow value (N_m). Analysis of molecular variance (AMOVA) indicated that the source of variation between population (94.60%, 90.69%) was much higher than that within populations (5.40%, 9.31%). The maximum likelihood phylogenetic tree and haplotype network showed that the genetic distance between SL and LK populations was short and formed a branch, while the other branch only consisted of ST haplotypes. A neutral test and mismatch distribution analysis suggested that P. elongata has not undergone a recent population expansion. For ST population, neutral test Tajima’s D and Fu’s F_s values were negative (P＜0.05), and the mismatch distribution was represented by a single peak curve, which revealed that P. elongata populations in the ST area had possibly experienced a population expansion or bottleneck effect during evolution. Overall, P. elongata belongs to the category that includes populations with a high haplotype diversity index and low nucleotide diversity index, whereas obvious genetic differentiation among the three populations were observed with the ST population having the lowest genetic diversity. Thus, we recommend that ST populations should be regarded as a management protection unit to prioritize their conservation, while, SL and LK populations should be managed and protected as a whole. Our results could provide scientific evidence for the conservation, sustainable development, and commercial utilization of germplasm resources of P. elongata.