The objective of this paper is to clarify the genetic relationship of japonica rice resources from different breeding units of Heilongjiang province. Genetic diversity and population structure of 104 japonica rice accessions, which were collected from eight different breeding units, were analyzed by 45 pairs of simple sequence repeat (SSR) molecular markers in this study. The results showed that all selected markers were applicable in clarification of the genetic diversity, yielding a total of 154 alleles. The alleles (Na) per locus ranged from 2 to 8 with the mean value of 3.42. The gene diversity indices (H) varied from 0.02 to 0.77 with an average of 0.38. The polymorphism information content (PIC) was changed from 0.02 to 0.73 with the average of 0.34. Furthermore, the genetic diversity of rice varieties from Heihe Branch of Heilongjiang Academy of Agricultural Sciences was the most abundant, while the Northeast Agricultural University was narrow, with the polymorphism information content (PIC) of 0.33 and 0.18, respectively. By UPGMA Cluster and genetic structure analysis, the rice cultivars from Heihe Branch of Heilongjiang Academy of Agricultural Sciences represented genetic differentiation and the other units, while a closer relationship between Rice institute of Heilongjiang Agricultural Reclamation Academy and Jiamusi Rice Research Institute of Heilongjiang Academy of Agricultural Sciences was observed. Thus, our results suggested a demand of enlarging the genetic basis in breeding for elite varieties in Helongjiang province, by taking use of the genetic resource from other units in future.