Nanorod hydroxyapatite (NRHA)/graphene oxide (GO) composites with weight ratios of 0.4, 1.5, and 5 have been fabricated by a facile ultrasonic-assisted method at room temperature and atmospheric pressure. The chemical structure properties and morphology of the composites were characterized by field emission source scanning electron microscope, X-ray diffraction, transmission electron microscopy, and high-resolution transmission electron microscopy. The results indicate that the NNRHA/GO composites have an irregular surface with different degree wrinkles and are stable, and NRHA are well combined with GO. In addition, the biomimetic mineralization mechanism of hydroxyapatite on the NNRHA/GO composites in simulated body fluid (SBF) is presented. The presence of a bone-like apatite layer on the composite surface indicate that the NNRHA/GO composites facilitate the nucleation and growth of hydroxyapatite crystals in SBF for biomimetic mineralization. Moreover, the NNRHA-1.5/GO composite and pure GO were cultured with MC3T3-E1 cells to investigate the proliferation and adhesion of cells. In vitro cytocompatibility evaluation demonstrated that the NNRHA/GO composite can act as a good template for the growth and adhesion of cells. Therefore, the NNRHA/GO composite could be applied as a GO-based, free-template, non-toxic, and bioactive composite to substitute for a damaged or defect bone.