Different microstructures of the same polymer-modified bitumen (PMB) were obtained by subjecting the bitumen modified with styrene-butadiene-styrene (SBS) copolymer to isothermal annealing at various temperatures. The effects of the morphology on the rheological properties of SBS-modified bitumen were investigated within the high-temperature range. The PMB microstructures were quantitatively evaluated using image analysis. A dynamic shear rheometer was used to measure the rheological parameters of the PMB samples and perform the multiple stress creep and recovery (MSCR) test. A quantitative basis could be established on which to discuss the relationship between the PMB morphology and rheology. The image analysis indicated that conditioning by isothermal annealing evidently led to a difference in the microstructure of the samples. Variation of the thermal history is demonstrated to be a practical way to vary the morphology of the PMB with the same raw materials and formulation. Compared with the two-phase morphology, the single-phase microstructure tended to have a narrower linear viscoelastic (LVE) region of the PMB. Within the LVE region, especially at low frequencies, the homogenous PMB can store more energy when experiencing loadings and is more elastic. Outside the LVE region, based on the MSCR test results, the homogenous morphology could assist in reaching a higher percentage of strain recovery after the creep period.