The 'Grain-for-Green' project on the Loess Plateau is the largest revegetation program in the world. However, revegetation-induced land use changes can influence both water and carbon cycles, and the diverse consequences were not well understood. Therefore, the reasonability and sustainability of revegetation measures are in question. This study quantifies the impacts of revegetation-induced land use conversions on the water and carbon cycles in a typical watershed on the Loess Plateau and identifies suitable areas where revegetation of forest or grassland could benefit both soil and water conservation and carbon sequestration. We used a coupled hydro-biogeochemical model to simulate the changes of a few key components in terms of water and carbon by designing a variety of hypothetical land use conversion scenarios derived from revegetation policy. Compared to the baseline condition (land use in 2000), both sediment yield and water yield decreased substantially when replacing steep cropland with forest or grassland. Converting cropland with slopes larger than 25°, 15°, and 6° to forest (CTF) would enhance the carbon sequestration with a negligible negative effect on soil water content, while replacing cropland with grassland (CTG) would result in a decline in net primary production but with a substantial increase in soil water content (3.8%–14.9%). Compared to the baseline, the soil organic carbon would increase by 0.9%–3.2% in CTF and keep relatively stable in CTG. Through testing a variety of hypothetical revegetation scenarios, we identified potential priority areas for CTF and CTG, where revegetation may be appropriate and potentially beneficial to conserving soil and water and enhancing carbon sequestration. Our study highlights the challenges in future water and carbon coupling management under revegetation policy, and our quantitative results and identification of potential areas for revegetation could provide information to policy makers for seeking optimal management on the Loess Plateau.