Pressure has been introduced into power systems owing to the intermittent and uncertain nature of renewable energy. As a result, energy resource aggregators are emerging in the electricity market to realize sustainable and economic advantages through distributed generation, energy storage, and demand response resources. However, resource aggregators face the challenge of dealing with the uncertainty of renewable energy generation and setting appropriate incentives to exploit substantial energy flexibility in the building sector. In this study, a risk-aware optimal dispatch strategy that integrates probabilistic renewable energy prediction and bi-level building flexibility engagements is proposed. A natural gradient boosting algorithm (NGBoost), which requires no prior knowledge of uncertain variables, was adopted to develop a probabilistic photovoltaic (PV) forecasting model. The lack of suitable flexibility incentives is addressed by a novel interactive flexibility engagement scheme that can take into account building users’ willingness and optimize the building flexibility provision. The chance-constrained programming method was applied to manage the supply–demand balance of the resource aggregator and ensure risk-aware decision-making in power dispatch. The case study results show the strong economic and environmental performance of the proposed strategy. The proposed strategy leads to a win–win situation in which profit increases through a load reduction of 13% and a carbon emission reduction of 3% is achieved for different stakeholders, which also shows a trade-off between the economic benefits and the risk of supply shortage.