Unlike the traditional fossil energy, wind, as the clean renewable energy, can reduce the emission of the greenhouse gas. To take full advantage of the environmental benefits of wind energy, wind power forecasting has to be studied to overcome the troubles brought by the variable nature of wind. Power forecasting for regional wind farm groups is the problem that many power system operators care about. The high-dimensional feature sets with redundant information are frequently encountered when dealing with this problem. In this paper, two kinds of feature set construction methods are proposed which can achieve the proper feature set either by selecting the subsets or by transforming the original variables with specific combinations. The former method selects the subset according to the criterion of minimal-redundancy-maximal-relevance (mRMR), while the latter does so based on the method of principal component analysis (PCA). A locally weighted learning method is also proposed to utilize the processed feature set to produce the power forecast results. The proposed model is simple and easy to use with parameters optimized automatically. Finally, a case study of 28 wind farms in East China is provided to verify the effectiveness of the proposed method.