In this paper, the genetic algorithm (GA) is applied to optimize a grid connected solar photovoltaic (PV)-wind-battery hybrid system using a novel energy filter algorithm. The main objective of this paper is to minimize the total cost of the hybrid system, while maintaining its reliability. Along with the reliability constraint, some of the important parameters, such as full utilization of complementary nature of PV and wind systems, fluctuations of power injected into the grid and the battery’s state of charge (SOC), have also been considered for the effective sizing of the hybrid system. A novel energy filter algorithm for smoothing the power injected into the grid has been proposed. To validate the proposed method, a detailed case study has been conducted. The results of the case study for different cases, with and without employing the energy filter algorithm, have been presented to demonstrate the effectiveness of the proposed sizing strategy.