Phase change materials are potential candidates for the application of latent heat storage. Herein, we fabricated porous capsules as shape-stable materials from cellulose-based polyelectrolyte complex, which were first prepared using cellulose 6-(N-pyridinium)hexanoyl ester as the cationic polyelectrolyte and carboxymethyl cellulose as the anionic polyelectrolyte to encapsulate polyethylene glycol by the vacuum impregnation method. Furthermore, the multi-walled carbon nanotube or graphene oxide, which were separately composited into the polyelectrolytes complex capsules to enhance thermal conductivity and light-to-thermal conversion efficiency. These capsules owned a typical core–shell structure, with an extremely high polyethylene glycol loading up to 34.33 g∙g^‒1. After loading of polyethylene glycol, the resulted cellulose-based composite phase change materials exhibited high thermal energy storage ability with the latent heat up to 142.2 J∙g^‒1, which was 98.5% of pure polyethylene glycol. Further results showed that the composite phase change materials demonstrated good form-stable property and thermal stability. Moreover, studies involving light-to-thermal conversion determined that composite phase change materials exhibited outstanding light-to-thermal conversion performance. Considering their exceptional comprehensive features, innovative composite phase change materials generated from cellulose presented a highly interesting choice for thermal management and renewable thermal energy storage.