Solid-state electrolytes (SSEs) can address the safety issue of organic electrolyte in rechargeable lithium batteries. Unfortunately, neither polymer nor ceramic SSEs used alone can meet the demand although great progress has been made in the past few years. Composite solid electrolytes (CSEs) composed of flexible polymers and brittle but more conducting ceramics can take advantage of the individual system for solid-state lithium metal batteries (SSLMBs). CSEs can be largely divided into two categories by the mass fraction of the components: “polymer rich” (PR) and “ceramic rich” (CR) systems with different internal structures and electrochemical properties. This review provides a comprehensive and in-depth understanding of recent advances and limitations of both PR and CR electrolytes, with a special focus on the ion conduction path based on polymer-ceramic interaction mechanisms and structural designs of ceramic fillers/frameworks. In addition, it highlights the PR and CR which bring the leverage between the electrochemical property and the mechanical property. Moreover, it further prospects the possible route for future development of CSEs according to their rational design, which is expected to accelerate the practical application of SSLMBs.