Current knowledge of the risk factors predicting the progression to severe coronavirus disease 2019 (COVID-19) among patients in community isolation who either are asymptomatic or only suffer from mild COVID-19 is very limited. Using a multivariable competing risk survival analysis, we herein identify several important predictors of progression to severe COVID-19—rather than to recovery—among patients in community isolation. A competing risk survival analysis was performed on time-to-event data from a cohort study of all COVID-19 patients (n = 1753) in the largest community isolation center in Wuhan, China, from opening to closing. The exposures were age, sex, respiratory symptoms, gastrointestinal symptoms, general symptoms, and computed tomography (CT) scan signs. The main outcomes were time to COVID-19 deterioration or recovery. The factors predicting progression to severe COVID-19 among the patients in community isolation were: male sex (hazard ratio (HR) = 1.29, 95% confidence interval (CI), 1.04–1.58, p = 0.018), young and old age, dyspnea (HR = 1.58, 95% CI, 1.24–2.01, p < 0.001), and CT signs of ground-glass opacity (HR = 1.39, 95% CI, 1.04–1.86, p = 0.024) and infiltrating shadows (HR= 1.84, 95% CI, 1.22–2.78, p = 0.004). The risk of progression was found to be lower among patients with nausea or vomiting (HR = 0.53, 95% CI, 0.30–0.96, p = 0.036) and headaches (HR = 0.54, 95% CI, 0.29–0.99, p = 0.046). Based on the results of this study, resource-poor settings, dyspnea, sex, and age can easily be used to identify mild COVID-19 patients who are at increased risk of progression. Looking for CT signs of ground-glass opacity and infiltrating shadows may be an affordable option to support triage decisions in resource-rich settings. Common and unspecific symptoms including headaches, nausea, and vomiting likely induced the selection for community isolation of COVID-19 patients who were relatively unlikely to deteriorate. Triage and prioritization outcomes could be boosted if strategies are incorporated to minimize the inefficient prioritization of harmless comorbidities.