Comorbid medical illnesses, such as obesity and diabetes, are associated with more severe COVID-19, hospitalization, and death. However, the role of the immune system in mediating these clinical outcomes has not been determined. We used multiparameter flow cytometry and systems serology to comprehensively profile the functions of T cells and antibodies targeting spike, nucleocapsid, and envelope proteins in a convalescent cohort of COVID-19 subjects who were either hospitalized (n = 20) or not hospitalized (n = 40). To avoid confounding, subjects were matched by age, sex, ethnicity, and date of symptom onset. Surprisingly, we found that the magnitude and functional breadth of virus-specific CD4+ T cell and antibody responses were consistently higher among hospitalized subjects, particularly those with medical comorbidities. However, an integrated analysis identified more coordination between polyfunctional CD4+ T cells and antibodies targeting the S1 domain of spike among subjects who were not hospitalized. These data reveal a functionally diverse and coordinated response between T cells and antibodies targeting SARS-CoV-2, which is reduced in the presence of comorbid illnesses that are known risk factors for severe COVID-19.