The live-attenuated yellow fever 17D (YF17D) vaccine is a model of acute viral infection that induces long-lasting protective immunity. Among immunocompetent adults, responses to YF17D vary significantly. To understand the sources of this variability, we investigate the influence of sex, age, human leukocyte antigen (HLA) type, and 20 prior infections on basal immune parameters and the cellular and antibody response to YF17D in 250 healthy young individuals. Multivariate regression found that sex and cytomegalovirus (CMV) infection significantly contribute to baseline immune variation but do not affect vaccine responses except for reduced YF17D-specific CD8+ frequencies in CMV-infected males. However, the abundance at baseline of non-specific cytokine-expressing T helper cells in circulation is associated with stronger vaccine responses, a state that smoking favors. Additionally, an elevated baseline level of interferon-stimulated CXCL10 is linked to poorer vaccination outcomes. Altogether, YF17D reactivity is conditioned by the baseline immune status independent of sex and CMV-related variations.