Retinoic acid (RA) signaling is a master regulator of vertebrate development with crucial roles in body axis orientation and tissue differentiation, including in the reproductive system. However, a mechanistic understanding of how RA signaling governs cell lineage identity is often missing. Here, leveraging prostate organoid technology, we show that RA signaling orchestrates the commitment of adult mouse prostate progenitors to glandular identity, epithelial barrier integrity, and specification of prostatic lumen. RA-dependent RARγ activation promotes the expression of Foxa1, which synergizes with the androgen pathway for luminal expansion, cytoarchitecture and function. FOXA1 mutations are common in prostate and breast cancers, though their pathogenic mechanism is incompletely understood. Combining functional genetics with structural modeling of FOXA1 folding and chromatin binding analyses, we discover that FOXA1F254E255 is a loss-of-function mutation compromising its transcriptional function and luminal fate commitment of prostate progenitors. Overall, we define RA as an instructive signal for glandular identity in adult prostate progenitors. Importantly, we identify cancer-associated FOXA1 indels affecting residue F254 as loss-of-function mutations promoting dedifferentiation of adult prostate progenitors.