Plant growth and crop yield are essentially determined by photosynthesis when considering carbon dioxide (CO2) availability. CO2 diffusion inside a leaf is one of the factors that dictate the CO2 concentrations in chloroplasts. Carbonic anhydrases (CAs) are zinc-containing enzymes that interconvert CO2 and bicarbonate ions (HCO3-), which, consequently, affect CO2 diffusion and thus play a fundamental role in all photosynthetic organisms. Recently, the great progress in the research in this field has immensely contributed to our understanding of the function of the β-type CAs; however, the analysis of α-type CAs in plants is still in its infancy. In this study, we identified and characterized the OsαCA1 gene in rice via the analysis of OsαCAs expression in flag leaves and the subcellular localization of its encoding protein. OsαCA1 encodes an α-type CA, whose protein is located in chloroplasts with a high abundance in photosynthetic tissues, including flag leaves, mature leaves, and panicles. OsαCA1 deficiency caused a significant reduction in assimilation rate, biomass accumulation, and grain yield. The growth and photosynthetic defects of the OsαCA1 mutant were attributable to the restricted CO2 supply at the chloroplast carboxylation sites, which could be partially rescued by the application of an elevated concentration of CO2 but not that of HCO3-. Furthermore, we have provided evidence that OsαCA1 positively regulates water use efficiency (WUE) in rice. In summary, our results reveal that the function of OsαCA1 is integral to rice photosynthesis and yield potential, underscoring the importance of α-type CAs in determining plant physiology and crop yield and providing genetic resources and new ideas for breeding high-yielding rice varieties.