Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) mRNA vaccines are effective at protecting from severe disease, but the protective antibodies wane rapidly even though SARS-CoV-2-specific plasma cells can be found in the bone marrow (BM). Here, to explore this paradox, we enrolled 19 healthy adults at 2.5-33 months after receipt of a SARS-CoV-2 mRNA vaccine and measured influenza-, tetanus- or SARS-CoV-2-specific antibody-secreting cells (ASCs) in long-lived plasma cell (LLPC) and non-LLPC subsets within the BM. Only influenza- and tetanus-specific ASCs were readily detected in the LLPCs, whereas SARS-CoV-2 specificities were mostly absent. The ratios of non-LLPC:LLPC for influenza, tetanus and SARS-CoV-2 were 0.61, 0.44 and 29.07, respectively. In five patients with known PCR-proven history of recent infection and vaccination, SARS-CoV-2-specific ASCs were mostly absent from the LLPCs. We show similar results with measurement for secreted antibodies from BM ASC culture supernatant. While serum IgG titers specific for influenza and tetanus correlated with IgG LLPCs, serum IgG levels for SARS-CoV-2, which waned within 3-6 months after vaccination, were associated with IgG non-LLPCs. In all, our studies suggest that rapid waning of SARS-CoV-2-specific serum antibodies could be accounted for by the absence of BM LLPCs after these mRNA vaccines.