Abstract
To meet the growing demand for high-capacity wireless communications, orbital angular momentum (OAM) multiplexing has garnered significant attention due to the orthogonality between OAM modes, which enables enhanced channel capacity. In this work, we propose and experimentally demonstrate a metasurface-based OAM mode-division multiplexing (OAM-MDM) system operating in the E-band. The system employs Fabry–Perot cavity meta-atoms that offer high transmission efficiency and precise phase control, enabling metasurfaces capable of multiplexing and demultiplexing two distinct OAM modes. We establish an electromagnetic-based effective channel model that characterizes the magnitude and phase variations between transmitted and received OAM modes from the radiated electric field. Specifically, the proposed effective wireless channel model captures not only the desired mode-to-mode transmission but also the inter-mode interference and represents these effects in a mathematically tractable form suitable for communication-theoretic analysis. Furthermore, the system performance is comprehensively evaluated by comparing the achievable rates derived from both simulations and experimental measurements under varying input power levels. Experimental results demonstrate that an achievable rate of up to 41.8 bits/s/Hz is attained at an input power of 4.9 dBm. This metasurface-based OAM-MDM system presents a promising approach for future high-capacity free-space communication. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1038/s41598-026-40149-7.