Abstract
In-band full-duplex (IBFD) communication systems offer a promising means of improving spectral efficiency by enabling simultaneous transmission and reception on the same frequency channel. Despite this advantage, self-interference (SI) remains a major challenge to their practical deployment. Among the different SI cancellation (SIC) techniques, this paper focuses on digital SIC methodologies tailored for multiple-input multiple-output (MIMO) wireless transceivers operating under digital beamforming architectures. Two distinct digital SIC approaches are evaluated, employing a generalized memory polynomial (GMP) model augmented with Itô-Hermite polynomial basis functions and a phase-normalized neural network (PNN) to effectively model the nonlinearities and memory effects introduced by transmitter and receiver hardware impairments. The robustness of the SIC is further evaluated under both single off-line training and closed-loop real-time adaptation, employing estimation techniques such as least squares (LS), least mean squares (LMS), and fast Kalman (FK) for model coefficient estimation. The performance of the proposed digital SIC techniques is evaluated through detailed simulations that incorporate realistic power amplifier (PA) characteristics, channel conditions, and high-order modulation schemes. Metrics such as error vector magnitude (EVM) and total bit error rate (BER) are used to assess the quality of the received signal after SIC under different signal-to-interference ratio (SIR) and signal-to-noise ratio (SNR) conditions. The results show that, for time-variant scenarios, a low-complexity adaptive SIC can be realized using a GMP model with FK parameter estimation. However, in time-invariant scenarios, an open-loop SIC approach based on PNN offers superior performance and maintains robustness across various modulation schemes.