Abstract
The polarized optical states in the transmission spectrum of a twisted-nematic Fabry-Pérot cavity with the distinctly broken Mauguin's waveguide regime have been theoretically and experimentally investigated. Specific features of the electric field-induced transformation of the polarization and spectral characteristics of eigenmodes of the neighboring series at the overlap resonant frequencies have been examined. It is demonstrated that the linear polarizations of eigenmodes at the cavity boundaries remain nearly orthogonal and their frequency trajectories reproduce the avoided crossing phenomenon. The experimental data are confirmed analytically and by the numerical simulation of light transmission through the investigated anisotropic multilayer with the use of a Berreman matrix method. The results obtained can be generalized to any materials with the helix response.