Abstract
Frustrated magnetic systems arising in geometrically constrained lattices represent rich platforms for exploring unconventional phases of matter, including fractional magnetization plateaus, incommensurate orders and complex domain dynamics. However, determining the microscopic spin configurations that stabilize such phases is a key challenge, especially when in-plane and out-of-plane spin components coexist and compete. Here, we combine neutron scattering and magnetic susceptibility experiments with simulations to investigate the emergence of field-induced fractional plateaus and the related criticality in a frustrated magnet holmium tetraboride (HoB(4)) that represents the family of rare earth tetraborides that crystalize in a Shastry-Sutherland lattice in the ab plane. We focus on the interplay between classical and quantum criticality near phase boundaries, as well as the role of material defects in the stabilization of the ordered phases. We find that simulations using classical annealing can explain certain observed features in the experimental Laue diffraction and the origin of multiple magnetization plateaus. Our results show that defects and out-of-plane interactions play an important role and can guide the route towards resolving microscopic spin textures in highly frustrated magnets.