Abstract
The collective patterns that emerge in schooling fish are often analyzed using models of self-propelled particles in unbounded domains. However, while schooling fish in both field and laboratory settings interact with domain boundaries, these effects are typically ignored. Here, we propose a model that incorporates geometric confinement, by accounting for both flow and wall interactions, into existing data-driven behavioral rules. We show that new collective phases emerge where the school of fish "follows the tank wall" or "double mills." Importantly, confinement induces repeated switching between two collective states, schooling and milling. We describe the group dynamics probabilistically, uncovering bistable collective states along with unintuitive bifurcations driving phase transitions. Our findings support the hypothesis that collective transitions in fish schools could occur spontaneously, with no adjustment at the individual level, and opens venues to control and engineer emergent collective patterns in biological and synthetic systems that operate far from equilibrium.