Canonical serine protease inhibitor proteins occupy the substrate-binding groove of their target enzyme via a surface loop. Unlike true substrates, inhibitors are cleaved by the target protease extremely slowly. Here, we applied an unbiased directed evolution approach to investigate which loop residues hamper proteolytic cleavage while maintaining high-affinity binding. As a protease inhibitor model system, we used human chymotrypsin C (CTRC) and Schistocerca gregaria protease inhibitor 2 (SGPI-2). We created an SGPI-2 library displayed on M13 phage by randomizing the binding loop amino acid positions, with the exception of the structurally indispensable Cys residues. We selected binding phage clones against active CTRC and the inactive mutant Ser195Ala. All CTRC-selected binders inhibited CTRC activity and also bound to the inactive Ser195Ala mutant, but the Ser195Ala-selected clones proved to be either inhibitors or substrates of active CTRC. Substrate-like behavior of SGPI-2 variants was associated with the absence of the P2 Thr, the residue next to the specificity determinant P1 amino acid. The selected SGPI-2 variants containing a P2 Thr bound strongly to CTRC even if the other loop residues deviated from the optimal inhibitory consensus sequence. In the absence of a P2 Thr, however, SGPI-2 variants became substrates unless all other loop residues were optimal for binding. Structural modeling confirmed that P2 Thr is important for organizing a stabilizing H-bond network. The observations indicate that binding loops of canonical serine protease inhibitors evolved amino acids not only to support tight binding to the target enzyme but also to inhibit proteolytic cleavage.