The aim of this study was to compare different antigen retrieval methods to improve the outcome of immunohistochemistry (IHC) performed on osteoarthritic (OA) cartilage obtained from total knee replacement operation. A voluminous and dense extracellular matrix of articular cartilage inhibits antibody penetration, and therefore, proteins present at low concentrations and masked during fixation may need antigen retrieval to enhance an IHC outcome. We focused on the IHC detection of a minor but diagnostically promising cartilage glycoprotein, CILP-2 (cartilage intermediate layer protein 2), to demonstrate the effect of four different protocols: (1) heat-induced epitope retrieval (HIER), (2) proteolytic-induced epitope retrieval applying proteinase K and hyaluronidase (PIER), (3) HIER combined with PIER, and (4) no antigen retrieval (control). A semi-quantitative staining assessment based on the CILP-2 staining extent was applied. Out of the tested antigen retrieval protocols, the best CILP-2 IHC staining results were achieved by PIER. Combining PIER with HIER did not improve CILP-2 staining in the given experimental setting. Rather the opposite, the application of heat reduced the positive effect of PIER on CILP-2 staining and resulted in the frequent detachment of sections from the slides. Our findings emphasize the need for proper adaptation of antigen retrieval protocols for IHC to maximize the quantitative evaluation of minor matrix proteins in OA articular cartilage samples.