Abstract
In this work I present two methods for measuring agreement in nominal and ordinal data. The measures, which employ Gower-type distances, are simple, intuitive, and easy to compute for any number of units and any number of coders. Influential units and/or coders are easily identified. I consider both one-way and two-way random sampling designs, and develop an approach to Bayesian inference for each. I apply the methods to simulated data and to two real datasets, the first from a one-way radiological study of congenital diaphragmatic hernia, and the second from a two-way study of psychiatric diagnosis. Finally, I consider agreement scales and suggest that Gaussian mutual information can perhaps provide a scale that is more useful than the scale most commonly used. The methods I propose are supported by my open source R package, goweragreement, which is available on the Comprehensive R Archive Network.