Sipilä, IMV;
Steele, J;
Dickens, L;
Martin, L;
(2023)
Bones of contention: a double-blind study of experts’ ability to classify sheep and goat astragali from images.
Archaeological and Anthropological Sciences
, 15
(12)
, Article 187. 10.1007/s12520-023-01865-7.
Preview |
Text
Martin_Sipila et al 2023 Bones of Contention Double blind study.pdf Download (2MB) | Preview |
Abstract
In zooarchaeology, animal bones are normally identified using comparative macro-morphological methods, which involve visual comparison of the bone with reference materials. However, recent work has oppugned the reliability of these methods. Although previous studies applying macro-morphological methods to identify sheep and goat bones have found low error rates, these results are based on small numbers of analysts and large numbers of different bone types and do not properly account for ambiguous ‘sheep/goat’ classifications. We present an extensive blind study of performance and reliability for binary macro-morphological species identification using just the astragalus. Each participant made independent comparative identifications on a random subset, including repeat presentations for consistency analysis. No sheep/goat category was offered. Instead, participants reported confidence scores on each sample. The participants also reported the reference materials used and indicated their regions of attention in each image. Findings indicate that neither the use of reference materials nor experience is a good predictor of accuracy, although more experienced analysts are found to be more consistent. Forcing binary classifications leads to a more transparent analysis but indicates lower performance scores than reported elsewhere, while corresponding confidence scores positively correlate with accuracy. Qualitative analysis of reported attention regions indicates that mistakes can occur when there is an overlap in the morphologies of the two species. We conclude that overreliance on reference materials impacts performance when the morphology of reference materials is not representative of the population variance, which is especially evident when the wider bone morphology is not adequately integrated into the classification decision.
Archive Staff Only
 |
View Item |
