The messy science of assessing working ability in dogs

By Carol Beuchat PhD

Dealing with the behavioral issues of dogs is a big business. Trainers and therapists abound, as do the professional suffixes tagged after the names of those who took some sort of course or were accredited by an organization.

I have no idea whether the majority of these people know what they're doing. I can tell you, though, that the scientific literature about behavior in dogs is a morass of tangled, insufficient, and often contradictory studies. In fact, in the field of canine behavior, science is marching forward very slowly, even as the ranks of lay behaviorists swell and practitioners claim to become ever more authoritative and confident of their expertise.​

How reliable are behavioral assessments of dogs? Are they repeatable, meaningful, and verifiable? This especially matters when dogs are being assessed for working ability, as in breeding programs for service, police, and other types of working dogs. Breeding and training these dogs is time consuming and expensive, and being able to accurately assess the usefulness of a potential working animal is the differences between success and failure.

A recent study evaluated the reliability and validity of various behavioral tests that are used to assess the traits of working dogs (Brady et al 2018). What the researchers found should give pause to anybody that professes to have a deep and true understanding of canine behavior.
The aim of this study was to assess the reliability and predictive validity of tests designed to assess relevant traits of working dogs. They reviewed many published, peer-reviewed studies and proceeded to evaluate only those that met strict criteria for focus, design, methods, and analysis. Of the thousands of behavior studies they reviewed, 16 were judged to meet the requirements of their analysis.

For the selected studies, they evaluated the reliability, repeatability, and predictive validity of the behavioral tests, both within and among "raters" (the individuals conducting the evaluation). They had an assortment of problems to deal with, like the use of different terminology for the same trait. More seriously, they found that data were often inadequately reported, statistical analyses were misinterpreted, and there was an overall lack of concordance in the various methods used for assessment.

Ultimately, they concluded that there is "a widespread lack of information relating to the reliability and validity of measures to assess behaviour and inconsistencies in terminologies, study parameters and indices of success...This review indicates that we are still not addressing concerns over the lack of standardisation (sic) amongst research on dog behavioural tests".
What this means is that we are probably not as good at assessing behavioral traits as we think we are or, at the very least, it's hard to tell how good we are because of limitations of the studies and the lack of standardization in terminology and protocol. Certainly, it should give pause to anybody relying on "experts" for advice, most of whom cannot lean on published studies to support claims about the validity and reliability of their assessments.

Buyer beware.
You can download a copy of the study below.
REFERENCES

Brady, K., N. Cracknell, H. Zulch, and D.S. Mills. 2018. A systematic review of the reliability and validity of behavioural tests used to assess behavioural characteristics important in working dogs. Frontiers in Veterinary Science 5(103);doi: 10.3389/fvets.2018.00103.

Are you interested in canine behavior?

You can learn about the genetic basis of behavior in dogs in ICB's online course.​

The Genetics of Behavior & Performance
Online course starts 4 March 2019
10 weeks, $125​

Learn more
To learn more about the genetics of dogs, check out
ICB's online courses

***************************************

Visit our Facebook Groups

ICB Institute of Canine Biology
...the latest canine news and research

ICB Breeding for the Future
...the science of animal breeding​

Continue reading...