Don't believe everything you read...

  • user warning: Table './apbc_org_uk_@002d_member/cache_filter' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_filter WHERE cid = '2:fb0ad108b5a9970d37b29b606dc80570' in /home/jbellapbc/public_html/includes/cache.inc on line 25.
  • user warning: Table './apbc_org_uk_@002d_member/cache_filter' is marked as crashed and last (automatic?) repair failed query: UPDATE cache_filter SET data = '<p><span style=\"font-size:12.0pt;\">Over the last twenty years the pet dog has finally been recognised as a worthy subject for scientific scrutiny and thus has followed an explosion in studies relating to dog behaviour and the way in which dogs and their owners interact. In the name of scientific progress it can be assumed that our knowledge and understanding of the world as it relates to our dogs has therefore&nbsp;</span><span style=\"font-size:12.0pt;\">also increased dramatically.</span></p>\n<div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">However, this is not necessarily the case. As recognised in the field of medicine and more recently veterinary science, where clinical skills have evolved an &lsquo;evidence-based&rsquo; approach, not all evidence is equal.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Our skills in how to evaluate the plethora of information are now more important than ever, especially in the light of use of the available information by dog trainers and behaviourists in order to support or refute their personal positions and beliefs, and justify their methods and actions.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">The hierarchy of evidence</span></b></div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Studies of populations are often used to search for patterns of individuals with certain characteristics. But what counts as good scientific evidence between a variable of interest and a desired or undesired outcome? The &lsquo;gold standard&rsquo; is considered to be from a randomised controlled trial, but these are few and far between in dog behaviour studies due to the nature of what we are studying. For example, it would not be feasible for a study to randomly assign dog owners to two groups, one being allowed to take their dog to training classes and the other not allowed! </span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Thankfully, observational studies, such as cohort, case-control and cross-sectional, can still give us useful information from simply observing what is naturally occurring in a study population, but only if they are conducted well.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Descriptions of a single, or series, of cases, are subject to bias as they are not compared to anything &lsquo;normal&rsquo;. As such, &lsquo;expert opinion&rsquo; alone is generally considered low evidence; not because they are not knowledgeable experts, but because of inevitable bias in what is seen. For example, a behaviourist may see many cases of aggression in terriers that were bred in a shed, leading them to formulate a hypothesis. However, they do not get to see the potentially many other non-aggressive terriers bred in a shed that may be quite happily living as pets. Hence the need for a comparison control population.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">Control yourself</span></b></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">When comparing individuals with a condition (cases) to those without (controls) it is important that the control population are truly representative of the cases.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">A good question to ask is would a control individual end up as a case if it was to have the condition of interest? If not, then the control is inadequate, and any differences found could be due to underlying differences between the groups being compared, rather than evidence of any association between the variables of interest.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">For example, many studies of &lsquo;which dogs bite?&rsquo; use a case population of dogs bites reported at a hospital and compare them to a &lsquo;registered&rsquo; sample of dogs in the local area. Firstly, we know that of the dogs that bit, only a proportion were usually registered, and secondly, there is likely to already be a bias in the presentation of dog bite cases to a hospital, for example, bites from larger dogs or to children.</span></div>\n<div style=\"line-height:normal\"><b>&nbsp;</b></div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">Measure twice, cut once</span></b></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">If something is to be investigated, it first needs to be measured. How a variable of interest is measured can affect the findings of a study. Measurements should be objective and repeatable. It should not matter who takes the measurement, or, for example, who defines the dog as aggressive, as it would be the same no matter who did when a clear &lsquo;case definition&rsquo; was used. Anything open to interpretation is very suspicious, especially if it is the dog owners themselves making the decisions, as their knowledge and interpretation of dog behaviour varies. As another example, identification of dog breeds by owners or bite victims can be problematic.</span></div>\n<div style=\"line-height:normal\"><b>&nbsp;</b></div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">Association does not imply causation</span></b></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Just because two things appear to be associated, it does not mean that one caused the other. A prime example is the commonly found association between consumption of alcohol and lung cancer, where the lung cancer is really caused by smoking. This is an example of &lsquo;confounding&rsquo;, where associations with a third variable on the causal pathway can confuse matters. It is possible to account for the effects of confounder variables during the design stage of a study, or during analysis using more advanced statistical techniques, however only <u>if</u> data on potential confounding variables is also collected from that population. Unfortunately this is not always the case.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Whenever a study appears to report an association, ask yourself if you can think of anything else that might be associated with the variable of interest, and the outcome, that they have not accounted for. For example, both small dogs and those that lie on the sofa may be reported to be more likely to bite, but small dogs are also more likely to lie on the sofa. Which is more likely to be the cause of aggression, being small or being on the sofa? Another problem is that of reverse causality, the &lsquo;chicken and egg&rsquo; situation, what came first, for example, neutering or aggression?. Longitudinal study designs can help with this but are rare because of the costs to run them for all that time.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">Size isn&rsquo;t everything</span></b></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">It is true that in population-based studies, large sample sizes are usually construed as a good thing, because the larger the sample, the more statistical power that is available to detect a &lsquo;real difference&rsquo; in the data and the less likelihood of &lsquo;false&rsquo; associations being erroneously reported. However, a large sample size means nothing if the design and analysis of the study is fundamentally flawed.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">The importance of sample size also depends on the research question; a few animals may be perfectly sufficient in order to demonstrate a theory or experimental idea. For example, the study by Bradshaw <i>et al</i> at the University of Bristol is often dismissed with the comment that it only contained 19 neutered male dogs. The study was designed to test the hypothesis that dogs live in a pack hierarchy with a leader. Thus in a group of domestic dogs living together, we would expect to be able to observe hierarchical relationships if they existed. It would not matter if the group consisted of 5 or 50 dogs. On the other hand, just because a cross-sectional study contains a few hundred dogs it does not mean that it is any good if the design of the study was not sufficiently robust.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">You say potato I say potato</span></b></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Even when presented with the same information, interpretation can differ. How any individual, be it a scientist or a dog trainer, interprets data, depends on their prior beliefs, and what they &lsquo;want&rsquo; to see. We are all human beings.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Coming back to our dogs on the sofa example, a dataset may show that dogs are more likely to bite if allowed on the sofa. This may be interpreted by a dog trainer who supports the dominance theory as evidence that spoiling dogs causes pack hierarchy problems.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">A trainer with other beliefs may interpret the data differently; perhaps for them it reflects a more physical owner-dog relationship that is provoking aggression, or the aggression caused the dog to be allowed on the sofa because nobody dared to argue with it.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">It may also be interpreted as a simple problem of exposure; if a dog is not on the sofa in the first instance, it is not going to be able to bite someone trying to move it. Both writers and readers of science should think carefully about their own interpretation of the data presented.</span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><b><span style=\"font-size:12.0pt;\">Nobody&rsquo;s perfect</span></b></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div>In a recent systematic review of the risk factors for human-directed dog aggression conducted by our team at the University of Liverpool, of over 200 published studies deemed appropriate for evaluation (i.e.not case histories or series), almost all were rejected due to one or more (important) weaknesses. Furthermore, the handful that were accepted for inclusion only provided moderate strength evidence; there were no high quality studies. This is important considering such research is often used to decide policy and future research funding.</div>\n<div>&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">Having said that, it does not mean that we know nothing! Science is a continually evolving process, building on more &lsquo;woolly&rsquo; ideas generated from previous studies, and testing them further. The study of companion animals is in its infancy, but we have a pretty good idea of where to start looking and are making fast progress. There is no doubt that there is a wealth of information out there to feast our eyes upon, but it requires careful consideration, critical evaluation and interpretation by the individual reader, with special attention to the type of information presented and the subsequent strength of the evidence. </span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><span style=\"font-size:12.0pt;\">So remember, <b>don&rsquo;t believe everything you read,&nbsp;</b></span></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\"><em>Dr Carri Westgarth, BSc PhD</em></div>\n<div style=\"line-height:normal\">&nbsp;</div>\n<div style=\"line-height:normal\">References:</div>\n<div style=\"line-height:normal\">\n<p class=\"ecxMsoNormal\" style=\"line-height: 20px; margin-top: 0px; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: \'Times New Roman\', serif; \"><span style=\"line-height: 19px; font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); \">Bradshaw JWS, Blackwell EJ, Casey RA:&nbsp;<b style=\"line-height: 19px; font-weight: bold; \">Dominance in domestic dogs-useful construct or bad habit?</b>&nbsp;<i style=\"line-height: 19px; font-style: italic; \">J Vet Behav-Clin Appl Res&nbsp;</i>2009,&nbsp;<b style=\"line-height: 19px; font-weight: bold; \">4</b>(3):135-144.</span></p>\n<p class=\"ecxMsoNormal\" style=\"line-height: 20px; margin-top: 0px; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: \'Times New Roman\', serif; \"><span style=\"line-height: 19px; font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); \">&nbsp;</span></p>\n<p class=\"ecxMsoNormal\" style=\"line-height: 20px; margin-top: 0px; margin-right: 0cm; margin-bottom: 0.0001pt; margin-left: 0cm; font-size: 12pt; font-family: \'Times New Roman\', serif; \"><span style=\"line-height: 19px; font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); \">Newman J, Westgarth C, Pinchbeck GL, Morgan KL, Dawson S, Christley RM:&nbsp;<b style=\"line-height: 19px; font-weight: bold; \">Human directed dog aggression: A systematic review</b>.&nbsp;<i style=\"line-height: 19px; font-style: italic; \">Proceedings of the</i><i style=\"line-height: 19px; font-style: italic; \">International Society for Anthrozoology 20<sup style=\"line-height: 15px; \">th</sup>&nbsp;Anniversary Conference, Human-Animal Interactions: Challenges and Rewards</i>, Indianapolis, Indiana, August 4-6 2011.</span></p>\n</div>\n<p>&nbsp;</p>\n</div>\n', created = 1493479662, expire = 1493566062, headers = '', serialized = 0 WHERE cid = '2:fb0ad108b5a9970d37b29b606dc80570' in /home/jbellapbc/public_html/includes/cache.inc on line 108.
  • user warning: Table './apbc_org_uk_@002d_member/cache_filter' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_filter WHERE cid = '3:2c28963416bf2aa0ec357ec0a5ea4422' in /home/jbellapbc/public_html/includes/cache.inc on line 25.
  • user warning: Table './apbc_org_uk_@002d_member/cache_filter' is marked as crashed and last (automatic?) repair failed query: SELECT data, created, headers, expire, serialized FROM cache_filter WHERE cid = '2:e19b2b051fc7afe6c6d4fe7bfa1c970e' in /home/jbellapbc/public_html/includes/cache.inc on line 25.
  • user warning: Table './apbc_org_uk_@002d_member/cache_filter' is marked as crashed and last (automatic?) repair failed query: UPDATE cache_filter SET data = '<p><a href=\"http://www.apbc.org.uk/help/regions\"><img align=\"center\" alt=\"\" src=\"/sites/default/files/images/locator_image.png\" /></a> <br />\n<a href=\"/help/regions\">Click here to find an APBC member in your area</a></p>\n', created = 1493479662, expire = 1493566062, headers = '', serialized = 0 WHERE cid = '2:e19b2b051fc7afe6c6d4fe7bfa1c970e' in /home/jbellapbc/public_html/includes/cache.inc on line 108.

Over the last twenty years the pet dog has finally been recognised as a worthy subject for scientific scrutiny and thus has followed an explosion in studies relating to dog behaviour and the way in which dogs and their owners interact. In the name of scientific progress it can be assumed that our knowledge and understanding of the world as it relates to our dogs has therefore also increased dramatically.

However, this is not necessarily the case. As recognised in the field of medicine and more recently veterinary science, where clinical skills have evolved an ‘evidence-based’ approach, not all evidence is equal.
 
Our skills in how to evaluate the plethora of information are now more important than ever, especially in the light of use of the available information by dog trainers and behaviourists in order to support or refute their personal positions and beliefs, and justify their methods and actions.
 
The hierarchy of evidence
Studies of populations are often used to search for patterns of individuals with certain characteristics. But what counts as good scientific evidence between a variable of interest and a desired or undesired outcome? The ‘gold standard’ is considered to be from a randomised controlled trial, but these are few and far between in dog behaviour studies due to the nature of what we are studying. For example, it would not be feasible for a study to randomly assign dog owners to two groups, one being allowed to take their dog to training classes and the other not allowed!
 
Thankfully, observational studies, such as cohort, case-control and cross-sectional, can still give us useful information from simply observing what is naturally occurring in a study population, but only if they are conducted well.
 
Descriptions of a single, or series, of cases, are subject to bias as they are not compared to anything ‘normal’. As such, ‘expert opinion’ alone is generally considered low evidence; not because they are not knowledgeable experts, but because of inevitable bias in what is seen. For example, a behaviourist may see many cases of aggression in terriers that were bred in a shed, leading them to formulate a hypothesis. However, they do not get to see the potentially many other non-aggressive terriers bred in a shed that may be quite happily living as pets. Hence the need for a comparison control population.
 
Control yourself
 
When comparing individuals with a condition (cases) to those without (controls) it is important that the control population are truly representative of the cases.
 
A good question to ask is would a control individual end up as a case if it was to have the condition of interest? If not, then the control is inadequate, and any differences found could be due to underlying differences between the groups being compared, rather than evidence of any association between the variables of interest.
 
For example, many studies of ‘which dogs bite?’ use a case population of dogs bites reported at a hospital and compare them to a ‘registered’ sample of dogs in the local area. Firstly, we know that of the dogs that bit, only a proportion were usually registered, and secondly, there is likely to already be a bias in the presentation of dog bite cases to a hospital, for example, bites from larger dogs or to children.
 
Measure twice, cut once
 
If something is to be investigated, it first needs to be measured. How a variable of interest is measured can affect the findings of a study. Measurements should be objective and repeatable. It should not matter who takes the measurement, or, for example, who defines the dog as aggressive, as it would be the same no matter who did when a clear ‘case definition’ was used. Anything open to interpretation is very suspicious, especially if it is the dog owners themselves making the decisions, as their knowledge and interpretation of dog behaviour varies. As another example, identification of dog breeds by owners or bite victims can be problematic.
 
Association does not imply causation
 
Just because two things appear to be associated, it does not mean that one caused the other. A prime example is the commonly found association between consumption of alcohol and lung cancer, where the lung cancer is really caused by smoking. This is an example of ‘confounding’, where associations with a third variable on the causal pathway can confuse matters. It is possible to account for the effects of confounder variables during the design stage of a study, or during analysis using more advanced statistical techniques, however only if data on potential confounding variables is also collected from that population. Unfortunately this is not always the case.
 
Whenever a study appears to report an association, ask yourself if you can think of anything else that might be associated with the variable of interest, and the outcome, that they have not accounted for. For example, both small dogs and those that lie on the sofa may be reported to be more likely to bite, but small dogs are also more likely to lie on the sofa. Which is more likely to be the cause of aggression, being small or being on the sofa? Another problem is that of reverse causality, the ‘chicken and egg’ situation, what came first, for example, neutering or aggression?. Longitudinal study designs can help with this but are rare because of the costs to run them for all that time.
 
Size isn’t everything
 
It is true that in population-based studies, large sample sizes are usually construed as a good thing, because the larger the sample, the more statistical power that is available to detect a ‘real difference’ in the data and the less likelihood of ‘false’ associations being erroneously reported. However, a large sample size means nothing if the design and analysis of the study is fundamentally flawed.
 
The importance of sample size also depends on the research question; a few animals may be perfectly sufficient in order to demonstrate a theory or experimental idea. For example, the study by Bradshaw et al at the University of Bristol is often dismissed with the comment that it only contained 19 neutered male dogs. The study was designed to test the hypothesis that dogs live in a pack hierarchy with a leader. Thus in a group of domestic dogs living together, we would expect to be able to observe hierarchical relationships if they existed. It would not matter if the group consisted of 5 or 50 dogs. On the other hand, just because a cross-sectional study contains a few hundred dogs it does not mean that it is any good if the design of the study was not sufficiently robust.
 
You say potato I say potato
 
Even when presented with the same information, interpretation can differ. How any individual, be it a scientist or a dog trainer, interprets data, depends on their prior beliefs, and what they ‘want’ to see. We are all human beings.
 
Coming back to our dogs on the sofa example, a dataset may show that dogs are more likely to bite if allowed on the sofa. This may be interpreted by a dog trainer who supports the dominance theory as evidence that spoiling dogs causes pack hierarchy problems.
 
A trainer with other beliefs may interpret the data differently; perhaps for them it reflects a more physical owner-dog relationship that is provoking aggression, or the aggression caused the dog to be allowed on the sofa because nobody dared to argue with it.
 
It may also be interpreted as a simple problem of exposure; if a dog is not on the sofa in the first instance, it is not going to be able to bite someone trying to move it. Both writers and readers of science should think carefully about their own interpretation of the data presented.
 
Nobody’s perfect
 
In a recent systematic review of the risk factors for human-directed dog aggression conducted by our team at the University of Liverpool, of over 200 published studies deemed appropriate for evaluation (i.e.not case histories or series), almost all were rejected due to one or more (important) weaknesses. Furthermore, the handful that were accepted for inclusion only provided moderate strength evidence; there were no high quality studies. This is important considering such research is often used to decide policy and future research funding.
 
Having said that, it does not mean that we know nothing! Science is a continually evolving process, building on more ‘woolly’ ideas generated from previous studies, and testing them further. The study of companion animals is in its infancy, but we have a pretty good idea of where to start looking and are making fast progress. There is no doubt that there is a wealth of information out there to feast our eyes upon, but it requires careful consideration, critical evaluation and interpretation by the individual reader, with special attention to the type of information presented and the subsequent strength of the evidence.
 
So remember, don’t believe everything you read, 
 
Dr Carri Westgarth, BSc PhD
 
References:

Bradshaw JWS, Blackwell EJ, Casey RA: Dominance in domestic dogs-useful construct or bad habit? J Vet Behav-Clin Appl Res 2009, 4(3):135-144.

 

Newman J, Westgarth C, Pinchbeck GL, Morgan KL, Dawson S, Christley RM: Human directed dog aggression: A systematic reviewProceedings of theInternational Society for Anthrozoology 20th Anniversary Conference, Human-Animal Interactions: Challenges and Rewards, Indianapolis, Indiana, August 4-6 2011.