Cat and Dog people under attack! Cynicism and ignorance passing as science journalism

According to an unpublished study that has attracted attention recently, cat people are more intelligent than dog people. Details of the study and its findings are provided in this press release, and it has been widely reported on many websites, such as Time and Huffington Post. As the study has not yet been published, I cannot comment in any depth on its quality, but from what little I have seen so far, I would say that it seems sound enough in terms of its scientific rigour. However, science journalist Faye Flam, writing for Knight Science Journalism at MIT, begs to differ. In a piece titled Cat people smarter than dog people? Study should be a hoax, but probably isn't, Flam takes the position that the study is meaningless rubbish designed to get publicity, and that thinking people should not take it seriously. However, Flam's critique demonstrates a complete lack of knowledge or insight into the topic, lacks substance and seems to be little more than an exercise in cheap cynicism.

Differences in personality traits between people who describe themselves either as "cat people" or "dog people" have been researched in a number of academic studies. For example, one well-known study found that cat people differed from dog people in all of the Big Five personality traits, being lower in extraversion, agreeableness, and conscientiousness, and higher in neuroticism and openness to experience. (I have discussed the implications of this particular study in an article on my blog on Psychology Today.) Another interesting and soon to be published paper on this topic looking at other personality traits can be viewed here. The study I focus on here, by Denise Guastello and colleagues, looked at yet another set of personality traits (Cattell's 16PF), and also appears to be the first study of its kind to examine whether cat and dog people differ in intelligence. The study surveyed 600 college students, and found that not only did cat and dog people differ on a variety of personality traits, in line with previous research, but that cat people scored higher than dog people in a measure of intelligence.

Although the details are somewhat sketchy, so far as I can tell, this seems like a valid research design with reasonable conclusions. Openness to experience is associated with intelligence and knowledge (e.g. see this article), and previous research has found that cat people are higher than dog people in this trait, so it does not seem all that surprising that they would be higher in intelligence as well. Of course, it is possible that the results found by Guastello and colleagues might turn out to be false positives or methodological artefacts, but in the absence of any evidence for this at this stage I am willing to give them the benefit of the doubt. On the other hand, Faye Flam seems inclined to be much more cynical than I am and goes on at some length about why she thinks the whole thing is a big joke. Unfortunately, she never provides any actual evidence why she thinks this is a reasonable position to take and shows considerable ignorance in the process.

Flam claims that there is a not so subtle formula one can use to whip up these kinds of studies that will grab international publicity. Apparently, doing substantive research that builds on existing knowledge does not enter into it, it's all about marketing I suppose. 

All you have to do is round up a few college students [Does 600 count as "a few"?], and divide them into two categories. It might be cat people vs. dog people, or baseball vs. soccer fans, or people who prefer fish tacos vs. those who prefer chicken tacos.

The implication here seems to be that the categories chosen by Guastello and colleagues for their study are actually arbitrary and meaningless in themselves, and chosen simply for being "cute." As I have noted, there is an existing body of literature - some of the relevant papers can even be found in a few seconds on google - that provides evidence supporting the validity of the concept of people self-identifying as cat or dog people, but Flam does not appear to have done even such a rudimentary search on the topic. She also fails to demonstrate that she knows anything at all about the literature on personality and intelligence testing.

Then give them some tests. Claim the tests reveal intelligence, or introversion/extroversion, or whatever seems sexy and attention-getting.

 Tests for assessing intelligence have been around for over a hundred years and have been intensively researched since then. Tests for assessing personality traits have been around for nearly as long and have also been intensively researched. The 16PF test that Guastello and colleagues used in their study was developed in the 1940s. Flam implies that such tests only "claim" to reveal these things, as if this is not really true, with the implication that they might as well be bogus and are only included so that one can generate some "sexy" but actually meaningless results. This completely ignores the thousands of research papers that have been written on the subject that show that these tests actually do measure meaningful aspects of behavior.

As long as the samples aren't really big, chance alone will ensure that the results will be slightly different. One group will always look a little smarter or more outgaining [sic, presumably she means "outgoing"] or whatever you claim you’re testing. If not, add more tests. That’s the nature of what scientists refer to as noise.

Well gosh darn it, if there was only some way of telling if group differences in test scores are nothing more than random chance fluctuations or real substantive differences! Oh wait a second, I just remembered there are these things called "statistical significance tests" that are designed exactly for this purpose. To be fair though, these tests are not perfect and it is true that if you perform multiple tests on the same sample some of the results might be statistically significant due to chance. Fortunately, there are procedures for correcting for multiple comparisons that are discussed in, oh I don't know, every basic statistics textbook ever. And what would Flam consider to be a "really big" sample size that would provide a valid test of whether or not the results are due to chance? There are statistical procedures for determining if a given sample size is adequate for the analyses being performed, but Flam does not say if she is aware of this or not.

Then assume your observed differences are not random noise but meaningful correlations and make up a story to explain it. The researchers in this case had a ready explanation for why the dog people were more “lively”…

 You mean the researchers actually attempted to provide an explanation for the results they observed?? Who do they think they are?? Oh, but wait a second, isn't this a normal part of science? Formulate a hypothesis, design a study to test it, analyse the results and then provide an explanation of what the results mean and whether or not they confirm the original hypothesis? I don't understand, what exactly is Flam's problem here? Here is the researcher's explanation that Flam quotes:

“It makes sense that a dog person is going to be more lively, because they're going to want to be out there, outside, talking to people, bringing their dog," Guastello said. "Whereas, if you're more introverted, and sensitive, maybe you're more at home reading a book, and your cat doesn't need to go outside for a walk.”

 Hmm, Guastello and colleagues gave some people who self-identified as cat people or dog people some well researched personality tests and the results indicated that they differed on the expected traits and then explain why they think this result makes sense. I fail to see what is so shockingly ridiculous or unscientific about this that Flam thinks the study deserves to be treated as some sort of hoax.

To be fair, Faye Flam does convey one nugget of real information that is worth noting.

Maybe the study was extremely carefully done, but there’s really no way to know. I can’t find a paper. It looks like it all started with a talk at a conference.

Flam is correct to point out that the study has not yet been published in a peer-reviewed journal and what information is currently available derives from a conference presentation. Hence, news sites, such as Time and Huffington Post, that have hyped the results may be jumping the gun as it is impossible to assess the study’s rigour at this stage. If Flam had just pointed this out and left it at that I would not have a problem. However, there is no justification for what amounts to a cynical and uninformed accusation that the research study was performed in an academically shoddy manner simply to gain attention, an accusation based on no evidence whatsoever. Flam waits until near the end of her critique to acknowledge that for all she knows the study might be a very good one. Since she has no evidence that the study is actually a bad one, then what justification does she have for such an extreme lack of charity to the authors?

I would think that "science journalism" should attempt to provide an informed and thoughtful discussion of scientific issues, but what we have here is nothing but crud bereft of substance or insight. "Science dismissal" would be a more apt term for it.

If you are interested in reading more in-depth articles on psychology why not check out my blog Unique - Like Everybody Else on Psychology Today. Also please consider following me on 

Facebook, Google Plus or Twitter.

© Scott McGreal. Please do not reproduce without permission. Brief excerpts may be quoted as long as a link to the original article is provided.