Monday 10 September 2007

20/20

Found via Kill Ten Rats:

http://www.overcomingbias.com/2007/08/hindsight-deval.html

Really interesting article talking about the way our brains automatically devalue statistics. The main point being that if you present people with a statistic and a mentally comfortable reason for that stat to be true (e.g Men are involved in proportionally fewer car accidents because they have better spatial awareness), then accepting the stat becomes much easier because the reasoning has been provided. Most people, especially when not actively engaging the cynical portion of their brain, will take the easy route of:

Do I agree with the premise (men have better spatial awareness)?
Do I agree with the reasoning (better spatial awareness -> better driving)?

If both those boxes are ticked, the stat (Men are involved in proportionally fewer car accidents) is automatically filed away as accepted.

The only way round this is to actively consider whether the reason given is the only factor. In the case of my rather hasty example, it is clearly not - there are a huge number of factors affecting driving ability and road safety - but it would be equally dishonest to say 'Women are involved in proportionally fewer car accidents because they are less aggressive drivers'. One of the two positions must be true, but to present it in such a simplistic manner is misleading.

Basically, next time you laugh at the 'obvious' conclusions published after research, it may be worth considering how obvious it is. And, more generally, applying this rule to any situation in which a statistic is used. Clearly it's not possible to question fully every statistic you read, but being a little cynical never hurt anyone*.

* This statement may not actually be true.

1 comment:

Zubon said...

Another point is that your brain will supply the explanation if it is missing. If you have a good explanation for why men are be safer drivers and why women are be safer drivers, you have rationalizations rather than reasons. I got really annoyed with my economics education when I realized that I had a perfectly good explanation for whatever happened, no matter what happened. If mutually exclusive stories are both plausible, we have a problem here.

On a side note, men have more crashes. The largest explanatory factor is that men drive more miles per year, but men still have more traffic crashes and fatalities even after taking that into account. The primary hypothesis is that men are far less risk averse, or possibly stupid.