Here is a non-peer-reviewed summary:

The most well known test-taking strategy advises students to never change an answer to a multiple-choice item. The justification of this theory is that the initial impression is usually correct. In fact, many faculty members advise their students not to change answers (Kussler, 1988). The literature states that, although most students believe in this theory, it is more common to see students change answers (Haase, Riley, Dunn, & Gaskins, 1992). In fact, not only do most students change at least one answer to a multiple-choice question, the overwhelming majority of these changes are from incorrect to correct answers (Benjamin, Cavell, & Shallenberger, 1984).

For peer-reviewed, try:

Effects of Student Confidence and Item Difficulty on Test Score Gains Due to Answer Changing, Philip H. Ramsey, Patricia P. Ramsey & Michael J. Barnes, Teaching of Psychology, Volume 14, Issue 4, 1987, DOI:10.1207/s15328023top1404_3

Significant gains, even for changes based on low confidence, were intepreted as suggesting previous cautions about answer changing are not warranted.

To quote Carl Sagan: “I try not to think with my gut. If I’m serious about understanding the world, thinking with anything besides my brain, as tempting as that might be, is likely to get me into trouble.”

The classic logic/instinct problem suddenly occurs to me here. How did it go? $100 is under one of three cups. You pick one. You’re shown that another is empty. Do you now want to change your choice? And there’s some dodgy probability calculations to rationalise changing.

You are probably thinking of the Monty Hall problem. Actually, the probability is not dodgy at all. If the person lifting the cups plays according to the rules (i.e. they always lift an empty cup, regardless of whether your initial choice was right or wrong), then changing your choice is provably the correct action, in terms of expected outcome.

One of the best ideas (imo) in it is: any critical decision should be made in the space of seven breaths.

Authoritative-sounding support for that say that your first instinct, after a bit of time thinking about important factors, is generally pretty good, and that any amount of time after that you just spending second-guessing yourself and going around in circles.

The classic logic/instinct problem suddenly occurs to me here. How did it go? $100 is under one of three cups. You pick one. You’re shown that another is empty. Do you now want to change your choice? And there’s some dodgy probability calculations to rationalise changing. Classic example of getting yourself into a lather of second-guessing, when you know full-well that your first guess was as good as any other. Better probably, since the guy running the scam probably knows you’ve guessed correctly and is trying to convince you to change so he keeps the money.

I’m sure I have heard of psych experiments where they’ve actually tested this second-guessing syndrome (possibly in the same conversation) that returned some sort of quantitative result supporting first guess vs second, but it was third-hand – can’t give you any references.

In my personal experience though, it seems to have been borne out to some extent, and it “feels right” – in the gut, you know?

Turns out it’s (1997) US$1.54 million.

The reference is in the blog post.

The study (IIRC it was a scientific study done somewhere) wasn’t designed to show that we’re bad at maths, but our gut feel is actually very good at solving problems with tons of variables, which is very difficult with “real maths”. — you can keep about 3 numbers in your “head”, but have a feel for many many numbers in your “gut”.

If you used the analysis of the air bag in your purchasing decision, you may optimise on that value rather than something else which may have a far greater impact on how safe the car is (like perhaps colour).

I don’t think you should discounting the replacement cost of airbags which fire accidentally or in non-life-vs-death situations as trivial – according to research, the 2001 cost of replacing a single airbag was NZ$1320, so if my car had 4 such airbags (driver+passenger, front+side), one good 16km/h jolt would effectively write off the vehicle (NZ$5280).

However, the same research shows both that there is subtantial benefit in the non life-vs-death case, and that you are also seriously undervaluing the dollar worth of the human life (unless your personal net assets are rather more substantial than they look). The paper puts a single life (death) at AZ$2.47 million dollars, based on “willingness to pay” settlement figures in legal claims.

The summary of the paper is (roughly) that it’s economically viable to retro-fit or replace airbags on vehicles in NZ up to 14 years old, but that it’s also far more cost effective if they come as standard on a new car….

Can you recommend any further reading there? I don’t doubt that humans are crap at rational decisions, but I would be interested in hearing how they’ve shown that attempts at carefully-considered “rational” decision-making turns out to be less effective than “gut feel”.

Thanks

That’s a funny anecdote, but it didn’t affect my decision for two reasons.

One is that hypothesis that car colour affects accident rates has had little scientific support.

The other is that I was choosing between several models of car rather than several instances of cars. The colour choices between the car models were effectively identical.

I am one of those people who gets angsty about such a big purchase, and I come over all analytical to overcome my fear.

Research shows that humans suck at making big decisions rationally. Best to obsess over the small stuff and listen to your gut on the big stuff; that gives you the biggest chances of making the right moves.

I have to say Scott Adams’ approach to the problem seems intelligent, though.