ArticlesI recently finished reading a book that I started reading over six months ago when I bought it up from a remaindered bin at 70% off.
The book is titled Reckoning With Risk: Learning to Live with Uncertainty, by Gerd Gigerenzer.
With a title like that, the publishers have made it sound like a self-help book, but it is not.
That’s not why I picked it up. Based on its blurb, I thought it might be a book describing the psychological hoops humans will jump through to convince themselves that they are in control in an uncertain world, but it is not.
Instead, it is a book similar to Innumeracy: Mathematical Illiteracy and its Consequences, by John Allen Paulos. It argues that misunderstandings about the assessment of risk affect serious decisions made in people’s lives.
He doesn’t spend much time on the inability for people to estimate the chance of low-risk activities (and use it, for example, to determine whether to wear a seat-belt.)
Instead, he focusses on the way the presenting the same information to people in a “natural frequencies” format is preferable to a “conditional probabilities” format. He gives a real-life example, where he was required to get an AIDS test for a U.S. green-card.
About 0.01 percent of men with no known risk behaviour are infected with HIV (base rate). If such a man has the virus, there is a 99.9 percent chance that the test result with be positive (sensitivity). If a man is not infected, there is a 99.99 percent chance that the test result will be negative (specificity).
The question he asks is “If I test positive in this HIV test, what is the chance that I will actually have HIV?”
Although all the information you need to answer that question is in the above paragraph, it is easier to reason about it if you present the same information in a different format.
Imagine 10,000 men who are not in any known risk category. One is infected (base rate) and will test positive with practical certainty (sensitivity). On the 9,999 men who are not infected, another one will also test positive (false positive rate). So we can expect two men will test positive.
From here, it is clear that if someone (in a low risk group) tests positive, there is only a 50-50 chance that they have the disease.
This simple idea of presenting the statistics in a different format to aid understanding is the key idea in the book. In fact, it is almost the only idea. I originally considered creating a tongue-in-cheek review of the book that went along the lines of:
In Chapter One, Gigerenzer introduces the idea that natural frequencies are a better way of presenting statistics. In Chapter Two, Gigerenzer argues that natural frequencies are a better way of presenting statistics. In Chapter Three, Gigerenzer gets into his stride, arguing that natural frequencies are a better way of presenting statistics. (etc.) Chapter Fourteen summarises the arguments in the previous chapters by suggesting that, perhaps, there is a better way of presenting statistics, called natural frequencies.
That’s why it took me six months to read. It was a little dull and repetitive.
I decided not to write the cynical review; it seemed too petty, and it wasn’t really fair. The author did introduce some other concepts too, and one of the reasons the work seemed so repetitive is that he provided copious evidence of experiments showing how people had completely misunderstood various statistics, and how simply re-presenting the same information helped people’s cognition immensely.
Examples included: Breast screening, AIDS counselling, Wife Battering, DNA fingerprinting, releasing violent people from jail, and less important examples, like the infamous Monty Hall puzzle.
What made me change my mind, and write this review? The case of Andrew Stimpson.
Andrew Stimpson has made news recently because of his miraculous “cure” from AIDS. He was tested positive for HIV several years ago, considered suicide, took no medication, partook in “dangerous” practices (e.g. unprotected sex with his HIV-positive boyfriend), and yet has recently tested negative for HIV.
The media hailed this as a miracle cure from AIDS, quoting the original diagnosticians as saying there was no room for error in the tests.
Bullshit!
I do not know whether Stimpson was properly informed of the chance that there could be a false positive. Gigerenzer described an experiment where a majority of German AIDS counsellors , when directly asked by a researcher pretending to be a patient, “incorrectly assured the client that false positives never occurred”. So, there is a good chance that Stimpson was not informed of the chance.
Even if he was told, there is a good chance, according to Gigerenzer, that Stimpson would not have been able to understand the risks (unless, of course, they had been presented in “natural frequencies”!)
Either way, it seems likely to me that an HIV-free Stimpson walked out of that clinic with the firm belief that he was HIV-positive, no two ways about it. Had he committed suicide, or subsequently contracted HIV through discarding safe practices, the situation would have been tragic.
According to the media, the clinic initially maintained their position that both tests were accurate.
Then doubt was raised about the second (negative) test, and Stimpson was urged to get another test, debating whether it was the appropriate test.
I would equally question the results of the first test. I believe Gigerenzer’s figures are more illustrative than definitive, but he rates an HIV-positive result amongst gay men (who have a base-rate of HIV-infection around 1.5% – I assume Germany and the UK have similar base-rates) as meaning there is “only” a 1 in 151 chance that you do not have the disease – bad news, for sure, but it leaves a big gap from the “miracle” that the papers describe.
The medical and the media professions would do well to reconsider how they present the information to their patients and their readers. Perhaps Gigerenzer was right in ramming the same point home several times in his book.
Comment by Pete on November 17, 2005
I’m interested to know how what you state above changes in light of the fact that Stimpson tested positive TWICE for aids before the THIRD test came up negative.
http://society.guardian.co.uk/health/story/0,7890,1642155,00.html
Of course, this is reported in the popular media (including SMH) so the facts could just be completely made up.
Have a great day!
Comment by Julian on November 17, 2005
Pete,
I have three quick and easy answers to your question:
a) The two false-positives may not have been independent. For example, if he suffered from syphilis or lyme disease during the first period that may lead to false-positives in the ELISA HIV test (Ref).
b) There could have, coincidentally, been two independent false-positives. That would take 1-in-151 chance up to 1-in-22,801. Much rarer, but hardly worthwhile of sustained international media coverage.
c) If may have been the third test which was a false-negative – apparently he is declining to be re-tested, which could be ascribed to a number of innocent or money-induced reasons.
However, rather than simply trot these answer out, I did some more research on HIV testing to try to sound more knowledgeable. It revealed that the test accuracy is likely to have been far higher than the examples given by Gigerenzer.
My guess (without evidence) is that the two positive tests that Stimpson had (referred to in that media report) were an initial screening ELISA test (specificity somewhere between 98.5% and 99.9%, (Ref and Ref) followed by the far more reliable Western Blot test. Counted in combination, this is a very accurate test (Specificity of >99.9994% (Ref)) That translates to “the chance of a false-positive identification in a low-prevalence setting is about 1 in 250 000.” (Ref) Stimpson presumably falls into a higher risk group, given that his boyfriend is HIV-positive, which would reduce that risk of a false positive even further.
To summarise – the 1-in-151 from Gigerenzer has become 1-in-22,801 if the initial test was simply repeated, or a less than one in a quarter-million chance if it was followed up by a Western Blot test. A one in a quarter-million chance probably counts as news-worthy, but given the number of HIV tests done every day, it is likely to happen every now and again. I wouldn’t get my hopes up that this individual represents the breakthrough required for a miracle cure for AIDS.
Comment by Sunny Kalsi on November 18, 2005
Well, we could look at this in reverse: Risk = damage * probability. “Damage” can be replaced by “reward” here, it’s just the negative. So we can say, what’s the risk of not testing him?
Consider the number of people that get tested for and have AIDS. Consider also that the odds of this guy actually having a cure is far higher than anyone else, and we can see some case for the hysteria. He’s like the Makybe Diva of AIDS research. A stupid horse doesn’t freaking matter, but the media covers it, as it will cover a (relatively) very good chance of having found a cure. If you were one of the scientists that wanted to test this guy, I’m sure your article would not convince you not to do the test.
At the very extreme, I think you’re questioning the very value of the tests themselves. No matter how many times the test is re-done, there is never a 100% chance of knowing, so why try?
The point I’m making here is, you’ve actually got to do the figures and come up with which of the odds are better (him never having had AIDS, him still having AIDS, or him having a cure). Then looking at the ratio (let’s say there’s a 1000 to 1 chance he’s got a cure) and deciding whether it’s worth it for us to be hysterical.
Because I for one love being hysterical.
Comment by Julian on November 19, 2005
Sunny,
I am not arguing that the tests have no value. They are of great value to society and to the individual patients.
I am not arguing that this patient should not come forward to be re-tested. I would gently encourage him to do so (but not pressure him to do so, it is a personal choice for him.) I would definitely pressure him not to risk the lives of others with unsafe practices (like unprotected sex).
I am arguing that medical tests are not 100% accurate, and it is important to understand this, for a number of reasons, including:
Getting a positive result is not a guaranteed death-sentence. Too many patients fuck up their lives only to find that they are HIV-negative after all. Presumably a percentage of people who elect to commit suicide have thrown away their life on a misunderstanding of test accuracy.
When someone does appear to be miraculously cured, to understand that there is a high chance that they never had the disease.
The miraculous cure story does occur on a regular basis (not just for AIDS, but for many diseases). Often the person gets tested once, drops out of the medical system, tries some remarkable quack cure, gets tested again years later and is “cured”, while the “doctors can’t explain it”. Guess what: the doctors can. There are many possible reasons, and one is called “specificity”. Meanwhile the quack cure is available for sale, falsely raising the hopes and draining the wallets of the real victims.
Perhaps you are right that we should work out the odds that he really has a miraculous cure that medical science can learn to reproduce and put into pill form. Understanding test accuracy can help us do this. I suspect it is much lower than 1000-to-1.
In the mean time, enjoying the fun of getting hysterical over dodgy media reports (or for that matter, getting mad and blogging about it) is a luxury of the well.
Comment by Alastair on July 11, 2006
On a related note, did you see the recent Schneier post where a similar application of Bayes’ theorem was used to prove the uselessness of NSA surveillance activity?
It’s interesting, because it’s so counterintuitive (like Monty Hall).
Comment by Alastair on July 11, 2006
Bloody hell, false positives are everywhere. What’s the bet that the 20% false positive rate of WGA is explainable in the same way?
Comment by Tsavo on April 12, 2007
Just concern to note some people do not believe, that one can be positive and then later be negative. We have seen
(here in Nairobi ) orphans born by HIV positive dead parents and when these young children are taken to chirdren homes some of them later are 1005 HIV negative while the origin tests indicate that they were 100% positive. May be the change of life style, does teh magic.
Tsavo
Comment by Julian on April 13, 2007
Tsavo,
You might be right. It may be that the change of lifestyle really does make orphans spontaneously eradicate the HIV from their system. It may be that all of understanding that we have about HIV from medical research over the past 30 years is wrong.
Alternatively, it may be that there is a more prosaic answer here: That the tests themselves gave false positives or false negatives. That the orphan’s records being kept by the authorities might be accidentally or deliberately confused. That the entire story is merely propagada from HIV-denialists.
Spend a minute to see if you can think of any other explanations.
You haven’t provided any references to follow up; until then, I know which way I would lean.
Comment by Dev Vaish on May 26, 2007
I enjoyed the book innumeracy and in particular the article on Baye’s theorem, applied to cancer or HIV/Aids testing. Now your web-page illustrates how false positives abound. I only wish these thoughts and ideas are communicated in a bigger way, so that exaggerated claims of any disease do not alarm the population. Flase positives can and do cause ‘trauma’ which can be avoided by better understanding (mathematical understanding)and the limitations of test results. Thanks
Comment by Dr Nick on August 6, 2011
The label AIDS denialist is a farce… And a term coined by the sheeple fear factor mass propoganda fake meds real cause of death Elite Zionist cash cow machine….
I know of numerous doctors who have been assaulted because their greed for getting clients to purchase expensive toxic HIV drugs caused the doctors and labs to “tamper” with perfectly healthy samples or simply watch the footy whilst eating lunch and incorrectly labelling and testing samples and deeming them positive. You idiots do not get it. The tests are designed to create more positives so more people buy the $4k a month AZT drugs which cause the stupid AIDS symptoms in the first place. People die of the drugs and the docs simply say “Doh, must have been the AIDS…” years of study at a university and graduating from medical school and that is the best they can come up with? This is the biggest scam and largest scale genocide in the history of man kind. Wives have been left, sons and daughters have killed themselves by jumping off bridges, families torn apart all for the sake of some stupid Big Pharma constantly mis diagnosing HIV a virus which doesnt even cause AIDS to perfectly healthy people. Im not apologizing for the fact that millions of people ARE NOT BUYING THE STORY
Comment by Dr Nick on August 6, 2011
Hey watch out there is a new BUTT SPRAY FLU BN1V1 that is caused by drinking too much Coke Zero but the media sweep that story under the rug because fake HIV and AIDS link is more important oh wait the flavour of the month is Swine Flu and Starfish Fever S1F1 UN number 328.
Just be weary of your doctors, if they say your result is positive don’t make the mistake my mate made. He jumped off one of the biggest bridges in the southern hemisphere after only one positive result which would have been caused by his Influenza shot. But the doctor was really rude to him as if he was a blight in human civilization. Do not ever let a doctor who knows nothing of the actual HIV to put you down. Walk out and get another test somewhere else!! Keep testing until you are negative because HIV and AIDS is a farce let alone the actual (without swearing) HIV ELISA, and Western Snot tests….the packaging even says they do not confirm HIV. Thats because IT DOES NOT EXIST. You have more chances of getting sick from a Large Quarter Pounder Meal
Comment by Julian on August 6, 2011
Dr Nick,
There are so many ways to reply to your answer.
I could take the easy way out, and just delete your comment forever.
I could take the help-the-ignorant approach: throw at you copious evidence that HIV causes AIDS. But you’ve already chosen to ignore that. It won’t convince you.
I could take the skeptical approach: Maybe you are right, but you haven’t provided any evidence. I will need to withhold my provisional acceptance of your claims until you provide some. That’s the approach I took with Tsavo above.
I could take the harsh approach: Dude, how idiotic do you need to be to post this crap to a blog whose most recent post is about how I am slowly becoming a more militant skeptic? That approach probably won’t help with you, but might help stop others follow in your footsteps.
I could take the caring but condescending approach: Mental illness can lead to paranoid delusions and belief in conspiracy theories. It must be troublesome dealing with your thoughts. Please get help.
Whichever answer you prefer, wear a condom.
Comment by Aristotle Pagaltzis on August 7, 2011
Ah, I’m so reminded of something I’ve seen before, Julian.