Thinking when you think you’re not thinking—again

I really enjoyed the 2006 Science paper by Dijksterhuis, Bos, Nordgren and van Baaren on deliberation without attention.  Then came Acker (2008) with a meta-review and the suggestion that there is “little evidence” of an advantage of deliberation without attention.

Today, from the latest issue of Judgment and Decision Making: Are complex decisions better left to the unconscious? Further failed replications of the deliberation-without-attention effect by Dustin P. Calvillo and Alan Penaloza.

The summary seems to be that deciding without deliberation, immediately after the stimulus is presented, might sometimes be better than deliberation.  But not with distraction post stimulus.

The moderators of the effect—there seems to be something going on in a few studies!—are still not well understood.

Reasoning to an interpretation before applying Bayes’ rule

What’s the point of Bayes’ rule?  This web page by Eliezer S. Yudkowsky gives a long intuitive explanation (thanks to Keith Frankish for pointing to it).  This blog post is an attempt at a slightly shorter version with a bit more maths, and a bit of rambling about interpretation.

The information in the example problem given there is as follows:

1. 1% of women at age forty who participate in routine screening have breast cancer.
2. 80% of women with breast cancer will get positive mammographies.
3. 9.6% of women without breast cancer will also get positive mammographies.

The task: A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

The general problem solved by Bayes’ rule is that if you know the probability of if A, then B, how do you work out the probability of if B, then A?  More precisely if you know P(B|A), what is P(A|B)?

Here B|A denotes the conditional event, a simultaenously easy and difficult concept.  One way to think of it is as follows.

Consider a fair die with six sides.  It’s thrown.  What’s the probability of a six given that a side showing an even number lands upwards? (Van Frassen, 1976 used an example like this to explain the conditional event interpretation of the natural language if-then.)  This is P(lands six|lands even).  The idea is that you only consider cases where it’s showing an even number (2, 4, or 6). Assuming they’re all equally probable, then P(lands six|lands even) = 1/3.

Interpretation

The first stage of solving problems like that above is interpretating the problem in the language of the mathematical theory you want to use.

Let denote “has cancer”, denote “does not have cancer”, denote “shows a positive test result”, and denote “shows a negative test result”.

Let’s take each item of information individually.

1% of women at age forty who participate in routine screening have breast cancer.

There’s a mix of information here: a percentage of people (1%), from a particular sub-population (women, aged 40, who participate in routine screening), and a property they have.  From the problem it is clear that the interpretation is supposed to be:

But one can imagine a more complicated formalisation, for instance if the population of interest contains women of many different ages, some, but not all, of whom were screened because they had some worry about their health.

Next sentence:

80% of women with breast cancer will get positive mammographies.

This is an instance of

X% of people with property A have property B

The intended interpretation is P(B|A) = X%, but this might not be obvious to all readers.  Take some:

Some people with property A have property B

If this is interpreted as an existential quantifier, then it also follows that some people with property B have property A.  The conditional event, B|A, is in general not reversable in this way, so would not be suitable for the interpretation of an existential “some”.  Consider the following statement:

All people with property A have property B

This is not (in general) reversable. The percentage quantifier (used in the problem description) is also not reversible.  So there’s quite a lot of trickiness involved in interpreting this innocent looking statement. Given some background knowledge (we know the article is about Bayes’ rule, and about conditional probabilities), the intended interpretation of the original information is:

The idea is that if we choose a person at random from the population of interest, who has cancer (i.e., we know for sure she has cancer), then the probability of her having a positive test result is .8.

Then similarly for the last sentence:

9.6% of women without breast cancer will also get positive mammographies.

The formalisation is:

Here is the summary:

Now the problem statement:

A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?

We have to infer . Note how this is a reversal of the conditional statements we encounted in the information given about the test.

Calculation

Now comes the calculation. A good place to start when thinking about conditional probability is the ratio formula for the probability of a condititional event:

Take an interpretatation of “If it is raining, then I have an umbrella” as the conditional event expression:

I have an umbrella  |  it is raining

The probability of this is the probability that I have an umbrella and it is raining, divided by the probability that it is raining.

This can easily be rewritten to

So if you know the probability of rain, and the probability that I have an umbrella when it rains, then you can multiply them to infer the probability that it is raining and I have an umbrella.

One step towards Bayes’ rule begins with:

1. 
2. [ in (this) probability theory, so it does not matter what order you write them]

From 2 we can infer , which slots into 1 to give

Now use the same variables as in the original problem

We can already fill in the numerator (top row) with and , but not yet the denominator (bottom row).

Let’s work a bit further then. We can infer as follows:

Which is easily calculated from the rewrite of the conditional probability above:

One more thing: .  So this gives:

Now we have everything we need:

.

Yet more scribbles

Some stuff on competence in reasoning over here.

The Wisdom of Many in One Mind

Averaging many people’s estimates of, e.g., when a famous event occurred tend to be better than asking any one arbitrary person.   Herzog and Hertwig (2009) investigated whether the average of two estimates from one person tended to be better than their first estimate, using the years of 40 historical events, e.g., when electricity was invented.

There were three conditions:

1. Repeated sampling: just giving an estimate twice.
2. So-called “dialectical” sampling (they cite Hegel here), where participants were told: “First, assume that your first estimate is off the mark. Second, think about a few reasons why that could be. Which assumptions and considerations could have been wrong? Third, what do these new considerations imply? Was the first estimate rather too high or too low? Fourth, based on this new perspective, make a second, alternative estimate.”
3. Pairing each participant’s guess with a random other participant.

Results are below:

The instruction to consider you were wrong increases accuracy beyond that with simple repeated measurement.  Best of all is averaging with another person.

Reference

Herzog, S. M. & Hertwig, R. (2009). The Wisdom of Many in One Mind: Improving Individual Judgments With Dialectical Bootstrapping. Psychological Science, 20, 231-237

Prover9 and Mace4

Just found two fantastic programs and a GUI for exploring first-order classical models and also automated proof, Prover9 and Mace4.  There are many other theorem provers and model checkers out there.  This one is special as it comes as a self-contained and easy to use package for Windows and Macs.

There are many impressive examples built in which you can play with.  To start easy, I gave it a little syllogism:

all B are A
no B are C

with existential presupposition, which is expressed simply:

exists x a(x).
exists x b(x).
exists x c(x).
all x (b(x) -> a(x)).
all x (b(x) -> -c(x)).

and asked it to find a model. Out popped a model with two individuals, named 0 and 1:

a(0).
- a(1).

b(0).
- b(1).

- c(0).
c(1).

So individual 0 is an A, a B, but not a C. Individual 1 is not an A, nor a B, but is a C.

Then I requested a counterexample to the conclusion no C are A:

a(0).
a(1).

b(0).
- b(1).

- c(0).
c(1).

The premises are true in this model, but the conclusion is false.

Finally, does the conclusion some A are not C follow from the premises?

2 (exists x b(x)) [assumption].
4 (all x (b(x) -> a(x))) [assumption].
5 (all x (b(x) -> -c(x))) [assumption].
6 (exists x (a(x) & -c(x))) [goal].
7 -a(x) | c(x). [deny(6)].
9 -b(x) | a(x). [clausify(4)].
10 -b(x) | -c(x). [clausify(5)].
11 b(c2). [clausify(2)].
12 c(x) | -b(x). [resolve(7,a,9,b)].
13 -c(c2). [resolve(10,a,11,a)].
16 c(c2). [resolve(12,b,11,a)].
17 $F. [resolve(16,a,13,a)].

Indeed it does. Unfortunately the proofs aren’t very pretty as everything is rewritten in normal forms.  One thing I want to play with is how non-classical logics may be embedded in this system.

A non-judgmental reconstruction of drunken logic

Simmons (2007) makes a helpful contribution to the logical modelling of real arguments by an addition of the shot glass modality to intuitionist logic.  A snippet:

Per Per Martin-Löf [7], something is true when witnessed by an object of knowledge, which lends itself to an obvious question of whether the truth of a proposition can be obviated by the presence of alcohol, seeing as alcohol has an clearly negative impact on one’s knowledge [1]. The possibility of the analytical truth of a proposition becoming questionable under the influence is also evidenced by discussion as to whether conference submissions that can be understood while drunk are novel enough to be worth accepting.

I think the following inference rule which I discovered while living in the homeland of Martin-Löf still requires further investigation:

Reference

Robert J. Simmons.  A non-judgmental reconstruction of drunken logic.  Presented at SIGBOVIK 2007, April 1, 2007. Winner of the Best Paper raffle. [PDF]

Reasoning about the news

Every now and again interesting (worrying) stories are released in the news which presumably people interpret in different ways. For instance recall when Tony Blair said the following (12 January 2007):

There are two types of nations similar to ours today. Those who do war fighting and peacekeeping and those who have, effectively, except in the most exceptional circumstances, retreated to the peacekeeping alone.

Britain does both. We should stay that way. But how do we gain the consent to do it?

Note the rhetoric of “retreat”, how the UK (“we”) “should” continue to engage in “war fighting”, and the problem of how “we” persuade others (who?—Brits who disagree? Other countries’ governments?) to allow the wars to take place. I remember reading this and thinking that a UK prime minister couldn’t possibly have allowed these words to leave his lips. But there it all still is on a pm.gov.uk website!

The current item in the news causing confusion (at least to me) relates to the downing of a plane in Iran. Initially the story was that an American plane, flying low to evade radar, was forced down. Its passengers and crew were questioned for a day before being allowed to fly on. It turned out that they had got lost, straying into Iranian airspace. Then it became an aid organisation’s plane (“not a military plane and did not belong to the United States”, said Al Alam). The Americans denied than an American plane had been downed (still consistent here!), but it was reported that five passengers (in the aid plane) were US military. Again from the US press release, “All U.S. aircraft supporting operations in Afghanistan and Iraq are accounted for”.

The “Diplomatic Editor” at the Telegraph writes: “The aircraft and all of its occupants were, it transpired, Hungarians.” Oops. According to news.trendaz.com, the count is four Hungarian soldiers and three crew from Jas Cargo, the company that rented the plane to the Hungarian Army. “The Hawker 800 aircraft was forced to land because there was a mistake in one figure in the written overflight permit,” they write. (It was a Falcon business jet yesterday.)

And… well so it continues.

A reasoning task

Any mathematical physicists out there? Can the “EASILY TESTABLE FORMULA” published by Tipler (2008) really be easily tested? Would the result say anything about the Many Worlds Interpretation?

Frank J. Tipler (2008). Testing Many-Worlds Quantum Theory By Measuring Pattern Convergence Rates, arXiv:0809.4422v1.

The only good thing about Dark Knight (game theory!)

Check out this posting at The Quantitative Peace on applying game theory to The Ferry Scene.

Binary decisions (updated)

I like this: icantdeci.de. You provide two answers for some problem you can’t solve and other users of the website are asked to choose for you. Meanwhile you make five choices for problems provided by others. Once you’ve finished, you get a nice barchart.  You can head back to check up on progress if you remember not to lose the URL.

Fantastic idea, except we don’t know much about the people who use the service. For instance I’m not sure how good they are arithmetic…

Now I wonder how to use this to work out how good the others are at making decisions before trusting them to make a decision.

Update: it’s looking better now.  26 votes, 69% were correct, almost above chance (by the binomial test, p = 0.08).  Maybe we need around 50 people to use this thing to do arithmetic for us.