Dysrationalia — if you disagree with me, off with your head

Some quotations from a critique of (an early version of) Stanovich’s theory by Sternberg (1994). Firstly, theory:

“Why do we need a theory? Because we’ve had too many fly-by-night constructs in the abilities business, and we don’t need more of them. We do need serious new constructs — and the way to present them is via a theory [...] — and construct validation of that theory.”

What is an irrational belief?

“In the real world, few problems truly lend themselves to the kind of deductive (rational) reasoning we learn in logic classes. The vast majority of problems are inductive, so that arguments can be stronger or weaker, but not logically valid or invalid. I am afraid that Stanovich has fallen into a trap that of labeling people as “dysrational” who have beliefs that he does not accept. And therein lies frightening potential for misuse. And if you disagree with me, off with your head. Here, it’s a joke. Historically, it’s not.”

Reference

Robert J. Sternberg (1994). What If the Construct of Dysrationalia Were an Example of Itself? Educational Researcher, 23, pp. 22-23+27

Mao Tse-tung on the psychology of problem solving and the empirical method

“You can’t solve a problem? Well, get down and investigate the present facts and its past history! When you have investigated the problem thoroughly, you will know how to solve it. Conclusions invariably come after investigation, and not before. Only a blockhead cudgels his brains on his own, or together with a group, to “find a solution” or “evolve an idea” without making any investigation. It must be stressed that this cannot possibly lead to any effective solution or any good idea.”

—Oppose Book Worship (May 1930)

:-)

Reasoning to interpretations

Have a look at the picture below, spotted in Salzburg:

What does the notice mean?  (Answer below cut.)

The rational agent

“… the rational agent is not simply the one who follows the normative canons of logic and probability theory, and neither is she the one who follows adapted heuristics for action choice or ‘somatic markers’. Rather the rational agent is the critically self-aware agent; the one who is aware why she acts, and who modifies her own behaviour according to her self-knowledge. As Karl Popper (1990) wrote, ‘A rationalist is simply someone for whom it is more important to learn than to be proved right’…”

Lambie, J. A. (2008). On the irrationality of emotion and the rationality of awareness. Consciousness and Cognition, 17, 946-971

Can psychology offer anything to the philosophy of logic?

What is logic? What is an “arbitrary variable”? What is a consequence relation? What is deduction? Why should the excluded middle be rejected? Are proofs mental constructions or are they Platonic entities out there somewhere?  These are the sorts of questions I associate with the philosophy of logic.  Other questions, related to the details of logics, are just mathematics, pursued (sometimes amateurishly) in philosophy departments, but moving increasingly to computer science and pure mathematics.

I think sociophilosophy is a way to get back to the philosophy of logic. How does the process of constructing new logics actually work? From where do the ideas come? What impact does this have on the status of concepts like proof?

Think how the debates work, in research groups, between student and supervisor, at workshops.  (Yes, here follows a rapid sketch of Lakatos.)   Conjectures suggested, refuted, adapted, proofs sketched, fixed, attacked, repaired, interactively… until a final proof is produced. The final proofs must have non-formal bits — otherwise there’d be infinite regress — so how on earth do mathematicians decide when something has really been proved? Why does proof work so much better than empirical methods? (Though sometimes things fall apart, e.g., the serious problem with Frege’s logic, discovered by Russell, or more recently the problem with Martin-Löf’s type theory logic, discovered by Girard.)

I think it’s revealing when you present a class with a bunch of sentences which just have in common that they are if-thens. Not everyone sees the same structure. Some can’t see beyond the superficial if-then. And, as history as shown, logicians and linguists also see different things in the very same natural language.

Though inventing logics is a human activity, there’s something Out There constraining what pops out. How does what’s out there do the constraining?

Now over to psychology.

Increasingly it’s seen as a necessary (and obvious) that psychologists of reasoning take ideas from logic (many still resist). Though not just off the shelf. Logicians don’t care about psychology. One of the earliest discoveries, the figural effect, is bizarre when approached from logic. Look at how automated theorem provers work, for instance: miles away from how people do things.  Obviously the maths logicians produce will not immediately model cognition. But there are important ideas in there. (And, trivially, given a fixed interpretation of a task, logics give the right answers to the questions.)

The relationship between psychologists and logicians is very similar to the relationship between biologists who care about how birds fly and aeronautical engineers who just want to build things that fly — by whatever means necessary.  The well-rehearsed story goes somewhat like this. Biologists can learn about flight from the physicists and engineers, e.g., lift, drag, air pressure, and so on.   They have no need to learn the details of the most recent jet engines. The engineers learned a bit by looking at the phenomena of flight. Wings still feature a lot, and nature got there first with that idea. However flapping the wings in a bird-like fashion was given up very early… And so on.

I think that when people wonder, why the hell should logic care about psychology, they’re in the same mode of thinking. And I agree. Engineers building reasoning systems have no reason to care about psychology, but they might get ideas — very important though primitive ideas which are very quickly generalised, polished, and forgotten.

But issues related to the philosophy of logic can be informed by psychology. For instance, how do people see and represent the world? How do they analyse what they see? How do constraints related to cognitive control and memory representations influence logical analyses.  How do processes of abstraction work?  How do groups of logicians get together to overcome their individual cognitive failings.  What is the relationship between amateur (often implicit) and professional logic, i.e., the role of specialisation and expertise.

In a few months I’ll be giving a talk on making these sorts of ideas more concrete (complete with data — yes data — which I think is at least tenuously connected to the issues), so comments and references very welcome!  (Thanks to Leon for prompting the writing of this post.)

Visuospatial representations and reasoning

We have a paper coming out (Fugard, Stewart, & Stenning, to appear) relating performance on the visuospatial Raven’s Advanced Progressive Matrices items, as determined by DeShon, Shah, and Weissbein (1995), to position on the sub-clinical autism spectrum, measured by the Autism-Spectrum Quotient.  The more you report having autistic traits, the better you are at the visuospatial items.  This result fits nicely with the enhanced perceptual processing theory of autism.  It also provides more evidence that Raven’s matrices load highly on g because the test is a package of many kinds of intelligence test.

The forthcoming QJEP paper by Borst and Kosslyn pinged my radar as they use DeShon and colleagues’ classification of the Advanced matrices to further clarify the representations involved in visual imagery tasks.  In brief, their main task requires participants to remember a two-dimensional array of dots.  The dots are removed and an arrow appears.  Participants are then asked whether or not the arrow points at one of the locations previously showing a dot.  The task is neat: for trials where the arrow is pointing at one of the dot locations, and when participants give the correct answer, the distance from arrow to dot is proportional to the response time.  This is consistent with a model of visuospatial representation which requires sequential scanning analogous to what one would do with one’s eyes if the array were still visible.

Back to the visuospatial items of the Raven: correlations were nearly .5 with performance on the dot-arrow task, compared to .04 between the dot-arrow task and Raven’s verbal-analytic items.

There were also correlations between paper form board and paper folding tasks, and visuospatial items (.42 and .52, respectively) again weaker (and p > .05) for the verbal-analytic items (.23 and .24).

Now, what’s the best way to come up with a process model of what’s going on in all of these tasks?  I think work by Maithilee Kunda for Raven’s matrices is very promising.  She and colleagues are coming up with algorithms which operate directly on the visual images in the Raven’s test.  These algorithms tend to work best for the visuospatial items.  A big challenge is to get such algorithms to work across a range of different tasks and to use the algorithms to generate new psychological tasks and predictions.

References

Borst, G. and Kosslyn, S. M. (in press). Individual differences in spatial mental imagery. The Quarterly Journal of Experimental Psychology.

DeShon, R. P., Chan, D., & Weissbein, D. A. (1995).  Verbal overshadowing effects on Raven’s Advanced Progressive Matrices: Evidence for multidimensional performance determinants. Intelligence, 21, 135-155

Fugard, A. J. B., Stewart, M. E., and Stenning, K. (to appear).  Visual/verbal-analytic reasoning bias as a function of self-reported autistic-like traits: a study of typically developing individuals solving Raven’s Advanced Progressive Matrices. Autism.

A ToM meta analysis

In case it’s useful, an approximation of the formula plotted in Figure 2B of Wellman, Cross, and Watson (2001):

ETA: already in the paper (p. 668):

Wellman, H. M., Cross, D., & Watson, J. (2001). Meta-Analysis of Theory-of-Mind Development: The Truth about False Belief.  Child Development, 72(3), 655–684

Some troubling and interesting things about investigating reasoning

Competence models are typically created and explored by a small number of experts.  Boole, Gentzen, Kolmogorov, Ramsey, De Finetti, …  The authority can often be shifted to the mathematics.   However, although non-experts can usually understand a statement of the theorem to proved, often they can’t understand the details of the proof.

There are problems with being an expert.  If you stare too long at the formalism, then you lose your intuition, and can’t see why someone would interpret a task “the wrong” way.  Often there are a priori non-obvious interpretations.

And who decides what constitutes a permissible interpretation?  Some obvious ideas for this are open to debate.  For instance, is it always reasonable for people to keep their interpretation constant across tasks?  Or is it rational to change your mind as you learn more about a problem?  Is it rational to be aware of when you change your mind?

To complicate things further, various measures loading on g predict interpretations.  Does that mean that those who have better cognitive ability can be thought of as having reasoned to the correct interpretation?

Recognizing textual entailment with natural logic

How do you work out whether a segment of natural language prose entails a sentence?

There are two extreme positions on how to model what’s going on.  One is to translate the natural language into a logic of some kind, then apply a theorem prover to draw conclusions.  The other is to use algorithms which work directly on the original text, using no knowledge of logic, for instance applying lexical or syntactic matching between premises and putative conclusion.

The main problem with the translation approach is that it’s very hard, as anyone who has tried manually to formalise some prose will agree.  The main problem with approaches processing the text in a shallow fashion is that they can be easilly tricked,  e.g., by negation, or systematically replacing quantifiers.

Bill MacCartney and Christopher D. Manning (2009) report some work from the space in between using so-called natural logics, which work by annotating the lexical elements of the original text in a way that allows inference. One example of such a logic familar to those in the psychology of reasoning community is described by Geurts (2003).

The general idea is finding a sequence of edits, guided by the logic, which try to transform the premises into the conclusion.  The edits are driven solely by the lexical items and require no context.

Seems promising for many cases, easily beating both the naive lexical comparisons and attempts automatically to formlalise and prove properties in first-order logic.

References

Bill MacCartney and Christopher D. Manning (2009). An extended model of natural logic.  The Eighth International Conference on Computational Semantics (IWCS-8), Tilburg, Netherlands, January 2009.

Geurts, B. (2003). Reasoning with quantifiers. Cognition, 86, 223-251.

Embracing individual differences

This week I finished teaching/facilitating a course entitled Embracing individual differences in thinking and reasoning.  I was asked to give the gist of what this was about.

There’s a bunch of individual differences in how people solve reasoning problems. One way of thinking about this is that some people are very good at reasoning problems and others are not so good, with a continuum in between. But there’s evidence that people are interpreting the tasks in different (and reasonable) ways, and succeeding in reasoning from their interpretation. We examined these sorts of issues on the course.

A simple example is the meaning of “some”, discussed by J S Mill in his 1867, An examination of Sir William Hamilton’s philosophy: And of the principal philosophical questions discussed in his writings. “I saw some of your children today” implicates that I didn’t see all of them (if I had, then I’d have said so), even though the “all” conclusion is compatible with some.

There are many other examples: the degree to which people are affected by how information is presented; whether people can suspend their beliefs and reason from premises which are obviously false; whether people are sensitive to alternative causes of effects, or factors which can disable a relationship between a cause and effect.

The waters are muddied somewhat by complicated relationships with intelligence. So for instance people with higher intelligence (for several of the standard psychological ways to operationalize the concept as IQ) are more likely to go for the normative answer on some (but not all) tasks. But then one can wonder what exactly the IQ tests are measuring.

Things get particularly interesting when people with clinical conditions, such as autism, actually are more likely to give the normative answer for some tasks. There’s a nice example of where their ability to do so was used as an argument for why the normative answer was wrong. One researcher blogged:

Autistics were shown to perform with enhanced logical consistency, avoiding irrational and irrelevant biases that distorted decision-making in their nonautistic controls. However, autistics’ enhanced performance in this study was interpreted by the authors as a litany of autistic failures, imbalances, impairments, deficits, reduced capacities, weaknesses, and impoverishments (several invocations of some of these), none of which were actually found. [...] In years to come, we can look forward to interventions designed to overcome this core autistic deficit and to ensure that autistics become as irrational as nonautistics.

There were plenty of issues to debate…