Security through obscurity

Vaughan at Mind Hacks writes:

“Almost every psychological test relies on the fact that the person being assessed has no foreknowledge of the material.”

Any test with good test-retest reliability does not require this, and there are a few of those around.  But irrespective of this, there is a hidden assumption that psychologists are somehow out to trick people into revealing their psychological properties.  Undergraduate students, for instance, often complain that personality questionnaires—and questionnaires in general—are rubbish because it’s obvious what they’re asking or what the answers “should” be.

I don’t think it’s a problem if what is being tested is transparent.  For instance take selection for a job or a university.  The point should be that a good selection process benefits both the candidate and the folk making the selection.  If you cheat your way into a job you’re not capable of doing by obsessively practicing a test, then it won’t be long until the pressures of performance will force you out.

This assumes that tests used for selection have predictive validity, of course. And… well you can imagine how this argument would continue, how some jobs might require people who are good at pretending, how validity might depend on people being motivated enough to try some practice IQ tests—acquisition of foreknowledge might (unknowingly to the tester) be part of the test—and so on and so forth…

For the application of clinical diagnosis, I find it quite frightening that tests should somehow trick patients into revealing their complaints. There is a diagnosis tool for autism spectrum conditions which works basically by tricking people into revealing how socially inept they are by various “social presses”, including during a period which appears to be a break between testing sections. The most frightening part of this was the obvious power trip the person who explained this test to me was on everytime she used it.

Be suspicious of tests which are designed to trick people.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s