What happens when you set targets – examples

(Updated 25/5/2016)

1. Ambulance response times (see Bevan and Hood, 2006)

ambulance target
Statistically unlikely spike exactly at the target suggesting something has happened to the data.

2. Phonics test scores (see Dorothy Bishop’s blog post)

phonics
A higher score is better performance. There’s a dip just below the pass mark of 32 and then a big spike, suggesting scores have been changed.

3. Call centre response times (see Caulcutt, 2004)

“The Times reported in October 2003 that the telecommunications regulator, Oftel, intended to investigate the workings of one of the newly established directory enquiry companies. According to the report: “Sixty call centre workers at the 118 118 directory enquiries service will be sacked in an attempt to head off a scandal over staff who deliberately gave out wrong numbers to boost their pay”. Why did they do this? It appears that the motivation was provided by a bonus system that rewarded employees for dealing with calls in less than 40 seconds.”

4. Final high school exam scores

Matura scores in Poland, 2013. To pass you need 30% or above.

4nnbyru

(See here, spotted via @MaxCRoser.)

5. “‘G4S cheats’ made 1,000 FAKE 999 calls to boost performance figures”

“Staff at scandal-hit G4S boosted performance figures by making hundreds of fake calls to a 999 centre run by the firm.

“Five employees have been suspended after allegedly making more than 1,000 “test calls” – many reportedly at quiet times when they could be picked up quickly.

“Without them G4S would have missed key targets of answering 92% of calls within 10 seconds in November and December 2015, so incurring a financial penalty.”

(Mirror article)

 

In general, Goodhart’s (1975) law applies: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”

 

References

Bevan, G., & Hood, C. (2006). What’s measured is what matters: targets and gaming in the English public health care system. Public administration, 84(3), 517-538.

Caulcutt, R. (2004). Managing by fact. Significance, 1(1), 36-38.

Goodhart, C. A. E. (1975). Monetary relationships: A view from Threadneedle Street. In Papers in Monetary Economics, Vol 1, Reserve Bank of Australia.

Advertisements

One comment

  1. Pingback: What I think’s wrong with adult mental health PbR | Figural Effect

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s