“Winwood Reade is good upon the subject,” said Holmes. “He remarks that, while the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty. You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.”
—The Sign of Four by Sir Arthur Conan Doyle (hat-tip MP)
(Updated 24 April 2015)
One of my day jobs is teaching psychology students how to do data analysis. Occasionally I quote famous statisticians, for instance to illustrate ways of thinking about analysis, the subjective nature of modeling data, and other fun things. I mention the likes of William Gosset (Guinness and t-tests), George Box (all models are wrong), and Bruno de Finetti (probabilities don’t exist).
Most—often all—of my students are women. Most of my current collection of quotations are from men. This is a problem. So, I’m currently looking for examples of famous female statisticians (broadly interpreted; including data scientists, economists, quantitative social scientists). Here’s my current list. Suggestions for others would be most welcome, especially if you have a quotation I can use (turns out that statisticians write in maths most of the the time so it can be hard to find nice quotes).
- Daphne Koller (Professor in Stanford University; wide range, e.g., conditional independence models, feature selection)
- Deirdre McCloskey (economist, writes on applied stats, e.g., regression; she is also transgender)
- Fiona Steele (Professor in Stats at LSE, e.g., multilevel modeling)
- Florence Nightingale (data visualisation and public health stats)
- Gertrude Mary Cox (experimental design and analysis of experiments)
- Helena Chmura Kraemer (Professor of Biostatistics in Psychiatry, Emerita, at Stanford)
- Hilary Mason (“enthusiastic member of the larger conspiracy to evolve the emerging discipline of data science”)
- Hilary Parker (data analyst at Etsy; PhD in biostatistics, genomics; useR)
- Irini Moustaki (Professor in Social Statistics at LSE)
- Jane Hillston (Professor of quantitative modelling at Edinburgh University; invented the stochastic process algebra, PEPA)
- Jennifer Neville (e.g., data mining for relational data such as networks/graphs)
- Juliet Popper Shaffer (work on corrections for multiple hypothesis testing)
- Pat Dugard (e.g., randomisation stats for single case and small-N multiple baseline studies)
- Rachel Schutt (Senior Vice President of Data Science at News Corp)
- Stella Cunliffe (worked in Guinness and first female president of RSS)
- Susan A. Murphy (e.g., clinical trial design; methods for multi-stage decision making)
- Victoria Stodden (e.g., reproducibility of models, codes)
Quotations—work in progress
“The newly mathematized statistics became a fetish in fields that wanted to be sciences. During the 1920s, when sociology was a young science, quantification was a way of claiming status, as it became also in economics, fresh from putting aside its old name of political economy, and in psychology, fresh from a separation from philosophy. In the 1920s and 1930s even the social anthropologists counted coconuts.”
—Deirdre McCloskey, The Trouble with Mathematics and Statistics in Economics
“The Cabinet Ministers, the army of their subordinates… have for the most part received a university education, but no education in statistical method. We legislate without knowing what we are doing. The War Office has some of the finest statistics in the world. What comes of them? Little or nothing. Why? Because the Heads do not know how to make anything of them. Our Indian statistics are really better than those of England. Of these no use is made in administration. What we want is not so much (or at least not at present) an accumulation of facts, as to teach men who are to govern the country the use of statistical facts.”
—Florence Nightingale in a letter to Benjamin Jowett, from Kopf, E. W. (1916). Florence Nightingale as statistician. Publications of the American Statistical Association, 15(116), 388–404.
“To understand God’s thoughts we must study statistics, for these are the measure of His purpose.”
“The statistician who supposes that his main contribution to the planning of an experiment will involve statistical theory, finds repeatedly that he makes his most valuable contribution simply by persuading the investigator to explain why he wishes to do the experiment.”
—Gertrude M Cox
“It is no use, as statisticians, our being sniffy about the slapdash methods of many sociologists unless we are prepared to try to guide them into more scientifically acceptable thought. To do this, there must be interaction between them and us.”
—Stella V Cunliffe (1976, p. 9). Interaction. Journal of the Royal Statistical Society. Series A (General), 139, 1–19.
… to everyone who sent suggestions!
Randomised controlled trials (RCTs) are great, the gold standard of empirical research. The only thing better than RCTs are systematic reviews of lots and lots of RCTs. (So the story goes.) The reader may have noticed that RCTs evaluating CBT for psychosis have been vigorously debated for many months after a review was published in the British Journal of Psychiatry (Jauhar et al., 2014). Maybe not everyone agrees that RCTs are great (disclosure: I have analysed a couple), but I think it’s fair to say they are unavoidable whether you are trying to design or demolish them.
High-quality psychotherapy research by Patricia Arean and Helena Chmura Kraemer sets out to be a “practical, step-by-step guide” to designing and running RCTs. So why bother with an RCT? Observational trials, the authors explain, might involve studying participants who choose one of two or more interventions of interest by simply observing how they get on. This is problematic as differences in outcomes might be due to whatever factors led to them ending up receiving an intervention rather than the effect it had. RCTs use randomisation to overcome this problem so that people differ only in terms of the intervention received. That’s about it for the “why”: don’t expect debate on the epistemology.
The book’s strengths emerge as it develops: it catalogues issues that should worry study investigators and the authors draw on their own experience to offer hints. The Delphi consensus-building approach is introduced to solve the problem of developing an intervention manual and examples are given of how to word a letter asking for feedback on the proposed result. Randomisation techniques are introduced including horror stories of how they have gone wrong and invalidated RCTs. Ideas are provided for control groups, e.g., waiting list, usual care, and “gold standard” controls, and their strengths and drawbacks. The importance of not using pilot study results to determine sample size choices is explained. Guidance is provided on the people required; for example you need three or more therapists, at least two research assistants in case one takes ill, and a good statistician amongst other people. The Appendix includes a sample budget justification. All practical advice.
The text runs to under 200 pages so this could never be a comprehensive guide to all aspects of RCTs. What this book does do well is provide a systematic menu of options and ideas for things to consider. It might possibly give some ideas of what to demolish too, should you be so inclined, but this book is really only for those who are already sold on RCTs and want to get on with the seemingly painful task of designing and running one.
Areán, P. A., & Kraemer, H. C. (2013). High-quality psychotherapy research: from conception to piloting to national trials. Oxford University Press.
Jauhar, S., McKenna, P. J., Radua, J., Fung, E., Salvador, R., & Laws, K. R. (2014). Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. The British Journal of Psychiatry, 204, 20–29. doi:10.1192/bjp.bp.112.116285
Stephen Senn (2002) explains the need for statistical power analysis.
“An astronomer does not know the magnitude of new stars until he has found them, but the magnitude of star he is looking for determines how much he has to spend on a telescope.”
Senn, S. J. (2002). Power is indeed irrelevant in interpreting completed studies. BMJ, 325, 1304–1304
Computing Cohen’s (1988) U1, U2, and U3 from d (Table 2.2.1, p. 22)
U3 = function(x) pnorm(x)
U2 = function(x) pnorm(x/2)
U1 = function(x) (2*U2(x) - 1) /U2(x)
cohenTab = data.frame(d = c(seq(0.0, 2.0, 0.1),
seq(2.2, 4.0, 0.2)))
cohenTab$U1 = U1(cohenTab$d) * 100
cohenTab$U2 = U2(cohenTab$d) * 100
cohenTab$U3 = U3(cohenTab$d) * 100
“Models of data have a deep inﬂuence on the kinds of theorising that researchers do. A structural equation model with latent variables named Shifting, Updating, and Inhibition (Miyake et al. 2000) might suggest a view of the mind as inter-connected Gaussian distributed variables. These statistical constructs are driven by correlations between variables, rather than by the underlying cognitive processes […]. Davelaar and Cooper (2010) argued, using a more cognitive-process-based mathematical model of the Stop Signal task and the Stroop task, that the inhibition part of the statistical model does not actually model inhibition, but rather models the strength of the pre-potent response channel. Returning to the older example introduced earlier of g (Spearman 1904), although the scores from a variety of tasks are positively correlated, this need not imply that the correlations are generated by a single cognitive (or social, or genetic, or whatever) process. The dynamical model proposed by van der Mass et al. (2006) shows that correlations can emerge due to mutually beneﬁcial interactions between quite distinct processes.”
Fugard, A. J. B & Stenning, K. (2013). Statistical models as cognitive models of individual differences in reasoning. Argument & Computation, 4(1), 89–102.
“It can be said that some of the questionnaires used in these surveys contain everything but the proverbial kitchen sink, and once such a questionnaire has been filled in by a sizable group its author has the ‘basic’ data at hand for a half dozen articles. If he is fortunate enough to have punched card equipment, it becomes the misfortune of his professional contemporaries to find the literature being filled with results of cross tabulations which are so lacking in rationale as to be nonsensical. The ‘hypothesis’ step in scientific reasoning and research seems to be all too frequently ignored by the users of these techniques.”
McNemar, Q. (1946). Opinion-attitude methodology. Psychological Bulletin, 43(4), 289–374.