During my (relatively ancient*) PhD, I interviewed many prominent (at the time) critics of Null Hypothesis Testing and advocates of methodological and statistical change in psychology, medicine and ecology. My interview with Paul Meehl took place in 2002. (He died in Feb 2003, which makes me feel extremely lucky and grateful, to both him and his wife, Leslie Yonce.) This is not the entire transcript, but bits that I’ve been thinking about while at SIPS 2017. Over coming months, I’m going to make an effort to post bits of other interview transcripts here. (Shockingly, my recordings are on cassette tapes.)
Everyone thinks I’m a Popperian; I want people to know I’m not. I’ve tried to explain this… (personal communication, August 2002)
On the persistence of NHST and inertia
…plain psychic inertia is a powerful factor in science, as it is in other areas of life—don’t underestimate it. When the issue is method, rather than substance, it makes it worse. If one has been thinking in a certain way since he was a senior in college ‘the way you test theories in psychology is refute H null’, there is a certain intellectual violence involved in telling a person, well, not that they’ve been a crook, but that they’ve been deceiving themselves (personal communication, August 2002).
On the 1999 APA Task Force on Statistical Inference report
I’ll tell you a story that might interest you. It is a sad commentary on our profession.
The Task Force appointed four outside consultants: Cronbach, Tukey, Mosteller and Meehl. In my letter of acceptance [to be a consultant] I wrote about NHST and the difference between a substantive theory and a statistical hypothesis. I said the ‘logical problem of inductive inference is bigger than the mathematical problems being debated, like how you best compute the power for example.’
The first draft [of the TFSI report] had nothing in it of what I had said.
So, I wrote another note, and reminded them of my first note. I said ‘if what I had to say on this question is all baloney seems to me you might want to tell me what is the matter with it.’
There was no response. The second draft had nothing [related to my comments].
Then, the quasi-final draft arrived, still no reference to anything I had said.
Finally, in my last letter, I was slightly irritated—I don’t have a real fragile ego so I wasn’t enraged, but I was hurt—I asked ‘I wonder why you appointed expert, outside consultants, if you won’t pay any attention to their input.’
Still no response! It is somewhat discourteous: You appoint somebody as an outside advisor and they put in the work. I don’t even know whether the chairman of the committee even circulated my stuff.
When I read the final report, most of the things were very obvious and trivial and should have been in there. For example, tell [the reader] whether you’ve got this population and tell whether people dropped out. Of course, I agree with all that. But on the hardest part of it, the whole problem of inductive inference in this context, what your general view of theory testing is, the philosophical aspects—they were practically missing. You would think that philosophers of science didn’t exist! (personal communication, August, 2002)
* 2006 is not really that long ago. But at SIPS it feels like it. Also, I’d argue quite violently against many of the claims I made in dissertation now, and against the language I used to make them. It really has been a *very* long decade.