RightWingProf offers up this horror story from a conference presentation for a supposedly quantitative research paper in education. There’s no way to quote all the methodological problems with the paper in a succinct way — there’s too many of them. But suffice to say that a little statistics and a little critical thinking would have gone a long way, although “Jeanne-Marie”‘s paper might not have survived the process. Just go read the article.
I’ve blogged on this sort of issue before, calling out proponents of the so-called Digital Native hypothesis to either produce real statistical evidence to support their claims or else stop making those claims in the first place.
I really think that people who care about research methodology need to create an organization whose members attend conferences where this faux-quantitative research is being presented, ask all kinds of irritating direct questions about the methodology employed in the paper being presented, capture the Q&A session on video or transcription, and then blog it. This sort of anti-scientific nonsense needs to be stopped, and public ridicule seems to be as good an approach as any.
How about calling it the Society for Quantitative Integrity in Research Methods — or SQUIRM for short? Who’s with me?