The State of Research Trials

[Griffin NoName]

False Positives: fraud and misconduct - this makes me mad.

[Bob in a quantum-state-of-faith]

False Positives: fraud and misconduct - this makes me mad.

That's one of the (many) problems with the soft sciences such as psychology & sociology.  Difficult to get real, unbiased data when you're talking about actual ... people (who are the data).

[Swatopluk]

I'd say any scientist that claims never to have 'massaged' data should be considered a liar as default.
This does not mean they're all frauds. The nature of data even in the 'hard' sciences makes it close to impossible to do without. Even if the theory is sound and measurements are taken meticilously, there will almost always be parts that do not fit and some will be far off. Any scientist worth the name will use his judgement in treating those outliers and in doing so commit a (usually lesser) sin.
One extremly common method is to repeat a measurement until it fits. Unless of course the 'outlier' turns out to be real thing. Then the choice is between looking for the real reason or to commit real fraud by suppressing the consistent deviation.

[Roland Deschain]

Fanelli found that "the odds of reporting a positive result were around five times higher among papers in the disciplines of psychology and psychiatry and economics and business compared with space science".

This pretty much explains how I feel about those fields. They may or may not be more complex than research in a hard science, as details vary greatly, but when it comes to studying people, subjectivity comes into play far more easily. I'd say it was worse in a field such as theology, where you're studying what is essentially unknowable, but that's not at issue here.

Science needs a far better way to ensure its checks and balances are stronger, such as changing the method of publication. As mentioned in the article, an online database of papers - each categorised, tagged, and linked to one another - would be one of the more useful methods, as if you want to look up research in one particular field, you instantly get papers doing the same research, either with the same, similar, or different results, and also other papers related to that one.

You'd be able to see at a glance whether results were replicated successfully or not, and where potential issues may lie. This would need to be a non-profit journal, with voluntary donations from every participating university or institution. It wouldn't stop people not submitting negative results, as evidenced by the number of cases coming from pharmaceutical companies, but at the least it would help to reduce a lot of the fraud, and make it far easier to spot plagiarism.

A program could spend its time scouring the database for these plagiarised papers, highlighting any that seem suspect, which would then be looked at by a human eye to see if it is indeed fraudulent. There could even be checklists assigned to each paper, detailing the types of methods reported as being used, along with other criteria. The potential for cross-referencing would be immense, as would the potential for spotting sloppy research methods.

Is that not a far better prospect for the future of research than what is currently starting to come out?

[Swatopluk]

Can't speak for other sciences but in some (I know about chemistry) the language and style has become so much standardized that plagiarism is difficullt to prove by text alone. And the 'positive better than negative' is, I think, even stronger in the hard sciences.

[Griffin NoName]

Can't speak for other sciences but in some (I know about chemistry) the language and style has become so much standardized that plagiarism is difficullt to prove by text alone.

Don't get me started on plagiarism I anticipate spending the whole of this week trying to squeeze my essay past the software.   Psychotherapy is a soft science, which is why I intend doing a qualitative research project. Even mental states are subjective despite so called gold standard measuring scales. What with Paychiatrists being close to G-d.

[Swatopluk]

My last comment was in part in recognition that we had that discussion already (about your trouble with insipid filters).

[Griffin NoName]

Yes, I recognised that, but still had to get on my soapbox, mainly because it is imminent agony. Let's hope my page numbers aren't matched this time.

[Swatopluk]

Avoid tha lattar 'e', it's joost ar too oftan. Also avoid 'and', 'or', 'the' and 'a'. Use 'it' sparingly.
And of course, never ever use 'I'.
The latter is not actually a joke (although it should be). Some journals do reject any use of 1st person Sg. (and at best tiny amounts of Pl.). 3rd person all the way and passive past tense wherever grammatically possible (occasionally even where it is not).

[Griffin NoName]