On Being a Scientist will provide you with an overview of scientific practice, what it means to be a scientist and allows you to become acquainted with academic conduct, thus meeting a demand for increased awareness in scientific integrity. This course is designed to inform you on topics as scientific integrity and social responsibilities of scientists. Broad questions, which are inseparably linked to these topics are discussed: namely regarding the nature of science and the societal role it fulfills. Course objectives: After this course you will: 1) Understand the basic principles of science, and know what is "not done". 2) Have a realistic image of science and scientists. 3) Recognize integrity dilemmas, know how to respond in clear cases, and have the skills to respond prudently in unclear cases. 4) Know and understand the differences and similarities of various disciplines. 5) Have a basic understanding of the role of science in society, realise your own societal responsibilities, and are able to take a position in societal issues where science plays a role. The course consists of a feature film, supported by short lectures, set to serve as a starting point for the discussions and assignments.
The scientific method. Part II
Loading...
來自 Universiteit Leiden 的課程
On Being a Scientist
9 個評分
從本節課中
Scientific activities
This module questions the notion of a single scientific method. It also discusses such issues as publication bias, confirmation bias and questionable research practices.
與講師見面
Bas HaringProf.dr.
Science
Frans van LunterenProf. dr.
Science
In the previous lecture,
I referred to John Loannidis' suggestion that most scientific results are false.
Now I want to explore the factors that have given rise to this pessimistic view.
So let us look at one of the main problems.
Nowadays, many experimental tests depends on Statistical Evidence.
When we pose a question to nature, nature hardly ever says yes or no or
even probably yes or probably no.
But rather, something like, if the effect that we are looking for
does not really exist, the probability of getting these results or
even stronger ones is pretty small.
Now this probability of a so-called false positive of getting results,
suggesting an effect that isn't really there, is known as the p-value.
In many fields, if the p-value is equal to or
smaller than 5%, the result is considered significant.
In other words, we conclude that the effect is real, but
such a conclusion may be highly premature.
For instance, you would expect that, in general,
only 5% of all published results is false.
But this is not the case, as a simple example can show.
Suppose that of all the hypotheses, we would like to test 1 in 10 is true.
Now imagine tests on 1,000 hypotheses,
100 of which are true, and therefore 900 are false.
With a false positive rate of 5%,
we may expect 5% of 900, or 45 false positives.
Suppose, furthermore, that the tests have a power of 80%.
That is, they confirm 80%, in this case,
80 of the 2 hypotheses, thereby producing 20 false negatives.
Not knowing what is false and what is true, the researchers will conclude
that 125 hypotheses are true, 45 of which are not.
As negative results are rarely published, the proportion of the published results,
which is false, will be much higher than 5%.
In this case, even more than 30%.
Now, in reality, the situation is probably even worse.
As researchers are only human,
they tend to be liable to what is known as confirmation bias.
In other words, they tend to find what they expect or hope to find.
Scientists have to make constant judgments about reliability of their data
during the experiments.
Results often evoke more trust if they fit theoretical predictions or expectations.
Whereas in case of unexpected results, we are more liable to re-run the experiment
or to find reasons why we should actually discard the unwelcome results.
And this, of course, may lead to selective use of data.
Confirmation bias may affect the judgement of the most scrupulous scientists.
Finally, not all scientists are that scrupulous.
Again, being only human, they may succumb to what is known as Publication Pressure.
As we have seen,
they need lots of publications in order to further their career.
Now in general, scientists need to produce
positive results in order to be able to publish their results.
It is, therefore, tempting to slightly improve your results, so
as to make them publishable.
And we have seen a clear example of such behavior in the film
where Ponter instructs Rebecca to polish her results.
Everybody does it.
Now there are many ways to improve your results, even without committing fraud.
That is, without fabricating or falsifying data.
Some ways of polishing your data are known as p-hacking.
There are many different varieties.
They usually involve selective use of data.
For instance, we can test a huge number of hypotheses about a single data set
by exhaustively searching for
combinations of variables that show a meaningful correlation.
This is also known as data dredging.
We can also try different tests on the same hypotheses, but
only report the most significant test.
Or we can only analyze an interesting subset of the data
in which a pattern can be found.
And finally, we can analyze a data set and decide on whether to collect more data or
not depending on the data collected so far.
Now such practices are far from rare, as recent surveys have shown.
They are known as questionable research practices or 'sloppy science'.
This is the gray area between fraud and responsible research conduct.
So maybe the suggestion that most
published results are false is not that far off the mark.
Indeed, recent attempts to reproduce generally accepted results in social
sciences and biomedical sciences failed in more than half of the cases.
Still, as fallible as science may be,
it is still the best way we have to produce knowledge.
But there's always room for improvement.
In fact, many steps are taken to improve the situation.
One way to do so is to make it easier to publish negative results.
Another one is to put less emphasis on the number of publications as a quality
measure.
Above all, as scientists, we need to be honest to others and to ourselves.
