This course covers commonly used statistical inference methods for numerical and categorical data. You will learn how to set up and perform hypothesis tests, interpret p-values, and report the results of your analysis in a way that is interpretable for clients or the public. Using numerous data examples, you will learn to report estimates of quantities in a way that expresses the uncertainty of the quantity of interest. You will be guided through installing and using R and RStudio (free statistical software), and will use this software for lab exercises and a final project. The course introduces practical tools for performing data analysis and explores the fundamental concepts necessary to interpret and report results for both categorical and numerical data
Small Sample Proportions
Loading...
來自 Duke University 的課程
Inferential Statistics
1233 個評分
從本節課中
Inference for Proportions
Welcome to Week Four of our course! In this unit, we’ll discuss inference for categorical data. We use methods introduced this week to answer questions like “What proportion of the American public approves of the job of the Supreme Court is doing?”.
與講師見面
Mine Çetinkaya-RundelAssociate Professor of the Practice
Department of Statistical Science
So far in this unit, we work with
largest samples, where the success/failure conditions was met.
But what if it's not met?
Then what comes to our rescue is inference via simulation.
We did a little bit of this earlier in the
class when we worked over the gender discrimination case example.
So in this video, were going to review how we
set up a simulation assuming that the null hypothesis is true.
Because, remember that if we're doing any sort of hypothesis
test where the ultimate goal is to get a p-value, the definition
of the p-value stays regardless of what type of method you're using.
It's always an observed or more extreme
outcome given that the null hypothesis is true.
So we want to make sure that throughout our hypothesis
test, we act as if that null hypothesis is true,
and what that means is that we set up a
simulation scheme which assumes that that null hypothesis is true.
Let's give a quick example.
Remember this guy, Paul the Octopus, who became famous for predicting
correctly the outcome of soccer games during the 2010 World Cup.
So the setup that he had was, he was given these
two boxes, with a little bit of food in each of them.
And these boxes had flags of the countries who were
playing against each other in the World Cup that day.
And basically
the, his predictions were assumed to be whichever
box he chose to get the food out of.
So, he was, he became famous because he was actually able
to predict all eight World Cup games and predicted them all correctly.
We want to see, does this provide
convincing evidence that Paul actually has psychic powers?
In other words, that he does better than just randomly guessing.
Because in his
setup he had only two countries to choose from, if he is
randomly guessing, he would be expected to get right 50% of the time.
So the null hypothesis, which claims that no he does not have psychic powers, he's
simply randomly guessing, would set the, the true proportion of success to 0.5.
If he's doing better than randomly guessing, then
the alternative hypothesis should say that p is greater
than 0.5.
Let's check to see if our conditions for inference are met here.
We also know that the sample size is eight, or our number of trials is eight,
and Paul the Octopus guessed all of them correctly so p hat is one or 100%.
Let's check to see if the conditions for inference are met here.
In terms of independence, it seems like we can assume
that his guesses are independent of each other from one
time to the other.
In terms of sample size and skew, we need to check our success/failure condition.
We have eight trials times 0.5, our null value, gives us four.
So that it appears that our success/failure is not met.
Meaning that the distribution of sample proportions
cannot be assumed to be nearly normal.
Which means that we cannot use methods that rely on the central limit
theorem and the normality of the sampling distribution to find our p-value.
And this is one, one when once again
the simulation based inference comes to our rescue.
So how do we do simulation based inference?
Let try, let's try to remind ourselves.
Remember the ultimate goal of the hypothesis test is a p-value.
And the p-value is the probability of observed or more extreme outcome given
that the null hypothesis is true.
So what we want to do is to devise a
simulation scheme that assumes that the null hypothesis is true.
And we want to repeat the simulation
many times and record the relevant sample statistic.
Finally we calculate the p-value as the proportion of
simulations that yield a result favorable to the alternative hypothesis.
For those of you that remember the examples we
did earlier in terms of inference via simulation, these steps should make sense.
For those of you who do not remember them, then please revisit these
steps once again after we go through the calculations for this particular example.
So given that our null value is 0.5, how do we set up a simulation scheme?
We can use a fair coin and label heads as successes.
This is our correct guesses.
We could also use tail, but in this case we're choosing to use heads.
And one simulation can be comprised of a flip of the coin
eight times, and recording the proportion of the heads, the correct guesses.
Remember we're trying to simulate whatever Paul did as many times as
possible, and we need to think of his eight trials as one batch.
And we want at each simulation to recreate
that batch of eight trials, and calculate his rate of success.
Remember his rate of success was one.
And we were going to try to see if we leave things up
to chance, what does the rate of success come out to be.
We repeat the simulation many times then,
recording the proportion of heads at each iteration.
Finally, we calculate the percentage of simulations where the simulated proportion
of heads is at least as extreme as the usual observation.
So let's take a look to see how we can actually do this.
In our first simulation, we said that we're going to flip a coin eight times,
so lets flip the coin once, and it seems like we get a head first.
So that would be a success.
We can flip the coin one more time, another head.
Another flip of the coin, another head.
Another flip of
the coin, a tail. Another flip of the coin, another head.
We have three more to go in order to get to our eight tosses.
One more, another head.
Another head, and lastly yet another head. So in this case, our
sample proportion, or the proportion of success, is seven over eight or 0.875.
We record this
number, and we're going to collect these on the
dot plot, dot plot at the bottom of the screen.
For the second simulation, we once again have eight slots, and
we toss the coin eight times and we record the outcomes.
And in this case we have three out of five heads.
So our proportion of success here would be 0.375, and
we record that number on our dot plot as well.
Another simulation,
yet another set of eight flips, and then we
will want to count how many of those were heads.
So that seems like five out of eight, and we want to record that number as well.
We can keep doing this forever, and we would want to do it as many
times as possible, but for illustrative purposes, we're
only going to really get to ten simulations.
So let's say that at each iteration we're
collecting these data the simulated p hat, and finally
when we get to the last simulation again we roll the coin eight more times.
Seems like we have six out of eight heads, for 0.75.
So this is what our simulated distribution looks like for p hat.
Obviously, if in fact we actually had done a lot of simulations, as
we should, the shape of the simulation would look maybe only slightly different.
We would definitely have more
observations, and the shape should probably follow something similar to this,
but ten simulations is definitely not sufficient to make a call.
However, based on this and based on the definition of
the p-value, as the probably of observed or more extreme outcome.
So in this case, our observed outcome was 100% success.
So the p-value can be defined as what is the probability of
100% or more success, which doesn't even exactly make sense,
given that the true rate of success was only 50%.
We don't have any data, any simulated sample proportions that
actually fit the bill, so based on this simulation, our p-value is zero.
It's probably usually a good idea to say that it's almost zero.
And chances are if we had actually done
this properly with about 10,000 or so simulations
we would get a number that's small, which
would probably also yield a rejection of the
null hypothesis, but it may not be exactly zero.
Of course, when we're thinking 10,000 simulations, we would never think
of doing that by hand and we would use R for it.
So, the first thing we want to do is we want to load our inference
function that you should have been using in the labs and
we're going to use in order to do the simulation test.
We can then define what data from Paul the
Octopus looks like, that's eight yesses and zero noes.
And finally, we can write our inference
function as where we're estimating a proportion,
we're doing a hypothesis test using a
simulation method, and we’re calling the outcome yes,
a success.
Our null value is 0.5 and our alternative
hypothesis is looking for parameter greater than 0.5.
We’re looking for evidence of a parameter greater than 0.5.
In this case, the p-value with 10,000 simulations, which is the default for this
function, comes out to be 0.0037, meaning
that again we would reject the null hypothesis.
Let's think here though, what does
rejection of the null hypothesis here mean?
Does it mean that we found evidence that Paul is psychic?
Probably not, and chances are we've made some sort of
an error where the null hypothesis should not have been rejected.
We had a pretty small sample size. It appeared to show a trend in a certain
way, and what that, those particular data yielded is a small
p-value based on which, yes, we would definitely reject the null hypothesis.
But we might be making a type one error rejecting a null
hypothesis that says this octopus simply randomly picks when we shouldn't have.
The possibility would be to try to collect a little more data from
Paul the Octopus, but unfortunately he passed
away shortly after he became a sensation.
