In-classproblems on hypothesis tests

Conceptual questions on hypothesis testing

Decide whether the following thedesigningfairy.comements aretrueor false. Explain your reasoning.

You are watching: As the test statistic becomes larger the p value

Problems:

a) A p-value of .08 is more evidenceagainstthe null hypothesis than a p-value of .04.

False. A small p-value means the value of the thedesigningfairy.comisticwe observed in the sample is unlikely to have occurred when thenull hypothesis is true. Hence, a .04 p-value means it is evenmore unlikely the observed thedesigningfairy.comistic would have occurred when the nullhypothesis is true than a .08 p-value. The smaller thep-value, the stronger the evidenceagainst the null hypothesis.

b) If two independent studies are doneon the same population with the purpose of testing the same hypotheses,the study with the larger sample size is more likely to have a smallerp-value than the study with the smaller sample size. (Hint:Consider if this is true in the case of the null hypothesis is trueand in the case of the null hypothesis is false.)

By definition, p-values take into considerationthe sample size, since the test thedesigningfairy.comistic has the standard error in thedenominator.Hence, when the null hypothesis is true, a small p-valueshouldbe equally likely regardless of sample size. However,whenthe null hypothesis is false, hypothesis tests done with largesamplesizes are more likely to reveal the false null, and hence more likelyto result in a small p-value.

c) The thedesigningfairy.comement, "the p-value is.003",is equivalent to the thedesigningfairy.comement, "there is a 0.3% probability that thenullhypothesis is true".

False. The null hypothesis is either true, or it is nottrue. Hence, the probability that it is true equals either zeroor one. The p-value is not interpreted as a probability that thenull hypothesis is true. It is the probability of observinga value of the test thedesigningfairy.comistic that is as or more extreme than what wasobserved in the sample, assuming the null hypothesis is true.

d) Even though you rejected the nullhypothesis,it may still be true.

True. Just by chance it is possible to get a samplethat produces a small p-value,even though the null hypothesis is true. This is called a Type Ierror. A Type II error is when the null hypothesis is notrejectedwhen it is in fact false.

e) Assuming the central limit theoremapplies,hypothesis tests are valid.

By valid, we mean that the p-value is an accurate summary of theevidence against the null hypothesis. False. The central limit theorem is needed for hypothesis teststo be valid. However, it is also necessary that the data becollected from random samples. Hypothesis tests will not remedypoorly collected data.

f) A researcher who tried to learnthedesigningfairy.comisticswithout taking a formal course does a hypothesis test and gets ap-valuethat .024. He says, "there is a 98.6% chance that thealternativehypothesis is false, so the null hypothesis is true."What, if anything, is wrong with his thedesigningfairy.comement?

False. The researcher is claimingthat (1 - p-value) is the probability that the alternative hypothesisis false.The p-value is not a probability of an alternative (or null) hypothesisbeing true or false.See the answer to part c.

g) You perform a hypothesis test using asamplesize of four units, and you do not reject the nullhypothesis.Your research colleague says this thedesigningfairy.comistical test provides conclusiveevidence that the null hypothesis is true. Do you agreeordisagreewith his conclusion? Explain your reasoning in three or lesssentences.With four units, the null hypothesis is unlikely to be rejectedbecause the variability in the sample mean will be large, i.e. thestandard error will be large. Hence,all we can say is that there is not enough data to determine whether ornot the null hypothesis is correct. In fact, whenever you fail toreject a null hypothesis, essentially you are saying that the evidencein the data does not contradict the null hypothesis. You nevercan conclude from a hypothesis test that the null hypothesis is true.

h) You are the head of the Food andDrugAdministration (F.D.A.), in charge of deciding whether new drugs areeffectiveand should be allowed to be sold to people. Apharmaceuticalcompany trying to win approval for a new drug they manufacture claimsthattheir drug is better than the standard drug at curing a certaindisease.The company bases this claim on a study in which they gave their drugto1000 volunteers with the disease. They compared thesevolunteersto a group of 1000 hospital patients who were treated with the standarddrug and whose information is obtained from existing hospitalrecords.The company found a "thedesigningfairy.comistically significant" difference between thepercentage of volunteers who were cured and the percentage of thecomparisongroup who were cured. That is, they did a thedesigningfairy.comisticalhypothesistest and rejected the null hypothesis that the percentages areequal.As director of the F.D.A., should you permit the new drug to besold?Explain your reasoning in three or less sentences.

You should not allow the drug to bemanufactured based onthis evidence. The study was not a randomized study, which meanstheremay be differences in the background charcteristics of the people whogotthe new drug and the people who got the old drug.Hypothesistests cannot fix poorly designed studies.

i) If you get a p-value of0.13, itmeans there is a 13% chance that the sample average equals thepopulation average.

False.In fact, it"s almostguaranteed thatthe sample average won"t exactly equal the population average, becausethe process of taking a random sample guarantees variability inthe sample average. Thep-value does not give the probability that the sample average equalsthe population average. See part c for the precisedefinition of p-values.j) If you get a p-value of0.13, it meansthere is a 13% chance that the sample average does not equal thepopulation average.

False.See the answer to part i.k) If you get a p-value of0.13, it means there is an 87% chancethat the sample average equals the population average.

False.Computing (1 - p-value) does not givethe probability that the sample average equals the population average.See the answer to part i.

l) If you get a p-value of0.13, it means there is an 87% chancethatthe sample average does not equal the population average.

False.See the answer to part k.

m) Ifyou get a p-value of 0.13, it means thatthe null hypothesis is true in 13% of all samples.

False. The null hypothesisis either true or false. Truth does not change with differentsamples, only the test thedesigningfairy.comistic (which is based on sample means andSEs) changes with different samples.

n) Ifyou get a p-value of 0.13,it means thatwhen the null hypothesis is true, a value of the test thedesigningfairy.comistic as ormore extreme than what was observed occurs in about 13% of all samples.

See more: Pirate Ports Of Call: Ecu Vs A&Amp;T, South Carolina Vs

True. This is a re-expression of the definition ofp-values. That is, saying there is a 13% chance of observingresults as or more extreme than what was observed is equivalent tosaying that you"d observe results as or more extreme than what wasobserved in 13% of (random) samples.