Impact of Sampling Techniques on Statistical Inference and Testing

This assignment delves into the critical role of sampling techniques in statistical inference and hypothesis testing. It emphasizes the trade-off between sample size, precision, and confidence levels. The document explores the concept of sampling distributions, highlighting how larger sample sizes lead to reduced variability and more precise estimates. It also discusses the relationship between probability and frequency, illustrating how increasing sample size allows relative frequency to approach true probability values. Furthermore, the assignment critiques the use of p-values in hypothesis testing, advocating for the use of confidence intervals to account for variability in sampling distributions. The analysis references key research papers and provides a comprehensive understanding of the nuances of statistical sampling.

Sampling techniques are often used for inferences and hypothesis testing. It is convenient when we
have large populations that can cost a lot in terms of money and/or time. But the use of samples does
not come free- a cost in terms of precision has to be paid. In simple words, we can never make any
conclusion with 100% confidence with the use of a sample. Higher confidence comes at the cost /loss of
precision.
The use of samples is associated with the use of a distribution that describes the statistic that we are
considering. Since every sample has a different statistic value we can draw a sampling distribution for
the statistic. The spread of this value is called the variability of the sampling distribution. This variability
declines as we use larger sample. As per (Frost, 2011)
‘Increasing the sample size is like increasing the resolution of a picture of the populations. With just a few
samples, the picture is fuzzy …if we collect a very large sample, the picture becomes sharp enough’ . The
largest possible size is the population itself, which implies that the least variation is found in a
population. Since we do not use the population we are saddled with higher variability. ‘As the sample
size increases, the chance of observing extreme values decreases and the observed values for the statistic
will group more closely around the mean of the sampling distribution’(Sampling in Statistical Inference).
Thus, more datapoints allow greater precision. We can be 100% sure that the mean of a population lies
between the maximum and minimum values, but such an estimate has zero precision. To narrow our
interval estimate we need to lose some confidence, giving us narrow intervals.
Probability and frequency are related through the relative frequency approach in statistics. The relative
frequency of an event becomes its probability as we allow the experiment to continue till infinity. This is
akin to saying that the sample size is very large. In 50 tosses we may get only 30 heads and 20 tails, but if
we toss the coin 1000 times we are likely to get close to 500 heads. Thus, as we increase the size of our
sample (number of tosses) the relative frequency approaches the true probability value of an event.
The use of p values becomes important as they tell us the probability of rejecting a null hypothesis.
But a simple value like this depends on variability in the sampling distribution which is used to
determine the p value. It is better to use estimates of confidence intervals rather than
hypothesis tests using p values. It is advisable to explain p in terms of where ‘the confidence
interval falls in relation to the null hypothesised value.’ (Cumming, 2010). Comparing p value
with significance level is incorrect as it ignores the variability in sampling distribution, which is

based on sample size. Thus, the relation between p value and sample size is ignored when we
use p value blindly with no reference to variability of the sampling distribution.

References
Boos, D. D. (2009, Aug 15). The varaibility of P values . Retrieved Sep 14, 2017, from
Stat.ncsu.edu: http://www.stat.ncsu.edu/information/library/papers/pvalues_2626.pdf
Cumming, G. (2010). Understanding, teaching and using p values. Retrieved Sep 12, 2017, from
Iase-web.org: https://iase-web.org/documents/papers/icots8/ICOTS8_8J4_CUMMING.pdf
Frost, J. (2011, Dec 15). Variability and Statistical Power. Retrieved Sep 16, 2017, from
Blog.minitab.com: http://blog.minitab.com/blog/adventures-in-statistics-2/variability-and-
statistical-power
Marley, S. (n.d.). Imprtance and effect of sample size. Retrieved Sep 17, 2017, from select-
statistics.co.uk: https://select-statistics.co.uk/blog/importance-effect-sample-size/