We use a special case of the Central Limit Theorem to ensure the distribution of the sample means will be nearly normal, regardless of sample size, provided the data come from a nearly normal distribution.
Central Limit Theorem for normal data
The sampling distribution of the mean is nearly normal when the sample observations are independent
and come from a nearly normal distribution. This is true for any sample size.
While this seems like a very helpful special case, there is one small problem. It is inherently difficult to verify normality in small data sets.
Caution: Checking the normality condition
We should exercise caution when verifying the normality condition for small samples. It is
important to not only examine the data but also think about where the data come from. For example,
ask: would I expect this distribution to be symmetric, and am I confident that outliers are rare?
You may relax the normality condition as the sample size goes up. If the sample size is 10 or more, slight skew is not problematic. Once the sample size hits about 30, then moderate skew is reasonable. Data with strong skew or outliers require a more cautious analysis.