Inferential Statistics
- Comparing Groups I

Further information on this topic can be found in Chapter 10 of:

Maths from Scratch for Biologists

Numerical ability is an essential skill for everyone studying the biological sciences but many students are frightened by the 'perceived' difficulty of mathematics, and are nervous about applying mathematical skills in their chosen field of study. Maths from Scratch for Biologists is a highly instructive, informal text that explains step by step how and why you need to tackle maths within the biological sciences. (Amazon.co.UK)

infer:     to conclude from evidence

There are two methods of reaching a statistical inference:

a) Estimation

In estimation, a sample from a population is studied and an inference is made about the population based on the sample.

The key to estimation is the probability with which particular values will occur during sampling - this allows the inference about the population to be made.

The values which occur are inevitably based on the sampling distribution of the population. The key to making an accurate inference about a population is random sampling, where:

each possible sample of the same size has the same probability of being selected from the population.

In real life, it is often difficult to take truly random samples from a population. Shortcuts are frequently taken, e.g. every third item on a list, or simply the first n results to be obtained. A better method is to use a table of random numbers or the Microsoft Excel RAND function.

Estimation is a relatively crude method of making population inferences. A much better method and the one which is normally used is:

b) Hypothesis Testing

To answer a statistical question, the question is translated into a hypothesis - a statement which can be subjected to test. Depending on the result of the test, the hypothesis is accepted or rejected.

The hypothesis tested is known as the null hypothesis (H₀). This must be a true/false statement.
For every null hypothesis, there is an alternative hypothesis (H_A).

Constructing and testing hypotheses is an important skill, but the best way to construct a hypothesis is not necessarily obvious:

• The null hypothesis has priority and is not rejected unless there is strong evidence against it.
• If one of the two hypotheses is 'simpler' it is given priority so that a more 'complicated' theory is not adopted unless there is sufficient evidence against the simpler one (Occam's Razor: "If there are two possible explanations always accept the simplest")
• In general, it is 'simpler' to propose that there is no difference between two sets of results than to say that there is a difference.

The outcome of a hypothesis testing is "reject H₀" or "do not reject H₀". If we conclude "do not reject H₀", this does not necessarily mean that the null hypothesis is true, only that there is insufficient evidence against H₀ in favour of H_A. Rejecting the null hypothesis suggests that the alternative hypothesis may be true.

In order to decide whether to accept or reject the null hypothesis, the level of significance (a) of the result is used (a = 0.05 or a = 0.01).
This allows us to state whether or not there is a "significant difference" (technical term!) between populations, i.e. whether any difference between populations is a matter of chance, e.g. due to experimental error, or so small as to be unimportant.

Procedure for hypothesis testing:

1. Define H₀ and H_A, based on the guidelines given above.
2. Choose a value for a. Note that this should be done before performing the test, not when looking at the result!
3. Calculate the value of the test statistic.
4. Compare the calculated value with a table of the critical values of the test statistic.
5. If the calculated value of the test statistic is LESS THAN the critical value from the table, accept the null hypothesis (H₀).
   If the absolute (calculated) value of the test statistic is GREATER THAN or EQUAL to the critical value from the table, reject the null hypothesis (H₀) and accept the alternative hypothesis (H_A).

Note that:

• A significance test can never prove a null hypothesis, only fail to disprove it.

• A very small P value (e.g. 0.001) does not signify a large effect - it signifies that the observed data are highly improbable given the null hypothesis. A very small P value can arise when an effect is tiny but the sample sizes are large. Conversely a larger P value can arise when the effect is large but the sample size is small.

Standard Scores (z Scores)

z scores define the position of a score in relation to the mean using the standard deviation as a unit of measurement.

Standard scores are therefore useful for comparing datapoints in different distributions.

z = (score - mean) / standard deviation

The z-score is the number of standard deviations that the sample mean departs from the population mean.

Since this technique normalizes distributions, z-scores can be used to compare data from different sets, e.g. a student's performance on two different exams (e.g. did Joe Blogg's performance on module 1 and module 2 improve or decline?):

• JoeB scored 71.2% on module 1 (mean = 65.4%, SD = 3.55)   z = (71.2 - 65.4) / 3.55 = 1.63

• JoeB scored 66.8% on module 2 (mean = 61.1%, SD = 2.54)   z = (66.8 - 61.1) / 2.54 = 2.24

• Conclusion: JoeB did better, relatively speaking, on module 2 than on module 1, even though his mark was lower on this course.

Note that the z-score is only informative when it refers to a normal distribution - calculating a z-score from a skewed dataset may not produce a meaningful number. Comparing z-scores for different distributions is also meaningless unless:

• The datasets being compared are as similar as possible (e.g. response to different doses of a drug under the same physiological conditions).

• The shapes of the distributions being compared are as similar as possible.

Comparing Two Populations

Biological systems are complex, with many different interacting factors.

To compensate for this, the most common experimental design in biology involves comparing experimental results with those obtained under control conditions.

To interpret this type of experiment, we must be able to make objective decisions about the nature of any differences between the experimental and control results - is there a statistically significant difference or are the results due to experimental error or random chance (sampling error)?

A frequently used test of statistical significance is the:

Student's t-test (t-test)

The Student's t-test (or simply t-test) was developed by William Gosset - "Student" in 1908. Gossett was a chemist at the Guiness brewery in Dublin and developed the t-test to ensure that each batch of Guiness was as similar as possible to every other batch! The t-test is used to compare two groups and comes in at least 3 flavours:

• Paired t-test: used when each data point in one group corresponds to a matching data point in the other group.

• Unpaired t-test: used whether or not the groups contain matching datapoints:
  • Two-sample assuming equal variances
  • Two-sample assuming unequal variances

Paired t-test: The paired t-test is used to investigate the relationship between two groups where there is a meaningful one-to-one correspondence between the data points in one group and those in the other, e.g. a variable measured at the same time points under experimental and control conditions. It is NOT sufficient that the two groups simply have the same number of datapoints!

The advantage of the paired t-test is that the formula procedure involved is fairly simple:

1. Start with the hypothesis (H₀) that the mean of each group is equal (H_A: the means are not equal). How do we test this? By considering the variance (standard deviation) of each group.

2. Set a value for a (significance level, e.g. 0.05).

3. Calculate the difference for each pair (i.e. the variable measured at the same time point under experimental and controlled conditions).

4. Plot a histogram of the differences between data pairs to confirm that they are normally distributed - if not, STOP!

5. Calculate the mean of all the differences between pairs (d_av) and the standard deviation of the differences (SD)

6. The value of t can then be calculated from the following formula:

where:
d_av is the mean difference, i.e. the sum of the differences of all the datapoints (set 1 point 1 - set 2 point 2, ...) divided by the number of pairs
SD is the standard deviation of the differences between all the pairs
N is the number of pairs.

N.B. The sign of t (+/-) does not matter, assume that t is positive.

7. The significance value attached to the resulting value of t can be looked up in a table of the t distribution (or obtained from appropriate software). To do this, you need to know the "degrees of freedom" (df) for the test. The degrees of freedom take account of the number of independent observations used in the calculation of the test statistic and are needed to find the true value in a probability table. For a paired t-test:

df = n-1   (number of pairs - 1)

Groan! Why do we need something as complicated as "degrees of freedom" ?

8. To look up t , you also need to determine whether you are performing a one-tailed or two-tailed test. In any statistical test we can never be 100% sure that we have to reject (or accept) the null hypothesis. There is, therefore, the possibility of making an error:

Falsely rejecting a true Ho is called a type I error (finding an innocent person guilty). The probability of committing a type I error is always equal to a.
Failure to reject a false Ho is called a type II error (finding a guilty person innocent). The probability of committing a type II error depends on the probability of retaining a false H₀.
The "power" of a statistical test refers to the probability of claiming that there is a significant difference when this is true.
As scientists are cautious, it is considered "worse" to make a type I error than a type II error - we thus reduce the possibility of making a type I error by having a stringent rejection limit, i.e. 5%. However, as we reduce the possibility of making one type of error, we increase the possibility of making the other type.
Whether you use a one- or two-tailed test depends on your testing hypothesis:

• One-tailed test: Used where there is some basis (e.g. previous experimental observation) to predict the direction of the difference, e.g. expectation of a significant difference between the groups.
• Two-tailed test: Used where there is no basis to assume that there may be a significant difference between the groups - this is the test most frequently used.
• Why are they called "tails"?

  Note that H_A states 'there is a difference .... ', it does not state why there is a difference or whether the difference between the two groups if greater or less than. If H_A had specified the nature of the difference, this would have been a one-tailed hypothesis. However, since H_A does not specify the nature of the difference, hence we can either accept a reduction or an increase. This is therefore a two-tailed hypothesis. For a variety of reasons two-tailed hypotheses are safer than one-tailed. Statistical tables are sometimes tabulated only for one-tailed hypotheses. To convert them to two-tailed, double a.

Critical values of t for Student's t distribution

9. If the calculated value of t is greater than the critical value, H₀ is rejected, i.e. there is evidence of a statistically significant difference between the groups.
   If the calculated value of t is less than the critical value, H₀ is accepted, i.e. there is no evidence of a statistically significant difference between the two groups.

Unpaired t-test: The unpaired t-test does not require that the two groups be paired in any way, or even of equal sizes. A typical example might be comparing a variable in two experimental groups of patients, one treated with drug A and one treated with drug B. Such situations are common in medicine where an accepted treatment already exists and it would not be ethical to withhold this from a control group. Here, we wish to know if the differences between the groups are "real" (statistically significant) or could have arisen by chance. The calculations involved in an unpaired t-test are slightly more complicated than for the paired test. Note that the unpaired t-test is equivalent to one-way ANOVA used to test for a difference in means between two groups. To perform an unpaired t-test:

1. Plot a histogram of the datasets to confirm that they are normally distributed - if not, STOP and use a non-parametric method!

2. Start with the hypothesis (H₀) "There is no difference between the populations of measurements from which samples have been drawn". (H_A: There is a difference).

3. Set a value for a (significance level, e.g. 0.05).

4. Check the variance of each group. If the variances are very different, you can already reject the hypothesis that the samples are from a common population, even though their means might be similar. However, it is still possible to go on and perform a t-test.
5. Calculate t using the formula:

where:
= means of groups A and B, respectively, and

6. Look up the value of t for the correct number of degrees of freedom and for a one- or two-tailed test (above).
   For an unpaired t-test:

df = (n_A + n_B) - 2

where n_A/B = the number of values in the two groups being compared. Remember that the sign of t (+/-) does not matter - assume that t is positive.

7. If the calculated value of t is greater than the critical value, H₀ is rejected, i.e. there is evidence of a statistically significant difference between the groups.
   If the calculated value of t is less than the critical value, H₀ is accepted, i.e. there is no evidence of a statistically significant difference between the groups.

Example:

Consider the data from the following experiment. A total of 12 readings were taken, 6 under control and 6 under experimental conditions:

• Are the datapoints for the control and experimental groups paired?
  - Nope, they are just replicate observations.

• Can we perform an unpaired t-test?
  - Are the data normally distributed?

Yes, approximately:

• H₀: "There is no difference between the populations of measurements from which samples have been drawn"
  (H_A: There is a difference).
  Set value of a = 0.05

• Are the variances of the two groups similar?
  - Yes (1.68 / 2.46) - less than 2-fold difference.
  Since all the requirements for a t-test have been met, we can proceed:

• t = 1.08

• Is this a one-tailed or a two-tailed test?
  - Two tailed, since we have no firm basis to assume that there is a difference between the groups.

• How many degrees of freedom are there?

df = (n_A-1)+(NB-1) = 10

• From the table of values for t, we can see that for a two-tailed test with df=10 and a=0.05, t would have to be 2.228 or greater for >5% (0.05) of pairs of samples to differ by the observed amount:

• If the calculated value of t is GREATER THAN the critical value of t (from the table), REJECT H₀.
  If the calculated value of t is LESS THAN the critical value of t (from the table), ACCEPT H₀.

• Since t_calc = 1.08 and t_crit = 2.228, the null hypothesis is accepted.

• CONCLUSION: There is no statistically significant difference (at the 95% confidence level) between the Experimental and the Control groups.

ANalysis Of VAriance: ANOVA

Student's t-test can only be used for comparison of two groups. Although it is possible to perform many pairwise comparisons to analyze all the possible combinations involving more than two groups, this is undesirable because: a) it is tedious, and b) it increases the possibility of type I errors. However, ANOVA can compare two or more groups:

Sir Ronald Fisher (1890-1962) developed the technique of ANOVA, which comes in various flavours:

• One-way (or one-factor) ANOVA: Tests the hypothesis that means from two or more samples are equal (drawn from populations with the same mean). Student's t-test is actually a particular application of one-way ANOVA (two groups compared) and results in the same conclusions.

• Two-way (or two-factor) ANOVA: Simultaneously tests the hypothesis that the means of two variables ("factors") from two or more groups are equal (drawn from populations with the same mean), e.g. the difference between a control and an experimental variable, or whether there is a difference between alcohol consumption and liver disease in several different countries. Does not include more than one sampling per group. This test allows comments to be made about the interaction between factors as well as between groups.

• Repeated measures ANOVA: Used when members of a random sample are measured under different conditions. As the sample is exposed to each condition, the measurement of the dependent variable is repeated. Using standard ANOVA is not appropriate because it fails to take into account correlation between the repeated measures, violating the assumption of independence. This approach can be used for several reasons, e.g. where research requires repeated measures, such as longitudinal research which measures each sample member at each of several ages - age is a repeated factor.

The F ratio

The F (Fisher) ratio compares the variance within sample groups ("inherent variance") with the variance between groups ("treatment effect") and is the basis for ANOVA:

F = variance between groups / variance within sample groups

You can perform various types of ANOVA analysis using the Excel Analysis ToolPak:

One-way ("Single Factor") ANOVA:

"Pain Score" for 3 Analgesics:
Aspirin:	Paracetemol (Acetaminophen):	Ibuprophen:	Control (no drug):
5	4	4	5
4	4	4	5
5	3	5	5
3	4	3	4
5	5	3	5
5	3	5	5
4	4	3	5

• The null hypothesis (H₀) is that there is no difference between the four groups being compared.
• In this example, with a significance level of 95% (a = 0.05), since the calculated value of F (3.23) is greater than F_crit (3.01), we reject the null hypothesis that the three drugs perform equally.
• A post-hoc comparison or individual pairwise comparisons would have to be be performed to determine which pair or pairs of means caused rejection of the null hypothesis.

Two-way ANOVA ("Two-factor without replication"):

• As always, the null hypothesis (H₀) is that there is no difference between the groups being compared.

• In this example, with a significance level of 95% (a = 0.05), the calculated value of F (10.57) for the table rows (Orchard 1 vs. Orchard 2) is greater than F_crit (2.82), so the hypothesis that there is no difference between the orchards is rejected.

• However, the calculated value of F (0.48) for the table columns (Bait 1 vs. Bait 2) is less than F_crit (2.82), so the hypothesis that there is no difference between the pheromone baits is accepted.

© MicrobiologyBytes 2007.