## Combining the results from several experiments

Suppose that we have carried out an experiment to compare a standard/reference
diet and 5 other diets (Standard, D2, D3, D4, D5 and D6). We find that D2
is better than the Standard diet, but given the variability and the numbers
of experimental units (animals) on each diet the difference was not
statistically significant at the 5% level, which we had previously decided to
be our cut-off point for considering an effect to be real (statistically
significant) or not. But nonetheless there was a *'tendancy'.*
Suppose that we had obtained:

D2 - Standard = 5.20 ± 3 Pr = ~.10 => -2 ln Pr = 4.6052
In a previous experimental trial the previous year another budding graduate
student had used the same Standard diet and D2, as well as other diets,
and obtained:

D2 - Standard = 5.00 ± 2.5 Pr = ~.055 => -2 ln Pr = 5.8008
In reviewing the literature we find that another researcher at another
University has published a paper and amongst other comparisons has also used
the Standard diet and D2, and obtained:

D2 - Standard = 4.5 ± 3 Pr = ~.18 => -2 ln Pr = 3.4296
None of these results are statistically significant, but they all tend in the
same direction; suggesting that Diet 2 is superior to the Standard.
Can we combine these results?

Yes, Fisher (1950) showed that * -2 ln P * is distributed as a
c with 2 degrees of freedom.
So, if the trials are indeed seperate, completely independent, we can add
these c together
to produce a pooled combined result.

See Steel, Torrie and Dickey, Chapter 20.5.

| |
-2 ln P (c^{2}) |

So | | 4.6052 |

| + | 5.8008 |

| + | 3.4296 |

| = | 13.8356 |

The critical, tabulated value for a c^{2}
with 6 degrees of freedom
at the 5% level is 12.6. Why 6 d.f.? Because we are combining 3 trials,
each with 2 degrees of freedom for the c^{2}.
It is irrelevant
how many experimental units (animals, or plants, or whatever) and degrees of
freedom there were for each of the trials. We are not combining the parameter
estimates, we are combining the probabilities.

From the above we would reject the Null Hypothesis, that the 2 diets are
equal and we would conclude that they differ; that D2 is indeed better
than the Standard diet.

Note the additional information that we have been able to glean, or extract,
from 3 non-significant results.

R.I. Cue ©

Department of Animal Science, McGill University

last update : 2010 May 4