Introduction
- Meta-analysis is a statistical technique for amalgamating,
summarising, and reviewing previous quantitative research. By
using meta-analysis, a wide variety of questions can be investigated, as
long as a reasonable body of primary research studies exist.
Selected parts of the reported results of primary studies are entered
into a database, and this "meta-data" is "meta-analyzed", in similar
ways to working with other data - descriptively and then inferentially
to test certain hypotheses.
- Meta analysis can be used as a guide to answer the question 'does
what we are doing make a difference to X?', even if 'X' has been
measured using different instruments across a range of different
people. Meta-analysis provides a systematic overview of
quantitative research which has examined a particular question.
- The appeal of meta analysis is that it in effect combines all the
research on one topic into one large study with many participants. The
danger is that in amalgamating a large set of different studies the
construct definitions can become imprecise and the results difficult to
interpret meaningfully.
- Not surprisingly, as with any research technique, meta-analysis has
its advantages and disadvantages. An advantage is its
objectivity, and yet like any research, ultimately its value
depends on making some qualitative-type contextualizations and
understandings of the objective data.
- Meta-analysis has been used to give helpful insight into:
- the overall effectiveness of interventions (e.g., psychotherapy,
outdoor education),
- the relative impact of independent variables (e.g., the effect of
different types of therapy), and
- the strength of relationship between variables.
- To get more introduction to meta-analysis, go to
Effect Sizes & Confidence Intervals
- Meta analysis reports findings in terms of effect sizes. The
effect size provides information about how much change is evident across
all studies and for subsets of studies.
- There are many different types of effect size, but they fall into
two main types:
- standardized mean difference (e.g., Cohen's d or Hedges
g) or
- correlation (e.g., Pearson's r)
- It is possible to convert one effect size into another, so each
really just offers a differently scaled measure of the strength of an
effect or a relationship.
- The standardised mean effect size is basically computed as the
difference score divided by the standard deviation of the scores.
- In meta-analysis, effect sizes should also be reported with:
- the number of studies and the number of effects used to create the
estimate.
- confidence intervals to help readers determine the consistency and
reliability of the mean estimated effect size.
- For more information about calculating effect sizes and confidence
intervals, see:
- Tests of statistical significance can also be conducted and on the
effect sizes.
- Different effect sizes are calculated for different constructs of
interest, as predetermined by the researchers based on what issues are
of interest in the research literature.
- Rules of thumb and comparisons with field-specific benchmarks can be
used to interpret effect sizes. According to an arbitrary but
commonly used interpretation of effect size by Cohen (1988), a
standardised mean effect size of 0 means no change, negative effect
sizes mean a negative change, with .2 a small change, .5 a moderate
change, and .8 a large charge. Wolf (1986), on the other hand, suggests
that .25 is educationally significant and .50 is clinically significant.
Using Effect Sizes in Primary Studies
- Meta-analysis methodologies, particularly effect sizes, are also
applicable to primary research. For example, effect sizes are
particularly useful in program evaluation studies. For more
information:
How to Conduct a Meta-analysis
Meta-analytic Studies of Psychological
Interventions
Hattie, J. (1992). Self-concept. NJ:
Lawrence Erlbaum.
Lipsey, M. W., & Wilson, D. B. (1993).
The efficacy of psychological, educational, and behavioral treatment.
American Psychologist, 48, 1181-1201.
Smith, M. L., Glass, G. V., & Miller, T. I.
(1980). The benefits of psychotherapy. Baltimore: Johns
Hopkins University Press.
Meta-analysis Methodology References
Bushman, B. J., & Wells, G. L. (2001).
Narrative impressions of literature: The availability bias and the
corrective properties of meta-analytic approaches. Personality and
Social Psychology Bulletin, 27, 1123-1130.
Cohen, J. (1988). Statistical power analysis
for the behavioral sciences (2nd ed.). New York: Academic Press.
Glass, G. V. (1976). Primary, secondary, and
meta-analysis of research. Educational Researcher, 5,
3-8.
Glass, G. V. (1977). Integrating findings: The
meta-analysis of research. Review of Research in Education,
5, 351-379.
Wolf, F. M. (1986). Meta-analysis: Quantitative
methods for research synthesis. Beverly Hills, CA:
Sage. |