Common effect meta-analysis with Stata

You can start reading about Stata at Section 2.4.6 of our book.

This is a meta-analysis where we consider all studies to have data drawn from distribution(s) where the contrast between intervention and control has the same true population value. Some writers call this fixed-effects, which we think is potentially confusing for beginners; others use that term to mean something else. Michael Borenstein calls it “fixed effect (singular)”, which is better, but we prefer common effect to avoid any confusion. The code here relates to Section 3.6.3.

Caveat: some meta-analysts, including Robert and Gian Luca, don’t believe that common effect meta-analysis (CE MA) is really a defensible model for studies, unless they were done in-house to exactly the same conditions. However, it is an important first step in learning about MA as a statistical model, rather than a mysterious estimator that has some justification in terms of a weighted average.

Each study reports an estimate of intervention effect and a standard error. These intervention effects, or contrasts will have asymptotically normal sampling distributions, but for small sample sizes in studies, you should consider a t-distribution instead (see below).

See Chapter 5 of the book if you have to calculate the standard errors from other statistics given. In another post, we show models for unreported standard errors, but in this example, we assume all the stats are known.

Simple contrast-based models

Let’s assume there are m studies. Each study reports a log odds ratio in a variable called logor, and its standard error in a variable called se_logor. We square the standard errors first, as Stata will expect variances as input to normal likelihoods. In this simple example, the rationale for doing a Bayesian analysis is to have an informative prior on theta.

gen se2 = se_logor^2

bayesmh logor, likelihood(normal(se2)) ///
               prior({logor:_cons}, ///
                     ADD_YOUR_PRIOR_HERE) ///
               nchains(2)

Note that bayesmh expects the task to be a form of regression, with a dependent variable, which in this case is logor. The parameter or unknown {logor:_cons} is the constant in the regression formula. Because there are no independent variables (also known as covariates or predictors), this is simply the overall mean, usually denoted by theta. By specifying se2 in the normal likelihood, we are fixing the variance rather than getting Stata to infer it.

This runs two chains with the default of 2500 burn-in iterations and 10,000 sampling iterations each.

Although this shows a log odds ratio as the contrast statistic, the same model is applicable to any statistic that: (1) has an asymptotically normal sampling distribution and the sample size is big enough to rely on this (otherwise, see Exact Likelihoods below), and (2) we are content to treat the standard errors as perfectly known. You could use a mean difference or log hazard ratio, for example.

Why don’t we give priors in our online code? Well, an important stimulus to writing our book was the widespread adoption of the network meta-analysis BUGS code given in 2011 by the NICE Decision Support Unit report. However, Robert’s scoping review of Bayesian MAs 2005-16 found that many authors of NMAs had simply copied and pasted code including the priors. Sometimes, what looks like a sensible “default prior” can be unexpectedly informative. So, we force you to think about the priors.

Sensible initial values here would be close to zero, in line with the clinical trial principle of equipoise, unless you have strong information otherwise, or are meta-analysing other study designs. For example, for two chains, -0.5 and 0.5.

bayesmh logor, likelihood(normal(se2)) ///
               prior({logor:_cons}, ///
                     ADD_YOUR_PRIOR_HERE) ///
               nchains(2) ///
               init1({logor:_cons} -0.5) ///
               init2({logor:_cons} 0.5)

Exact likelihoods and small sample sizes

When studies have small sample sizes (n), we can easily switch from the asymptotic normal likelihood to an “exact” alternative. Remember that “exact” has a specific meaning in statistical theory and does not imply that the normal is unreliable.

Means have a Student’s t sampling distribution with small numbers, and we can use that for the likelihood. Counts of how many participants had an event will come from a binomial distribution, and if the study counts events at a location or over a period of time, like road traffic accidents, it might come from a Poisson distribution. If you are uncertain about which sampling distribution is right for your evidence base, ask a statistician (they don’t have to be familiar with Bayesian methods to know this).

Let’s start with the mean differences (md) and their standard errors (se_md) first. The t-distribution will have n[j]-2 degrees of freedom, so we can supply that as transformed data. We need to send md, se_md (or precision), and n (or df). As before, there is one line per study in the data.

gen se2 = se_md^2
gen df = n-2

bayesmh md, likelihood(t(se2, df)) ///
            prior({md:_cons}, ///
                  ADD_YOUR_PRIOR_HERE) ///
               nchains(2) ///
               init1({md:_cons} -0.5) ///
               init2({md:_cons} 0.5)

For log odds ratios, Alan Agresti suggested that the asymptotics behave well in small samples. Our experimentation with simulated data suggests that, although the empirical sampling distribution can become discrete and lumpy for rare events and very small sample sizes, it still has a symmetric normal-like shape. Using the normal to infer the true population log odds ratio is justifiable. It is also possible to deal with each arm’s proportion of events using an arm-based model, which is addressed in another post.

Other statistics might need to be assessed by simulating pseudo-studies and looking at the distribution of their stats. The same applies to well-known statistics on unusually-distributed outcome variables. This will be the subject of a future post.

Uncertainty in standard errors

Because the t-distribution is the exact likelihood for a mean’s sampling distribution when the standard deviation is not known (and it never is outside of textbooks), we can use it on sampling distributions as likelihoods. The standard deviation of a sampling distribution is called a standard error. Just remember to set df to n-2, as above.

Other “fully Bayesian” models have been suggested, going back to 2000, where sampling distributions are used for both the mean given the standard deviation, and the standard deviation alone. These can be useful for imputing missing (unreported) study statistics, but in most circumstances, they will not add more information compared to the simpler exact likelihood approach. We present one in the post of arm-based models.

tags: #stata, #stata-repo, #repo