This page describes the computational methods used in Quantitative Synthesis (QNS).
Data Analyzed #
Any study included (adjudicated, in dual screening) during Screening and with Meta-Analytical extraction marked “Complete” is analyzed on QNS. If dual extraction is configured for the nest, only adjudicated data is analyzed in QNS.
Analyses #
Summary (Pooled Estimates) #
Any study with one or more arms extracted are analyzed in the summary table. The rows of the summary table correspond to individual study arms, or an intervention. Columns are estimates (singular for a study arms or pooled for interventions) of the data element being analyzed.
For continuous mean & standard deviation (SD) and dichotomous data elements, pooled estimates are computed via the Inverse Variance method. When random effects are specified, between-study variance is estimated using the Dersimonian-Laird estimator. For dichotomous data, the Haldane-Anscambe correction is conditionally applied for 0 event counts. For continuous mean & SD, missing SDs are imputed. All inferential statistics, at the arm and intervention level, are computed using Normal approximations (logit transformed, for dichotomous data). For data elements with means but not an SD configured, the arithmetic mean is computed, and no inferential statistics are computed.
For medians, pooled estimates & inferentials are computed as the Weighted Median of Medians, described in McGrath et al.
Sources of Computational Methods
All computational methods are implemented as a transcription, with minor modifications, of the peer-reviewed work in the R packages meta, metafor, and metamedian. As these are open source, our modifications are open source, made publicly available for audit & modification in the Javascript package shukra. Pooled estimates specifically are computed by the pooling module. To ensure correctness, all methods in shukra, and therefore used in QNS, are tested for equality to estimates computed in the upstream packages.
Network Meta-Analysis (NMA) #
Any multi-arm study with two or more arms extracted are analyzed in NMA. If the user-configured intervention grouping assigns the same intervention to two or more arms in the study, they are pooled; if it assigns the same intervention to all arms in the study, the study is discarded from the NMA. Only continuous data elements with mean & SD configured, or dichotomous data elements are eligible for NMA; arms where SD is not extracted are discarded from the analysis.
Before performing the NMA, connected components of the intervention graph are identified. Each connected component is analyzed in a separate NMA, and the results of the NMAs are combined in outputs and visuals. No effect estimates will be provided for interventions that belong to different connected components.
Direct effects are computed as Mean Differences (continuous data elements) or Odds Ratios (dichotomous data elements) between all pairs of within-study arms. Consistent indirect effects are computed using the Aitken estimator (described in Senn et. al), with an pairwise-effect weighting correction for multi (3+) arm studies (described in Rücker et. al, 2014). For full details on computation of the Aitken estimator see Rücker et. al, 2012. When random effects are specified, between-study variance is estimated using the DerSimonian-Laird estimator. All inferential statistics, at the study (used in forest plots) and intervention level, are computed using Normal approximations (logit transformed, for dichotomous data) on the consistent treatment effects. Intervention rankings are computed with P-scores, described in Rücker and Schwarzer, 2015.
Note: when encountering a 0 value in dichotomous data (for example, an event rate of 0 in one of the study arms), a continuity correction of 0.5 is applied to all cells containing a 0 to allow for the calculation of effect sizes and to incorporate the study into the meta-analysis without skewing the results. This adjustment helps in estimating the effect size and its variance, facilitating the inclusion of studies with no events in one or both arms in the analysis.
Funnel plots are generated using all contrasts available on the selected intervention. “Comparison-adjusted” effects (Chaimani et. al) center all direct effects at a common value using their summary effect estimates, making trending and symmetry analysis feasible on the plot.
Sources of Computational Methods
All computational methods are implemented as a transcription, with modification, of the peer-reviewed work in the R packages netmeta and metafor. As these are open source, our modifications are open source, made publicly available for audit & modification in the Javascript package shukra. NMAs specifically are computed by the nma module. To ensure correctness, all methods in shukra, and therefore used in QNS, are tested for equality to estimates computed in the upstream packages.
