Minimum MSE Weighted Estimator to Make Inferences for a Common Risk Ratio across Sparse Meta-Analysis Data ()
1. Introduction
Frequently, biostatisticians would like to evaluate the effects of treatments or risk factors in terms of risk difference, relative risk (risk ratio), and/or odds ratio between two independent sample groups (e.g., treatment or control, presence or absence of a risk factor) and binary outcomes (e.g., disease or non-disease, success or failure, dead or alive) in a 2 × 2 table. Let
be the probability of outcome in the treatment or exposed group and
be the probability of outcome in the control or unexposed group. On comparing two independent groups, the popular effect parameters are defined by the risk difference
, the relative risk
, or the odds ratio
. Obviously, all three effect sizes are related to the estimation of proportion p on each arm. It has widely been known that the conventional proportion estimator
is a good choice for estimating p in general, but may not be in sparse data. For example, in sparse data coping with the small number of events X and the small sample size n, the variance of
, estimated by
, can cause a problem with a value of 0 when
or
. To solve the problem, a continuity correction term c is often added to each cell of each group in the 2 × 2 table, yielding to
in each arm. Yate [1] first used the continuity correction of 0.5 in the approximation of a discrete distribution to a continuous one in 1934. And it seems that the correction value of 0.5 has been used extensively until now, for examples: Lane [2], Stijnen et al. [3], White et al. [4], Lui and Lin [5], Sankey et al. [6], Gart and Zwefel [7], Walter [8], and Cox [9] used value 0.5 adjustment for zero observations in each cell of the 2 × 2 table. Another choice of c for this class such as 0.25, 0.5 and 1 had been suggested by Li and Wang [10]; 1/6 by Turkey [11] and Sánchez-Meca and Marin-Martinez [12]; Böhning and Viwatwongkasem [13] showed that the simple adjusted estimate
with
performed surprisingly well with the smallest average MSE. In addition, Agresti and Caffo [14] also suggested a value of 1 to solve the zero observations; however, a correction value of 2 was recommended by MaClave and Sincich [15].
Our focus of interest is not on the simple adjusted estimate
, but on the logarithm of
instead, leading to the final interest to the logarithm of the risk ratio estimate,
. The reason is that we are interested in the logarithm function, because it is widely known that the risk ratio has a non-symmetric and rather right-skewed distribution about the null value of 1; consequently, the natural logarithm of the risk ratio is needed to transform to be more reliable for the normal distribution. For estimating
, Pettigrew, Gart and Thomas [16] proposed the moment estimator
with the bias and the first four cumulants by means of asymptotic Taylor’s series expansion. Likewise, we re-derive and expand on the details of the solution properties for the adjusted estimator of the log-risk ratio,
.
Therefore, the first objective of the study is to find the optimal points
based upon continuity correction of this adjusted log-risk ratio estimator under various settings, such as the smallest bias, the smallest average bias, the smallest MSE, and the smallest average MSE.
Secondly, after obtaining the optimal value of
which is the best choice when considering the smallest bias and/or the smallest MSE of
. The next focus of interest is concerned with sparse data in meta-analysis studies that combine the various risk ratios from k studies to produce a single summary risk ratio. Under a common risk ratio overall studies or homogeneity of risk ratios across k studies, the second aim of the study is to find the optimal weights
of the pooled estimate
that minimize the MSE of
subject to the constraint on
where
,
,
. Indeed, these optimal weights
actually come from the roughly similar formula to the work of Viwatwongkasem et al. [19] that was used for the risk difference in multi-center studies.
Finally, the final objective of the study of interest is about comparing the performance of well-known summary effect estimators, such as the Mantel-Haenszel (MH) estimator stated with its variance estimate by Greenland and Robin [20] and the weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator to the new adjusted summary effect estimator
with various adjustment points
in terms of point estimation and hypothesis testing via simulation studies.
The rest of the paper is organized as follows. Section 2 contains the derivative methods and the results of the adjusted log-risk ratio estimator. Section 3 discusses on the methodology and the outcomes of the minimum MSE weights of the adjusted summary relative risk estimator. Section 4 states on the other well-known estimators and tests as the comparative candidates to compare the performances of making inference for a common relative risk across meta-analysis studies. Section 5 consists of the simulation plans for studying the performances in senses of the estimation, the type I errors and the power of the tests. Section 6 contains the results of Section 5. Section 7 is about the discussion and the recommendation.
2. Adjusted Estimator of the Logarithm of the Risk Ratio
In a single study (
), since the conventionally popular estimator of the logarithm of risk ratio, obtained by
, may have a problem when data are sparse. To solve the problem, a continuity correction term
is often added to each cell of the 2 × 2 table, leading to the adjusted estimate as
. In addition, the adjusted risk ratio estimator can be further found as
. Due to the work of Pettigrew, Gart and Thomas [16], we re-derive and extend the results to get the expectation, bias, variance, and MSE of
in the following:
(1)
(2)
(3)
(4)
when
and
.
The first setting to find the optimal point
that minimizes the smallest bias of
is investigated. The first derivatives of the bias of
with respect to
and
are given by
(5)
(6)
Setting
and
, the critical roots
are obtained as follows:
,
(7)
,
(8)
The sufficient conditions of the second-order derivatives of the bias of
to guarantee the solutions of (7) and (8) being minimum points are that the determinants
and
, evaluated at critical points, are all positive where
and
. Unfortunately, it is impossible to find the minimum point
such that
has the smallest bias for all
. In addition, the solution to (7) and (8) is practically inflexible because it cannot be applied when
and
equal 0.5. Therefore, we further consider the other settings, such as the smallest average bias and the smallest MSE, to find the optimal points
. However, these settings still fail, they cannot provide the optimal points
.
Another successful setting for finding the optimal values
is coming from an average MSE or equivalently known as Bayes risk. Suppose that the squared error loss function is given by
. The risk as the expectation of loss or the MSE of
in such this case, is given by
. The prior distribution is usually constructed based on the previously scientific knowledge or the prior observed data; especially, in this paper we set the prior distribution based on Table 1. For given the prior uniform density
over [0, 1] × [0, 1], the Bayes risk of
(or the average MSE of
) with respect to the Euclidean loss function is obtained as
Table 1. High sparse data of a multi-center clinical trial in CALGB study (Cancer and Leukemia Group B) from Lipsitz et al. [17] and Cooper et al. [18].
*Either
or
is 0, leading to the infinity value problem of the natural logarithm.
(9)
The first and second order of partial derivatives of
with respect to
and
are as follows:
,
,
,
,
. Unfortunately, the result of Bayes risk shows that the critical point
is not a minimum point. With the conditions of
and
, the critical point
is a saddle point since
. However, in particular case, we let
and try again to find the minimum point. Fortunately, with the condition of
, the optimal point c with the smallest average MSE is obtained by
; equivalently, with the condition of
, the minimum point is
. The solution of
or
looks well and can be considered as an appropriate one in practice.
3. Minimum MSE Weights of the Adjusted Summary Relative Risk Estimator
Under a common risk ratio overall k studies or homogeneity of risk ratios across k studies, the optimal weights
are investigated to minimize the MSE of
of the form
, subject to the constraint on
, where
,
and
. The MSE of
, used under a true common risk ratio
across all k studies, is given by
To find the optimal weights
with the constraint on
, we form the auxiliary function
in extending
under the Lagrange’s method where
is a Lagrange multiplier as follows:
From the previous section, we have the following results:
and
The partial derivatives with respect to
and
yield
Setting
,
and solving the equations, it yields
where
,
,
.
More details of derivation can be found from the work of Viwatwongkasem et al. [19] that used the minimum MSE weights for the risk difference in multi-center studies. After replacing the unknown parameter quantities with their sample estimates, the minimum MSE weighted estimate is obtained in the following:
(10)
where
,
,
,
,
where
,
,
,
Assuming that a normal approximation is reliable, the minimum MSE weights Z-test for testing
, (
) is
We will reject
at
level for two-sided test if
where
is the upper
percentile of the standard normal distribution. Alternatively, reject
when the p-value (p) is less than or equal to
where
and
is the cumulative standard normal distribution.
4. Other Well-Known Estimators and Tests for Making Inference for a Common Relative Risk
Under a common risk ratio or homogeneity of risk ratios across k studies, we wish to compare the performance of the minimum MSE weighted estimator adjusted by various points
,
,
with the other well-known summary risk ratio estimators, such as the Mantel-Haenszel (MH) estimator and the weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator via a simulation study. According to these well-known estimators, we will present briefly both estimators.
Mantel-Haenszel Weights (MH)
For Mantel-Haenszel (MH) relative risk estimator overall centers/studies from binomial data, the estimator has been proposed by
(11)
where
.
The variance estimator of the log-relative risk of Mantel-Haenszel was proposed by Green and Robin [20] based on unconditional binomial distribution is given by
(12)
where:
,
,
,
and
. Note that under a binomial sparse-data model, the Mantel-Haenszel relative risk is consistent in sparse stratification.
Assuming that a normal approximation is valid, the Mantel-Haenszel’s Z-test for testing
, (
) is
We will reject
at
level for two-sided test if
where
is the upper
percentile of the standard normal distribution. Alternatively, reject
when the p-value (p) is less than or equal to
where
and
is the cumulative standard normal distribution.
Weighted least square (WLS) estimator
The weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator of the log-relative risk overall centers/studies is
(13)
where
(14)
Practically, the weights
are often replaced by their estimates.
The variance of the summary estimator
is given by
(15)
Assuming that a normal approximation is valid, the weighted least square Z-test for testing
, (
) is
We will reject
at
level for two-sided test if
where
is the upper
percentile of the standard normal distribution. Alternatively, reject
when the p-value (p) is less than or equal to
where
and
is the cumulative standard normal distribution.
5. Simulation Plan for the Estimation, Studying the Type I Error and the Power of the Test
We present here a simulation study using the following designs:
Parameters: Let the common relative risk be some constants (RR = 1, 2 and 4) and generate the baseline proportion
in the control arm for the jth study from a uniform distribution in which the range corresponds to the values of RR. If
, then
; if
, then
; and if
, then
. The correspondent proportion risks in the exposure arm is
. The sample size
and
in each study are fixed and varied as 4, 8, 16, 32 and 100. The number of studies is 1, 2, 4, 8, 16, 32 and 100.
Statistic 1 (estimation of RR): Generate
and
from the binomial distribution with parameter
and
for each study j (
). All summary estimates are calculated. The procedure is replicated 5000 times. From these replicates, we compute bias, variance, and mean square error (MSE) for the adjusted relative risk estimator with the proposed weight to compare the performance with Mantel-Haenszel (MH) estimator and the weighted least square estimator.
Statistic 2 (studying the type I error under
): Generate
and
from the binomial distribution with parameter
and
and replicate these generations 5000 times for every procedure. From these replicates, the number of null hypothesis rejections when
is true under three Z-tests is counted for the actual type I error.
Statistic 3 (studying the power of the test under
): Before comparing the power of tests, all test statistics should be calibrated to handle the same type I error rate under the null hypothesis. Under the alternative hypothesis with the random effects model, the powers of three candidate tests are compared. We need to revise the parameter setting for studying the power of the test. Let
be a uniform distribution over [0, 0.25] and we assume that
follows as
where
is a uniform
random variable for a given
, or equivalently, U is a uniform over (0, 1). Note that these parameter settings provide:
and
. Consequently, we still have
. Binomial variates
and
are also generated with parameter
and
, respectively. All proposed test statistics under this alternative hypothesis are computed and replicated 5000 times. From these replicates, the number of null hypothesis rejection is counted for the power of the test.
6. Results from Simulation Studies
6.1. Comparative Performance for Point Estimation
Under a constant of relative risk (RR = 1, 2 and 4), the performance in terms of bias, variance, and mean square error (MSE) of several summary relative risk estimators are compared. Results show that increasing k can decrease the variance and the MSE of all estimators and the increase of both
can also decrease the variance of all estimators while fixing k. The unbalance cases of
(
and
) have less affected on the order performance of MSE estimators. The summary adjusted relative risk estimator in meta-analysis study of size k has shrinkage estimator to be a simple adjusted relative risk estimator in one single study case. The optimal point (
) providing the bias, variance, and MSE of
, adjusted by
is identical to
. By these, the numerical evidence has confirmed the derivation process of finding the root c and it is very useful in practice.
For a single center study (
), regardless of a true value of RR, the proposed estimator adjusted by
performs the best with smallest MSE.
For a multi-center study of size k, when
, the WLS estimator is the best in sense of the smallest MSE ignoring the sample size
. In case
,
the proposed estimator adjusted by c performs the best with the smallest MSE. Another issue, when
, the MH estimator achieves the smallest MSE when sample sizes are small. The MSE of the WLS estimator and the proposed estimator adjusted by c are close together when the sample sizes are moderate and large (
). For the case
and
or 16, the MH estimator well performs generally with the smallest MSE. Some comparisons of the performances of all summary estimators when
and
are depicted in Figures 1(a)-(f) for bias; Figures 2(a)-(f) for variance; and Figures 3(a)-(f) for MSE.
In summary for the performance of estimators, regardless of the true values of RR, the MH estimator achieves the best performance with the smallest MSE when the study size is rather large (
) and the sample sizes within each study are small. The MSE of WLS estimator and the proposed-weight estimator adjusted by
,
,
are close together and they are the best when the sample sizes are moderate to large (
) while the study size is rather small.
6.2. Studying the Type I Error
A type I error comparison between the tests is considered by comparing the actual (empirical) type I error (
) with the nominal level of significance. In this study, the evaluation of the ability to control type I error probability for two-sided tests is based on Cochran limits as follows.
At
significant level, the actual
value is between [0.005, 0.015].
At
significant level, the actual
value is between [0.04, 0.06].
At
significant level, the actual
value is between [0.08, 0.12].
If the actual type I error or the empirical alpha lies within those Cochran limits, then the statistical test can control type I error rate.
For a single study (
), regardless of the true values of RR, it is unfortunate that almost all tests shown in Table 2 cannot control type I error rate. There are some type I error rates lying in the Cochran limits.
For a meta-analysis study of size k, also displayed in Table 2, regardless of the true values of RR, the Mantel-Haenszel’s Z-Test can control type I error rate when the sample size either in treatment or control group is moderate to large (
or
). In addition, the weighted least square Z-test and the proposed weight Z-test adjusted by
,
,
can handle type I error rate when both
are large. But the proposed Z-test adjusted by
cannot control type I error rate into Cochran’s limit at almost all situations.
6.3. Studying the Power of the Test
Usually, the empirical power of the tests will be compared under the same type I error value. In summary, the Mantel-Haenszel Z-test performs best when
is moderate to large with satisfying the type I error value within Cochran’s range limit, regardless of the study size k. The inverse variance weighted Z-test is good when
is large. For the proposed Z-test adjusted by
,
, the Z-tests perform well under the same type I error value when k is small (
) and
is large (
). The results in Table 3 illustrate the comparison of the power of tests under the same rate of the type I error.
Figure 1. Bias comparison of relative risk between well-known estimators and adjusted relative risk estimators at k = 1, k = 4 and k = 16 for n1 = 4 and n2 = 4, 8, 16 and 32 ((a)-(c)) and at k = 1, k = 4 and k = 16 for n1 = 8 and n2 = 8, 16 and 32 ((d)-(f)).
Figure 2. Variance comparison of relative risk between well-known estimators and adjusted relative risk estimators at k = 1, k = 4 and k = 16 for n1 = 4 and n2 = 4, 8, 16 and 32 ((a)-(c)) and at k = 1, k = 4 and k = 16 for n1 = 8 and n2 = 8, 16 and 32 ((d)-(f)).
Figure 3. MSE comparison of relative risk between well-known estimators and adjusted relative risk estimators at k = 1, k = 4 and k = 16 for n1 = 4 and n2 = 4, 8, 16 and 32 ((a)-(c)) and at k = 1, k = 4 and k = 16 for n1 = 8 and n2 = 8, 16 and 32 ((d)-(f)).
Table 2. Comparison of the empirical type I error for testing
in a center study (
) and multi-center study (
, 16 and 32) at 5% significant level.
Bold Values denote that the statistical tests can control the type I error.
Table 3. Comparison of the empirical power of test (percent).
#denotes that the statistical tests cannot control type I error rate under
.
7. Discussion and Recommendation
The main question rises which continuity correction values are the best choice for the adjusted relative risk estimator in a center study and multi-center study with sparse data. Due to the conventional continuity correction, most of investigators such as Yate [1], Lane [3], Stijnen et al., [4], Lui and Lin [6], Sankey et al., [7], Gart and Zwefel [8], Walter [9] and Cox [10], suggest to use
. However, in this study with the smallest average MSE of
, the optimal point
can perform the best for the point estimation. The minimum point
of
agrees with the suggestion of Turkey [12], which is very useful and the most appropriate in a practice way.
For estimation of fixing
(
) in a center (
), regardless of a true value of RR, the proposed estimator adjusted by
performs the best with the smallest MSE. For a meta-analysis study of size k, in general the MH estimator achieves the smallest MSE when the sample size
is small while the study size is rather large (
). The MSE of the WLS estimator and the proposed estimator adjusted by the various values of c are closed together and they are the best when the sample sizes are moderate to large (
and
) while the study size is rather small. This finding is consonant with the work of Viwatwongkasem et al. [19]. Since the true value of RRis usually not available in practice as mentioned earlier, we suggest to choose the proposed relative risk estimator adjusted by
that can minimize the Bayes risk with respect to uniform prior (0, 1) and Euclidean loss function.
For the empirical power of the test under
, regardless of the study size k, the MH Z-test performs the best with the highest power when both
are moderate to large. The inverse-variance weighted Z-test is good when
is large. In accordance with Soulakova and Bright [21], the empirical power of the test will increase when the sample sizes increase. For the power of the proposed Z-test adjusted by
,
, the Z-test has the higher performance under the same type I error when k is small (
) and
is large (
).
If we don’t know information about parameter RR, we recommend to use the adjusted estimator
by using continuity correction defined by
, or 1/3, or 1/2 in a center study. For a multi-center study of size k, we recommend to use adjusted
defined by
, or 1/3, or 1/2, including optimal weights
.
Generally, the effect of exposures or the effect of treatments with binary outcomes covers the risk difference, the relative risk, and the odds ratio. Obviously, all three conventional effect estimators have the same problem of the zero values in sparse data. The conventional proportion estimate of
is in need of replacement by
to solve this problem. Therefore, the recommendation for a further study is to use these ideas, such as the smallest MSE, the smallest Bayes risk to fine the appropriate point
in estimating the odds ratio parameter.
Acknowledgements
The study was partially supported for publication by the Faculty of Public Health, Mahidol University, Bangkok, Thailand.