Minimum MSE Weighted Estimator to Make Inferences for a Common Risk Ratio across Sparse Meta-Analysis Data

Abstract

The paper aims to discuss three interesting issues of statistical inferences for a common risk ratio (RR) in sparse meta-analysis data. Firstly, the conventional log-risk ratio estimator encounters a number of problems when the number of events in the experimental or control group is zero in sparse data of a 2 × 2 table. The adjusted log-risk ratio estimator with the continuity correction points  based upon the minimum Bayes risk with respect to the uniform prior density over (0, 1) and the Euclidean loss function is proposed. Secondly, the interest is to find the optimal weights of the pooled estimate  that minimize the mean square error (MSE) of  subject to the constraint on  where , , . Finally, the performance of this minimum MSE weighted estimator adjusted with various values of points  is investigated to compare with other popular estimators, such as the Mantel-Haenszel (MH) estimator and the weighted least squares (WLS) estimator (also equivalently known as the inverse-variance weighted estimator) in senses of point estimation and hypothesis testing via simulation studies. The results of estimation illustrate that regardless of the true values of RR, the MH estimator achieves the best performance with the smallest MSE when the study size is rather large  and the sample sizes within each study are small. The MSE of WLS estimator and the proposed-weight estimator adjusted by , or , or are close together and they are the best when the sample sizes are moderate to large (and) while the study size is rather small.

Share and Cite:

Viwatwongkasem, C. , Srisawad, S. , Soontornpipit, P. , Sillabutra, J. , Satitvipawee, P. , Kitidamrongsuk, P. and Chootrakool, H. (2022) Minimum MSE Weighted Estimator to Make Inferences for a Common Risk Ratio across Sparse Meta-Analysis Data. Open Journal of Statistics, 12, 49-69. doi: 10.4236/ojs.2022.121004.

1. Introduction

Frequently, biostatisticians would like to evaluate the effects of treatments or risk factors in terms of risk difference, relative risk (risk ratio), and/or odds ratio between two independent sample groups (e.g., treatment or control, presence or absence of a risk factor) and binary outcomes (e.g., disease or non-disease, success or failure, dead or alive) in a 2 × 2 table. Let p 1 be the probability of outcome in the treatment or exposed group and p 2 be the probability of outcome in the control or unexposed group. On comparing two independent groups, the popular effect parameters are defined by the risk difference R D = p 1 p 2 , the relative risk R R = p 1 / p 2 , or the odds ratio O R = p 1 / ( 1 p 1 ) p 2 / ( 1 p 2 ) . Obviously, all three effect sizes are related to the estimation of proportion p on each arm. It has widely been known that the conventional proportion estimator p ^ = X / n is a good choice for estimating p in general, but may not be in sparse data. For example, in sparse data coping with the small number of events X and the small sample size n, the variance of p ^ , estimated by p ^ ( 1 p ^ ) n , can cause a problem with a value of 0 when X = 0 or X = n . To solve the problem, a continuity correction term c is often added to each cell of each group in the 2 × 2 table, yielding to p ^ c = ( X + c ) / ( n + 2 c ) in each arm. Yate [1] first used the continuity correction of 0.5 in the approximation of a discrete distribution to a continuous one in 1934. And it seems that the correction value of 0.5 has been used extensively until now, for examples: Lane [2], Stijnen et al. [3], White et al. [4], Lui and Lin [5], Sankey et al. [6], Gart and Zwefel [7], Walter [8], and Cox [9] used value 0.5 adjustment for zero observations in each cell of the 2 × 2 table. Another choice of c for this class such as 0.25, 0.5 and 1 had been suggested by Li and Wang [10]; 1/6 by Turkey [11] and Sánchez-Meca and Marin-Martinez [12]; Böhning and Viwatwongkasem [13] showed that the simple adjusted estimate p ^ c = ( X + c ) / ( n + 2 c ) with c = 1 performed surprisingly well with the smallest average MSE. In addition, Agresti and Caffo [14] also suggested a value of 1 to solve the zero observations; however, a correction value of 2 was recommended by MaClave and Sincich [15].

Our focus of interest is not on the simple adjusted estimate p ^ c = ( X + c ) / ( n + 2 c ) , but on the logarithm of p ^ c instead, leading to the final interest to the logarithm of the risk ratio estimate, θ ^ c = log R R ^ c = log p ^ c 1 log p ^ c 2 = log ( X 1 + c 1 n 1 + 2 c 1 ) log ( X 2 + c 2 n 2 + 2 c 2 ) . The reason is that we are interested in the logarithm function, because it is widely known that the risk ratio has a non-symmetric and rather right-skewed distribution about the null value of 1; consequently, the natural logarithm of the risk ratio is needed to transform to be more reliable for the normal distribution. For estimating log ( p + c / n 1 + 2 c / n ) , Pettigrew, Gart and Thomas [16] proposed the moment estimator log ( X + c n + 2 c ) with the bias and the first four cumulants by means of asymptotic Taylor’s series expansion. Likewise, we re-derive and expand on the details of the solution properties for the adjusted estimator of the log-risk ratio, θ ^ c = log R R ^ c = log p ^ c 1 log p ^ c 2 = log ( X 1 + c 1 n 1 + 2 c 1 ) log ( X 2 + c 2 n 2 + 2 c 2 ) .

Therefore, the first objective of the study is to find the optimal points ( c 1 , c 2 ) based upon continuity correction of this adjusted log-risk ratio estimator under various settings, such as the smallest bias, the smallest average bias, the smallest MSE, and the smallest average MSE.

Secondly, after obtaining the optimal value of ( c 1 , c 2 ) which is the best choice when considering the smallest bias and/or the smallest MSE of θ ^ c . The next focus of interest is concerned with sparse data in meta-analysis studies that combine the various risk ratios from k studies to produce a single summary risk ratio. Under a common risk ratio overall studies or homogeneity of risk ratios across k studies, the second aim of the study is to find the optimal weights f ^ j of the pooled estimate θ ^ w = j = 1 k θ ^ c j f ^ j that minimize the MSE of θ ^ w subject to the constraint on j = 1 k f ^ j = 1 where θ ^ c j = log ( R R ^ c j ) = log ( p ^ c 1 j ) log ( p ^ c 2 j ) , p ^ c 1 j = ( X 1 j + c 1 ) / ( n 1 j + 2 c 1 ) , p ^ c 2 j = ( X 2 j + c 2 ) / ( n 2 j + 2 c 2 ) . Indeed, these optimal weights f ^ j actually come from the roughly similar formula to the work of Viwatwongkasem et al. [19] that was used for the risk difference in multi-center studies.

Finally, the final objective of the study of interest is about comparing the performance of well-known summary effect estimators, such as the Mantel-Haenszel (MH) estimator stated with its variance estimate by Greenland and Robin [20] and the weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator to the new adjusted summary effect estimator θ ^ w with various adjustment points ( c 1 , c 2 ) in terms of point estimation and hypothesis testing via simulation studies.

The rest of the paper is organized as follows. Section 2 contains the derivative methods and the results of the adjusted log-risk ratio estimator. Section 3 discusses on the methodology and the outcomes of the minimum MSE weights of the adjusted summary relative risk estimator. Section 4 states on the other well-known estimators and tests as the comparative candidates to compare the performances of making inference for a common relative risk across meta-analysis studies. Section 5 consists of the simulation plans for studying the performances in senses of the estimation, the type I errors and the power of the tests. Section 6 contains the results of Section 5. Section 7 is about the discussion and the recommendation.

2. Adjusted Estimator of the Logarithm of the Risk Ratio

In a single study ( k = 1 ), since the conventionally popular estimator of the logarithm of risk ratio, obtained by θ ^ = log R R ^ = log p ^ 1 log p ^ 2 = log ( X 1 n 1 ) log ( X 2 n 2 ) , may have a problem when data are sparse. To solve the problem, a continuity correction term ( c 1 , c 2 ) is often added to each cell of the 2 × 2 table, leading to the adjusted estimate as θ ^ c = log R R ^ c = log p ^ c 1 log p ^ c 2 = log ( X 1 + c 1 n 1 + 2 c 1 ) log ( X 2 + c 2 n 2 + 2 c 2 ) . In addition, the adjusted risk ratio estimator can be further found as R R ^ c = p ^ c 1 p ^ c 2 = ( X 1 + c 1 ) / ( n 1 + 2 c 1 ) ( X 2 + c 2 ) / ( n 2 + 2 c 2 ) . Due to the work of Pettigrew, Gart and Thomas [16], we re-derive and extend the results to get the expectation, bias, variance, and MSE of θ ^ c in the following:

E ( θ ^ c ) = ( log p 1 log p 2 ) + B ( θ ^ c ) (1)

B ( θ ^ c ) = 2 c 1 1 + p 1 { 1 2 ( 2 c 1 ) } 2 n 1 p 1 + 6 c 1 2 + 6 ( 2 c 1 p 1 ) 2 + 12 c 1 q 1 + 4 q 1 ( q 1 p 1 ) 9 q 1 2 12 ( n 1 p 1 ) 2 [ 2 c 2 1 + p 2 { 1 2 ( 2 c 2 ) } 2 n 2 p 2 + 6 c 2 2 + 6 ( 2 c 2 p 2 ) 2 + 12 c 2 q 2 + 4 q 2 ( q 2 p 2 ) 9 q 2 2 12 ( n 2 p 2 ) 2 ] + O ( n 3 ) (2)

V ( θ ^ c ) = q 1 n 1 p 1 + q 1 ( 1 2 c 1 ) + 1 2 q 1 2 ( n 1 p 1 ) 2 + q 2 n 2 p 2 + q 2 ( 1 2 c 2 ) + 1 2 q 2 2 ( n 2 p 2 ) 2 + O ( n 3 ) (3)

M S E ( θ ^ c ) = q 1 n 1 p 1 + ( c 1 2 c 1 p 1 ) ( c 1 2 c 1 p 1 q 1 ) + q 1 ( 1 2 c 1 ) + 3 4 q 1 2 ( n 1 p 1 ) 2 + q 2 n 2 p 2 + ( c 2 2 c 2 p 2 ) ( c 2 2 c 2 p 2 q 2 ) + q 2 ( 1 2 c 2 ) + 3 4 q 2 2 ( n 2 p 2 ) 2 + O ( n 3 ) (4)

when q 1 = 1 p 1 and q 2 = 1 p 2 .

The first setting to find the optimal point ( c 1 , c 2 ) that minimizes the smallest bias of θ ^ c is investigated. The first derivatives of the bias of θ ^ c with respect to c 1 and c 2 are given by

B ( θ ^ c ) c 1 = 1 + ( 1 + n 1 ) p 1 2 n 1 p 1 2 + c 1 ( 1 + 4 p 1 2 ) n 1 2 p 1 2 (5)

B ( θ ^ c ) c 2 = [ 1 + ( 1 + n 2 ) p 2 2 n 2 p 2 2 + c 2 ( 1 + 4 p 2 2 ) n 2 2 p 2 2 ] (6)

Setting B ( θ ^ c ) c 1 = 0 and B ( θ ^ c ) c 2 = 0 , the critical roots ( c 1 , c 2 ) are obtained as follows:

c 1 = 1 + p 1 n 1 p 1 + 2 n 1 p 1 2 1 + 4 p 1 2 , p 1 0.5 (7)

c 2 = 1 + p 2 n 2 p 2 + 2 n 2 p 2 2 1 + 4 p 2 2 , p 2 0.5 (8)

The sufficient conditions of the second-order derivatives of the bias of θ ^ c to guarantee the solutions of (7) and (8) being minimum points are that the determinants D 1 and D 2 , evaluated at critical points, are all positive where D 1 = 2 B ( θ ^ c ) c 1 2 and D 2 = | 2 B ( θ ^ c ) c 1 2 2 B ( θ ^ c ) c 1 c 2 2 B ( θ ^ c ) c 1 c 2 2 B ( θ ^ c ) c 2 2 | . Unfortunately, it is impossible to find the minimum point ( c 1 , c 2 ) such that θ ^ c = log ( X 1 + c 1 n 1 + 2 c 1 ) log ( X 2 + c 2 n 2 + 2 c 2 ) has the smallest bias for all ( p 1 , p 2 ) . In addition, the solution to (7) and (8) is practically inflexible because it cannot be applied when p 1 and p 2 equal 0.5. Therefore, we further consider the other settings, such as the smallest average bias and the smallest MSE, to find the optimal points ( c 1 , c 2 ) . However, these settings still fail, they cannot provide the optimal points ( c 1 , c 2 ) .

Another successful setting for finding the optimal values ( c 1 , c 2 ) is coming from an average MSE or equivalently known as Bayes risk. Suppose that the squared error loss function is given by Loss = ( θ ^ c θ ) 2 . The risk as the expectation of loss or the MSE of θ ^ c in such this case, is given by Risk = M S E ( θ ^ c ) = E ( θ ^ c θ ) 2 . The prior distribution is usually constructed based on the previously scientific knowledge or the prior observed data; especially, in this paper we set the prior distribution based on Table 1. For given the prior uniform density g ( p 1 , p 2 ) = 1 [ 0 , 1 ] × 1 [ 0 , 1 ] over [0, 1] × [0, 1], the Bayes risk of θ ^ c (or the average MSE of θ ^ c ) with respect to the Euclidean loss function is obtained as

Bayesrisk = m ( c 1 , c 2 ) = 0 1 0 1 M S E ( θ ^ c ) g ( p 1 , p 2 ) d p 1 d p 2

Table 1. High sparse data of a multi-center clinical trial in CALGB study (Cancer and Leukemia Group B) from Lipsitz et al. [17] and Cooper et al. [18].

*Either X 1 j or X 2 j is 0, leading to the infinity value problem of the natural logarithm.

m ( c 1 , c 2 ) = 0 1 0 1 q 1 n 1 p 1 + ( c 1 2 c 1 p 1 ) ( c 1 2 c 1 p 1 q 1 ) + q 1 ( 1 2 c 1 ) + 3 4 q 1 2 ( n 1 p 1 ) 2 + q 2 n 2 p 2 + ( c 2 2 c 2 p 2 ) ( c 2 2 c 2 p 2 q 2 ) + q 2 ( 1 2 c 2 ) + 3 4 q 2 2 ( n 2 p 2 ) 2 d p 1 d p 2

m ( c 1 , c 2 ) = ( 1 + c 1 + 3 c 1 2 ) n 2 2 n 1 n 2 2 + n 1 2 ( 3.25 c 2 3 c 2 2 + n 2 ) n 1 2 n 2 2 (9)

The first and second order of partial derivatives of m ( c 1 , c 2 ) with respect to c 1 and c 2 are as follows: m ( c 1 , c 2 ) c 1 = 1 + 6 c 1 n 1 2 , m ( c 1 , c 2 ) c 2 = 1 6 c 2 n 2 2 , 2 m ( c 1 , c 2 ) c 1 2 = 6 n 1 2 , 2 m ( c 1 , c 2 ) c 2 2 = 6 n 2 2 , 2 m ( c 1 , c 2 ) c 1 c 2 = 0 . Unfortunately, the result of Bayes risk shows that the critical point ( c 1 , c 2 ) = ( 1 6 , 1 6 ) is not a minimum point. With the conditions of D 1 = 2 m ( c 1 , c 2 ) c 1 2 and D 2 = | 2 m ( c 1 , c 2 ) c 1 2 2 m ( c 1 , c 2 ) c 1 c 2 2 m ( c 1 , c 2 ) c 1 c 2 2 m ( c 1 , c 2 ) c 2 2 | , the critical point ( c 1 , c 2 ) is a saddle point since D 2 = 36 n 1 2 n 2 2 < 0 . However, in particular case, we let c = c 1 = c 2 and try again to find the minimum point. Fortunately, with the condition of n 2 > n 1 , the optimal point c with the smallest average MSE is obtained by c = 1 6 ; equivalently, with the condition of n 1 > n 2 , the minimum point is c = 1 6 . The solution of c = 1 6 or c = 1 6 looks well and can be considered as an appropriate one in practice.

3. Minimum MSE Weights of the Adjusted Summary Relative Risk Estimator

Under a common risk ratio overall k studies or homogeneity of risk ratios across k studies, the optimal weights f j are investigated to minimize the MSE of θ ^ w of the form θ ^ w = j = 1 k θ ^ c j f j , subject to the constraint on j = 1 k f j = 1 , where θ ^ c j = log ( R R ^ c j ) = log ( p ^ c 1 j ) log ( p ^ c 2 j ) , p ^ c 1 j = ( X 1 j + c 1 ) / ( n 1 j + 2 c 1 ) and p ^ c 2 j = ( X 2 j + c 2 ) / ( n 2 j + 2 c 2 ) . The MSE of θ ^ w = j = 1 k θ ^ c j f j , used under a true common risk ratio θ across all k studies, is given by

M S E ( θ ^ w ) = E ( θ ^ w θ ) 2 = E ( j = 1 k θ ^ c j f j θ ) 2

To find the optimal weights f j with the constraint on j = 1 k f j = 1 , we form the auxiliary function ϕ in extending MSE ( θ ^ w ) under the Lagrange’s method where λ is a Lagrange multiplier as follows:

ϕ = E ( j = 1 k θ ^ c j f j θ ) 2 + λ ( j = 1 k f j 1 )

ϕ = E ( j = 1 k θ ^ c j f j ) 2 2 θ E ( j = 1 k θ ^ c j f j ) + θ 2 + λ ( j = 1 k f j 1 )

ϕ = V a r ( j = 1 k θ ^ c j f j ) + ( E ( j = 1 k θ ^ c j f j ) ) 2 2 θ E ( j = 1 k θ ^ c j f j ) + θ 2 + λ ( j = 1 k f j 1 )

ϕ = j = 1 k f j 2 V a r ( θ ^ c j ) + ( j = 1 k f j E ( θ ^ c j ) ) 2 2 θ ( j = 1 k f j E ( θ ^ c j ) ) + θ 2 + λ ( j = 1 k f j 1 )

ϕ = j = 1 k f j 2 V a r ( θ ^ c j ) + ( j = 1 k f j E ( θ ^ c j ) θ ) 2 + λ ( j = 1 k f j 1 )

From the previous section, we have the following results:

V j = V a r ( θ ^ c j ) = q 1 j n 1 j p 1 j + q 1 j ( 1 2 c 1 ) + 1 2 q 1 j 2 ( n 1 j p 1 j ) 2 + q 2 j n 2 j p 2 j + q 2 j ( 1 2 c 2 ) + 1 2 q 2 j 2 ( n 2 j p 2 j ) 2 + O ( n 3 )

and

E j = E ( θ ^ c j ) = ( log p 1 j log p 2 j ) + ( 2 c 1 1 + p 1 j { 1 2 ( 2 c 1 ) } 2 n 1 j p 1 j 2 c 2 1 + p 2 j { 1 2 ( 2 c 2 ) } 2 n 2 j p 2 j ) + O ( n 2 )

The partial derivatives with respect to λ and f j yield

ϕ λ = j = 1 k f j 1

ϕ f j = 2 f j V j + 2 ( j = 1 k f j E j θ ) E j + λ

Setting ϕ λ = 0 , ϕ f j = 0 and solving the equations, it yields

λ = 2 a 2 b ( j = 1 k f j E j θ ) a

f j = ( V j 1 ( 1 + τ j θ ) a ) ( V j 1 τ j a + j = 1 k τ j E j V j 1 ) ( j = 1 k V j 1 E j ( 1 + τ j θ ) a )

where a = j = 1 k 1 V j = j = 1 k V j 1 , b = j = 1 k E j V j , τ j = a E j b .

More details of derivation can be found from the work of Viwatwongkasem et al. [19] that used the minimum MSE weights for the risk difference in multi-center studies. After replacing the unknown parameter quantities with their sample estimates, the minimum MSE weighted estimate is obtained in the following:

θ ^ w = j = 1 k θ ^ c j f ^ j (10)

where

θ ^ c j = log ( R R ^ c j ) = log ( p ^ c 1 j ) log ( p ^ c 2 j ) , p ^ c 1 j = ( X 1 j + c 1 ) / ( n 1 j + 2 c 1 ) ,

p ^ c 2 j = ( X 2 j + c 2 ) / ( n 2 j + 2 c 2 ) , q ^ c 1 j = 1 p ^ c 1 j , q ^ c 2 j = 1 p ^ c 2 j

f ^ j = ( V ^ j 1 ( 1 τ ^ j θ ^ p o o l ) a ^ ) ( V ^ j 1 τ ^ j a ^ + j = 1 k τ ^ j E ^ j V ^ j 1 ) + ( j = 1 k E ^ j V ^ j 1 ( 1 + τ ^ j θ ^ p o o l ) a ^ )

E ^ j = ( log p ^ c 1 j log p ^ c 2 j ) + ( { c 1 0.5 + ( 0.5 2 c 1 ) p ^ c 1 j } n 1 j p ^ c 1 j { c 2 0.5 + ( 0.5 2 c 2 ) p ^ c 2 j } n 2 j p ^ c 2 j ) + O ( n 2 )

V ^ j = q ^ c 1 j n 1 j p ^ c 1 j + q ^ c 1 j ( 1 2 c 1 ) + 0.5 q ^ c 1 j 2 ( n 1 j p ^ c 1 j ) 2 + q ^ c 2 j n 2 j p ^ c 2 j + q ^ c 2 j ( 1 2 c 2 ) + 0.5 q ^ c 2 j 2 ( n 2 j p ^ c 2 j ) 2 + O ( n 3 )

θ ^ p o o l = log ( p ^ 1 ) log ( p ^ 2 )

where

p ^ 1 = j = 1 k n 1 j p ^ 1 j j = 1 k n 1 j = j = 1 k X 1 j j = 1 k n 1 j , p ^ 2 = j = 1 k n 2 j p ^ 2 j j = 1 k n 2 j = j = 1 k X 2 j j = 1 k n 2 j

a ^ = j = 1 k 1 V ^ j = j = 1 k V ^ j 1 , b ^ = j = 1 k E ^ j V ^ j , τ ^ j = a ^ E ^ j b ^

Assuming that a normal approximation is reliable, the minimum MSE weights Z-test for testing H 0 : θ = θ 0 , ( θ 0 = log R R 0 ) is

Z c w = j = 1 k f ^ j θ ^ c j θ 0 j = 1 k f ^ j 2 V ^ a r ( θ ^ c j | H 0 ) = j = 1 k f ^ j θ ^ c j θ 0 j = 1 k f ^ j 2 V ^ j

We will reject H 0 at α level for two-sided test if | Z c w | > Z α / 2 where Z α / 2 is the upper 100 ( α t h ) percentile of the standard normal distribution. Alternatively, reject H 0 when the p-value (p) is less than or equal to α where p = 2 ( 1 Φ ( | Z c w | ) ) and Φ ( Z ) is the cumulative standard normal distribution.

4. Other Well-Known Estimators and Tests for Making Inference for a Common Relative Risk

Under a common risk ratio or homogeneity of risk ratios across k studies, we wish to compare the performance of the minimum MSE weighted estimator adjusted by various points c = 1 / 6 , c = 1 / 3 , c = 1 / 2 with the other well-known summary risk ratio estimators, such as the Mantel-Haenszel (MH) estimator and the weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator via a simulation study. According to these well-known estimators, we will present briefly both estimators.

Mantel-Haenszel Weights (MH)

For Mantel-Haenszel (MH) relative risk estimator overall centers/studies from binomial data, the estimator has been proposed by

R R ^ M H = j = 1 k n 2 j X 1 j / N j j = 1 k n 1 j X 2 j / N j (11)

where N j = n 1 j + n 2 j .

The variance estimator of the log-relative risk of Mantel-Haenszel was proposed by Green and Robin [20] based on unconditional binomial distribution is given by

V ^ [ log R R ^ M H ] = j = 1 k D j ( j = 1 k R j ) ( j = 1 k S j ) (12)

where:

D j = ( n 2 j n 1 j t j x 1 j x 2 j N j ) / N j 2 , R j = x 1 j n 2 j / N j , S j = x 2 j n 1 j / N j , t j = x 1 j + x 2 j and N j = n 1 j + n 2 j . Note that under a binomial sparse-data model, the Mantel-Haenszel relative risk is consistent in sparse stratification.

Assuming that a normal approximation is valid, the Mantel-Haenszel’s Z-test for testing H 0 : θ = θ 0 , ( θ 0 = log R R 0 ) is

Z M H = log R R ^ M H θ 0 V ^ ( log R R ^ M H | H 0 )

We will reject H 0 at α level for two-sided test if | Z M H | > Z α / 2 where Z α / 2 is the upper 100 ( α t h ) percentile of the standard normal distribution. Alternatively, reject H 0 when the p-value (p) is less than or equal to α where p = 2 ( 1 Φ ( | Z M H | ) ) and Φ ( Z ) is the cumulative standard normal distribution.

Weighted least square (WLS) estimator

The weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator of the log-relative risk overall centers/studies is

θ ^ W L S = log R R ^ W L S = j = 1 k w j log R R ^ j / j = 1 k w j (13)

where

log R R ^ j = log p ^ 1 j log p ^ 2 j = log ( X 1 j n 1 j ) log ( X 2 j n 2 j )

w j = 1 V [ log R R ^ j ] = ( 1 p 1 j n 1 j p 1 j + 1 p 2 j n 2 j p 2 j ) 1 (14)

Practically, the weights w j are often replaced by their estimates.

w ^ j = 1 V ^ [ log R R ^ j ] = ( 1 p ^ 1 j n 1 j p ^ 1 j + 1 p ^ 2 j n 2 j p ^ 2 j ) 1 = ( 1 X 1 j 1 n 1 j + 1 X 2 j 1 n 2 j )

The variance of the summary estimator θ ^ W L S is given by

V ( θ ^ W L S ) = j = 1 k w j 2 V [ log R R ^ j ] ( j = 1 k w j ) 2 = j = 1 k w j 2 ( 1 w j ) ( j = 1 k w j ) 2 = 1 j = 1 k w j (15)

Assuming that a normal approximation is valid, the weighted least square Z-test for testing H 0 : θ = θ 0 , ( θ 0 = log R R 0 ) is

Z W L S = j = 1 k w ^ j log R R ^ j / ( j = 1 k w ^ j ) θ 0 1 / ( j = 1 k w ^ j )

We will reject H 0 at α level for two-sided test if | Z W L S | > Z α / 2 where Z α / 2 is the upper 100 ( α t h ) percentile of the standard normal distribution. Alternatively, reject H 0 when the p-value (p) is less than or equal to α where p = 2 ( 1 Φ ( | Z W L S | ) ) and Φ ( Z ) is the cumulative standard normal distribution.

5. Simulation Plan for the Estimation, Studying the Type I Error and the Power of the Test

We present here a simulation study using the following designs:

Parameters: Let the common relative risk be some constants (RR = 1, 2 and 4) and generate the baseline proportion p 2 j in the control arm for the jth study from a uniform distribution in which the range corresponds to the values of RR. If R R = 1 , then p 2 j ~ U ( 0 , 0.9 ) ; if R R = 2 , then p 2 j ~ U ( 0 , 0.45 ) ; and if R R = 4 , then p 2 j ~ U ( 0 , 0.23 ) . The correspondent proportion risks in the exposure arm is p 1 j = p 2 j × R R . The sample size n 1 j and n 2 j in each study are fixed and varied as 4, 8, 16, 32 and 100. The number of studies is 1, 2, 4, 8, 16, 32 and 100.

Statistic 1 (estimation of RR): Generate x 1 j and x 2 j from the binomial distribution with parameter ( p 1 j , n 1 j ) and ( p 2 j , n 2 j ) for each study j ( j = 1 , 2 , , k ). All summary estimates are calculated. The procedure is replicated 5000 times. From these replicates, we compute bias, variance, and mean square error (MSE) for the adjusted relative risk estimator with the proposed weight to compare the performance with Mantel-Haenszel (MH) estimator and the weighted least square estimator.

Statistic 2 (studying the type I error under H 0 : R R = R R 0 ): Generate x 1 j and x 2 j from the binomial distribution with parameter ( p 1 j , n 1 j ) and ( p 2 j , n 2 j ) and replicate these generations 5000 times for every procedure. From these replicates, the number of null hypothesis rejections when H 0 is true under three Z-tests is counted for the actual type I error.

TheactualtypeIerror = Numberofrejectionsof H 0 when H 0 istrue Numberofreplicates ( 5000 times )

Statistic 3 (studying the power of the test under H 1 ): Before comparing the power of tests, all test statistics should be calibrated to handle the same type I error rate under the null hypothesis. Under the alternative hypothesis with the random effects model, the powers of three candidate tests are compared. We need to revise the parameter setting for studying the power of the test. Let p 2 j be a uniform distribution over [0, 0.25] and we assume that θ j = log R R j follows as θ j = 0.1 + U m = 0.1 + m m ( 2 U 1 ) where U m is a uniform [ m m , m m ] random variable for a given m m = 0.2 , 0.4 , 0.6 , or equivalently, U is a uniform over (0, 1). Note that these parameter settings provide: E ( θ j ) = 0.1 and V a r ( θ j ) = ( 2 × m m ) 2 / 12 . Consequently, we still have p 1 j = p 2 j × R R . Binomial variates x 1 j and x 2 j are also generated with parameter ( p 1 j , n 1 j ) and ( p 2 j , n 2 j ) , respectively. All proposed test statistics under this alternative hypothesis are computed and replicated 5000 times. From these replicates, the number of null hypothesis rejection is counted for the power of the test.

Theactualpowerofthetest = Numberofrejectionsof H 0 when H 1 istrue Numberofreplicates ( 5000 times )

6. Results from Simulation Studies

6.1. Comparative Performance for Point Estimation

Under a constant of relative risk (RR = 1, 2 and 4), the performance in terms of bias, variance, and mean square error (MSE) of several summary relative risk estimators are compared. Results show that increasing k can decrease the variance and the MSE of all estimators and the increase of both n i j can also decrease the variance of all estimators while fixing k. The unbalance cases of n i j ( i = 1 , 2 and j = 1 , , k ) have less affected on the order performance of MSE estimators. The summary adjusted relative risk estimator in meta-analysis study of size k has shrinkage estimator to be a simple adjusted relative risk estimator in one single study case. The optimal point ( c = c 1 = c 2 ) providing the bias, variance, and MSE of θ ^ w , adjusted by c = 1 / 6 is identical to c = 1 / 6 . By these, the numerical evidence has confirmed the derivation process of finding the root c and it is very useful in practice.

For a single center study ( k = 1 ), regardless of a true value of RR, the proposed estimator adjusted by c = 1 / 3 performs the best with smallest MSE.

For a multi-center study of size k, when R R = 1 , the WLS estimator is the best in sense of the smallest MSE ignoring the sample size n i j . In case R R = 2 , k = 4 the proposed estimator adjusted by c performs the best with the smallest MSE. Another issue, when R R = 2 , k = 16 , the MH estimator achieves the smallest MSE when sample sizes are small. The MSE of the WLS estimator and the proposed estimator adjusted by c are close together when the sample sizes are moderate and large ( n i j 16 ). For the case R R = 4 and k = 4 or 16, the MH estimator well performs generally with the smallest MSE. Some comparisons of the performances of all summary estimators when n 1 j = 4 , 8 and n 2 j = 4 , 8 , 16 , 32 are depicted in Figures 1(a)-(f) for bias; Figures 2(a)-(f) for variance; and Figures 3(a)-(f) for MSE.

In summary for the performance of estimators, regardless of the true values of RR, the MH estimator achieves the best performance with the smallest MSE when the study size is rather large ( k 16 ) and the sample sizes within each study are small. The MSE of WLS estimator and the proposed-weight estimator adjusted by c = 1 / 6 , c = 1 / 3 , c = 1 / 2 are close together and they are the best when the sample sizes are moderate to large ( n i j 16 ) while the study size is rather small.

6.2. Studying the Type I Error

A type I error comparison between the tests is considered by comparing the actual (empirical) type I error ( α ^ ) with the nominal level of significance. In this study, the evaluation of the ability to control type I error probability for two-sided tests is based on Cochran limits as follows.

At α = 0.01 significant level, the actual α ^ value is between [0.005, 0.015].

At α = 0.05 significant level, the actual α ^ value is between [0.04, 0.06].

At α = 0.10 significant level, the actual α ^ value is between [0.08, 0.12].

If the actual type I error or the empirical alpha lies within those Cochran limits, then the statistical test can control type I error rate.

For a single study ( k = 1 ), regardless of the true values of RR, it is unfortunate that almost all tests shown in Table 2 cannot control type I error rate. There are some type I error rates lying in the Cochran limits.

For a meta-analysis study of size k, also displayed in Table 2, regardless of the true values of RR, the Mantel-Haenszel’s Z-Test can control type I error rate when the sample size either in treatment or control group is moderate to large ( n 1 j 16 or n 2 j 16 ). In addition, the weighted least square Z-test and the proposed weight Z-test adjusted by c = 1 / 6 , c = 1 / 2 , c = 1 can handle type I error rate when both n i j are large. But the proposed Z-test adjusted by c = 2 cannot control type I error rate into Cochran’s limit at almost all situations.

6.3. Studying the Power of the Test

Usually, the empirical power of the tests will be compared under the same type I error value. In summary, the Mantel-Haenszel Z-test performs best when n i j is moderate to large with satisfying the type I error value within Cochran’s range limit, regardless of the study size k. The inverse variance weighted Z-test is good when n i j is large. For the proposed Z-test adjusted by c = 1 , c = 2 , the Z-tests perform well under the same type I error value when k is small ( k 4 ) and n i j is large ( n i j 32 ). The results in Table 3 illustrate the comparison of the power of tests under the same rate of the type I error.

Figure 1. Bias comparison of relative risk between well-known estimators and adjusted relative risk estimators at k = 1, k = 4 and k = 16 for n1 = 4 and n2 = 4, 8, 16 and 32 ((a)-(c)) and at k = 1, k = 4 and k = 16 for n1 = 8 and n2 = 8, 16 and 32 ((d)-(f)).

Figure 2. Variance comparison of relative risk between well-known estimators and adjusted relative risk estimators at k = 1, k = 4 and k = 16 for n1 = 4 and n2 = 4, 8, 16 and 32 ((a)-(c)) and at k = 1, k = 4 and k = 16 for n1 = 8 and n2 = 8, 16 and 32 ((d)-(f)).

Figure 3. MSE comparison of relative risk between well-known estimators and adjusted relative risk estimators at k = 1, k = 4 and k = 16 for n1 = 4 and n2 = 4, 8, 16 and 32 ((a)-(c)) and at k = 1, k = 4 and k = 16 for n1 = 8 and n2 = 8, 16 and 32 ((d)-(f)).

Table 2. Comparison of the empirical type I error for testing H 0 : R R = R R 0 in a center study ( k = 1 ) and multi-center study ( k = 4 , 16 and 32) at 5% significant level.

Bold Values denote that the statistical tests can control the type I error.

Table 3. Comparison of the empirical power of test (percent).

#denotes that the statistical tests cannot control type I error rate under H 0 : R R = 1 .

7. Discussion and Recommendation

The main question rises which continuity correction values are the best choice for the adjusted relative risk estimator in a center study and multi-center study with sparse data. Due to the conventional continuity correction, most of investigators such as Yate [1], Lane [3], Stijnen et al., [4], Lui and Lin [6], Sankey et al., [7], Gart and Zwefel [8], Walter [9] and Cox [10], suggest to use c = 0.5 . However, in this study with the smallest average MSE of θ ^ c = log R R ^ c , the optimal point ( c 1 , c 2 ) = ( 1 / 6 , 1 / 6 ) can perform the best for the point estimation. The minimum point ( c 1 , c 2 ) = ( 1 / 6 , 1 / 6 ) of θ ^ c agrees with the suggestion of Turkey [12], which is very useful and the most appropriate in a practice way.

For estimation of fixing θ ( θ = log R R ) in a center ( k = 1 ), regardless of a true value of RR, the proposed estimator adjusted by c = c 1 = c 2 = 1 / 3 performs the best with the smallest MSE. For a meta-analysis study of size k, in general the MH estimator achieves the smallest MSE when the sample size n i j is small while the study size is rather large ( k 16 ). The MSE of the WLS estimator and the proposed estimator adjusted by the various values of c are closed together and they are the best when the sample sizes are moderate to large ( n 1 j 16 and n 2 j 16 ) while the study size is rather small. This finding is consonant with the work of Viwatwongkasem et al. [19]. Since the true value of RRis usually not available in practice as mentioned earlier, we suggest to choose the proposed relative risk estimator adjusted by c = 1 / 6 that can minimize the Bayes risk with respect to uniform prior (0, 1) and Euclidean loss function.

For the empirical power of the test under H 1 : θ j = 0.1 + U m ( m m , m m ) , regardless of the study size k, the MH Z-test performs the best with the highest power when both n i j are moderate to large. The inverse-variance weighted Z-test is good when n i j is large. In accordance with Soulakova and Bright [21], the empirical power of the test will increase when the sample sizes increase. For the power of the proposed Z-test adjusted by c = 1 , c = 2 , the Z-test has the higher performance under the same type I error when k is small ( k 4 ) and n i j is large ( n i j 32 ).

If we don’t know information about parameter RR, we recommend to use the adjusted estimator θ ^ by using continuity correction defined by c 1 = c 2 = 1 / 6 , or 1/3, or 1/2 in a center study. For a multi-center study of size k, we recommend to use adjusted θ ^ w defined by c 1 = c 2 = 1 / 6 , or 1/3, or 1/2, including optimal weights f ^ j .

Generally, the effect of exposures or the effect of treatments with binary outcomes covers the risk difference, the relative risk, and the odds ratio. Obviously, all three conventional effect estimators have the same problem of the zero values in sparse data. The conventional proportion estimate of p ^ = X / n is in need of replacement by p ^ c = ( X + c ) / ( n + 2 c ) to solve this problem. Therefore, the recommendation for a further study is to use these ideas, such as the smallest MSE, the smallest Bayes risk to fine the appropriate point ( c 1 , c 2 ) in estimating the odds ratio parameter.

Acknowledgements

The study was partially supported for publication by the Faculty of Public Health, Mahidol University, Bangkok, Thailand.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Yates, F. (1934) Contingency Tables Involving Small Numbers and the Chi-Squared Test. Supplement to the Journal of the Royal Statistical Society, 1, 217-235.
https://doi.org/10.2307/2983604
[2] Lane, P.W. (2013) Meta-Analysis of Incidence of Rare Events. Statistical Methods in Medical Research, 22, 117-132.
https://doi.org/10.1177/0962280211432218
[3] Stijnen, T., Hamza, T.H. and Özdemir, P. (2010) Random Effects Meta-Analysis of Event Outcome in the Framework of the Generalized Linear Mixed Model with Applications in Sparse Data. Statistics in Medicine, 29, 3046-3067.
https://doi.org/10.1002/sim.4040
[4] White, I.R., Daniel, R. and Royston, P. (2010) Avoiding Bias Due to Perfect Prediction in Multiple Imputation of Incomplete Categorical Variables. Computational Statistics and Data Analysis, 54, 2267-2275.
https://doi.org/10.1016/j.csda.2010.04.005
[5] Lui, K.J. and Lin, C.D. (2003) A Revisit on Comparing the Asymptotic Interval Estimators of Odds Ratio in a Single 2 × 2 Table. Biometrical Journal, 45, 226-237.
https://doi.org/10.1002/bimj.200390008
[6] Sankey, S.S., Weissfeld, L.A., Fine, M.J. and Kapoor, W. (1996) An Assessment of the Use of the Continuity Correction for Sparse Data in Meta-Analysis. Communications in Statistics—Simulation and Computation, 25, 1031-1056.
https://doi.org/10.1080/03610919608813357
[7] Gart, J.J. and Zweifel, J.R. (1967) On the Bias of Various Estimators of the Logit and Its Variance with Application to Quantal Bioassay. Biometrika, 54, 181-187.
https://doi.org/10.1093/biomet/54.1-2.181
[8] Walter, S.D. (1975) The Distribution of Levin’s Measure of Attributable Risk. Biometrika, 62, 371-372.
https://doi.org/10.1093/biomet/62.2.371
[9] Cox, D.R. (1970) The Continuity Correction. Biometrika, 57, 217-219.
https://doi.org/10.1093/biomet/57.1.217
[10] Li, L. and Wang, X. (2017) Meta-Analysis of Rare Binary Events in Treatment Groups with Unequal Variability. Statistical Methods in Medical Research, 28, 263-274.
https://doi.org/10.1177/0962280217721246
[11] Tukey, J.W. (1977) Exploratory Data Analysis. Addison-Wesley Publishing Company, Boston.
[12] Sánchez-Meca, J. and Marín-Martínez, F. (2000) Testing the Significance of a Common Risk Difference in Meta-Analysis. Computational Statistics and Data Analysis, 33, 299-313.
https://doi.org/10.1016/S0167-9473(99)00055-9
[13] Böhning, D. and Viwatwongkasem, C. (2005) Revisiting Proportion Estimators. Statistical Methods in Medical Research, 14, 147-169.
https://doi.org/10.1191/0962280205sm393oa
[14] Agresti, A. and Caffo, B. (2000) Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result from Adding Two Successes and Two Failures. The American Statistician, 54, 280-288.
https://doi.org/10.1080/00031305.2000.10474560
[15] McClave, J.T. and Sincich, T.T. (2012) Statistics. Pearson Education, London.
[16] Pettigrew, H.M., Gart, J.J. and Thomas, D.G. (1986) The Bias and Higher Cumulants of the Logarithm of a Binomial Variate. Biometrika, 73, 425-435.
https://doi.org/10.1093/biomet/73.2.425
[17] Lipsitz, S.R., Dear, K.B.G., Laird, N.M. and Molenberghs, G. (1998) Tests for Homogeneity of the Risk Difference When Data Are Sparse. Biometrics, 54, 148-160.
[18] Cooper, M.R., Dear, K.B.G., McIntyre, O.R., Ozer, H., Ellerton, J.Cannellos, G., Duggan, B. and Schiffer, C. (1993) A Randomized Clinical Trial Comparing Melphalan/Prednisone with and without α-2b Interferon in Newly-Diagnosed Patients with Multiple Myeloma: A Cancer and Leukemia Group B Study. Journal of Clinical Oncology, 11, 155-160.
https://doi.org/10.1200/JCO.1993.11.1.155
[19] Viwatwongkasem, C., Jitthavech, J., Böhning, D. and Lorchirachoonkul, V. (2012) Minimum MSE Weights of Adjusted Summary Estimator of Risk Difference in Multi-Center Studies. Open Journal of Statistics, 2, 48-59.
https://doi.org/10.4236/ojs.2012.21006
[20] Greenland, S. and Robin, J.M. (1985) Estimation of a Common Effect Parameter from Sparse Follow-up Data. Biometrics, 45, 55-68.
[21] Soulakova, J.N. and Bright, B.C. (2013) Applications of Asymptotic Confidence Intervals with Continuity Corrections for Asymmetric Comparisons in Noninferiority Trials. Pharmaceutical Statistics, 12, 147-155.
https://doi.org/10.1002/pst.1566

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.