Cross Validation Based Model Averaging for Varying-Coefficient Models with Response Missing at Random

Huixin Li, Xiuli Wang^*

School of Mathematics and Statistics, Shandong Normal University, Jinan, China.

DOI: 10.4236/jamp.2024.123047                

Abstract

In this paper, a model averaging method is proposed for varying-coefficient models with response missing at random by establishing a weight selection criterion based on cross-validation. Under certain regularity conditions, it is proved that the proposed method is asymptotically optimal in the sense of achieving the minimum squared error.

Keywords

Response Missing at Random, Model Averaging, Asymptotic Optimality, B-Spline Approximation

1. Introduction

In the past two decades, there have been more and more studies on model averaging and model selection. Model selection is to select a final model before estimation, but due to the uncertainty of the model, there will be a deviation caused by the selection of wrong model. Model averaging is to estimate a large number of candidate models, and then give greater weight to the sub-models with better prediction results, that is, to select the model after estimation, and the candidate model does not require the inclusion of the real model, which can effectively overcome the shortcomings of the model selection method. Therefore, model averaging is increasingly used in various fields.

Most of the model averaging methods have been devoted to the frequency model averaging, and mainly in parametric models. For example, Hansen (2007) [1] , Zhang et al. (2008) [2] , Wan et al. (2010) [3] proposed the Mallows-type model averaging (MMA) method. To further improve the method, Zhang et al. (2016) [4] proposed a weight selection criterion based on the Kullback-Leibler loss with penalty term for generalized linear models and generalized linear mixed-effects models. Using nonparametric kernel smoothing method to estimate nonparametric functions, Li et al. (2018) [5] , Zhang and Wang (2019) [6] extended the Mallows-type model averaging method to semiparametric models. Recently, Xia (2021) [7] extended the Mallows-type model averaging method to varying-coefficient models using B-spline to estimate the nonparametric functions. However, the above mentioned literatures focus on the case where the data is completely observed. In statistical studies such as economics, market research and medical research, missing data is a common phenomenon. According to the different missing mechanisms, it can be divided into three types, namely Missing completely at random (MCAR), Missing at random (MAR) and Missing not ant random (MNAR). Little attention has been paid to model averaging with missing data. One of the reasons is that the complexity of missing data makes it very challenging to extend existing methods with complete data to missing data. The study of model averaging with missing data is still a relatively new topic and needs further development.

In this paper, we focus on model averaging for varying-coefficient models with response missing at random (MAR), which has been widely studied in the field of missing data. As far as we know, there are few literatures on the model averaging with missing response. For example, Sun et al. (2014) [8] developed a model average estimation approach for linear regression models with response missing at random under a local misspecification framework, called smoothed focused vector information criteria (SFVIC). Zeng et al. (2018) [9] extended the SFVIC to varying-coefficient partially linear models with response missing at random by employing the profile least-squares estimation process and inverse probability weighted method to estimate regression coefficients of the models under the framework of local misspecification. However, these researches are limited to the local misspecification framework, and the model averaging without local misspecification framework is relatively small. For example, based on the weighted delete-one cross-validation (WDCV) criterion, Xie et al. (2021) [10] proposed a two-step model averaging procedure for high-dimensional linear regression models with missing response without local misspecification framework, and under certain conditions, they proved the WDCV criterion asymptotically achieved the lowest possible prediction loss. But their criterion required the covariates used in different candidate models cannot overlap. Based on the Mallows-type model averaging method, Wei et al. (2021) [11] established the HRC_p criterion of linear models with missing response, which improved the shortcomings of MMA in estimating the error covariance matrix. Wei and Wang (2021) [12] developed a linear models averaging procedure with response missing at random by establishing a cross-validation-based weight choice criterion, and proved its asymptotic optimality. The three literatures mentioned above are limited to parametric linear models.

Inspired by Xia (2021) [7] , Wei and Wang (2021) [12] , and Wei et al. (2021) [11] , in this paper we shall extend the Jackknife model averaging (JMA) method (e.g. Hansen and Racine (2012) [13] and Zhang et al. (2013) [14] ) to varying-coefficient models with response missing at random and heteroscedasticity error. The varying-coefficient model is a common type of non-parametric model. It has no fixed regression function and is more flexible in form. It can further fit complex data and is more widely used in various fields. Following Xia (2021) [7] , we use B-spline to estimate the nonparametric varying-coefficient functions, and use the inverse probability weighted (IPW) method to deal with the missing data. In order to avoid the “curse of dimensionality”, we assume a parametric model for propensity score function. Compared with previous articles, our method has the following advantages. Firstly, we use B-spline to estimate the varying-coefficient functions, which results in less computational burden compared with the local kernel function method. Secondly, we choose the weight vector based on JMA method. Compared with the MMA method, our method does not need to estimate the covariance matrix of the error and is more suitable for the heteroscedasticity models, which makes it easier to deal with real data. In the theoretical proof, we do not need to consider the relationship between the estimated error covariance matrix and the real error covariance matrix, which makes our theoretical proof easier. Finally, our method can deal with three cases for candidate models (Nested case, Marginal case and Full case), not just the case where the candidate models are nested. Under certain conditions, our proposal is asymptotically optimal in the sense of achieving the lowest possible squared error, which shows that our method is feasible. The finite sample performance of our proposal is studied by numerical simulations and compared with some related methods. The simulation results display that our proposal is feasible.

The rest of this paper is organized as follows. Section 2 and Section 3 describe the method of model averaging estimator and weight selection respectively. Some conditions and theoretical result for asymptotic optimality are given in Section 4. The proofs of the main results are given in Section 5.

2. Model Framework and Model Averaging Estimation Method

We consider the following varying-coefficient model:

$\{\begin{array}{l}{Y}_i={\mu }_i+{\epsilon }_i\hfill \\ {\mu }_i={\displaystyle {\sum }_{j=1}^p{X}_{ij}{\beta }_j\left(U_i\right)}\hfill \end{array},i=1,2,\cdots ,n,$ (1)

where $Y_i$ is response variable, the index variable $U_i$ is a scalar and ${\epsilon }_i$ is a random error with $E\left({\epsilon }_i|X_i\mathrm{,}{U}_i\right)=0$ and $Var\left({\epsilon }_i|X_i\mathrm{,}{U}_i\right)={\sigma }_i^2$ . We assume ${\mu }_i=E\left(Y_i|X_i\mathrm{,}{U}_i\right)={\displaystyle {\sum }_{j\mathrm{<mo>\; =\; </mo>1}}^p}{X}_{ij}{\beta }_j\left(U_i\right)=X_i^{\text{T}}\beta \left(U_i\right)$ , where $X_i={\left(X_{i1}\mathrm{,}\cdots \mathrm{,}{X}_{ip}\right)}^{\text{T}}$ is a p-dimensional covariate, $\beta \left(U_i\right)={\left({\beta }_1\left(U_i\right)\mathrm{,}\cdots \mathrm{,}{\beta }_p\left(U_i\right)\right)}^{\text{T}}$ corresponds to the vector of the unknown coefficient functions.

Suppose that $X_i$ and $U_i$ are completely observable, and $Y_i$ may be missing, where ${\delta }_i=1$ if $Y_i$ is observed, ${\delta }_i=0$ otherwise. In this paper, we assume that $Y_i$ is missing at random (MAR), that is

$P\left({\delta }_i=1|Y_i\mathrm{,}{X}_i\mathrm{,}{U}_i\right)=P\left({\delta }_i=1|X_i\mathrm{,}{U}_i\right)\mathrm{<mo>\; :\; </mo>}=\pi \left(X_i\mathrm{,}{U}_i\right)\mathrm{.}$ (2)

Let $Z_{\pi \mathrm{,}i}={\left\{\pi \left(X_i\mathrm{,}{U}_i\right)\right\}}^{-1}{\delta }_i{Y}_i\text{,}i=\mathrm{1,2,}\cdots \mathrm{,}n$ , using model (1) and MAR assumption, we can obtain

$\{\begin{array}{l}{Z}_{\pi \mathrm{,}i}={\mu }_i+{\epsilon }_{\pi \mathrm{,}i}\hfill \\ {\mu }_i=E\left(Z_{\pi \mathrm{,}i}|X_i\mathrm{,}{U}_i\right)\hfill \end{array}\mathrm{,}i=\mathrm{1,2,}\cdots \mathrm{,}n\text{,}$ (3)

where $E\left({\epsilon }_{\pi \mathrm{,}i}|X_i\mathrm{,}{U}_i\right)=0$ , $Var\left({\epsilon }_{\pi \mathrm{,}i}|X_i\mathrm{,}{U}_i\right)={\sigma }_{\pi \mathrm{,}i}^2$ , ${\sigma }_{\pi \mathrm{,}i}^2=\left[{\left\{\pi \left(X_i\mathrm{,}{U}_i\right)\right\}}^{-1}-1\right]{\mu }_i^2+{\left\{\pi \left(X_i\mathrm{,}{U}_i\right)\right\}}^{-1}{\sigma }_i^2$ .

We consider a series of approximate models. In particular, we use $M_n$ candidate model $\left\{M_1\mathrm{,}\cdots \mathrm{,}{M}_{M_n}\right\}$ to approximate the real data generating process of Y. The mth candidate model $M_m$ contains $k_m$ covariates, that is, $k_m=\left|M_m\right|$ is the cardinality of $M_m$ . The total number of all possible candidate models is $M_n=2^p$ , which contains an empty model that excludes all covariates. From model (3), under the mth model, we have

$Z_{\pi ,i}={\displaystyle \underset{j=1}{\overset{k_m}{\sum }}}{X}_{i{j}_{\left(m\right)}}{\beta }_{j_{\left(m\right)}}\left(U_i\right)+{\epsilon }_{\pi ,i}.$ (4)

The purpose of this paper is to construct an asymptotically optimal model average estimator of the conditional mean $\mu ={\left({\mu }_1\mathrm{,}{\mu }_2\mathrm{,}\cdots \mathrm{,}{\mu }_n\right)}^{\text{T}}$ with missing response under MAR assumption.

Note that $\left\{Z_{\pi \mathrm{,}i}\mathrm{,}{X}_i\mathrm{,}{U}_i\right\}$ is fully observed, if the selection probability function $\pi \left(X_i\mathrm{,}{U}_i\right)$ is known, the estimator of ${\beta }_{j_{\left(m\right)}}\left(u\right)$ can be obtained by using B-spline to approximate the coefficient function for $\left\{Z_{\pi \mathrm{,}i}\mathrm{,}{X}_i\mathrm{,}{U}_i\right\}$ . Specifically, we first approximate each ${\beta }_{j_{\left(m\right)}}(\cdot )$ with a function in a polynomial spline space. Without losing generality, we assume that U has a compact set $U=\left[0,\text{1}\right]$ . Let $B\left(u\right)={\left(B_1\left(u\right)\mathrm{,}\cdots \mathrm{,}{B}_{L_n}\left(u\right)\right)}^{\text{T}}$ is a B-spline function basis of order r, where $L_n=J_n+r+1$ , $J_n$ is the number of interior knots and increases with the increase of sample size n. According to the B-spline theory in De Boor (2001) [15] , there exists a parameter vector ${\gamma }_{j\left(m\right)}$ satisfying

${\beta }_{j_{\left(m\right)}}\left(u\right)\approx B{\left(u\right)}^{\text{T}}{\gamma }_{j\left(m\right)}\mathrm{,}$

where ${\beta }_{j\left(m\right)}\left(u\right)$ is the jth component of ${\beta }_{\left(m\right)}\left(u\right)$ . So the mth model can be re-expressed as

$Z_{\pi ,i}={\displaystyle \underset{j=1}{\overset{k_m}{\sum }}}{X}_{i{j}_{\left(m\right)}}B{\left(U_i\right)}^{\text{T}}{\gamma }_{j\left(m\right)}+{\epsilon }_{\pi ,i}\approx {\displaystyle \underset{j=1}{\overset{k_m}{\sum }}}{W}_{i\left(m\right)}{\gamma }_{j\left(m\right)}+{\epsilon }_{\pi ,i},$

where $W_{i\left(m\right)}={\left(X_{i1}B{\left(U_i\right)}^{\text{T}}\mathrm{,}{X}_{i2}B{\left(U_i\right)}^{\text{T}}\mathrm{,}\cdots \mathrm{,}{X}_{i{k}_m}B{\left(U_i\right)}^{\text{T}}\right)}^{\text{T}}$ , $i=\mathrm{1,2,}\cdots \mathrm{,}n$ is a $k_m{L}_n$ dimensional column vector, $B\left(U_i\right)={\left(B_1\left(U_i\right)\mathrm{,}\cdots \mathrm{,}{B}_{L_n}\left(U_i\right)\right)}^{\text{T}}$ .

Therefore, ${\beta }_{j\left(m\right)}\left(u\right)$ can be obtained by solving the following least squares optimization problem

$\underset{{\gamma }_{j\left(m\right)}}{\min }{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left[Z_{\pi ,i}-{\displaystyle \underset{j=1}{\overset{k_m}{\sum }}}{W}_{i\left(m\right)}{\gamma }_{j\left(m\right)}\right]}^2.$ (5)

Let $Z_{\pi }={\left(Z_{\pi \mathrm{,1}}\mathrm{,}{Z}_{{\pi }_2}\mathrm{,}\cdots \mathrm{,}{Z}_{{\pi }_n}\right)}^{\text{T}}$ , ${\gamma }_{\left(m\right)}={\left({\gamma }_{j\left(m\right)}^{\text{T}}\mathrm{,}j\in M_m\right)}^{\text{T}}$ is a vector of length $k_m{L}_n$ , then the solution of (5) is given by

${\hat{\gamma }}_{\left(m\right)}={\left[W_{\left(m\right)}^{\text{T}}{W}_{\left(m\right)}\right]}^{-1}{W}_{\left(m\right)}^{\text{T}}{Z}_{\pi }\mathrm{,}$

where $W_{\left(m\right)}={\left(W_{1\left(m\right)}\mathrm{,}\cdots \mathrm{,}{W}_{n\left(m\right)}\right)}^{\text{T}}$ is a $n\times k_m{L}_n$ -dimensional matrix. Thus, the estimator of ${\beta }_{\left(m\right)}\left(u\right)$ is ${\hat{\beta }}_{\left(m\right)}\left(u\right)={\left({\hat{\beta }}_{j\left(m\right)}\left(u\right)\mathrm{,}j\in M_m\right)}^{\text{T}}$ with ${\hat{\beta }}_{j\left(m\right)}\left(u\right)=B{\left(u\right)}^{\text{T}}{\hat{\gamma }}_{j\left(m\right)}$ .

For the mth candidate model, the estimator of $\mu$ is then obtained as

${\hat{\mu }}_{\pi \left(m\right)}=W_{\left(m\right)}{\left[W_{\left(m\right)}^{\text{T}}{W}_{\left(m\right)}\right]}^{-1}{W}_{\left(m\right)}^{\text{T}}{Z}_{\pi }=P_{\left(m\right)}{Z}_{\pi }\text{,}$ (6)

where $P_{\left(m\right)}=W_{\left(m\right)}{\left[W_{\left(m\right)}^{\text{T}}{W}_{\left(m\right)}\right]}^{-1}{W}_{\left(m\right)}^{\text{T}}$ . From (6), we know that ${\hat{\mu }}_{\pi \left(m\right)}$ is linearly related to $Z_{\pi }$ , we can define the model average estimator of $\mu$ as the weighted average of ${\hat{\mu }}_{\pi \left(m\right)}$ , that is

${\hat{\mu }}_{\pi }\left(\omega \right)={\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\omega }_m{\hat{\mu }}_{\pi \left(m\right)}={\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\omega }_m{P}_{\left(m\right)}{Z}_{\pi }=P\left(\omega \right)Z_{\pi }\text{,}$ (7)

where $P\left(\omega \right)={\displaystyle {\sum }_{m=1}^{M_n}}{\omega }_m{P}_{\left(m\right)}$ , $\omega ={\left({\omega }_1\mathrm{,}\cdots \mathrm{,}{\omega }_{M_n}\right)}^{\text{T}}$ is the weight vector, which satisfies

$H_n=\left\{\omega \in {\left[\mathrm{0,1}\right]}^{M_n}\mathrm{<mo>\; :\; </mo>}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\omega }_m=1\right\}\mathrm{.}$

3. Weight Selection

According to Hansen and Racine (2012) [13] , we give the details for the jackknife selection (also known as leave-one-out cross validation) of $\omega$ as follows.

The jackknife estimator of the mth model is denoted by

${\tilde{\mu }}_{\pi \left(m\right)}={\left({\tilde{\mu }}_{\pi \left(m\right)}^1\mathrm{,}{\tilde{\mu }}_{\pi \left(m\right)}^2\mathrm{,}\cdots \mathrm{,}{\tilde{\mu }}_{\pi \left(m\right)}^n\right)}^{\text{T}}$ ,

where ${\tilde{\mu }}_{\pi \left(m\right)}^i=W_{\left(m\right)}^i{\left[W_{\left(m\right)}^{\left(-i\right)}{}^{\text{T}}{W}_{\left(m\right)}^{\left(-i\right)}\right]}^{-1}{W}_{\left(m\right)}^{\left(-i\right)}{}^{\text{T}}{Z}_{\pi }^{\left(-i\right)}$ , and $W_{\left(m\right)}^{\left(-i\right)}$ and $Z_{\pi }^{\left(-i\right)}$ are defined as matrices $W_{\left(m\right)}$ and column vectors $Z_{\pi }$ with the ith row deleted. After some calculations, we can derive the following relationship:

${\tilde{P}}_{\left(m\right)}=D_{\left(m\right)}\left(P_{\left(m\right)}-I_n\right)+I_n\text{,}$

where $D_{\left(m\right)}$ is an $n\times n$ diagonal matrix and the ith diagonal element is ${\left(1-P_{\left(m\right)\mathrm{,}ii}\right)}^{-1}$ with $P_{\left(m\right)\mathrm{,}ii}$ the i-th diagonal element of $P_{\left(m\right)}$ , then ${\tilde{\mu }}_{\pi }\left(\omega \right)=\tilde{P}\left(\omega \right)Z_{\pi }$ , $\tilde{P}\left(\omega \right)={\displaystyle {\sum }_{m=1}^{M_n}}{\omega }_m{\tilde{P}}_{\left(m\right)}$ . Therefore, we obtain the following JMA criterion for selecting the weight vector $\omega$ :

$C{V}_{\pi }\left(\omega \right)={\Vert Z_{\pi }-{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert }^2\mathrm{.}$ (8)

In fact, $\pi \left(X_i\mathrm{,}{U}_i\right)$ in (8) is unknown and need to be modeled. Following Wei and Wang (2021) [12] , we assume that $\pi \left(X\mathrm{,}U\mathrm{<mo>\; ;\; </mo>}\alpha \right)$ is a parametric model of $\pi \left(X\mathrm{,}U\right)$ , where $\pi \left(\cdot \mathrm{<mo>\; ;\; </mo>}\alpha \right)$ is a known function and can be correctly specified while $\alpha$ is an unknown parameter vector. Denote ${\hat{\alpha }}_n$ as the maximum likelihood estimate (MLE) of $\alpha$ , $\hat{\pi }\left(X\mathrm{,}U\right)=\pi \left(X\mathrm{,}U\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)$ . Replacing $\pi \left(X\mathrm{,}U\right)$ in $C{V}_{\pi }\left(\omega \right)$ with $\hat{\pi }\left(X\mathrm{,}U\right)$ , we can get the following criterion:

$C{V}_{\hat{\pi }}\left(\omega \right)={\Vert Z_{\hat{\pi }}-{\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)\Vert }^2\mathrm{.}$ (9)

Then the optimal weight vector ${\hat{\omega }}_{\hat{\pi }}$ is defined as

${\hat{\omega }}_{\hat{\pi }}=\underset{\omega \in H_n}{\arg \min }C{V}_{\hat{\pi }}\left(\omega \right).$ (10)

Thus, the model average estimator of $\mu$ is ${\hat{\mu }}_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)$ , and named as the varying-coefficient jackknife model average estimator (VC-JMA).

4. Asymptotically Optimality

In this section, we investigate the asymptotic optimality of the proposed estimator ${\hat{\mu }}_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)$ . We first give some necessary symbols. Define

$L_{\pi }\left(\omega \right)={\Vert \mu -{\hat{\mu }}_{\pi }\left(\omega \right)\Vert }^2\text{,}{R}_{\pi }\left(\omega \right)=E\left\{L_{\pi }\left(\omega \right)|X\mathrm{,}U\right\}\text{,}$ (11)

as the loss function and risk function of ${\hat{\mu }}_{\pi }\left(\omega \right)$ , respectively. ${\Omega }_{e_{\pi }}=\text{diag}\left({\sigma }_{\pi \mathrm{,1}}^2\mathrm{,}\cdots \mathrm{,}{\sigma }_{\pi \mathrm{,}n}^2\right)$ . Let ${\xi }_{\pi }=\underset{w\in H_n}{\inf }{R}_{\pi }\left(\omega \right)$ , ${\omega }_m^0$ be a $M_n$ -dimensional unit vector whose the mth element is one and the other elements are zeros and $X_u={\left(X_{\left(1\right)}\mathrm{,}{X}_{\left(2\right)}\mathrm{,}\cdots \mathrm{,}{X}_{\left(M_n\right)}\right)}^{\text{T}}$ .

Next, we give some conditions which are required for proving the asymptotic optimality of ${\hat{\mu }}_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)$ . All the limiting processes discussed here and throughout the paper are with respect to $n\to \infty$ .

(C1) The MLE ${\hat{\alpha }}_n$ of $\alpha$ is $\sqrt{n}$ consistent and satisfies the regularity conditions of asymptotic normality. $\pi \left(X\mathrm{,}U\mathrm{<mo>\; ;\; </mo>}\alpha \right)$ is bounded away from 0 and twice continuously diferentiable with respect to $\alpha$ . For all $\alpha$ ’s in a neighborhood of ${\alpha }_0$ , ${\max }_{1\le i\le n}\left|\frac{\partial \pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}\alpha \right)}{\partial \alpha }\right|=O_p\left(1\right)$ , where ${\alpha }_0$ is the true value of ${\hat{\alpha }}_n$ .

(C2) ${\lambda }_{\max }\left\{{\Omega }_{e_{\pi }}\right\}\le C_e$ , $p_{\left(m\right)\mathrm{,}ii}\le C_p{k}_m{n}^{-1}$ , a.s., where $C_e$ and $C_p$ are constant.

(C3) $r_u{\xi }_{\pi }^{-1}\stackrel{\text{a}\text{.s}\text{.}}{\to }0$ , where $r_u$ is the rank of $X_u$ .

(C4) $M_n{\xi }_{\pi }^{-2G}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\left(R_{\pi }\left({\omega }_m^0\right)\right)}^G\stackrel{\text{a}\text{.s}\text{.}}{\to }0$ , ${\mu }^{{\rm T}}\mu n^{-1}=O\left(1\right)$ , $E\left({\epsilon }_{\pi \mathrm{,}i}^{4G}|X_i\mathrm{,}{U}_i\right)\le C_v\le \infty$ , a.s., where G is an integer satisfying $1\le G<\infty$ .

(C5) Functions ${\beta }_j\left(u\right)\mathrm{,}j=\mathrm{1,}\cdots \mathrm{,}p$ belong to a class of functions $B$ , whose rth detivatives ${\beta }_j^{\left(r\right)}\left(u\right)$ s exist and are Lipschitz of order ${\beta }_0$ .

(C6) The density $f_U$ of U is bounded away from 0 and infinity on its support.

Remark 1 As Zhang et al. (2013) [14] pointed out, condition (C2) is commonly used in the study of asymptotic optimality of cross-validation methods. The condition (C3) is based on Zhang et al. (2013) [14] ’s condition (22). For a detailed explanation of the condition, see Zhang et al. (2013) [14] . Conditions (C4) is commonly used in model averaging literature, see for example Wan et al. (2010) [3] , Zhang et al. (2013) [14] and Zhu et al. (2019) [16] .

Remark 2 Conditions (C5) and (C6) are two common assumptions in approximating nonparametric coefficient functions with B-spline basis functions, which can be seen in Fan et al. (2014) [17] .

With these conditions, the following theorem states the asymptotic optimality of ${\hat{\mu }}_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)$ .

Theorem 1 Suppose that conditions (C1) - (C6) hold, then

$\frac{L_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)}{\underset{\omega \in H_n}{\inf }{L}_{\hat{\pi }}\left(\omega \right)}\stackrel{\text{p}}{\to }1\text{,}$ (12)

where $L_{\hat{\pi }}\left(\omega \right)={\Vert \mu -{\hat{\mu }}_{\hat{\pi }}\left(\omega \right)\Vert }^2$ .

Theorem 1 implies that our proposed estimator ${\hat{\mu }}_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)$ is asymptotically optimal because the selected weight vector ${\hat{\omega }}_{\hat{\pi }}$ yields a squared error that is asymptotically identical to that of the infeasible optimal weight vector.

5. Theorem Proof

Before proving Theorem 1, we first introduce some symbols used in the proof process. We use c to represent a universal positive constant, which can take different values in different situations. For any matrix A, we define ${\lambda }_{\max }\left\{A\right\}$ as the maximum singular value of A. Let ${\tilde{\xi }}_{\pi }={\inf }_{\omega \in H_n}{\tilde{R}}_{\pi }\left(\omega \right)$ . According to the definitions of $L_{\pi }\left(\omega \right)$ and $R_{\pi }\left(\omega \right)$ in (11), we define the loss function and risk function of ${\tilde{\mu }}_{\pi }\left(\omega \right)$ in (8) by

${\tilde{L}}_{\pi }\left(\omega \right)={\Vert \mu -{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert }^2\text{,}{\tilde{R}}_{\pi }\left(\omega \right)=E\left\{{\tilde{L}}_{\pi }\left(\omega \right)\mathrm{<mo>\; |\; </mo>}X\mathrm{,}U\right\}\text{,}$

respectively, and then obtain

${\tilde{R}}_{\pi }\left(\omega \right)={\Vert \tilde{A}\left(\omega \right)\mu \Vert }^2+tr\left[\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right]\text{,}$

where $\tilde{A}=1-\tilde{P}\left(\omega \right)$ . Let $Q_{\left(m\right)}$ be an $n\times n$ diagonal matrix and the ith diagonal element is $P_{\left(m\right)\mathrm{,}ii}{\left(1-P_{\left(m\right)\mathrm{,}ii}\right)}^{-1}$ . Then according to the definition of $D_{\left(m\right)}$ given in Section 2.2, we obtain

${\tilde{P}}_{\left(m\right)}=P_{\left(m\right)}-Q_m{A}_{\left(m\right)}\text{,}$

where $A_{\left(m\right)}=I_n-P_{\left(m\right)}$ .

Next, give the four Lemmas needed to prove Theorem 1.

Lemma 1 Let

${\omega }^{\mathrm{<mo>\; *\; </mo>}}=\underset{\omega \in H_n}{\arg \min }\left({\tilde{L}}_{\pi }\left(\omega \right)+a_{\pi }\left(\omega \right)\right)\text{,}$

if, as $n\to \infty$ ,

$\underset{\omega \in H_n}{\sup }\frac{\left|a_{\pi }\left(\omega \right)\right|}{{\tilde{R}}_{\pi }\left(\omega \right)}\stackrel{\text{p}}{\to }0\text{,}\underset{\omega \in H_n}{\sup }\left|\frac{{\tilde{L}}_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1\right|\stackrel{\text{p}}{\to }0\text{,}$

and there exists a constant c such that

$\underset{\omega \in H_n}{\inf }{\tilde{R}}_{\pi }\left(\omega \right)\ge c>0\text{,}$

almost surely, then

$\frac{{\tilde{L}}_{\pi }\left({\omega }^{\mathrm{<mo>\; *\; </mo>}}\right)}{{\inf }_{\omega \in H_n}{\tilde{L}}_{\pi }\left(\omega \right)}\stackrel{\text{p}}{\to }1.$

The proof of Lemma 1 refers to Zhang et al. (2013) [14] .

Lemma 2 If (C4) is satisfied, we have

$\underset{\omega \in H_n}{\sup }\left|\frac{{\tilde{L}}_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\mathrm{.}$ (13)

Proof of Lemma 2: Note that

$\begin{array}{c}\left|{\tilde{L}}_{\pi }\left(\omega \right)-{\tilde{R}}_{\pi }\left(\omega \right)\right|=\left|{\Vert \mu -{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert }^2-{\Vert \tilde{A}\left(\omega \right)\mu \Vert }^2-tr\left\{\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right\}\right|\\ =\left|{\Vert \tilde{P}\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right\}-2{\mu }^{\text{T}}\tilde{A}{\left(\omega \right)}^{\text{T}}\tilde{P}\left(\omega \right){\epsilon }_{\pi }\right|\mathrm{.}\end{array}$

Therefore, to prove (13), it suffices to prove that

$\underset{\omega \in H_n}{\sup }\frac{\left|{\Vert \tilde{P}\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right\}\right|}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\text{,}$ (14)

$\underset{\omega \in H_n}{\sup }\frac{\left|{\mu }^{\text{T}}\tilde{A}{\left(\omega \right)}^{\text{T}}\tilde{P}\left(\omega \right){\epsilon }_{\pi }\right|}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\mathrm{.}$ (15)

By Zhang et al. (2013) [14] , under the condition (C4), we have

$M_n{\tilde{\xi }}_{\pi }^{-2G}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\left({\tilde{R}}_{\pi }\left({\omega }_m^0\right)\right)}^G\stackrel{\text{a}\text{.s}\text{.}}{\to }0.$

We use conditions (C4), Chebyshev inequality and Theorem 2 of Whittle (1960) [18] to prove (14) and (15), respectively, as follows:

$\begin{array}{l}P\left\{\underset{\omega \in H_n}{\sup }\left|{\Vert \tilde{P}\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right\}\right|/{\tilde{R}}_{\pi }\left(\omega \right)>\delta |X\mathrm{,}U\right\}\\ \le P\left\{\underset{\omega \in H_n}{\sup }{\displaystyle \underset{t=1}{\overset{M_n}{\sum }}}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\omega }_t{\omega }_m\left|{\epsilon }_{\pi }^{\text{T}}{\tilde{P}}_{\left(t\right)}^{\text{T}}{\tilde{P}}_{\left(t\right)}{\epsilon }_{\pi }-tr\left\{{\tilde{P}}_{\left(t\right)}^{\text{T}}{\Omega }_{e_{\pi }}{\tilde{P}}_{\left(m\right)}\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ \le P\left\{\underset{1\le t\le M_n}{\max }\underset{1\le m\le M_n}{\max }\left|{\tilde{P}}_{\left(t\right)}^{\text{T}}{\tilde{P}}_{\left(t\right)}{\epsilon }_{\pi }-tr\left\{{\tilde{P}}_{\left(t\right)}^{\text{T}}{\Omega }_{e_{\pi }}{\tilde{P}}_{\left(m\right)}\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ =P\{\left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_1^0\right)}^{\text{T}}\tilde{P}\left({\omega }_1^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_1^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_1^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ \cup \left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_1^0\right)}^{\text{T}}\tilde{P}\left({\omega }_2^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_1^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_2^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\end{array}$

$\begin{array}{l}\cup \dots \cup \left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_1^0\right)}^{\text{T}}\tilde{P}\left({\omega }_{M_n}^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_1^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_{M_n}^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ \cup \left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_2^0\right)}^{\text{T}}\tilde{P}\left({\omega }_1^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_2^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_1^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ \cup \left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_2^0\right)}^{\text{T}}\tilde{P}\left({\omega }_2^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_2^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_2^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\end{array}$

$\begin{array}{l}\cup \cdots \cup \left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_2^0\right)}^{\text{T}}\tilde{P}\left({\omega }_{M_n}^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_2^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_{M_n}^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ \cup \cdots \cup \left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_{M_n}^0\right)}^{\text{T}}\tilde{P}\left({\omega }_{M_n}^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_{M_n}^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_{M_n}^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\}\end{array}$

$\begin{array}{l}\le {\displaystyle \underset{t=1}{\overset{M_n}{\sum }}}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}P\left\{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_t^0\right)}^{\text{T}}\tilde{P}\left({\omega }_m^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_t^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_m^0\right)\right\}\right|>\delta {\tilde{\xi }}_{\pi }|X\mathrm{,}U\right\}\\ \le {\displaystyle \underset{t=1}{\overset{M_n}{\sum }}}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}\frac{E\left({\Vert {\epsilon }_{\pi }^{\text{T}}\tilde{P}{\left({\omega }_t^0\right)}^{\text{T}}\tilde{P}\left({\omega }_m^0\right){\epsilon }_{\pi }-tr\left\{\tilde{P}{\left({\omega }_t^0\right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left({\omega }_m^0\right)\right\}\Vert }^{2G}|X\mathrm{,}U\right)}{{\delta }^{2G}{\tilde{\xi }}_{\pi }^{2G}}\\ \le c{\delta }^{-2G}{\tilde{\xi }}_{\pi }^{-2G}{\displaystyle \underset{t=1}{\overset{M_n}{\sum }}}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\left(tr\left\{\tilde{P}{\left({\omega }_t^0\right)}^{\text{T}}\tilde{P}\left({\omega }_m^0\right)\right\}\right)}^G\\ \le c{\delta }^{-2G}{\tilde{\xi }}_{\pi }^{-2G}{\displaystyle \underset{t=1}{\overset{M_n}{\sum }}}{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\left(tr\left\{\tilde{P}{\left({\omega }_m^0\right)}^{\text{T}}\tilde{P}\left({\omega }_m^0\right)\right\}\right)}^G\\ \le c{\delta }^{-2G}{\tilde{\xi }}_{\pi }^{-2G}{M}_n{\displaystyle \underset{m=1}{\overset{M_n}{\sum }}}{\left({\tilde{R}}_{\pi }\left({\omega }_m^0\right)\right)}^G\stackrel{\text{a}\text{.s}\text{.}}{\to }0.\end{array}$

Therefore, we have

$\begin{array}{l}P\left\{\underset{\omega \in H_n}{\sup }\left|{\Vert \tilde{P}\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right\}\right|/{\tilde{R}}_{\pi }\left(\omega \right)>\delta \right\}\\ =E\left\{P\left\{\underset{\omega \in H_n}{\sup }\left|{\Vert \tilde{P}\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{\tilde{P}{\left(\omega \right)}^{\text{T}}{\Omega }_{e_{\pi }}\tilde{P}\left(\omega \right)\right\}\right|/{\tilde{R}}_{\pi }\left(\omega \right)>\delta \mathrm{<mo>\; |\; </mo>}X\mathrm{,}U\right\}\right\}=o_p\left(1\right)\mathrm{.}\end{array}$

(14) is proven. The proof process of (15) is similar to that of (14), and hence the proof of the Lemma 2.

Lemma 3 If (C1) - (C4) are satisfied, then

$\underset{\omega \in H_n}{\sup }\left|\frac{L_{\hat{\pi }}\left(\omega \right)}{R_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\mathrm{.}$ (16)

Proof of Lemma 3: By Lemma 1 of Wei and Wang (2021) [12] and Cauchy-Schwarz inequality, we have

$\begin{array}{c}\left|\frac{L_{\hat{\pi }}\left(\omega \right)}{R_{\pi }\left(\omega \right)}-1\right|\le \left|\frac{L_{\pi }\left(\omega \right)}{R_{\pi }\left(\omega \right)}-1\right|+\frac{2{\left\{L_{\pi }\left(\omega \right)\right\}}^{1/2}\cdot \Vert {\hat{\mu }}_{\pi }\left(\omega \right)-{\hat{\mu }}_{\hat{\pi }}\left(\omega \right)\Vert }{R_{\pi }\left(\omega \right)}\\ +\frac{{\Vert {\hat{\mu }}_{\pi }\left(\omega \right)-{\hat{\mu }}_{\hat{\pi }}\left(\omega \right)\Vert }^2}{R_{\pi }\left(\omega \right)}\text{,}\end{array}$

$\begin{array}{c}{\Vert {\hat{\mu }}_{\pi }\left(\omega \right)-{\hat{\mu }}_{\hat{\pi }}\left(\omega \right)\Vert }^2={\Vert P\left(\omega \right)Z_{\pi }-P\left(\omega \right)Z_{\hat{\pi }}\Vert }^2\\ \le {\left[{\lambda }_{\max }\left\{P\left(\omega \right)\right\}\right]}^2{\Vert Z_{\pi }-Z_{\hat{\pi }}\Vert }^2\le {\Vert Z_{\pi }-Z_{\hat{\pi }}\Vert }^2\mathrm{.}\end{array}$

Therefore, to prove (16), it suffices to show that

$\underset{\omega \in H_n}{\sup }\left|\frac{L_{\pi }\left(\omega \right)}{R_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\text{,}$ (17)

$\underset{\omega \in H_n}{\sup }\frac{{\Vert Z_{\pi }-Z_{\hat{\pi }}\Vert }^2}{R_{\pi }\left(\omega \right)}=o_p\left(1\right)\mathrm{.}$ (18)

Note that

$\begin{array}{c}\left|\frac{L_{\pi }\left(\omega \right)}{R_{\pi }\left(\omega \right)}-1\right|=\left|\frac{{\Vert P\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{P{\left(\omega \right)}^2{\Omega }_{e_{\pi }}\right\}-2{\mu }^{\text{T}}A\left(\omega \right)P\left(\omega \right){\epsilon }_{\pi }}{R_{\pi }\left(\omega \right)}\right|\\ \le \left|\frac{{\Vert P\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{P{\left(\omega \right)}^2{\Omega }_{e_{\pi }}\right\}}{R_{\pi }\left(\omega \right)}\right|+2\left|\frac{{\mu }^{\text{T}}A\left(\omega \right)P\left(\omega \right){\epsilon }_{\pi }}{R_{\pi }\left(\omega \right)}\right|\mathrm{.}\end{array}$

Therefore, to prove (17), it suffices to show that

$\underset{\omega \in H_n}{\sup }\left|\frac{{\mu }^{\text{T}}A\left(\omega \right)P\left(\omega \right){\epsilon }_{\pi }}{R_{\pi }\left(\omega \right)}\right|=o_p\left(1\right)\text{,}$ (19)

$\underset{\omega \in H_n}{\sup }\left|\frac{{\Vert P\left(\omega \right){\epsilon }_{\pi }\Vert }^2-tr\left\{P{\left(\omega \right)}^2{\Omega }_{e_{\pi }}\right\}}{R_{\pi }\left(\omega \right)}\right|=o_p\left(1\right)\mathrm{.}$ (20)

The proof steps of (19) and (20) are similar to those of (14) and (15), respectively. The detailed process is omitted here, so it is established under (C4) condition, i.e. (17) is proven, and next prove (18).

According to (C1) and Cauchy-Schwarz inequality, we have

$\begin{array}{l}\underset{\omega \in H_n}{\sup }\frac{{\Vert {\text{Z}}_{\pi }-{\text{Z}}_{\hat{\pi }}\Vert }^2}{R_{\pi }\left(\omega \right)}\le {\xi }_{\pi }^{-1}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left\{\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)}-\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\right)}\right\}}^2\cdot {\left({\mu }_i+{\epsilon }_i\right)}^2\\ \le {\xi }_{\pi }^{-1}{\left\{\underset{1\le i\le n}{\max }\left|\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)}-\frac{1}{\pi \left({\text{X}}_i\mathrm{,}{U}_i\right)}\right|\right\}}^2\cdot {\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left({\mu }_i+{\epsilon }_i\right)}^2\\ \le {\xi }_{\pi }^{-1}{\left\{\sqrt{n}\cdot \underset{1\le i\le n}{\max }\left|\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)}-\frac{1}{\pi \left({\text{X}}_i\mathrm{,}{U}_i\right)}\right|\right\}}^2\cdot c\cdot \left(\frac{1}{n}{\mu }^{\text{T}}\mu +\frac{1}{n}{\epsilon }^{\text{T}}\epsilon \right)\mathrm{.}\end{array}$

According to (C3) and (C4), to prove (18), it suffices to prove that

${\left(n{\xi }_{\pi }\right)}^{-1}{\epsilon }^{\text{T}}\epsilon =o_p\left(1\right)\text{,}$ (21)

$\sqrt{n}\cdot \underset{1\le i\le n}{\max }\left|\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)}-\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\right)}\right|=O_p\left(1\right)\mathrm{.}$ (22)

By (C2) - (C4) and Markov inequality, for any $\forall \delta >0$ , we have

$\begin{array}{l}P\left\{{\left(n{\xi }_{\pi }\right)}^{-1}{\epsilon }^{{\rm T}}\epsilon >\delta |X\mathrm{,}U\right\}\\ \le {\left(n{\xi }_{\pi }\delta \right)}^{-1}E\left\{{\epsilon }^{{\rm T}}\epsilon \mathrm{<mo>\; |\; </mo>}X\mathrm{,}U\right\}\le {\left(n{\xi }_{\pi }\delta \right)}^{-1}tr\left\{W\right\}\\ \le {\xi }_{\pi }^{-1}{\delta }^{-1}{\lambda }_{\max }\left\{{\Omega }_{e_{\pi }}\right\}\le C_e\cdot {\xi }_{\pi }^{-1}{\delta }^{-1}\stackrel{\text{a}\text{.s}\text{.}}{\to }0.\end{array}$

This together with the dominated convergence theorem indicates ${\left(n{\xi }_{\pi }\right)}^{-1}{\epsilon }^{\text{T}}\epsilon =o_p\left(1\right)$ .

We perform Taylor expansion of ${\left\{\pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)\right\}}^{-1}$ around the true value ${\alpha }_0$ and then obtain

$\begin{array}{l}\sqrt{n}\cdot \underset{1\le i\le n}{\max }\left|\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}{\hat{\alpha }}_n\right)}-\frac{1}{\pi \left(X_i\mathrm{,}{U}_i\right)}\right|\\ =\sqrt{n}\cdot \underset{1\le i\le n}{\max }\left|\frac{{\left(-{\frac{\partial \pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}\alpha \right)}{\partial \alpha }|}_{\alpha ={\alpha }_{n\mathrm{,}{X}_i}}\right)}^{\text{T}}}{\pi \left(X_i\mathrm{,}{U}_i\mathrm{,}{\hat{\alpha }}_n\right)\cdot \pi \left(X_i\mathrm{,}{U}_i\right)}\cdot \left({\hat{\alpha }}_n-{\alpha }_0\right)\right|\\ \le c\cdot \underset{1\le i\le n}{\max }\left|{\frac{\partial \pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}\alpha \right)}{\partial \alpha }|}_{\alpha ={\alpha }_{n\mathrm{,}{X}_i}}\right|\cdot \sqrt{n}\cdot \Vert {\hat{\alpha }}_n-{\alpha }_0\Vert \text{,}\end{array}$

where the last inequality is due to (C1) and Cauchy-Schwarz inequality, and ${\alpha }_{n\mathrm{,}{X}_i}$ is a linear combination between ${\hat{\alpha }}_n$ and ${\alpha }_0$ . Since ${\hat{\alpha }}_n$ is MLE, by (C1) we have $\sqrt{n}\cdot \Vert {\hat{\alpha }}_n-{\alpha }_0\Vert =O_p\left(1\right)$ . Because of the consistency of ${\hat{\alpha }}_n$ and (C1), we have ${\max }_{1\le i\le n}\left|{\frac{\partial \pi \left(X_i\mathrm{,}{U}_i\mathrm{<mo>\; ;\; </mo>}\alpha \right)}{\partial \alpha }|}_{\alpha ={\alpha }_{n\mathrm{,}{X}_i}}\right|=O_p\left(1\right)$ . These results imply (22), this completes the proof of (18) and hence the proof of the Lemma 3.

Lemma 4 If (C2) - (C4) are satisfied, then

$\frac{{\tilde{L}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\inf }_{\omega \in H_n}{\tilde{L}}_{\pi }\left(\omega \right)}-1=o_p\left(1\right)\mathrm{.}$ (23)

Proof of Lemma 4: After some careful calculations, we have

$C{V}_{\hat{\pi }}\left(\omega \right)={\tilde{L}}_{\pi }\left(\omega \right)+2a_{\hat{\pi }}\left(\omega \right)+{\Vert Z_{\hat{\pi }}-\mu \Vert }^2\text{,}$

where

$\begin{array}{c}{a}_{\hat{\pi }}\left(\omega \right)={\epsilon }_{\pi }^{\text{T}}\tilde{A}\left(\omega \right)\mu -{\epsilon }_{\pi }^{\text{T}}\tilde{P}\left(\omega \right){\epsilon }_{\pi }+\frac{1}{2}{\Vert {\tilde{\mu }}_{\pi }\left(\omega \right)-{\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)\Vert }^2\\ +{\left(Z_{\hat{\pi }}-Z_{\pi }\right)}^{\text{T}}\left\{\mu -{\tilde{\mu }}_{\pi }\left(\omega \right)\right\}+{\left(Z_{\hat{\pi }}-Z_{\pi }\right)}^{\text{T}}\left\{{\tilde{\mu }}_{\pi }\left(\omega \right)-{\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)\right\}\\ +{\left\{\mu -{\tilde{\mu }}_{\pi }\left(\omega \right)\right\}}^{\text{T}}\left\{{\tilde{\mu }}_{\pi }\left(\omega \right)-{\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)\right\}+{\epsilon }_{\pi }^{\text{T}}\tilde{P}\left(\omega \right)\left(Z_{\pi }-Z_{\hat{\pi }}\right)\mathrm{.}\end{array}$

Because ${\Vert Z_{\hat{\pi }}-\mu \Vert }^2$ is independent of $\omega$ , then

${\hat{\omega }}_{\hat{\pi }}=\underset{\omega \in H_n}{\arg \min }\left\{{\tilde{L}}_{\pi }\left(\omega \right)+2a_{\hat{\pi }}\left(\omega \right)\right\}\mathrm{.}$

By Lemma 1, we know that, to prove (23) it suffices to prove (13) and

$\underset{\omega \in H_n}{\sup }\frac{\left|a_{\hat{\pi }}\left(\omega \right)\right|}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\mathrm{.}$ (24)

Since (13) has been proved, the following only need to prove (24). By Cauchy-Schwarz inequality, we have

$\begin{array}{c}\left|a_{\hat{\pi }}\left(\omega \right)\right|\le \left|{\epsilon }_{\pi }^{\text{T}}\tilde{A}\left(\omega \right)\mu \right|+\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}\left(\omega \right){\epsilon }_{\pi }\right|+\frac{1}{2}{\Vert {\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)-{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert }^2\\ +\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert {\left\{{\tilde{L}}_{\pi }\left(\omega \right)\right\}}^{1/2}+\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert \cdot \Vert {\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)-{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert \\ +{\left\{{\tilde{L}}_{\pi }\left(\omega \right)\right\}}^{1/2}\Vert {\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)-{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert +\Vert {\left\{\tilde{P}\left(\omega \right)\right\}}^{\text{T}}{\epsilon }_{\pi }\Vert \cdot \Vert Z_{\hat{\pi }}-Z_{\pi }\Vert \text{.}\end{array}$

So to prove (24), it suffices to verify

$\underset{\omega \in H_n}{\sup }\frac{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{A}\left(\omega \right)\mu \right|}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\text{,}$ (25)

$\underset{\omega \in H_n}{\sup }\frac{\left|{\epsilon }_{\pi }^{\text{T}}\tilde{P}\left(\omega \right){\epsilon }_{\pi }\right|}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\text{,}$ (26)

$\underset{\omega \in H_n}{\sup }\frac{{\epsilon }_{\pi }^{\text{T}}\tilde{P}\left(\omega \right){\left\{\tilde{P}\left(\omega \right)\right\}}^{\text{T}}{\epsilon }_{\pi }}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\text{,}$ (27)

$\underset{\omega \in H_n}{\sup }\frac{{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)$ (28)

$\underset{\omega \in H_n}{\sup }\frac{{\Vert {\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)-{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert }^2}{{\tilde{R}}_{\pi }\left(\omega \right)}=o_p\left(1\right)\text{.}$ (29)

The proofs of (25), (26) and (27) are similar to (14), and hold under the condition (C4).

$\begin{array}{c}\underset{\omega \in H_n}{\sup }\frac{{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2}{{\tilde{R}}_{\pi }\left(\omega \right)}=\underset{\omega \in H_n}{\sup }\frac{{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2}{R_{\pi }\left(\omega \right)}\left\{\frac{R_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1+1\right\}\\ \le \underset{\omega \in H_n}{\sup }\frac{{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2}{R_{\pi }\left(\omega \right)}\cdot \underset{\omega \in H_n}{\sup }\left|\frac{R_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1\right|+\underset{\omega \in H_n}{\sup }\frac{{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2}{R_{\pi }\left(\omega \right)}\mathrm{.}\end{array}$ (30)

By Zhang et al. (2013) [14] , under (C2) - (C4), we have

$\underset{\omega \in H_n}{\sup }\left|\frac{R_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\mathrm{,}$

which together with (18) implies (28), and next prove (29).

By Lemma 1 of Wei and Wang (2021) [12] , we have

$\begin{array}{c}{\Vert {\tilde{\mu }}_{\hat{\pi }}\left(\omega \right)-{\tilde{\mu }}_{\pi }\left(\omega \right)\Vert }^2={\Vert \tilde{P}\left(\omega \right)\left(Z_{\hat{\pi }}-Z_{\pi }\right)\Vert }^2\le {\lambda }_{\max }\left\{{\left\{\tilde{P}\left(\omega \right)\right\}}^{{\rm T}}\tilde{P}\left(\omega \right)\right\}{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2\\ \le {\left[{\lambda }_{\max }\left\{\tilde{P}\left(\omega \right)\right\}\right]}^2{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2\le {\left(1+\tilde{p}\right)}^2{\Vert Z_{\hat{\pi }}-Z_{\pi }\Vert }^2\mathrm{.}\end{array}$ (31)

By Zhang et al. (2013) [14] , we know that, under (C2), $\tilde{p}\stackrel{\text{a}\text{.s}\text{.}}{\to }0$ . This together with (31) as well as (28) implies (29), and (25) - (29) together prove (24). This completes the proof of the Lemma 4.

Proof of Theorem 1: Firstly, we note that

$\begin{array}{l}\frac{L_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\inf }_{\omega \in H_n}{L}_{\hat{\pi }}(\omega )}-1=\underset{\omega \in H_n}{\sup }\left\{\frac{L_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)}{L_{\hat{\pi }}\left(\omega \right)}-1\right\}\\ =\underset{\omega \in H_n}{\sup }\left\{\frac{L_{\hat{\pi }}\left({\hat{\omega }}_{\hat{\pi }}\right)}{R_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}\frac{R_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\tilde{R}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}\frac{{\tilde{R}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\tilde{L}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}\frac{{\tilde{L}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\tilde{L}}_{\pi }\left(\omega \right)}\frac{{\tilde{L}}_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}\frac{{\tilde{R}}_{\pi }\left(\omega \right)}{R_{\pi }\left(\omega \right)}\frac{R_{\pi }\left(\omega \right)}{L_{\hat{\pi }}\left(\omega \right)}-1\right\}\\ \le \underset{\omega \in H_n}{\sup }\left(\frac{L_{\hat{\pi }}\left(\omega \right)}{R_{\pi }\left(\omega \right)}\right)\underset{\omega \in H_n}{\sup }\left(\frac{R_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}\right)\underset{\omega \in H_n}{\sup }\left(\frac{{\tilde{R}}_{\pi }\left(\omega \right)}{{\tilde{L}}_{\pi }\left(\omega \right)}\right)\underset{\omega \in H_n}{\sup }\left(\frac{{\tilde{L}}_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}\right)\\ \times \underset{\omega \in H_n}{\sup }\left(\frac{{\tilde{R}}_{\pi }\left(\omega \right)}{R_{\pi }\left(\omega \right)}\right)\underset{\omega \in H_n}{\sup }\left(\frac{R_{\pi }\left(\omega \right)}{L_{\hat{\pi }}\left(\omega \right)}\right)\frac{{\tilde{L}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\inf }_{\omega \in H_n}{\tilde{L}}_{\pi }\left(\omega \right)}-1.\end{array}$

Therefore, to prove Theorem 1, it suffices to prove that

$\underset{\omega \in H_n}{\sup }\left|\frac{R_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\text{,}$ (32)

$\underset{\omega \in H_n}{\sup }\left|\frac{{\tilde{L}}_{\pi }\left(\omega \right)}{{\tilde{R}}_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\text{,}$ (33)

$\underset{\omega \in H_n}{\sup }\left|\frac{L_{\hat{\pi }}\left(\omega \right)}{R_{\pi }\left(\omega \right)}-1\right|=o_p\left(1\right)\text{,}$ (34)

$\frac{{\tilde{L}}_{\pi }\left({\hat{\omega }}_{\hat{\pi }}\right)}{{\inf }_{\omega \in H_n}{\tilde{L}}_{\pi }\left(\omega \right)}-1=o_p\left(1\right)\mathrm{.}$ (35)

According to Zhang et al. (2013) [14] , we know that (32) holds under (C2) - (C4). Then according to Lemma 2, Lemma 3 and Lemma 4, we know that (33) - (35) hold, so the Theorem 1 is proved.

6. Conclusions

In this paper, we extend the JMA method to the nonparametric varying-coefficient models with response missing at random. Firstly, we use the inverse probability weighted method to deal with the missing data, then use B-spline to estimate the nonparametric functions, and finally use jackknife method to select weight $\omega$ . Under certain conditions, the asymptotic optimality of our method is proved.

In this paper, we only consider the varying-coefficient models. Undoubtedly, it is meaningful to extend our ideas to more complex models, such as partially linear models and semiparametric varying-coefficient models. However, this is a very challenging research. Second, for this artical, we assume that the parametric model of the selection probability function is correctly specified, and further research is needed to develop a model averaging method that is robust against the misspecification of the selection probability function. These will be interesting future research directions.

Acknowledgements

The research is supported by NSF projects (ZR2021MA077) of Shandong Province of China.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Hansen, B. (2007) Least Squares Model Averaging. Econometrica, 75, 1175-1189. https://doi.org/10.1111/j.1468-0262.2007.00785.x
[2]	Zhang, X., Wan, A.T.K. and Zou, G. (2008) Least Squares Model Combining by Mallows Criterion. https://doi.org/10.2139/ssrn.1272288
[3]	Wan, A.T.K, Zhang, X. and Zou, G. (2010) Least Squares Model Averaging by Mallows Criterion. Journal of Econometrics, 156, 277-283. https://doi.org/10.1016/j.jeconom.2009.10.030
[4]	Zhang, X., Yu, D., Zou, G. and Liang, H. (2016) Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models. Journal of the American Statistical Association, 111, 1775-1790. https://doi.org/10.1080/01621459.2015.1115762
[5]	Li, J., Xia, X., Wong, W.K. and Nott, D. (2018) Varying-Coefficient Semiparametric Model Averaging Prediction. Biometrics, 74, 1417-1426. https://doi.org/10.1111/biom.12904
[6]	Zhang, X. and Wang, W. (2019) Optimal Model Averaging Estimation for Partially Linear Models. Statistica Sinica, 29, 693-718. https://doi.org/10.5705/ss.202015.0392
[7]	Xia, X. (2021) Model Averaging Prediction for Nonparametric Varying-Coefffcient Models with B-Spline Smoothing. Statistical Papers, 62, 2885-2905. https://doi.org/10.1007/s00362-020-01218-9
[8]	Sun, Z., Su, Z. and Ma, J. (2014) Focused Vector Information Criterion Model Selection and Model Averaging Regression with Missing Response. Metrika, 77, 415-432. https://doi.org/10.1007/s00184-013-0446-8
[9]	Zeng, J., Cheng, W., Hu, G. and Rong, Y. (2018) Model Averaging Procedure for Varying-Coefficient Partially Linear Models with Missing Responses. Journal of the Korean Statistical Society, 47, 379-394. https://doi.org/10.1016/j.jkss.2018.04.004
[10]	Xie, J., Yan, X. and Tang, N. (2021) A Model-Averaging Method for High-Dimensional Regression with Missing Responses At Random. Statistica Sinica, 31, 1005-1026. https://doi.org/10.5705/ss.202018.0297
[11]	Wei, Y., Wang, Q. and Liu, W. (2021) Model Averaging for Linear Models with Responses Missing at Random. Annals of the Institute of Statistical Mathematics, 73, 535-553. https://doi.org/10.1007/s10463-020-00759-y
[12]	Wei, Y. and Wang, Q. (2021) Cross-Validation-Based Model Averaging in Linear Models with Response Missing at Random. Statistics and Probability Letters, 171, 108990. https://doi.org/10.1016/j.spl.2020.108990
[13]	Hansen, B. and Racine, J. (2012) Jackknife Model Averaging. Journal of Econometrics, 167, 38-46. https://doi.org/10.1016/j.jeconom.2011.06.019
[14]	Zhang, X., Wan, A.T.K. and Zou, G. (2013) Model Averaging by Jackknife Criterion in Models with Dependent Data. Journal of Econometrics, 174, 82-94. https://doi.org/10.1016/j.jeconom.2013.01.004
[15]	De Door, C. (2001) A Practical Guide to Splines. Springer, New York.
[16]	Zhu, R., Wan, A.T.K., Zhang, X. and Zou, G. (2019) A Mallows-Type Model Averaging Estimator for the Varying-Coefficient Partially Linear Model. Journal of the American Statistical Association, 114, 882-892. https://doi.org/10.1080/01621459.2018.1456936
[17]	Fan, J., Ma, Y. and Dai, W. (2014) Nonparametric Independent Screening in Sparse Ultra-High Dimensional Varying Coefficient Models. American Statistical Association, 109, 1270-1284. https://doi.org/10.1080/01621459.2013.879828
[18]	Whittle, P. (1960) Bounds for the Moments of Linear and Quadratic Forms in Independent Variables. Theory of Probability & Its Applications, 5, 302-305. https://doi.org/10.1137/1105028