Kernel-Based Partial Conditional Mean Dependence

Zhentao Tian^*, Zhongzhan Zhang

School of Mathematics, Statistics and Mechanics, Beijing University of Technology, Beijing, China.

DOI: 10.4236/ojs.2025.153015                

Abstract

We introduce the Kernel-based Partial Conditional Mean Dependence, a scalar-valued measure of conditional mean dependence of $Y$ given $X$ , while adjusting for the nonlinear dependence on $Z$ . Here $X$ , $Y$ and $Z$ are random elements from arbitrary separable Hilbert spaces. This measure extends the Kernel-based Conditional Mean Dependence. As the estimator of the measure is developed, the concentration property of the estimator is proved. Numerical results demonstrate the effectiveness of the new dependence measure in the context of dependence testing, highlighting their advantages in capturing nonlinear partial conditional mean dependencies.

Keywords

Partial Conditional Mean Dependence, Hilbert Space, High Dimension, Test of Independence

1. Introduction

Before constructing a regression model, it is important to determine whether the covariate $X$ has an effect on the response $Y$ . As pointed out by [1], in most cases, we are more concerned with the conditional mean of the response. Thus, the conditional mean dependence has received attention, which measures the departure of $E\left[Y|X\right]$ from $E\left[Y\right]$ . When $X$ has no effect on the conditional mean of $Y$ , i.e., $E\left[Y|X\right]=E\left[Y\right]$ , $X$ should not be included in a conditional mean regression model. In practice, based on historical analysis or domain knowledge, some covariates related to response will be known. Our aim is to determine whether $X$ has contribution on the conditional mean of $Y$ after controlling the affect from the known variable $Z$ .

Regarding partial dependence, work has been increasing recently. An intuitive approach to measure partial conditional mean dependence is

$E{\left[E\left(Y|X,Z\right)-E\left(Y|Z\right)\right]}^2.$ (1)

Based on a plug-in estimator of equation (1), [2] developed a partial conditional mean independence test. However, as pointed out by [3], under the null hypothesis, i.e., when quantity (1) equals zero, the test statistic has a degenerate distribution. To deal with the degenerate limit distribution, [3] developed a significance test based on the black-box learner, [4] proposed a general framework to evaluate feature importance, and [5] considered measuring the partial dependence based on the decomposition formula of the conditional variance. These methods combine machine learning with sample splitting. Therefore, to a certain extent, they suffer from the loss of power caused by sample splitting. Another issue is that they only consider scalar responses and cannot handle vector or functional responses. In the field of vector or functional data analysis, conditional mean regression is an important analytical tool (see [6] for a regression model with vector response, see [7]-[9]) for regression models with function response among others), and it is necessary to consider the partial conditional mean dependence for vector or functional response.

To our knowledge, among these tools for partial conditional mean dependence, the Partial Martingale Difference Divergence (pMDD), as introduced in [10], is currently the only one applicable to response variables in Hilbert space. pMDD is a scalar-valued measure of conditional mean dependence of $Y$ given $X$ , adjusting for the nonlinear dependence on $Z$ , where $X$ , $Y$ and $Z$ are random vectors of arbitrary dimensions. It extends the martingale difference divergence (MDD) introduced in [11]. However, as shown in [12] [13], the performance of MDD suffers from the curse of dimensionality. Let $\left(X^{\prime },Y^{\prime }\right)$ be an independent copy of $\left(X,Y\right)$ , and let $\text{MDD}\left(Y|X\right)$ be the martingale difference divergence of $Y$ given $X$ . When $X=\left(X_1,\cdots ,X_p\right)\in R^p$ and $Y\in R$ , [12] shows that

$\text{MDD}\left(Y|X\right)\approx \frac{1}{\sqrt{\tau }}{\displaystyle \sum }_{i=1}^p\mathrm{cov}\left(X_i,Y\right),$

where $\tau =E{\Vert X-X^{\prime }\Vert }^2$ , and $\mathrm{cov}\left(X_i,Y\right)$ is the covariance of $X_i$ and $Y$ . Since the covariance only captures the linear dependence, the martingale difference divergence may have less power when it is employed to detect nonlinear relationships, especially in the cases of high dimensions. pMDD, as an extension of MDD, will suffer from the curse of dimensionality for the same reason. This phenomenon can be found in the numerical results in Section 4.

In this paper, we introduce a new tool to measure the partial conditional mean dependence for vector or functional responses. The numerical experiments in [13] demonstrated the advantages of kernel-based conditional mean dependence over MDD in identifying nonlinear relationships and handling high-dimensional variables. This prompts us to develop a tool based on kernel methods for measuring partial conditional mean dependence. Our development follows that in [10], so we name our tool as Kernel-based Partial Conditional Mean Dependence. Simulation results show that Kernel-based Partial Conditional Mean Dependence has an advantage over Partial Martingale Difference Divergence in identifying the nonlinear dependence of $Y$ ’s conditional mean on $X$ after controlling for $Z$ .

The rest of the paper is organized as follows. In Section 2, we review the kernel-based conditional mean dependence measure. In Section 3, we explore the procedure of constructing Kernel-based Partial Conditional Mean Dependence, and give its sample analogy. A group of finite sample simulation studies is carried out in Section 4. In Section 5, some discussions are included. All technical proofs are presented in the Appendix.

2. Kernel-Based Conditional Mean Dependence

Before formally introducing the Kernel-based Partial Conditional Mean Dependence, it is necessary to review a tool for measuring conditional mean Dependence—Kernel-based Conditional Mean Dependence.

As proposed by [13], the kernel-based conditional mean dependence (KCMD) is defined as

$\text{KCMD}\left(Y|Z\right)=E\langle Y-EY,Y^{\prime }-EY\rangle K_Z\left(Z,Z^{\prime }\right),$

where $\mathcal{Y}$ , $\mathcal{Z}$ are separable Hilbert spaces, $Y$ , $Z$ are random elements valued in $\mathcal{Y}$ , $\mathcal{Z}$ , respectively. $\left(Z^{\prime },Y^{\prime }\right)$ is an independent copy of $\left(Z,Y\right)$ , $K_Z\left(\cdot ,\cdot \right)$ is a characteristic kernel (For details on characteristic kernels, please refer to [14] [15]) defined on $\mathcal{Z}\times \mathcal{Z}$ . We specifically point out that in this article, the kernel function $K_Z\left(\cdot ,\cdot \right)$ used in KCMD is not fixed. Its subscript $Z$ indicates that the form and domain of the kernel function depend on $Z$ and the space in which $Z$ is located. Through the paper, $\langle \cdot ,\cdot \rangle$ and $\Vert \cdot \Vert$ represent inner products and norms respectively. Kernel-based conditional mean dependence (KCMD) is intended to measure departure from the relationship

$E\left(Y|Z\right)=E\left(Y\right)\text{almost}\text{surely}$

for $Y\in \mathcal{Y}$ and $Z\in \mathcal{Z}$ . Lemma 1 below summarizes the fundamental properties of KCMD.

Lemma 1. Suppose that $K_Z\left(\cdot ,\cdot \right)$ is a positive definite and bounded characteristic kernel. Then $\text{KCMD}\left(Y|Z\right)$ is well defined, and

a) $\text{KCMD}\left(Y|Z\right)\ge 0$ ;

b) $\text{KCMD}\left(Y|Z\right)=0$ if and only if $E\left(Y|Z\right)=E\left(Y\right)$ , almost surely.

Denote

${\varphi }_Z\left(z,z^{\prime }\right)=K_Z\left(z,z^{\prime }\right)-E{K}_Z\left(z,Z^{\prime }\right)-E{K}_Z\left(Z,z^{\prime }\right)+E{K}_Z\left(Z,Z^{\prime }\right),$ (2)

${\psi }_Y\left(y,y^{\prime }\right)=\langle y-EY,y^{\prime }-EY\rangle =\langle y,y^{\prime }\rangle -E\langle y,Y\rangle -E\langle Y,y^{\prime }\rangle +E\langle Y,Y^{\prime }\rangle ,$

as shown in [13], one can give another expression of the KCMD as follows:

$\text{KCMD}\left(Y|Z\right)=E\left[{\varphi }_Z\left(Z,Z^{\prime }\right){\psi }_Y\left(Y,Y^{\prime }\right)\right].$ (3)

Using the expression, we can provide an unbiased estimator of $\text{KCMD}\left(Y|Z\right)$ . It is closely related to the so-called $\mathcal{U}$ -centred matrix.

$\left(\tilde{A}\cdot \tilde{B}\right)=\frac{1}{n\left(n-3\right)}{\displaystyle \sum }_{i\ne j}{\tilde{A}}_{ij}{\tilde{B}}_{ij}$ (4)

and $\left|\tilde{A}\right|={\left(\tilde{A}\cdot \tilde{A}\right)}^{1/2}$ as the norm of $\tilde{A}$ . Theorem 1 in [16] shows that the linear span of all matrices in $H_n$ is a Hilbert space with inner product defined in (4). Using the $\mathcal{U}$ -centered matrices, we can construct an estimator of $\text{KCMD}\left(Y|Z\right)$ . Given independent and identically distributed (i.i.d) observations ${\left(Z_i,Y_i\right)}_{i=1}^n$ from the joint distribution of $\left(Z,Y\right)$ , an unbiased estimator of $\text{KCMD}\left(Y|Z\right)$ provided in [13] is defined as

$\widehat{\text{KCMD}}\left(Y|Z\right)=\frac{1}{n\left(n-3\right)}{\displaystyle \sum }_{i\ne j}{\tilde{A}}_{ij}{\tilde{B}}_{ij}.$ (5)

Here, ${\tilde{A}}_{ij}$ and ${\tilde{B}}_{ij}$ are the $\mathcal{U}$ -centred versions of matrixes $A$ and $B$ respectively, and the $\left(i,j\right)$ -th elements of $A$ is $a_{ij}=K_Z\left(Z_i,Z_j\right)$ , the $\left(i,j\right)$ -th elements of $B$ is $b_{ij}={\Vert Y_i-Y_j\Vert }^2/2$ . [13] has shown that the estimator (5) is unbiased and admits a U-statistic expression

$\widehat{\text{KCMD}}\left(Y|Z\right)=\frac{1}{C_n^4}{\displaystyle \sum }_{i<j<s<t}h\left(V_i,V_j,V_s,V_t\right),$ (6)

with the kernel

$h\left(V_i,V_j,V_s,V_t\right)=\frac{1}{4!}{\displaystyle \sum }_{\left(u,v,q,r\right)}^{\left(i,j,s,t\right)}\left(a_{uv}{b}_{uv}-a_{uv}{b}_{uq}-a_{uv}{b}_{vr}+a_{uv}{b}_{qr}\right),$

where $V_i=\left(Z_i,Y_i\right)$ and the sum is over all 4! permutations of $\left(i,j,s,t\right)$ .

3. Kernel-Based Partial Conditional Mean Dependence

In this section, we introduce the kernel-based partial conditional mean Dependence (partial KCMD), which can measure the conditional mean dependence of a response $Y$ given a predictor variable $X$ after controlling some variable $Z$ , where $X$ is an random element valued in the separable Hilbert space $\mathcal{X}$ .

3.1. Population Partial KCMD

For any symmetric function $f\left(g,g^{\prime }\right)$ defined on $\mathcal{G}\times \mathcal{G}$ , define $D$ as a $\mathcal{U}$ -centered operator, and

$Df\left(g,g^{\prime }\right)=f\left(g,g^{\prime }\right)-Ef\left(g,G^{\prime }\right)-Ef\left(G,g^{\prime }\right)+Ef\left(G,G^{\prime }\right),$

where $G$ is a random element valued in Hilbert space $\mathcal{G}$ , $G^{\prime }$ is an independent copy of $G$ . The functional class $ℱ=\left\{Df:f\text{is}\text{a}\text{symmetric}\text{function}\right\}$ is a linear space. Define the inner product on $ℱ$ as

$D{f}_1\circ D{f}_2=ED{f}_1\left(G,G^{\prime }\right)D{f}_2\left(G,G^{\prime }\right).$

It can be verified that the map satisfies the conditions of an inner product. In addition, define the norm on $ℱ$ as $\Vert Df\Vert =\sqrt{Df\circ Df}$ . Take $\mathcal{G}=\mathcal{Z}\times \mathcal{Y}$ . Let $K_Z\left(\left(z,y\right),\left(z^{\prime },y^{\prime }\right)\right)=K_Z\left(z,z^{\prime }\right)$ and ${\varphi }_Z\left(\left(z,y\right),\left(z^{\prime },y^{\prime }\right)\right)={\varphi }_Z\left(z,z^{\prime }\right)$ for any $\left(z,y\right),\left(z^{\prime },y^{\prime }\right)\in \mathcal{Z}\times \mathcal{Y}$ . Then ${\varphi }_Z\in ℱ$ . Similarly, ${\psi }_Y\in ℱ$ . Thus $\text{KCMD}\left(Y|Z\right)$ in equation (3) can also be written as

$\text{KCMD}\left(Y|Z\right)={\varphi }_Z\circ {\psi }_Y.$ (7)

This implies that the KCMD measures the conditional mean dependence of $Y$ on $Z$ through the inner product of ${\varphi }_Z$ and ${\psi }_Y$ in a linear space. The cosine value

$\text{cos}\left({\varphi }_Z,{\psi }_Y\right)=\frac{{\varphi }_Z\circ {\psi }_Y}{\Vert {\varphi }_Z\Vert \cdot \Vert {\psi }_Y\Vert }$

measures the strength of conditional mean dependence.

Lemma 2. Suppose $\Vert {\varphi }_Z\Vert \ne 0$ and $\Vert {\psi }_Y\Vert \ne 0$ , then ${\psi }_Y$ can be decomposed into two orthogonal parts,

${\psi }_Y=\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z+\left({\psi }_Y-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\right).$ (8)

The first term in (8) represents the part of $Y$ ’s conditional mean that is affected by $Z$ , the second term represents the part that can not be interpreted by $Z$ , and in a sense it corresponds to $U=Y-E\left(Y|Z\right)$ since $E\left(U|Z\right)=0$ . Next we define $W=\left(X,Z\right)$ and $\mathcal{W}=\mathcal{X}\times \mathcal{Z}$ . One way to measure the additional contribution of $X$ to the conditional mean of $Y$ controlling for $Z$ , is to measure $E\left(U|W\right)$ .

Inspired by [10], we provide definitions for the Kernel-based Partial Conditional Mean Dependence and the Kernel-based Partial Conditional Mean Correlation. Define ${\varphi }_W$ similar to ${\varphi }_Z$ , replacing $K_Z\left(z,z^{\prime }\right)$ with $K_W\left(w,w^{\prime }\right)$ in ${\varphi }_Z$ is ${\varphi }_W$ .

Definition 1. The population partial KCMD of $Y$ given $X$ , after controlling for the effect of $Z$ , i.e., $\text{pKCMD}\left(Y|X;Z\right)$ is defined as

$\text{pKCMD}\left(Y|X;Z\right)={\varphi }_W\circ \left({\psi }_Y-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\right).$

If $\Vert {\varphi }_Z\Vert =0$ , then we define $\text{pKCMD}\left(Y|X;Z\right)={\varphi }_W\circ {\psi }_Y$ .

The population Kernel-based Partial Conditional Mean Correlation(pKCMC) is defined as

$\text{pKCMC}\left(Y|X;Z\right)=\frac{\text{pKCMD}\left(Y|X;Z\right)}{\Vert {\varphi }_W\Vert \cdot \Vert {\psi }_Y-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\Vert }.$

If $\Vert {\varphi }_W\Vert \cdot \Vert {\psi }_Y-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\Vert =0$ , then we define $\text{pKCMC}\left(Y|X;Z\right)=0$ .

After performing some straightforward calculations, we obtain an equivalent expression for pKCMD, which is given by

$\begin{array}{c}\text{pKCMD}\left(Y|X;Z\right)={\varphi }_W\circ {\psi }_Y-\frac{\left({\varphi }_Z\circ {\psi }_Y\right)\left({\varphi }_W\circ {\varphi }_Z\right)}{{\Vert {\varphi }_Z\Vert }^2}\\ =\text{KCMD}\left(Y|W\right)-\frac{\text{KCMD}\left(Y|Z\right)\text{HSIC}\left(Z,W\right)}{\text{HSIC}\left(Z,Z\right)}.\end{array}$ (9)

The $\text{HSIC}\left(Z,W\right)$ is the Hilbert-Schmidt Independence Criterion(HSIC) between $Z$ and $W$ , it measures the dependence between these two random elements. The kernel functions used in $\text{HSIC}\left(Z,W\right)$ are $K_Z\left(\cdot ,\cdot \right):\mathcal{Z}\times \mathcal{Z}\to R$ and $K_W\left(\cdot ,\cdot \right):\mathcal{W}\times \mathcal{W}\to R$ , which, like in KCMD, depend on the variables in their subscripts. The content about HSIC can be found in [15] [17]-[19] and so on. We reviewed the specific form of HSIC in the Appendix and derive the last equation of (9). When the conditional mean of $Y$ does not depend on $Z$ or $Z$ is a constant, we have $\text{KCMD}\left(Y|Z\right)=0$ or $\text{HSIC}\left(Z,Z\right)=0$ . As a result, we have $\text{pKCMD}\left(Y|X;Z\right)=\text{KCMD}\left(Y|W\right)=\text{KCMD}\left(Y|X\right)$ .

3.2. Sample pKCMD

Given the sample ${\left(X_i,Y_i,Z_i\right)}_{i=1}^n$ , we want to define sample partial KCMD, denoted as ${\text{pKCMD}}_n\left(Y|X;Z\right)$ as the sample analog of population partial KCMD. Let $W_i=\left(X_i,Z_i\right)$ , and define $\tilde{C}$ be $n\times n$ matrix with entries ${\tilde{C}}_{ij}$ ,

${\tilde{C}}_{ij}=\{\begin{array}{ll}{c}_{ij}-c_{i\cdot }-c_{\cdot j}+c_{\cdot \cdot },\hfill & i\ne j\hfill \\ 0,\hfill & i=j\hfill \end{array},$

where $c_{ij}=K_W\left(W_i,W_j\right)$ , and $c_{i\cdot }$ , $c_{\cdot j}$ and $c_{\cdot \cdot }$ are defined similarly to $a_{i\cdot }$ , $a_{\cdot j}$ and $a_{\cdot \cdot }$ .

Definition 2. Given a random sample from the joint distribution $\left(X,Y,Z\right)$ , the sample partial kernel-based conditional mean dependence of $Y$ given $X$ , after controlling for the effect of $Z$ , is given by

${\text{pKCMD}}_n\left(Y|X;Z\right)=\left(\tilde{C}\cdot \tilde{B}\right)-\frac{\left(\tilde{A}\cdot \tilde{B}\right)\left(\tilde{A}\cdot \tilde{C}\right)}{\left(\tilde{A}\cdot \tilde{A}\right)},$

assuming $\left(\tilde{A}\cdot \tilde{A}\right)\ne 0$ and $\left(\tilde{C}\cdot \tilde{B}\right)$ otherwise. The sample partial kernel-based conditional mean correlation is given by

${\text{pKCMC}}_n\left(Y|X;Z\right)=\frac{{\left|\tilde{A}\right|}^2{\text{pKCMD}}_n\left(Y|X;Z\right)}{\left|\tilde{C}\right|\cdot \left|{\left(\left|\tilde{A}\right|\right)}^2\tilde{B}-\left(\tilde{A}\circ \tilde{B}\right)\tilde{A}\right|}$

If $\left|\tilde{C}\right|\cdot \left|{\left(\left|\tilde{A}\right|\right)}^2\tilde{B}-\left(\tilde{A}\circ \tilde{B}\right)\tilde{A}\right|=0$ , then we define ${\text{pKCMC}}_n\left(Y|X;Z\right)=0$ .

We next outline theoretical properties of the sample pKCMD. Analogous results hold for the sample pKCMC, which we omit discussing further here.

Theorem 1. If one of the following two conditions holds,

a) $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ are bounded kernels, $E\Vert Y\Vert <\infty$ ;

b) $E{K}_Z^2\left(Z,Z^{\prime }\right)<\infty$ , $E{K}_W^2\left(W,W^{\prime }\right)<\infty$ , and $E{\Vert Y\Vert }^2<\infty$ .

Then, as $n\to \infty$ , we have ${\text{pKCMD}}_n\left(Y|X;Z\right)\to \text{pKCMD}\left(Y|X;Z\right)$ a.s..

We also show that ${\text{pKCMD}}_n\left(Y|X;Z\right)$ is concentrated. To obtain the bounds of the deviation ${\text{pKCMD}}_n\left(Y|X;Z\right)-\text{pKCMD}\left(Y|X;Z\right)$ , we impose the following condition.

(C1) There exists a constant $s_0>0$ such that for all $0<s\le 2s_0$ , $E\exp \left(s{\Vert Y\Vert }^2\right)<\infty$ .

Condition (C1) follows immediately when $Y$ is bounded uniformly, or when it has a Gaussian distribution. Condition (C1) is widely used in statistical research, for example, in [11] [20] [21], to analyze the theoretical properties of feature screening.

Theorem 2. If $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ are bounded kernels, $E\Vert Y\Vert <\infty$ , and Condition (C1) holds, then for any $ϵ>0$ , there exist constants $0<\beta <1/2$ , $r_1>0$ and $r_2>0$ such that

$\begin{array}{l}P\left(\left|{\text{pKCMD}}_n\left(Y|X;Z\right)-\text{pKCMD}\left(Y|X;Z\right)\right|\ge ϵ\right)\\ \le O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right).\end{array}$

Take $ϵ=\eta n^{-\gamma }$ with $0<\gamma <1/2$ and a constant $\eta >0$ . According to Theorem 2, there exists $0<\beta +\gamma <1/2$ , such that $P\left(\left|{\text{pKCMD}}_n\left(Y|X;Z\right)-\text{pKCMD}\left(Y|X;Z\right)\right|\ge \eta n^{-\gamma }\right)=o\left(1\right)$ . This implies that ${\text{pKCMD}}_n\left(Y|X;Z\right)$ is concentrated, and the deviation between ${\text{pKCMD}}_n\left(Y|X;Z\right)$ and $\text{pKCMD}\left(Y|X;Z\right)$ is less than $\eta n^{-\gamma }$ with probability at least $1-O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right)$ .

4. Simulation

In the section, we examine tests of the null hypothesis of zero pKCMD. When calculating pKCMD, we need to choose kernel functions $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ . We use Gaussian kernels for $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ , defined as

$K_Z\left(z_i,z_j\right)=\text{exp}\left\{-{\Vert z_i-z_j\Vert }^2/\left(2{\sigma }_Z^2\right)\right\}$

and

$K_W\left(w_i,w_j\right)=\text{exp}\left\{-{\Vert w_i-w_j\Vert }^2/\left(2{\sigma }_W^2\right)\right\},$

respectively. For the choice of bandwidths ${\sigma }_Z$ and ${\sigma }_W$ for these kernels, we can use median heuristic ([13]). We compare our proposed method with pMDD introduced in [10]. We use permutation to obtain the critical values and take the permutation number as 300. The permutation method is described in Section 5 of [10].

The simulations take into account varying sample sizes and levels of dependence between the two random variables to evaluate the performance of tests. For each setting, the empirical sizes or powers of the tests (represented by the proportions of rejections) are recorded through 1000 repetitions at different significance levels.

Example 1 Generate the i.i.d. sample of $\left(X,Y,Z\right)$ from the following model: $X=\left(X_1,\cdots ,X_p\right)$ , $Z=\left(Z_1,\cdots ,Z_p\right)$ , $Y=\cos \left(Z\right)$ , where $\cos \left(Z\right)=\left(\cos \left(Z_1\right),\cdots ,\cos \left(Z_p\right)\right)$ . Consider two scenarios:

1) $X_i~N\left(0,1\right)$ , and $Z_i~N\left(0,1\right)$ .

2) $X_1,\cdots ,X_p$ are independent and identically distributed (i.i.d.) random variables from the Cauchy distribution with location parameter 0 and scale parameter 1. $Z_1,\cdots ,Z_p$ are i.i.d. random variables from the standard normal distribution.

In this example, $X$ and $Z$ are independent of each other, and $Y$ depends only on $Z$ . Thus, after controlling for the third random vector $Z$ , the conditional mean of $Y$ is independent of $X$ . From Table 1, both methods can reasonably control the type-I error rate.

Table 1. Empirical size of the two tests for Example 1 with $n=50$ and $p=5$ .

Scenario	Method	$\alpha =0.01$	$\alpha =0.05$	$\alpha =0.10$
(1)	pKCMD	0.014	0.050	0.098
	pMDD	0.011	0.048	0.105
(2)	pKCMD	0.009	0.051	0.091
	pMDD	0.010	0.059	0.111

Example 2 Generate the i.i.d. sample of $\left(X,Y,Z\right)$ from the following model: $X=\left(X_1,\cdots ,X_p\right)~N\left(0,I_p\right)$ , $Z=\left(Z_1,\cdots ,Z_p\right)~N\left(0,I_p\right)$ , $Y=f\left(0.6X+Z\right)$ , where $N\left(0,I_p\right)$ is a multivariate normal distribution with zero mean and identity covariance matrix $I_p$ , and $f\left(x\right)=\left(f\left(x_1\right),\cdots ,f\left(x_p\right)\right)$ for any $x=\left(x_1,\cdots ,x_p\right)$ .

We consider the following four relationships for $f\left(x\right)$ : a) $f\left(x\right)=x$ , b) $f\left(x\right)=x^2$ , c) $f\left(x\right)=\text{sin}\left(x\right)$ , and d) $f\left(x\right)=\text{cos}\left(x\right)$ .

Table 2. Empirical powers of the two tests for Example 2 with $p=2$ .

Relationship	$\alpha$	Method	$n=10$	$n=15$	$n=20$	$n=25$	$n=30$
$f\left(x\right)=x$	0.01	pKCMD	0.271	0.492	0.664	0.780	0.843
		pMDD	0.424	0.667	0.810	0.900	0.939
	0.05	pKCMD	0.470	0.669	0.801	0.878	0.908
		pMDD	0.606	0.784	0.893	0.943	0.972
	0.10	pKCMD	0.573	0.749	0.856	0.911	0.933
		pMDD	0.698	0.836	0.924	0.955	0.979
$f\left(x\right)=x^2$	0.01	pKCMD	0.050	0.102	0.158	0.288	0.421
		pMDD	0.028	0.051	0.055	0.085	0.111
	0.05	pKCMD	0.165	0.287	0.398	0.538	0.688
		pMDD	0.101	0.141	0.166	0.223	0.273
	0.10	pKCMD	0.270	0.420	0.551	0.696	0.810
		pMDD	0.180	0.238	0.284	0.346	0.420
$f\left(x\right)=\text{sin}\left(x\right)$	0.01	pKCMD	0.235	0.480	0.694	0.835	0.909
		pMDD	0.241	0.474	0.676	0.793	0.897
	0.05	pKCMD	0.456	0.692	0.843	0.923	0.953
		pMDD	0.465	0.673	0.828	0.904	0.957
	0.10	pKCMD	0.583	0.778	0.905	0.949	0.966
		pMDD	0.602	0.772	0.886	0.940	0.972
$f\left(x\right)=\text{cos}\left(x\right)$	0.01	pKCMD	0.044	0.087	0.158	0.251	0.401
		pMDD	0.027	0.052	0.051	0.084	0.097
	0.05	pKCMD	0.157	0.276	0.383	0.543	0.679
		pMDD	0.089	0.143	0.168	0.217	0.255
	0.10	pKCMD	0.254	0.416	0.542	0.707	0.816
		pMDD	0.166	0.230	0.279	0.356	0.428

This example compares the empirical powers of pKCMD and pMDD across different functional relationships, significance levels, and sample sizes. According to Table 2, for the linear function $f\left(x\right)=x$ , pMDD consistently outperforms pKCMD, showing higher sensitivity. In the quadratic $f\left(x\right)=x^2$ and cosine $f\left(x\right)=\text{cos}\left(x\right)$ relationships, pKCMD generally demonstrates superior power, especially at larger samples, indicating better non-linear effect detection. For $f\left(x\right)=\text{sin}\left(x\right)$ relationship, both tests perform comparably well, with slight advantages for pMDD at lower significance levels. Overall, pMDD excels with linear relationships, while pKCMD is preferable for non-linear ones, particularly with larger sample sizes.

Example 3 Consider the model in Example 2, set $f\left(x\right)=\cos \left(x\right)$ , $n=400$ .

Table 3 compares the empirical powers of pKCMD and pMDD at varying significance levels $\alpha$ and varying dimension $p$ , with $n=400$ . pKCMD consistently outperforms pMDD across all settings, with its power decreasing more gradually as $p$ increases. At lower significance levels ($\alpha =0.01$ ), pKCMD maintains high power even with $p=20$ , while pMDD’s power drops sharply. At higher $\alpha$ , both tests improve, but pKCMD remains superior, especially for larger $p$ . Overall, according to the power shown in Table 3, pKCMD is more robust and powerful, particularly in high-dimensional contexts.

Table 3. Empirical powers of the two tests for Example 3 with $n=400$ .

$\alpha$	Method	$p=10$	$p=20$	$p=30$	$p=40$	$p=50$
0.01	pKCMD	1.000	0.960	0.670	0.399	0.281
	pMDD	0.933	0.385	0.183	0.101	0.091
0.05	pKCMD	1.000	0.995	0.863	0.638	0.506
	pMDD	0.986	0.647	0.413	0.262	0.222
0.10	pKCMD	1.000	0.999	0.937	0.761	0.662
	pMDD	0.998	0.771	0.552	0.394	0.336

Example 4 Consider two models with the formula $Y\left(t\right)=f\left(X\left(t\right)\right)+g\left(Z\left(t\right)\right)$ , where $Z\left(t\right)$ is generated by Wiener process. Two processes for $X$ , including Ornstein-Uhlenbeck process (OU) and Gaussian process with exponential variogram (VP), are employed, and they are generated by rproc2f data function in R package fda.usc with default parameters. The models for generating $Y\left(t\right)$ are as follows:

1) $Y\left(t\right)=1.5X^2\left(t\right)+Z\left(t\right)$ , $t\in \left[0,1\right]$ .

2) $Y\left(t\right)=2\text{cos}\left(X\left(t\right)\right)+0.5Z^3\left(t\right)$ , $t\in \left[0,1\right]$ .

Table 4 and Table 5 reveal the performance of pKCMD and pMDD in detecting partial conditional mean dependencies under two distinct models of $Y\left(t\right)$ . For the quadratic model in (1), pKCMD consistently outperforms pMDD across varying sample sizes and significance levels, with its empirical power improving significantly as the sample size increases. For instance, at $\alpha =0.05$ , pKCMD achieves a power of 0.943 and 0.952 for $n=50$ with the OU and VP processes, respectively, while pMDD attains only 0.741 and 0.591. This indicates that pKCMD is more effective in capturing the quadratic relationship, and the choice between the OU and VP processes does not substantially affect the relative performance of the tests. For the cosine model in (2), similar trends are observed, with pKCMD maintaining higher power than pMDD across all conditions. The empirical power of both tests increases with the sample size. Although pKCMD shows slightly higher power for the OU process compared to the VP process at larger sample sizes, the difference is not substantial, suggesting that the tests’ power is generally robust to the underlying stochastic process of $X\left(t\right)$ .

Overall, our analysis shows that the tests based on Kernel-based Partial Conditional Mean Dependence perform effectively in most of the situations we examined. As the sample size grows, the power of these tests increases. When compared to pMDD, our pKCMD test proves to be more efficient at capturing nonlinear relationships. Importantly, even when the dimension of variables increases, the decline in our test’s power is significantly slower compared to other tests.

Table 4. Empirical powers of the two tests for Example 4 (1).

$\alpha$	Method	$n=30$		$n=50$
		OU	VP	OU	VP
0.01	pKCMD	0.606	0.561	0.884	0.863
	pMDD	0.216	0.237	0.449	0.389
0.05	pKCMD	0.777	0.760	0.943	0.952
	pMDD	0.409	0.402	0.741	0.591
0.10	pKCMD	0.835	0.848	0.957	0.973
	pMDD	0.543	0.498	0.845	0.689

Table 5. Empirical powers of the two tests for Example 4 (2).

$n$	Method	$\alpha =0.01$		$\alpha =0.05$		$\alpha =0.10$
		OU	VP	OU	VP	OU	VP
30	pKCMD	0.388	0.274	0.526	0.416	0.593	0.519
	pMDD	0.139	0.116	0.320	0.230	0.438	0.328
50	pKCMD	0.472	0.378	0.583	0.542	0.643	0.628
	pMDD	0.206	0.147	0.428	0.309	0.529	0.429
70	pKCMD	0.575	0.459	0.671	0.588	0.730	0.671
	pMDD	0.350	0.219	0.541	0.415	0.638	0.548
90	pKCMD	0.639	0.516	0.743	0.658	0.785	0.742
	pMDD	0.435	0.338	0.644	0.550	0.711	0.633

5. Real Data

In this section, we consider exploring the Tecator dataset contained in R package fda.usc. This dataset includes values of a 100-channel spectrum (of wavelength 850 - 1050 mm) of absorbance ($Z$ ), water content ($X_1$ ), fat content ($Y$ ), and protein content ($X_2$ ) for 215 meat samples. This dataset has been widely studied in functional data analysis, and this literature mainly focuses on characterizing the influence of functional covariate $Z$ on scalar response $Y$ . Our goal is to determine whether the other two variables, $X_1$ and $X_2$ , have an impact on the conditional mean of the response after controlling for the influence of $Z$ . For $X_1$ , the p-values computed by pKCMD and pMDD all are 0.000, for $X_2$ , we obtain the same results. These values means that when constructing a conditional mean regression model, $X_1$ and $X_2$ should also be considered. [22] has already considered the influence of $X_1$ , $X_2$ , and $Z$ on $Y$ through a semi-functional partial linear regression.

6. Conclusion

This paper introduces pKCMD, a novel test for detecting partial conditional mean dependencies in Hilbert spaces, extending existing measures. We derive equivalent expressions for pKCMD at both population and sample levels. Numerical experiments show that pKCMD consistently outperforms pMDD across sample sizes and significance levels, particularly in nonlinear relationships. From the simulation results, compared with pMDD, pKCMD is more robust and performs better in high-dimensional situations without more computational loss. Overall, pKCMD is a competitive, reliable method for analyzing partial conditional mean dependencies, especially in nonlinear settings. How to choose the optimal kernel function in dependency testing is an important issue. This is our future research direction. In addition, we are also considering extending pKCMD to other tasks, such as conditional independence test, goodness-of-fit test, and feature screening, to broaden its applicability.

Appendix

Appendix A1. Hilbert-Schmidt Independent Criterion and Equation (9)

Definition A1. (HSIC) The Hilbert-Schmidt Independent Criterion of random elements $Z\in \mathcal{Z}$ and $W\in \mathcal{W}$ is defined as

$\begin{array}{c}\text{HSIC}\left(Z,W\right)=E\left[K_Z\left(Z,Z^{\prime }\right)K_W\left(W,W^{\prime }\right)\right]+E\left[K_Z\left(Z,Z^{\prime }\right)\right]E\left[K_W\left(W,W^{\prime }\right)\right]\\ -2E\left\{E\left[K_Z\left(Z,Z^{\prime }\right)|Z\right]E\left[K_W\left(W,W^{\prime }\right)|W\right]\right\},\end{array}$

where $\left(Z^{\prime },W^{\prime }\right)$ is an independent copy of $\left(Z,W\right)$ , $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ are two kernel functions.

Note that in Definition A1, the kernel functions $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ are mutable and depend on $Z$ and $W$ , respectively. HSIC was developed to test the independence of $Z$ and $W$ , and has many good properties. a) It is non negative; b) If $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ are characteristic, then $\text{HSIC}\left(Z,W\right)=0$ if and only if $Z$ and $W$ are independent. Using the symbol ${\varphi }_Z$ defined in (2), symbol ${\varphi }_W$ is similar to ${\varphi }_Z$ , but the kernel function used in $K_W\left(\cdot ,\cdot \right)$ is ${\varphi }_W$ , we have

$\begin{array}{l}{\varphi }_Z\circ {\varphi }_W\\ =E{\varphi }_Z\left(Z,Z^{\prime }\right){\varphi }_W\left(W,W^{\prime }\right)\\ =E(\left\{K_Z\left(Z,Z^{\prime }\right)-E\left[K_Z\left(Z,Z^{\prime }\right)|Z\right]-E\left[K_Z\left(Z,Z^{\prime }\right)|Z^{\prime }\right]+E{K}_Z\left(Z,Z^{\prime }\right)\right\}\\ \times \left\{K_W\left(W,W^{\prime }\right)-E\left[K_W\left(W,W^{\prime }\right)|W\right]-E\left[K_W\left(W,W^{\prime }\right)|W^{\prime }\right]+E{K}_W\left(W,W^{\prime }\right)\right\})\\ =E\left[K_Z\left(Z,Z^{\prime }\right)K_W\left(W,W^{\prime }\right)\right]+E\left[K_Z\left(Z,Z^{\prime }\right)\right]E\left[K_W\left(W,W^{\prime }\right)\right]\\ -2E\left\{E\left[K_Z\left(Z,Z^{\prime }\right)|Z\right]E\left[K_W\left(W,W^{\prime }\right)|W\right]\right\}\\ =\text{HSIC}\left(Z,W\right),\end{array}$

thus (9) holds.

Appendix A2. The Proofs of Main Results

Proof of Lemma 1. The Lemma 1 is the Proposition 1 in [13]. □

Proof of Lemma 2. Because

$\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\left(z,z^{\prime }\right)\in ℱ,$

and

$\begin{array}{l}\left({\psi }_Y-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\right)\left(\left(y,z\right),\left(y^{\prime },z^{\prime }\right)\right)\\ ={\psi }_Y\left(y,y^{\prime }\right)-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\left(z,z^{\prime }\right)\in ℱ,\end{array}$

their inner product is

$\begin{array}{l}\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\circ \left({\psi }_Y-\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }{\varphi }_Z\right)\\ =\frac{\Vert {\psi }_Y\Vert \text{cos}\left({\varphi }_Z,{\psi }_Y\right)}{\Vert {\varphi }_Z\Vert }\left({\varphi }_Z\circ {\psi }_Y\right)-\frac{{\Vert {\psi }_Y\Vert }^2{\left[\text{cos}\left({\varphi }_Z,{\psi }_Y\right)\right]}^2}{{\Vert {\varphi }_Z\Vert }^2}\left({\varphi }_Z\circ {\varphi }_Z\right)\\ ={\Vert {\psi }_Y\Vert }^2\text{cos}\left({\varphi }_Z,{\psi }_Y\right)\times \frac{\left({\varphi }_Z\circ {\psi }_Y\right)}{\Vert {\varphi }_Z\Vert \cdot \Vert {\psi }_Y\Vert }-{\Vert {\psi }_Y\Vert }^2{\left[\text{cos}\left({\varphi }_Z,{\psi }_Y\right)\right]}^2\\ =0,\end{array}$

thus these two parts in (8) are orthogonal. □

Proof of Theorem 1. a) When $K_Z\left(\cdot ,\cdot \right)$ and $K_W\left(\cdot ,\cdot \right)$ are bounded kernels. We suppose that $K_Z\left(\cdot ,\cdot \right)<M<\infty$ and $K_W\left(\cdot ,\cdot \right)<M<\infty$ . First, we consider $\left(\tilde{A}\cdot \tilde{B}\right)$ in ${\text{pKCMD}}_n\left(Y|X;Z\right)$ . Because $\left(\tilde{A}\cdot \tilde{B}\right)$ is an unbiased estimate of ${\varphi }_Z\circ {\psi }_Y$ and can be written as an U-statistic by equation (6), and

$\begin{array}{c}E\left|h\left(V_1,V_2,V_3,V_4\right)\right|\le E\left|a_{uv}\left(b_{uv}-b_{uq}-b_{vr}+b_{qr}\right)\right|\\ \le ME\left|b_{uv}-b_{uq}-b_{vr}+b_{qr}\right|\\ \le 4ME\left|b_{uv}\right|\\ \le 4ME\Vert Y\Vert E\Vert Y^{\prime }\Vert \\ <\infty .\end{array}$

By the strong law of large number for U-statistics,

$\left(\tilde{A}\cdot \tilde{B}\right)\to {\varphi }_Z\circ {\psi }_Y,a.s\mathrm{..}$

Similarly, we have

$\left(\tilde{C}\cdot \tilde{B}\right)\to {\varphi }_W\circ {\psi }_Y,a.s.,$

$\left(\tilde{A}\cdot \tilde{C}\right)\to {\varphi }_Z\circ {\varphi }_W,a.s.,$

$\left(\tilde{A}\cdot \tilde{A}\right)\to {\varphi }_Z\circ {\psi }_Z,a.s\mathrm{..}$

According to the continuous mapping theorem, we have $\text{pKCMD}\left(Y|X;Z\right)\to \text{pKCMD}\left(Y|X;Z\right)$ a.s..

b) For general kernel.

$\begin{array}{c}E\left|h\left(V_1,V_2,V_3,V_4\right)\right|\le E\left|a_{uv}\left(b_{uv}-b_{uq}-b_{vr}+b_{qr}\right)\right|\\ \le E{a}_{uv}{b}_{uv}+E{a}_{uv}{b}_{uq}+E{a}_{uv}{b}_{vr}+E{a}_{uv}{b}_{qr}\\ \le 2E{a}_{uv}^2+2E{b}_{uv}^2\\ \le 2E{K}_Z^2\left(Z_u,Z_v\right)+2{\left(E{\Vert Y\Vert }^2\right)}^2\\ \left(\text{because}E{K}_Z^2\left(Z,Z^{\prime }\right)<\infty \text{and}E{\Vert Y\Vert }^2<\infty \right)\\ <\infty .\end{array}$

Similar to (a), we can also prove that $\text{pKCMD}\left(Y|X;Z\right)\to \text{pKCMD}\left(Y|X;Z\right)$ a.s.. This completes the proof. □

Proof of Theorem 2. Denote $w={\varphi }_Z\circ {\psi }_Y$ and $\widehat{w}=\widehat{\text{KCMD}}\left(Y|Z\right)$. By employing the Markov inequality, we obtain that for any $ϵ>0$ and $t>0$ ,

$\begin{array}{c}P\left(\widehat{w}-w>ϵ\right)=P\left(\text{exp}\left(\widehat{w}t-wt\right)>\text{exp}\left(ϵt\right)\right)\\ \le \text{exp}\left(-ϵt\right)E\exp \left(\widehat{w}t-wt\right)\\ =\text{exp}\left(-ϵt-wt\right)E\exp \left(\widehat{w}t\right).\end{array}$

Note that $\widehat{w}$ admits a U-statistic expression

$\widehat{w}=\frac{1}{\left(\begin{array}{l}n\hfill \\ 4\hfill \end{array}\right)}{\displaystyle \sum }_{i<j<s<t}h\left(V_i,V_j,V_s,V_t\right),$

with the kernel $h\left(V_i,V_j,V_s,V_t\right)=\frac{1}{4!}{\displaystyle {\sum }_{\left(u,v,q,r\right)}^{\left(i,j,s,t\right)}\left(a_{uv}{b}_{uv}-a_{uv}{b}_{uq}-a_{uv}{b}_{vr}+a_{uv}{b}_{qr}\right)}$ , where

$V_i=\left(Z_i,Y_i\right)$ and the sum is over all 4! permutations of $\left(i,j,s,t\right)$ . Following [23] (Section 5.1.6), we write

${\widehat{w}}_k={\left(n!\right)}^{-1}{\displaystyle \sum }_{n!}{\displaystyle \sum }_{i=1}^{m}h\left(V_{\left(4i-3\right)},V_{\left(4i-2\right)},V_{\left(4i-1\right)},V_{\left(4i\right)}\right)/m,$

where where ${{\displaystyle \sum }}_{n!}$ denotes the summation over all possible permutations of $\left(1,\cdots ,n\right)$ , $V_{\left(i\right)}$ is the $i$ -th element under the permutation, $m=\lfloor n/4\rfloor$ is the integer part of $n/4$ . Denote $h_i=h\left(V_{\left(4i-3\right)},V_{\left(4i-2\right)},V_{\left(4i-1\right)},V_{\left(4i\right)}\right)$ , write $\widehat{w}={\widehat{w}}_1+{\widehat{w}}_2$ , where

${\widehat{w}}_1={\left(n!\right)}^{-1}{\displaystyle \sum }_{n!}{\displaystyle \sum }_{i=1}^m{h}_{i}I\left(\left|h_i\right|\le M_0\right)/m,$

${\widehat{w}}_2={\left(n!\right)}^{-1}{\displaystyle \sum }_{n!}{\displaystyle \sum }_{i=1}^m{h}_{i}I\left(\left|h_i\right|>M_0\right)/m,$

and $M_0>0$ . Correspondingly, its population counterpart can also be decomposed as $w=E{h}_{1}I\left(\left|h_1\right|\le M_0\right)+E{h}_{1}I\left(\left|h_1\right|>M_0\right)=w_1+w_2$ .

Jensen inequality yields

$\begin{array}{c}E\exp \left({\widehat{w}}_{1}t\right)=E\exp \left(t{\left(n!\right)}^{-1}{\displaystyle \sum }_{n!}{\displaystyle \sum }_{i=1}^m{h}_{i}I\left(\left|h_i\right|\le M_0\right)/m\right)\\ \le E\exp \left(\frac{t}{m}{\displaystyle \sum }_{i=1}^m{h}_{i}I\left(\left|h_i\right|\le M_0\right)\right)\\ =E^m\exp \left(\frac{t}{m}{h}_{i}I\left(\left|h_i\right|\le M_0\right)\right),\end{array}$

so

$P\left({\widehat{w}}_1-w_1>ϵ\right)\le \text{exp}\left(-ϵt\right)E^m\exp \left(\frac{t}{m}{h}_{i}I\left(\left|h_i\right|\le M_0\right)-\frac{t}{m}{w}_1\right).$

$E{h}_{i}I\left(\left|h_i\right|\le M_0\right)=w_1$ , and $\left|h_{i}I\left(\left|h_i\right|\le M_0\right)\right|\le M_0$ , apply the Lemma 5.6.1.A of [23],

$E\exp \left(\frac{t}{m}{h}_{i}I\left(\left|h_i\right|\le M_0\right)-\frac{t}{m}{w}_1\right)\le \text{exp}\left(\frac{t^2{M}_0^2}{2m^2}\right).$

So,

$P\left({\widehat{w}}_1-w_1>ϵ\right)\le \text{exp}\left(\frac{t^2{M}_0^2}{2m}-ϵt\right).$

Furthermore, $P\left(\left|{\widehat{w}}_1-w_1\right|>ϵ\right)\le 2\exp \left(\frac{t^2{M}_0^2}{2m}-ϵt\right)$ by the symmetry of the U-statistic.

Now we turn to ${\widehat{w}}_2$ . $w_{k2}^2=E^2{h}_{1}I\left(\left|h_1\right|>M_0\right)\le E{h}_1^{2}P\left(\left|h_1\right|>M_0\right)\le \frac{E{h}_1^{2}E{\left|h_1\right|}^{q_1}}{M_0^{q_1}}$

for any $q_1\in N$ . By the assumption that $K_Z\left(\cdot ,\cdot \right)$ is bounded, there exists a positive constant $M$ such that $\left|a_{uv}\right|<M$ , and

$\left|h_1\right|=\left|h\left(V_i,V_j,V_s,V_t\right)\right|\le a_{uv}\left|b_{uv}-b_{uq}-b_{vr}+b_{qr}\right|\le M\left({\Vert Y_u\Vert }^2+{\Vert Y_v\Vert }^2+{\Vert Y_q\Vert }^2+{\Vert Y_r\Vert }^2\right)$ ,

this yields $E{\left|h_1\right|}^{q_1}\le 4^{q_1-1}{A}^{q_1}E{\Vert Y\Vert }^{2q_1}<\infty$ by condition (C1).

Thus, if we choose $M_0=n^{\beta }$ for $0<\beta <1/2$ , then $\left|w_2\right|\le ϵ/2$ for sufficiently large $n$ . Hence, $P\left(\left|{\widehat{w}}_2-w_2\right|\ge ϵ\right)\le P\left(\left|{\widehat{w}}_2\right|\ge ϵ/2\right)$ .

$\begin{array}{c}P\left(\left|{\widehat{w}}_2\right|\ge ϵ/2\right)\le P\left({\displaystyle {\cup }_{i=1}^n\left\{{\Vert Y_i\Vert }^2>\frac{M_0}{4M}\right\}}\right)\\ \le {\displaystyle \sum }_{i=1}^{n}P\left({\Vert Y_i\Vert }^2>\frac{M_1}{4M}\right)\\ =nP\left({\Vert Y_i\Vert }^2>\frac{M_0}{4M}\right)\\ \le nC\exp \left\{-r_2{n}^{\beta }\right\},\end{array}$

for $r_2\in \left(0,2s_0\right]$ .

$\begin{array}{c}P\left(\left|\widehat{w}-w\right|>2ϵ\right)\le P\left(\left|{\widehat{w}}_1-w_1\right|>ϵ\right)+P\left(\left|{\widehat{w}}_2-w_2\right|>ϵ\right)\\ \le \text{exp}\left(\frac{t^2{n}^{2\beta }}{2m}-ϵt\right)+nC\exp \left\{-r_2{n}^{\beta }\right\},\end{array}$

choosing $t=ϵm/n^{2\beta }$ , $P\left(\left|\widehat{w}-w\right|\ge ϵ\right)\le 2\exp \left(-\frac{{ϵ}^{2}m}{2n^{2\beta }}\right)+nC\exp \left\{-r_2{n}^{\beta }\right\}$ , because $m=\lfloor n/4\rfloor$ , we have

$P\left(\left|{\widehat{w}}_k-w_k\right|\ge ϵ\right)\le 2\exp \left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+nC\exp \left\{-r_2{n}^{\beta }\right\},$

where the constants $r_1$ satisfy $n{r}_1=\frac{m{\eta }^2}{2}$ . Immediately, we have

$P\left(\left|{\widehat{w}}_k-w_k\right|\ge ϵ\right)=O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right).$

Thus, we obtain

$P\left(\left|\left(\tilde{C}\cdot \tilde{B}\right)-\text{KCMD}\left(Y|W\right)\right|\ge ϵ\right)\le O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right),$

$P\left(\left|\left(\tilde{A}\cdot \tilde{B}\right)-\text{KCMD}\left(Y|Z\right)\right|\ge ϵ\right)\le O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right).$

Because these kernels are bounded, according Theorem 3 in [24], we have

$P\left(\left|\left(\tilde{A}\cdot \tilde{C}\right)-\text{HSIC}\left(Z,W\right)\right|\ge ϵ\right)\le O\left(\text{exp}\left(-r_{3}n{ϵ}^2\right)\right),$

$P\left(\left|\left(\tilde{A}\cdot \tilde{A}\right)-\text{HSIC}\left(Z,Z\right)\right|\ge ϵ\right)\le O\left(\text{exp}\left(-r_{3}n{ϵ}^2\right)\right).$

$\begin{array}{l}P\left(\left|{\text{pKCMD}}_n\left(Y|X;Z\right)-\text{pKCMD}\left(Y|X;Z\right)\right|\right)\ge ϵ\\ \le P\left(\left|\left(\tilde{C}\cdot \tilde{B}\right)-\frac{\left(\tilde{A}\cdot \tilde{B}\right)\left(\tilde{A}\cdot \tilde{C}\right)}{\left(\tilde{A}\cdot \tilde{A}\right)}-\text{KCMD}\left(Y|W\right)+\frac{\text{KCMD}\left(Y|Z\right)\text{HSIC}\left(Z,W\right)}{\text{HSIC}\left(Z,Z\right)}\right|\ge ϵ\right)\\ \le P\left(\left|\left(\tilde{C}\cdot \tilde{B}\right)-\text{KCMD}\left(Y|W\right)\right|\ge ϵ/2\right)+P\left(\left|\left(\tilde{A}\cdot \tilde{B}\right)-\text{KCMD}\left(Y|Z\right)\right|\ge C_1ϵ\right)\\ +P\left(\left|\left(\tilde{A}\cdot \tilde{C}\right)-\text{HSIC}\left(Z,W\right)\right|\ge C_2ϵ\right)+P\left(\left|\left(\tilde{C}\cdot \tilde{C}\right)-\text{HSIC}\left(Z,Z\right)\right|\ge C_3ϵ\right)\\ \le O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right)+O\left(\text{exp}\left(-r_{3}n{ϵ}^2\right)\right)\\ \le O\left(\text{exp}\left(-r_1{n}^{1-2\beta }{ϵ}^2\right)+n\exp \left\{-r_2{n}^{\beta }\right\}\right).\end{array}$

This completes the proof of theorem. □

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Cook, R.D. and Li, B. (2002) Dimension Reduction for Conditional Mean in Regression. The Annals of Statistics, 30, 455-474. https://doi.org/10.1214/aos/1021379861
[2]	Williamson, B.D., Gilbert, P.B., Carone, M. and Simon, N. (2020) Nonparametric Variable Importance Assessment Using Machine Learning Techniques. Biometrics, 77, 9-22. https://doi.org/10.1111/biom.13392
[3]	Dai, B., Shen, X. and Pan, W. (2024) Significance Tests of Feature Relevance for a Black-Box Learner. IEEE Transactions on Neural Networks and Learning Systems, 35, 1898-1911. https://doi.org/10.1109/tnnls.2022.3185742
[4]	Williamson, B.D., Gilbert, P.B., Simon, N.R. and Carone, M. (2022) A General Framework for Inference on Algorithm-Agnostic Variable Importance. Journal of the American Statistical Association, 118, 1645-1658. https://doi.org/10.1080/01621459.2021.2003200
[5]	Cai, L., Guo, X. and Zhong, W. (2024) Test and Measure for Partial Mean Dependence Based on Machine Learning Methods. Journal of the American Statistical Association, 120, 833-845. https://doi.org/10.1080/01621459.2024.2366030
[6]	Welsh, A.H. and Yee, T.W. (2006) Local Regression for Vector Responses. Journal of Statistical Planning and Inference, 136, 3007-3031. https://doi.org/10.1016/j.jspi.2004.01.024
[7]	Scheipl, F., Staicu, A. and Greven, S. (2015) Functional Additive Mixed Models. Journal of Computational and Graphical Statistics, 24, 477-501. https://doi.org/10.1080/10618600.2014.901914
[8]	Sun, X., Du, P., Wang, X. and Ma, P. (2018) Optimal Penalized Function-on-Function Regression under a Reproducing Kernel Hilbert Space Framework. Journal of the American Statistical Association, 113, 1601-1611. https://doi.org/10.1080/01621459.2017.1356320
[9]	Sun, Y. and Wang, Q. (2020) Function-on-Function Quadratic Regression Models. Computational Statistics & Data Analysis, 142, Article ID: 106814. https://doi.org/10.1016/j.csda.2019.106814
[10]	Park, T., Shao, X. and Yao, S. (2015) Partial Martingale Difference Correlation. Electronic Journal of Statistics, 9, 1492-1517. https://doi.org/10.1214/15-ejs1047
[11]	Shao, X. and Zhang, J. (2014) Martingale Difference Correlation and Its Use in High-Dimensional Variable Screening. Journal of the American Statistical Association, 109, 1302-1318. https://doi.org/10.1080/01621459.2014.887012
[12]	Zhang, X., Yao, S. and Shao, X. (2018) Conditional Mean and Quantile Dependence Testing in High Dimension. The Annals of Statistics, 46, 219-246. https://doi.org/10.1214/17-aos1548
[13]	Lai, T., Zhang, Z. and Wang, Y. (2021) A Kernel-Based Measure for Conditional Mean Dependence. Computational Statistics & Data Analysis, 160, Article ID: 107246. https://doi.org/10.1016/j.csda.2021.107246
[14]	Fukumizu, K., Gretton, A., Schölkopf, B. and Sriperumbudur, B.K. (2009) Characteristic Kernels on Groups and Semigroups. In: Advances in Neural Information Processing Systems, Vol. 21, Curran Associates, 473-480.
[15]	Gretton, A., Bousquet, O., Smola, A. and Schölkopf, B. (2005) Measuring Statistical Dependence with Hilbert-Schmidt Norms. In: Jain, S., Simon, H.U. and Tomita, E., Eds., Lecture Notes in Computer Science, Springer, 63-77. https://doi.org/10.1007/11564089_7
[16]	Székely, G.J. and Rizzo, M.L. (2014) Partial Distance Correlation with Methods for Dissimilarities. The Annals of Statistics, 42, 2382-2412. https://doi.org/10.1214/14-aos1255
[17]	Albert, M., Laurent, B., Marrel, A. and Meynaoui, A. (2022) Adaptive Test of Independence Based on HSIC Measures. The Annals of Statistics, 50, 858-879. https://doi.org/10.1214/21-aos2129
[18]	Balasubramanian, K., Sriperumbudur, B. and Lebanon, G. (2013) Ultrahigh Dimensional Feature Screening via RKHS Embeddings. Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, Vol. 31, 126-134.
[19]	Manfoumbi Djonguet, T.K., Mbina Mbina, A. and Nkiet, G.M. (2024) Testing Independence of Functional Variables by an Hilbert-Schmidt Independence Criterion Estimator. Statistics & Probability Letters, 207, Article ID: 110016. https://doi.org/10.1016/j.spl.2023.110016
[20]	Li, R., Zhong, W. and Zhu, L. (2012) Feature Screening via Distance Correlation Learning. Journal of the American Statistical Association, 107, 1129-1139. https://doi.org/10.1080/01621459.2012.695654
[21]	Wu, Y. and Yin, G. (2015) Conditional Quantile Screening in Ultrahigh-Dimensional Heterogeneous Data. Biometrika, 102, 65-76. https://doi.org/10.1093/biomet/asu068
[22]	Aneiros-Pérez, G. and Vieu, P. (2006) Semi-Functional Partial Linear Regression. Statistics & Probability Letters, 76, 1102-1110. https://doi.org/10.1016/j.spl.2005.12.007
[23]	Serfling, R.J. (1980) Approximation Theorems of Mathematical Statistics. Wiley. https://doi.org/10.1002/9780470316481
[24]	Song, L., Smola, A., Gretton, A., Borgwardt, K. and Bedo, J. (2012) Feature Selection via Dependence Maximization. Journal of Machine Learning Research, 13, 1393-1434.