Function-on-Partially Linear Functional Additive Models

Jinyou Huang, Shuang Chen^*

School of Sciences, Hebei University of Technology, Tianjin, China.

DOI: 10.4236/jamp.2020.81001                

Abstract

We consider a functional partially linear additive model that predicts a functional response by a scalar predictor and functional predictors. The B-spline and eigenbasis least squares estimator for both the parametric and the nonparametric components proposed. In the final of this paper, as a result, we got the variance decomposition of the model and establish the asymptotic convergence rate for estimator.

Keywords

Functional Data Analysis, Functional Principal Component Analysis, Partial Linear Regression Models, Penalized B-Splines, Variance Model

1. Introduction

Function data are infinite dimensional vectors in functional space, the study of functional data helps people to further understand data changes in finance, medicine, etc. The study of fucntional response can date back to the work Ramsay and Silverman (2005) [1] where the model the functional concurrent model considers current response relate to the current values of the covariates. Wong et al. (2018) [2] investigated a class of partially linear functional additive models (PLFAM) that predicts a scalar response by both parametric effects of a multivariate predictor and nonparametric effects of a multivariate functional predictor. Further an additive function-on-function regression is established using principal component basis functions and b-spline basis functions by Janet S. Kim et al. (2016) [3]. Luo, R., Qi, X. (2017) [4] consider functional linear regression models with a functional response and multiple functional predictors, with the goal of finding the best finite-dimensional approximation to the signal part of the response function. This paper extends function on function regression, considering functional partially linear additive models that predict a functional response by a scalar predictor and functional predictors. Specifically we consider the model

$Y\left(t\right)=\mu \left(t\right)\gamma \left(t\right)+{\displaystyle \underset{j=1}{\overset{p}{\sum }}}{\displaystyle {\int }_{{\tau }_x}}{\beta }_j\left(s,t\right)x_j\left(s\right)\text{d}s+\epsilon \left(t\right)$ (1)

where $Y\left(t\right)$ is the response function defined in an interval ${\tau }_y$, For convenience, we assume that the response has zero-mean, the predictive curves. $x_1\left(s\right)\mathrm{,}{x}_2\left(s\right)\mathrm{,}\cdots \mathrm{,}{x}_p\left(s\right)$ are defined in ${\tau }_x$. where ${\tau }_y$ and ${\tau }_x$ are compact intervals. $\mu \left(t\right)$ is scalar predictor defined in an interval ${\tau }_y$. $\gamma \left(t\right)$ is scalar coefficient, $\epsilon \left(t\right)$ is the noise function with mean zero and unknown autocovariance function $A\left(t\mathrm{,}{t}^{\prime }\right)$ and is independent of the predictor and the response. In the previous papers, these focused on function-on-function, scalar-on-function, and scalar-mixed data. There is no paper devoted to function-on-mixed data. That is to say, the functional dependent variable analyzes the regression model of the mixed data. This paper considers the actual situation is closer to the actual demand. Considering that the real world data cannot be all the functional data, there will definitely be a part of the scalar in the model. From this perspective, expand the new model. This paper uses a different estimation method from the previous. The general method adopted by these articles is to use non-parametric methods such as spline function to approximate the model generation estimation parameters, and at the same time minimize the objective function. In the past, the model was generally complicated and inconvenient to calculate. They consider fitting with B-splines in three directions at the same time, and there will be over-fitting when doing so. In this paper, the problem is optimized by using the eigenbasis and orthogonal B-spline, that is, using the special feature base of $Y\left(t\right)$ and an orthogonal B-spline to fit the model. While ensuring the fitting effect, the complexity of the model is reduced, and the model is not over-fitting. We use the classical formula of probability to express the error term of the model from the perspective of variance, and the method of resampling is used to approximate the error of the model. At the same time, assume from the article hypothesis, we establish the asymptotic convergence rate for estimator.

2. Model

In this section, we consider the estimation of coefficients in a partial linear regression model of a function. In this paper, the model is estimated by using the eigenbasis spline and the B-spline function. Note ${\varphi }_k$ is the corresponding eigenfunctions of $y\left(t\right)$, and

${\displaystyle {\int }_0^1}{\varphi }_n\left(t\right){\varphi }_m\left(t\right)\text{d}t=\{\begin{array}{l}1,n=m\\ 0,n\ne m\end{array}$ (2)

Let

$h\left(t\right)=\mu \left(t\right)\gamma \left(t\right)$ (3)

we project $h\left(t\right)$ onto the eigenbasis of $y\left(t\right)$,

$H_k={\displaystyle {\int }_{\tau y}}h\left(t\right){\varphi }_k\left(t\right)\text{d}t$ (4)

then we can express $h\left(t\right)$ as

$h\left(t\right)={\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{H}_k{\varphi }_k\left(t\right)$ (5)

For the function coefficient term part of (1), we estimate it by means of B-spline and feature base. For convenience we write model (1) as

$y\left(t\right)=\mu \left(t\right)\gamma \left(t\right)+{\displaystyle {\int }_{\tau x}}X{\left(s\right)}^{\text{T}}\beta \left(s\mathrm{,}t\right)\text{d}s+\epsilon \left(t\right)$ (6)

where $X\left(s\right)={\left(x_1\left(s\right)\mathrm{,}\cdots \mathrm{,}{x}_2\left(s\right)\right)}^{\text{T}}$ is the p-dimensional vector of predictive functions, $\beta \left(s\mathrm{,}t\right)={\beta }_1\left(s\mathrm{,}t\right)\mathrm{,}\cdots \mathrm{,}{\beta }_p\left(s\mathrm{,}t\right)$ is the p-dimensional vector of coefficient functions. Let

$F\left(s\mathrm{,}t\right)=X{\left(s\right)}^{\text{T}}\beta \left(s\mathrm{,}t\right)$ (7)

Transform $F\left(s\mathrm{,}t\right)$ in the same way as $h(t)$

$G_k\left(s\right)={\displaystyle {\int }_{\tau y}}F\left(s\mathrm{,}t\right){\varphi }_k\left(t\right)\text{d}t$ (8)

So we can put $F\left(s\mathrm{,}t\right)$ similar expression of Karhunen-Loeve expansion

$F\left(s\mathrm{,}t\right)={\displaystyle \underset{k=0}{\overset{\infty }{\sum }}}{G}_k\left(s\right){\varphi }_k\left(t\right)\text{d}t$ (9)

In this paper, on spectral decomposition method projecting the corresponding function to the orthogonal eigenbasis of y, we estimate the parameter of the model can be performed without punished the complexity of the t direction. Generally, we preset the maximum variance number similar to the method in the functional principal component analysis, usually 85%. Next we will use the Orthogonal b-spline method (2010) [5] [6] to represent the parameter items in the s direction.

$G_k\left(s\right)={\displaystyle \underset{l=1}{\overset{M}{\sum }}}{B}_l\left(s\right){\theta }_{l\mathrm{.}k}$ (10)

$F\left(s\mathrm{,}t\right)={\displaystyle \underset{k=0}{\overset{\infty }{\sum }}}{\displaystyle \underset{l=1}{\overset{M}{\sum }}}{B}_l\left(s\right){\theta }_{l.k}{\varphi }_t\left(t\right)$ (11)

where $B_l\left(s\right)$ are or-thogonalized B-spline bases of dimensions M, ${\theta }_{l\mathrm{.}k}$ is unknown parameters, we get

$y\left(t\right)={\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{H}_k{\varphi }_k\left(t\right)+{\displaystyle {\int }_{\tau x}}{\displaystyle \underset{k=0}{\overset{\infty }{\sum }}}{\displaystyle \underset{l=1}{\overset{M}{\sum }}}{B}_l\left(s\right){\varphi }_k\left(t\right)\text{d}s+\epsilon \left(t\right)$ (12)

In the t direction perform truncation K for the preset maximum variance, we get

$y\left(t\right)\approx {\displaystyle \underset{k=1}{\overset{K}{\sum }}}{H}_k{\varphi }_k\left(t\right)+{\displaystyle {\int }_{\tau x}}{\displaystyle \underset{k=0}{\overset{K}{\sum }}}{\displaystyle \underset{l=1}{\overset{M}{\sum }}}{B}_l\left(s\right){\theta }_{l\mathrm{,}k}{\varphi }_k\left(t\right)\text{d}s$ (13)

We denote $Z=\left(Z_1\mathrm{,}{Z}_2\mathrm{,}\cdots \mathrm{,}{Z}_p\right)$, $Z_l$ as ${\displaystyle {\int }_{\tau x}}{B}_l\left(s\right)\text{d}s$, let ${\Phi }_k=\left({\theta }_{1k}\mathrm{,}{\theta }_{2k}\mathrm{,}\cdots \mathrm{,}{\theta }_{pk}\right)$.

Then we can get the simplified expression of $y(t)$

$y\left(t\right)\approx {\displaystyle \underset{k=1}{\overset{K}{\sum }}}{H}_k{\varphi }_k\left(t\right)+{\displaystyle \underset{k=1}{\overset{k}{\sum }}}Z{\Phi }_k{\varphi }_k\left(t\right)$ (14)

3. Parameter Estimation

We assume that with probability 1, the trajectory of $x_n$ is contained in a Hilbert space ${\chi }_n$, with inner product. We will focus on the case that ${\chi }_j$ ’s are $L^2$ functional spaces and the inner products are $\langle f\mathrm{,}g\rangle ={\displaystyle \int }f\left(t\right)g\left(t\right)\text{d}t$ for any $f\mathrm{,}g\in {\chi }_n$. We use quadratic penalties for the direction s and control the roughness in the direction t by the preset number of orthogonal basis functions. so the loss function is as follow

$L\left(H_k\mathrm{,}{\Phi }_k\right)=\left(y\left(t\right)-{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{h}_k{\varphi }_k\left(t\right)-{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{Z}^{\text{T}}{\Phi }_k{\varphi }_k\left(t\right)\right)$ (15)

We use the least squares penalty to punish the curvature of the functional parameter, considering the penalty term

${\displaystyle \iint }{\left(\frac{{\partial }^{2}F\left(s\mathrm{,}t\right)}{\partial s^2}\right)}^2\text{d}s\text{d}t={\displaystyle \iint }{\left(\frac{{\partial }^2{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{G}_k\left(s\right){\varphi }_k\left(t\right)}{\partial s^2}\right)}^2\text{d}s\text{d}t={\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\displaystyle \int }{\left(\frac{{\partial }^2{G}_k\left(s\right)}{\partial s^2}\right)}^2\text{d}s$ (16)

${\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\displaystyle \int }{\left(\frac{{\partial }^2{G}_k\left(s\right)}{\partial s^2}\right)}^2\text{d}s=\lambda {\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\displaystyle \int }{\left(\frac{{\partial }^2{B}_l\left(s\right)}{\partial s^2}\right)}^2{\theta }_{l\mathrm{,}k}^2\text{d}s=\lambda {\Phi }_k^{\text{T}}\text{P}{\Phi }_k$ (17)

where $\text{P}={\displaystyle \underset{l=1}{\overset{M}{\sum }}}{\displaystyle \int }\frac{{\partial }^2{B}_l\left(s\right)}{\partial s^2}\text{d}s$. Then we can get the penalized criterion

${\Vert y\left(t\right)-{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{H}_k{\varphi }_k\left(t\right)+{\displaystyle \underset{k=1}{\overset{k}{\sum }}}{Z}^{\text{T}}{\Phi }_k{\varphi }_k\left(t\right)\Vert }^{\text{2}}+\lambda {\Phi }_k\text{P}{\Phi }_k$ (18)

Similarly $F\left(s\mathrm{,}t\right)$ and $h\left(t\right)$ to process $y\left(t\right)$. We have Karhunen-Loeve expansion on y(t)

$y\left(t\right)={\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\xi }_k{\varphi }_k\left(t\right)+e_t$ (19)

where $e_t$ is zero-mean error, and the variables ${\xi }_k$ are the FPCA (Functional principal component) scores of Y. And note that ${\displaystyle \int }{\varphi }_k{\left(t\right)}^2\text{d}t=1$, so the criterion can be simplify written as

${\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\Vert {\xi }_k-H_k+Z^{\text{T}}{\Phi }_k\Vert }^{\text{2}}+\lambda {\Phi }_k\text{P}{\Phi }_k$ (20)

So the estimation process for the parameters $H_k$, ${\Phi }_k$ is as follows.

1) We get the function curve $\hat{y}\left(t\right)$ for smooth sample points, and we demean $\hat{Y}\left(t\right)$ for get a curve ${\hat{y}}^c\left(t\right)$ with a mean of 0. The point K is determined by the pre-specified percentage of variance.

2) We use FPCA(Functional Principal Component) to estimate eigenbasis ${\hat{\varphi }}_k$, and we project $\hat{y}\left(t\right)$ onto the corresponding eigenbasis ${\hat{\varphi }}_k$ to obtain an estimation of the principal component score ${\hat{\xi }}_k={\displaystyle {\int }_{\tau y}}{\hat{y}}^c\left(t\right)\hat{\varphi }\left(t\right)\text{d}t$.

3) We get a two-step estimation for $H_k$, ${\Phi }_k$

a) Obtain estimation of ${\Phi }_k$, $k=1,2,3,\cdots ,K$, of the coefficients by minimizing the penalized criterion(19) with respect to ${\Phi }_k$ and $H_k$, The estimated value we get for ${\Phi }_k$ is

${\hat{\Phi }}_k=\frac{Z\left({\hat{\xi }}_k-H_k\right)}{Z{Z}^{\text{T}}+\lambda \text{P}}$ (21)

b) Obtain estimation of $H_k$ by minimizing

$\arg \min {\displaystyle \underset{k=1}{\overset{K}{\sum }}}\Vert {\hat{\xi }}_k-H_k-Z^{\text{T}}{\hat{\Phi }}_k\Vert$ (22)

So the estimation of ${\Phi }_k$ is ${\hat{\Phi }}_k$ and ${\hat{H}}_k$ is ${\hat{H}}_k$. We can get

$\hat{y}\left(t\right)={\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{\varphi }}_k\left(t\right)\left({\hat{H}}_k+Z^{\text{T}}{\hat{\Phi }}_k\right)$ (23)

4. Error Variance Decomposition

In this section, we get error variance decomposition. Let $A\left(t\mathrm{,}{t}^{\prime }\right)$ be the variance function of $\epsilon \left(t\right)$.we estimate $A\left(t\mathrm{,}{t}^{\prime }\right)$.

1) obtain residuals by fit (14), $e_j=y\left(t_j\right)-\hat{y}\left(t_j\right)$.

2) Use FPCA for the residual function, to approximate the infinite covariance function using the estimated finite covariance function.

Let $x_{new}$, ${\mu }_{new}$ be new observation independent of X and $\epsilon \left(t\right)$, and fit function-on-partially linear functional additive models,then we measure the uncertainty in the prediction by predicting error ${\hat{y}}_{new}\left(t\right)-y_{new}\left(t\right)$.Base on Ruppert et al. (2009) [7]

$\mathrm{var}\left\{{\hat{y}}_{new}\left(t\right)-y_{new}\left(t\right)\right\}=\mathrm{var}\left\{{\hat{y}}_{new}\left(t\right)\right\}+\mathrm{var}\left\{{\epsilon }_{new}\left(t\right)\right\}$ (24)

We estimate the expression of $\mathrm{var}\left\{{\epsilon }_{new}\left(t\right)\right\}$ by estimating $\mathrm{var}\left\{\epsilon \left(t\right)\right\}$ Then we estimate $\mathrm{var}\left\{{\hat{y}}_{new}\left(t\right)\right\}$ using the classical variance formula

$\mathrm{var}\left\{{\hat{\epsilon }}_0\left(t\right)\right\}=E_{\pi }\left[\mathrm{var}\left\{{\hat{\epsilon }}_0\left(t\right)|\pi \right\}\right]+{\mathrm{var}}_{\pi }\left[E\left\{{\hat{\epsilon }}_0\left(t\right)|\pi \right\}\right]$ (25)

In this paper, we assume that $\pi$ is a known parameter set containing estimated arbitrary parameter values and corresponding parameter variance and covariance functions. So we will expand $\mathrm{var}\left\{{\hat{\epsilon }}_0\left(t\right)|\pi \right\}$ of the formula (24) as follows

$\underset{}{{\displaystyle \mathrm{var}}}\left\{{\hat{y}}_{new}\left(t\right)|\pi \right\}=\underset{}{{\displaystyle \mathrm{var}}}{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{\varphi }}_k\left(t\right)\left({\hat{H}}_{new\mathrm{,}k}+Z_{new}^{\text{T}}{\hat{\Phi }}_k\right)$ (26)

If we write $\frac{Z}{Z^{\text{T}}Z+\lambda \text{P}}$ as $m_{\lambda }$, $Z^{\text{T}}{m}_{\lambda }$ is recorded as $J_{\lambda }$, then ${\hat{\Phi }}_k=\frac{Z\left({\hat{\xi }}_k-H_k\right)}{Z^{\text{T}}Z+\lambda \text{P}}=m_{\lambda }\left({\hat{\xi }}_k-H_k\right)$ and ${\hat{H}}_k+Z_{new}^{\text{T}}{\hat{\Phi }}_k={\hat{H}}_k+Z_{new}^{\text{T}}{m}_{\lambda }\left({\hat{\xi }}_k-{\hat{H}}_k\right)=\left(1-J_{\lambda }\right){\hat{H}}_k+J_{\lambda }{\hat{\xi }}_k$. So we expand the expression (25) as follow

$\begin{array}{c}\text{var}\left\{{\hat{y}}_{new}\left(t\right)|\pi \right\}={\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{\varphi }}_k\left(t\right)\text{var}\left({\hat{H}}_k+Z_{new}^{\text{T}}{\hat{\Phi }}_k\right){\hat{\varphi }}_k\left(t\right)\\ +{\displaystyle \underset{k\ne k^{\prime }}{\sum }}{\hat{\varphi }}_k\left(t\right)\text{cov}\left({\hat{H}}_k+Z_{new}^{\text{T}}{\hat{\Phi }}_k\mathrm{,}{\hat{H}}_{k^{\prime }}+Z_{new}^{\text{T}}{\hat{\Phi }}_{k^{\prime }}\right){\hat{\varphi }}_k\left(t\right)\end{array}$ (27)

It can be seen that all the variances and covariance functions in the expression are represented by the variance and covariance functions of the existing estimates. Then we focus on the second item of expression (24), the second term is ${\mathrm{var}}_{\pi }\left[E\left\{{\hat{\epsilon }}_0\left(t\right)|\pi \right\}\right]$. This paper hopes to obtain an approximation form of the finite term of the variance function under finite samples by resampling the bootstrap method. We perform the bootstrap algorithm through the following steps;

1) For $q=1,2,3,\cdots ,Q$.

2) Resample the samples to get Q groups different sample.

3) Smooth the resample points and get the corresponding curve $X_j^q\left(s\right)\mathrm{,}{\mu }^q\left(t\right)\mathrm{,}{y}^q\left(t\right)$.

4) Obtain the de-mean ${\left({\hat{y}}^c\right)}^q\left(t\right)$ and the corresponding feature basis function ${\hat{\varphi }}_k^q$, and obtain the corresponding principal component score ${\hat{\xi }}_k^q$.

5) Fit the function-on-partially linear functional additive models to obtain the corresponding parameter estimation.

6) Get prediction function ${\hat{y}}_{new}^q\left(t\right)$ to any new observation curve $x_{new}\left(t\right)\mathrm{,}{y}_{new}\left(t\right)$ and calculate ${\hat{A}}^q\left(t\mathrm{,}{t}^{\prime }\right)$.

7) Get $\mathrm{var}\left\{{\hat{y}}_{new}^q\left(t\right)|\pi \right\}$ by expression (24).

Thus the approximate value of the finite term of $A\left(t\mathrm{,}{t}^{\prime }\right)$ is found $\hat{A}\left(t\mathrm{,}{t}^{\prime }\right)=\frac{{\displaystyle \underset{q=1}{\overset{Q}{\sum }}}{\hat{A}}^q\left(t\mathrm{,}{t}^{\prime }\right)}{Q}$. and the variance estimate of the error

$\begin{array}{l}\widehat{\text{var}}\left\{{\hat{y}}_{new}\left(t\right)-y_{new}\left(t\right)\right\}\\ =\frac{{\displaystyle \underset{q=1}{\overset{Q}{\sum }}}\widehat{\text{var}}\left\{y_{new}^q\left(t\right)\mathrm{<mo>\; |\; </mo>}{\pi }^q\right\}}{Q}+\frac{{\displaystyle \underset{q=1}{\overset{Q}{\sum }}}{\left\{y_{new}^b\left(t\right)-{\bar{y}}_{new}\left(t\right)\right\}}^2}{Q}+\frac{{\displaystyle \underset{q=1}{\overset{Q}{\sum }}}{\hat{A}}^q\left(t\mathrm{,}{t}^{\prime }\right)}{Q}\end{array}$ (28)

5. Basic Assumptions and Convergence Properties

In this section,we study the asymptotic properties of $F\left(t\right)$, $h\left(t\right)$, where

$\hat{h}\left(t\right)={\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{H}}_k\hat{\varphi }\left(t\right)$ (29)

$\begin{array}{c}F\left(t\right)={\displaystyle {\int }_{\tau x}}F\left(s\mathrm{,}t\right)\text{d}s={\displaystyle {\int }_{\tau x}}{\displaystyle \underset{k=1}{\overset{K}{\sum }}}G\left(s\right)\varphi \left(t\right)\text{d}s\\ ={\displaystyle {\int }_{\tau x}}{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\displaystyle \underset{l=1}{\overset{M}{\sum }}}{B}_l\left(s\right){\theta }_{l\mathrm{,}k}\varphi \left(t\right)\text{d}s\\ ={\displaystyle \underset{k=1}{\overset{k}{\sum }}}Z{\Phi }_k{\varphi }_k\left(t\right)\end{array}$ (30)

5.1. Basic Assumption

1) For the square integrable random function $\tilde{X}\left(s\right)$ satisfies $E{\Vert \tilde{X}\left(s\right)\Vert }^q<\infty ,2<q<\infty$ ;

2) For each k, there is $E\left[{\tilde{U}}_{ik}\right]\le c{\lambda }_k^2$, where c is a constant and c can take different values in different expressions;

3) For the eigenvalue ${\lambda }_k$, satisfied $c^{-1}{k}^{-s}\le {\lambda }_k\le c{k}^{-s},k\ge 1,s>1$ ;

4) For the coefficient $H_k$, assume there is a constant t, and $t>\frac{s}{3}+1$, which make $H_k$ satisfied $H_k<c{k}^{-s}$ ;

5) For the parameter K, assume $K=\frac{1}{n^{s+3t}}$

6) The number of knots $N_n=n^r$, with $\frac{1}{2q+1}\le r\le \frac{1}{2}$.

7) The distribution of $F\left(t\right)$ and $h\left(t\right)$ is absolutely continuous and its density is bounded on $\tau y$ ;

8) The joint density function of $x\left(s\right)$ is bounded on $\tau x$ ;

9) Covariance function ${{\displaystyle \mathrm{cov}}}_{}\left(y\left(t\right),y\left(t^{\prime }\right)\right)$ of $y\left(t\right)$ is bounded on interval $\tau y$

Conditions (1)-(5) are common in functional PCA literature and conditions (6)-(9) are common in nonparametric regression literature.

5.2. Related Lemma

Lemma 1. If $\tilde{F}\left(t\right)$ is spline expansion of $F\left(t\right)$, it is known by Chen [8] that there is $n>2,h=\frac{1}{N}$, we can get

${\Vert \tilde{F}\left(t\right)-F\left(t\right)\Vert }^2=O\left(h^2\right)<O\left(h^n\right)$ (31)

5.3. Convergence Properties

Consider the model (6)

$y\left(t\right)=\mu \left(t\right)\gamma \left(t\right)+{\displaystyle {\int }_{\tau x}}X{\left(s\right)}^{\text{T}}\beta \left(s\mathrm{,}t\right)\text{d}s+\epsilon \left(t\right)$ (32)

Use the principal component of $y\left(t\right)$ for the model (6) to represent the following expression

$\begin{array}{c}{\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{\xi }_k{\varphi }_k\left(t\right)={\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{H}_k{\varphi }_k\left(t\right)+{\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{Z}^{\text{T}}{\Phi }_k{\varphi }_k\left(t\right)+\epsilon \left(t\right)\\ ={\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}\left(H_k\mathrm{,}{\Phi }_k\right){\left(\mathrm{1,}Z\right)}^{\text{T}}{\varphi }_k\mathrm{<mo\; stretchy="false">\; (\; </mo>}t\mathrm{<mo\; stretchy="false">\; )\; </mo>}+\epsilon \left(t\right)\\ ={\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{D}_k{Z}_0{\varphi }_k\left(t\right)+\epsilon \left(t\right)\end{array}$ (33)

where $D_k=\left(H_k\mathrm{,}{\Phi }_k\right)$ and $Z_0=\left(\mathrm{1,}Z\right)$. We can get an estimation by least squares estimation of ${\hat{D}}_k=\left({\hat{H}}_k\mathrm{,}{\hat{\Phi }}_k\right)=\frac{Z_0{\hat{\xi }}_k}{Z_0{Z}_0^{\text{T}}}$, let ${\tilde{D}}_k=\left(H_k\mathrm{,}{\tilde{\Phi }}_k\right)$, where $\tilde{F}\left(t\right)={\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\tilde{\Phi }}_k{Z}^{\text{T}}\hat{\varphi }\left(t\right)$ is spline expansion of $F\left(t\right)$.

Theorem 2. Under the conditions of (1)-(5), we can get

${\Vert \hat{h}\left(t\right)-h\left(t\right)\Vert }^2=O\left(n^{\left(-\frac{3t-1}{s+3t}\right)}\right)\mathrm{,}n\to \infty$ (34)

Proof. Use Cauchy-Buniakowsky-Schwarz Inequality we can get

$\begin{array}{l}{\Vert \hat{h}\left(t\right)-h\left(t\right)\Vert }^2={\Vert {\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{H}}_k{\hat{\varphi }}_k\left(t\right)-{\displaystyle \underset{k=1}{\overset{\infty }{\sum }}}{H}_k{\varphi }_k\left(t\right)\Vert }^2\\ \le 2{\Vert {\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{H}}_k{\hat{\varphi }}_k\left(t\right)-{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{H}_k{\varphi }_k\left(t\right)\Vert }^2+2{\Vert {\displaystyle \underset{k=K+1}{\overset{\infty }{\sum }}}{H}_k{\varphi }_k\left(t\right)\Vert }^2\\ \le 4{\Vert {\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\hat{H}}_k\left({\hat{\varphi }}_k\left(t\right)-{\varphi }_k\left(t\right)\right)\Vert }^2+4{\Vert {\displaystyle \underset{k=1}{\overset{K}{\sum }}}\left({\hat{H}}_k-H_k\right){\hat{\varphi }}_k\left(t\right)\Vert }^2+2{\Vert {\displaystyle \underset{k=K+1}{\overset{\infty }{\sum }}}{H}_k{\varphi }_k\left(t\right)\Vert }^2\\ \le 8K{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\left({\hat{H}}_k\right)}^2{\Vert {\hat{\varphi }}_k\left(t\right)-{\varphi }_k\left(t\right)\Vert }^2+4{\displaystyle \underset{k=1}{\overset{K}{\sum }}}{\left({\hat{H}}_k-H_k\right)}^2+2{\displaystyle \underset{k=K+1}{\overset{\infty }{\sum }}}{H}_k^2\end{array}$ (35)

Assume from the article hypothesis, we have

${\Vert \hat{h}\left(t\right)-h\left(t\right)\Vert }^2=O\left(n^{\left(-\frac{3t-1}{s+3t}\right)}\right)\mathrm{,}n\to \infty$ (36)

☐

Theorem 3. Available under the conditions of (6)-(9), and let $\tilde{F}\left(t\right)$ is spline expansion of $F\left(t\right)$ we can get

${\Vert \hat{F}\left(t\right)-F\left(t\right)\Vert }_2^2=O\left(\frac{MK}{n}+{\rho }^2\right)$ (37)

Proof.

${\Vert \hat{F}\left(t\right)-F\left(t\right)\Vert }^2\le 2\Vert \hat{F}\left(t\right)-\tilde{F}\left(t\right)\Vert +2\Vert \tilde{F}\left(t\right)-F\left(t\right)\Vert$ (38)

Consider the latter item in the text, because $\tilde{F}\left(t\right)$ is a spline expansion of $F\left(t\right)$, by lemma we have

$\Vert \tilde{F}\left(t\right)-F\left(t\right)\Vert =O\left(h^2\right)$ (39)

Now consider the former item

$\begin{array}{c}{\Vert \hat{F}\left(t\right)-\tilde{F}\left(t\right)\Vert }^2={\Vert {\displaystyle \underset{i=1}{\overset{k}{\sum }}}{Z}^{\text{T}}{\hat{\Phi }}_k{\hat{\varphi }}_k\left(t\right)-{\displaystyle \underset{i=1}{\overset{k}{\sum }}}{Z}^{\text{T}}{\tilde{\Phi }}_k{\hat{\varphi }}_k\left(t\right)\Vert }^2\\ \le {\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\Vert Z^{\text{T}}{\hat{\Phi }}_k-Z^{\text{T}}{\tilde{\Phi }}_k\Vert }^2\le {\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\Vert {\hat{\Phi }}_k-{\tilde{\Phi }}_k\Vert }^2\end{array}$ (40)

Then consider

${\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\Vert {\hat{F}}_k-{\tilde{\Phi }}_k\Vert }^2\le {\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\Vert \frac{Z_{\text{0}}\epsilon }{Z_0{Z}_0^{\text{T}}}\Vert }^2\le {\displaystyle \underset{i=1}{\overset{k}{\sum }}}{\Vert \frac{Z_0\epsilon }{n}\Vert }^2$ (41)

Since $E\left[B_l\left(s_i\right)B_l\left(s_j\right){\epsilon }_i{\epsilon }_j\right]=\mathrm{0,}i\ne j$,

$E\Vert \frac{Z_0\epsilon }{n}\Vert =\frac{1}{n^2}E\left[\epsilon Z_0^{\text{T}}{Z}_0\epsilon \right]=\frac{1}{n^2}{\displaystyle \underset{l=1}{\overset{M}{\sum }}}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}E{\left[B_l\left(s_i\right){\epsilon }_i\right]}^2\le \frac{M}{n}$ (42)

Get ${\Vert \hat{F}\left(t\right)-F\left(t\right)\Vert }^2=O\left(\frac{MK}{n}+{\rho }^2\right)$ by the above formula.☐

6. Conclusion

This paper is a model extension of the function-to-function regression. The scalar variable is added to the dependent variable, which extends the application scope of the model. For real-life data, real data should include scalar data and function data, and the model used in this paper can be better explained. In this paper, the model is estimated by using the eigenbasis spline function and the orthogonal B-spline function. When the loss function is punished, the complexity of the t direction is controlled by controlling the preset principal component in the t direction, which reduces the complexity. It is more practical. At the same time, the variance decomposition of the error under the finite sample size is given. The approximation is performed by the resampling bootstrap method. Finally, the convergence properties of the estimated parameters are studied.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Billheimer, D. (2010) Functional Data Analysis, 2nd Edition Edited by J. O. Ramsay and B. W. Silverman. Biometrics, 63, 300-301. https://doi.org/10.1111/j.1541-0420.2007.00743_1.x
[2]	Wong, R.K.W., Li, Y. and Zhu, Z. (2018) Partially Linear Functional Additive Models for Multivariate Functional Data. Journal of the American Statistical Association.
[3]	Kim, J.S., Staicu, A.M., Maity, A., et al. (2017) Additive Function-on-Function Regression. Journal of Computational and Graphical Statistics, 27. https://doi.org/10.1080/10618600.2017.1356730
[4]	Luo, R. and Qi, X. (2016) Function-on-Function Linear Regression by Signal Compression. Journal of the American Statistical Association.
[5]	Chen, G.L. and Wang, Z.J. (2010) The Multivariate Partially Linear Model with B-Spline. Chinese Journal of Applied Probability and Statistics.
[6]	Benko, M., Hardle, W. and Kneip, A. (2009) Common Functional Principal Components. Annals of Statistics, 37, 1-34. https://doi.org/10.1214/07-AOS516
[7]	Ruppert, D., Wand, M.P. and Carroll, R.J. (2009) Semiparametric Regression during 2003C2007. Electronic Journal of Statistics, 3, 1193-1256. https://doi.org/10.1214/09-EJS525
[8]	Chen, T.P. (1985) Convergence and Asymptotic Expansion of Spline Functions. Chinese Science Bulletin, 30, 1361-1364.