Fourth-Order Predictive Modelling: I. General-Purpose Closed-Form Fourth-Order Moments-Constrained MaxEnt Distribution

Dan Gabriel Cacuci

Center for Nuclear Science and Energy, Department of Mechanical Engineering, University of South Carolina, Columbia, SC, USA.

DOI: 10.4236/ajcm.2023.134024                

Abstract

This work (in two parts) will present a novel predictive modeling methodology aimed at obtaining “best-estimate results with reduced uncertainties” for the first four moments (mean values, covariance, skewness and kurtosis) of the optimally predicted distribution of model results and calibrated model parameters, by combining fourth-order experimental and computational information, including fourth (and higher) order sensitivities of computed model responses to model parameters. Underlying the construction of this fourth-order predictive modeling methodology is the “maximum entropy principle” which is initially used to obtain a novel closed-form expression of the (moments-constrained) fourth-order Maximum Entropy (MaxEnt) probability distribution constructed from the first four moments (means, covariances, skewness, kurtosis), which are assumed to be known, of an otherwise unknown distribution of a high-dimensional multivariate uncertain quantity of interest. This fourth-order MaxEnt distribution provides optimal compatibility of the available information while simultaneously ensuring minimal spurious information content, yielding an estimate of a probability density with the highest uncertainty among all densities satisfying the known moment constraints. Since this novel generic fourth-order MaxEnt distribution is of interest in its own right for applications in addition to predictive modeling, its construction is presented separately, in this first part of a two-part work. The fourth-order predictive modeling methodology that will be constructed by particularizing this generic fourth-order MaxEnt distribution will be presented in the accompanying work (Part-2).

Keywords

Maximum Entropy Principle, Fourth-Order Predictive Modeling, Data Assimilation, Data Adjustment, Reduced Predicted Uncertainties, Model Parameter Calibration

1. Introduction

Scientific progress stems from the judicious combination of experimental information with results produced by computational models, neither of which are perfectly accurate. On the one hand, computations are afflicted by errors stemming from numerical procedures, uncertain model parameters, boundary/initial conditions and/or imperfectly known physical processes. On the other hand, results of measurements are afflicted by experimental errors, which means that around any reported experimental value there always exists a range of values that may also be plausibly representative of the true but unknown value of the measured quantity. Extracting “best estimate” values for model parameters and predicted results, together with “best estimate” uncertainties for these parameters and results requires the combination of experimental and computational data and their uncertainties.

The earliest systematic activities aimed at extracting best-estimate values for model parameters were initiated in the mid-1960s [1] [2] , in the course of evaluating neutron cross sections by using time-independent reactor physics models and experiments for evaluating, computationally and experimentally, so-called “integral quantities” (also called “system responses”) such as reaction rates and multiplication factors. A decade later, these activities had reached conceptual maturity under the name of “cross section adjustment” methodology [3] [4] . This methodology employs a user-defined least-square functional for combining uncertainties in the model parameters with uncertainties in the experimental data, subject to the hard constraint represented by the reactor physics model linearized with respect to the model parameters. The resulting “adjusted” neutron cross sections (model parameters) and their “adjusted” uncertainties were subsequently employed in the respective reactor physics model to predict improved “model responses,” such as improved reaction rates and reaction rate ratios, reactor multiplication factors, Doppler coefficients. By the late 1970s, adjoint neutron fluxes were introduced and used [5] [6] to compute efficiently the first-order response sensitivities, which were used as weighting functions in the least squares adjustment procedure. It is important to note that all of these works dealt with the time-independent linear neutron transport or diffusion equation, as encountered in reactor physics and shielding, for which the corresponding adjoint equations were already known and readily available.

In the late 1980s and during the 1990s, the fundamental concepts underlying the above-mentioned “data adjustment” methodology were “rediscovered” in the geophysical sciences while developing the so-called “data assimilation” procedure, in that the concepts underlying “data assimilation” are the same as those underlying the previously developed “data adjustment” procedure. Since then, numerous works have been published on “data assimilation;” the most representative can be found cited in the books by Lewis et al. [7] , Lahoz et al. [8] , and Cacuci et al. [9] . The data assimilation procedures also minimize, in a least-squares procedure, a “user-defined quadratic functional” which is meant to represent the discrepancies between computed and measured responses. The main differences between the data adjustment and data assimilation methods are as follows: 1) the data adjustment method are aimed at time-dependent models/systems, whereas the data assimilation methods are aimed at time-dependent models/systems; 2) the data adjustment methods can adjust/calibrate simultaneously model parameters and model responses, whereas the data assimilation methods can adjust/calibrate only model responses and initial conditions.

In the late 1990s, Cacuci and Ionescu-Bujor [10] [11] have developed a predictive modeling methodology which significantly generalizes and extends the “data adjustment and/or assimilation” methods by dispensing with the need to minimize the a priori user-defined “cost functional”, and by replacing the respective least-squares minimization procedure with the “principle of maximum entropy” (MaxEnt), originally formulated by Jaynes [12] , to attain optimal compatibility of the available information, while simultaneously ensuring minimal spurious information content. This methodology has been extended by Cacuci [13] to include the predictive modeling of coupled multi-physics systems, leading to the development of the “BERRU-PM” mathematical framework [14] for obtaining best-estimate optimal results with reduced predicted uncertainties. The “BERRU-PM” (Best-Estimate Results with Reduced Uncertainties Predictive Modeling) methodology provides a probabilistic description of possible future outcomes based on all recognized errors and uncertainties, along with a quantitative indicator, constructed from sensitivity and covariance matrices, for determining the consistency (agreement or disagreement) among the a priori computational and experimental data for parameters and responses. This consistency indicator measures, in the corresponding metric, the deviations between the experimental and nominally computed responses, and can be evaluated directly from the originally given data (i.e., originally available parameters and responses, together with their original uncertainties), once the response sensitivities have become available.

The data adjustment and assimilation methodologies, as well as the BERRU-PM methodology, incorporate only 1^st-order sensitivities of the model responses with respect to the model’s parameters. In generalizing the BERRU-PM methodology, this limitation has been removed by Cacuci [15] [16] by having recently conceived the “2^nd-BERRU-PM” methodology, which can incorporate arbitrarily-high order sensitivities of model responses with respect to model parameters. The essential contributions of the second- and higher-order sensitivities for reducing predicted uncertainties in the model response have been illustrated in [17] [18] [19] by applying the 2nd-BERRU-PM methodology [15] [16] to the polyethylene-reflected plutonium (PERP) OECD/NEA reactor physics benchmark [20] . The response of interest for this benchmark was the leakage of neutrons through the benchmark’s outer surface, which was computed [21] using the neutron transport Boltzmann equation, involving 21,976 imprecisely known parameters, the solution of which is representative of “large-scale computations.” As detailed in [21] , the first- through fourth-order sensitivities of this benchmark’s leakage response to the benchmark’s parameters were most efficiently computed using the adjoint sensitivity analysis methodology conceived by Cacuci [22] [23] and subsequently generalized by Cacuci [24] [25] to enable the computation of arbitrarily-high order sensitivities of model responses to model parameters.

Although the 2^nd-BERRU-PM methodology can incorporate arbitrarily high sensitivities, this methodology is limited to considering just second-order moments (hence the designation “2^nd-”) of the experimentally measured and computed model responses. The third-order moments (skewness) and fourth-order moments (kurtosis) of the distribution of measured and computed responses cannot be considered within the framework of the 2^nd-BERRU-PM methodology. Furthermore, the “output” produced by the 2^nd-BERRU-PM methodology is limited to yielding optimal best-estimate values for the means and covariances (i.e., the first- and second- moments) of the best-estimate predicted distribution of responses and parameters.

On the other hand, skewness and kurtosis play an essential role in determining the asymmetries of distributions. It is therefore paramount to generalize the 2^nd-BERRU-PM methodology to enable the incorporation of third- and fourth-order moments of measured and computed responses, as well as to enable the computation of skewness and kurtosis of the best-estimate predicted posterior distribution of calibrated model parameters and responses. However, such a generalization can be achieved only if the underlying moment-constrained MaxEnt distribution could be correspondingly generalized to include third- and fourth-order moments of the distributions of measured and computed model responses.

Various variants of the “moment constrained maximum entropy problem” arise in areas as diverse as solid-state physics [26] , econometrics [27] , statistical description of gas flows [28] , weather and climate prediction [29] [30] [31] . It appears that the most efficient computational algorithm currently available for solving the “multidimensional moment constrained maximum entropy problem” is the algorithm devised by Abramov [32] , which is practically capable of solving two-dimensional problems with moments of order up to 8, three-dimensional problems with moments of order up to 6, and four-dimensional problems of order up to 4. But being able to solve “four-dimensional problems of order up to 4” is woefully insufficient for realistic problems, which are characterized by multivariate quantities of interest having dimensions much larger than just “four-dimensional”. On the other hand, the unparalleled efficiency of 2^nd-BERRU-PM methodology for handling large-scale systems was demonstrated by means of the OECD/NEA reactor physics benchmark [20] involving 21,976 imprecisely known parameters. This computational efficiency was enabled by the analytical expressions for the first-and second-order moments produced by the underlying posterior second-order MaxEnt distribution within the 2^nd-BERRU-PM methodology. It is therefore logical to attempt to extend to fourth-order the mathematical concepts and algorithms underlying the moment-constrained MaxEnt distribution within the 2^nd-BERRU-PM methodology. This extension has been accomplished as will be presented in this work.

This work is structured as follows: Section 2 presents the novel, closed-form, fourth-order moment-constrained MaxEnt probabilistic representation of the distribution of an imprecisely measured or computed multivariate quantity, the components of which will be called “responses.” Such responses are produced by computations, measurements or both. The first four moments of the unknown multivariate distribution of these response, namely their mean values, variances/ covariances, skewness, and kurtosis, are assumed to be known. Section 3 concludes this work by outlining the crucial role which the analytical expression of fourth-order moment-constrained MaxEnt distribution presented in this work will play in constructing the mathematical framework of the 4th-BERRU-PM methodology, which will be presented in the accompanying Part 2 [33] .

2. Construction of the Fourth-Order Moment-Constrained Maximum Entropy (MaxEnt) Representation of Uncertain Multivariate Quantities

The components of the imprecisely known multivariate quantity of interest will be called “system responses.” Such system responses usually stem from measurements. Also, a computed response is equivalent to a “measurement” when the details (equations, parameters, etc.) underlying the computational model are unavailable. The case when the details of the computational model are available will be analyzed in the accompanying Part 2 [33] . In order to introduce the mathematical definitions of the known information (i.e., the moments of the unknown distribution of system responses), the unknown distribution of measured model system responses will be denoted as $p_e\left(r\right)$ , $r\triangleq {\left(r_1,\cdots ,r_{TR}\right)}^{†}$ , where $r_i$ denotes the “i^th-system response”, $i=1,\cdots ,TR$ , and where TR denotes the total number of system responses of interest. The unknown distribution $p_e\left(r\right)$ is formally defined on a TR-dimensional real-valued domain $D_e\subset {ℝ}^{TR}$ . The letter “e” will be used either as a superscript or a superscript to indicate “experimentally obtained” (e.g., measured or computed with an inaccessible model) quantities. Matrices will be denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “ $\triangleq$ ” will be used to denote “is defined as” or “is by definition equal to.” Transposition will be indicated by a dagger ( $†$ ) superscript.

In this work, it is considered that the first four moments, namely: mean values, variances/covariances, triple correlations (skewness) and quadruple correlations (kurtosis), of the unknown distribution of system responses are available. The formal mathematical definitions of the first four moments, considered to be known, of the unknown distribution $p_e\left(r\right)$ , are as follows:

(i) Known mean/expectation values, denoted as $r_i^e$ , for the system responses $r_i$ , where $i=1,\cdots ,TR$ :

$r_i^e\triangleq {\displaystyle \underset{D_e}{\int }{r}_i{p}_e\left(r\right)\text{d}r};r^e\triangleq {\left(r_1^e,\cdots ,r_i^e,\cdots ,r_{TR}^e\right)}^{†};i=1,\cdots ,TR$ . (1)

(ii) Known covariances, denoted as $c_{ij}^e$ , for two system responses $r_i$ and $r_j$ , where $i,j=1,\cdots ,TR$ :

$c_{ij}^e\triangleq \mathrm{cov}{\left(r_i,r_j\right)}_e\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)p_e\left(r\right)\text{d}}r;i,j=1,\cdots ,TR;$ (2)

The covariances $\mathrm{cov}{\left(r_i,r_j\right)}_e$ , $i,j=1,\cdots ,TR$ , of the system responses are considered to be components of the $TR\times TR$ -dimensional covariance matrix of

system responses, which will be denoted as $C_{rr}^e\triangleq {\left[\mathrm{cov}{\left(r_i,r_j\right)}_e\right]}_{TR\times TR}\triangleq {\left[c_{ij}^e\right]}_{TR\times TR}$ .

(iii) Known triple correlations, denoted as $t_{ijk}^e$ , for three system responses $r_i$ , $r_j$ , $r_k$ , where $i,j,k=1,\cdots ,TR$ :

$t_{ijk}^e\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)p_e\left(r\right)\text{d}}r;i,j,k=1,\cdots ,TR;$ (3)

(iv) Known quadruple correlations, denoted as $q_{ijk\mathcal{l}}^e$ , for four system responses $r_i$ , $r_j$ , $r_k$ , $r_{\mathcal{l}}$ , where $i,j,k,\mathcal{l}=1,\cdots ,TR$ :

$q_{ijk\mathcal{l}}^e\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)\left(r_{\mathcal{l}}-r_{\mathcal{l}}^e\right)p_e\left(r\right)\text{d}}r;i,j,k,\mathcal{l}=1,\cdots ,TR.$ (4)

When an unknown distribution, such as $p_e\left(r\right)$ , needs to be reconstructed from a finite number of its known moments, the principle of maximum entropy (MaxEnt) originally formulated by Jaynes [12] provides the optimal compatibility with the available information, while simultaneously ensuring minimal spurious information content, yielding an estimate of a probability density with the highest uncertainty among all densities satisfying the known moment constraints. According to the MaxEnt principle, such a MaxEnt probability density would satisfy the “available information” provided in Equations (1)-(4), without implying any spurious information or hidden assumptions, if the following conditions are satisfied:

(a) $p_e\left(r\right)$ maximizes the Shannon [34] information entropy, S, which is defined below:

$S=-{\displaystyle \underset{D_e}{\int }{p}_e\left(r\right)\ln \left[p_e\left(r\right)\right]\text{d}r}$ , (5)

(b) $p_e\left(r\right)$ satisfies the four “moments constraints” defined by Equations (1) through (4);

(c) $p_e\left(r\right)$ satisfies the following normalization condition:

${\displaystyle \underset{D_e}{\int }{p}_e\left(r\right)\text{d}r}=1.$ (6)

The MaxEnt distribution $p_e\left(r\right)$ is obtained as the solution of the variational problem $\partial H\left(p_e\right)/\partial p_e=0$ , where the entropy (Lagrangian functional) $H\left(p_e\right)$ is defined as follows:

$\begin{array}{l}H\left(p_e\right)=-{\displaystyle \underset{D_e}{\int }{p}_e\left(r\right)\ln \left[p_e\left(r\right)\right]\text{d}r}-c_0\left[{\displaystyle \underset{D_r}{\int }{p}_e\left(r\right)\text{d}r}-1\right]\\ -{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\beta }_i{\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)p_e\left(r\right)\text{d}}r}-\frac{1}{2}{\displaystyle \underset{i,j=1}{\overset{TR}{\sum }}{\lambda }_{ij}\left[{\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)p_e\left(r\right)\text{d}}r-c_{ij}^e\right]}\\ -{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}\left[{\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)p_e\left(r\right)\text{d}}r-t_{ijk}^e\right]}\\ -{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}\left[{\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)\left(r_{\mathcal{l}}-r_{\mathcal{l}}^e\right)p_e\left(r\right)\text{d}r-q_{ijk\mathcal{l}}^e}\right].\end{array}$ (7)

In Equation (7), the quantities $c_0$ , ${\beta }_i$ , ${\lambda }_{ij}$ , ${\theta }_{ijk}$ , and ${\omega }_{ijk\mathcal{l}}$ denote the respective Lagrange multipliers, and the factor 1/2 has been introduced for subsequent computational convenience. Introducing the variable:

$z\triangleq r-r^e\triangleq {\left(z_1,\cdots ,z_{TR}\right)}^{†};r^e\triangleq {\left(r_1^e,\cdots ,r_{TR}^e\right)}^{†};z_i\triangleq r_i-r_i^e;$ (8)

and solving the equation $\partial H\left(p_e\right)/\partial p_e=0$ yields the following expression for the resulting MaxEnt distribution $p_e\left(z\right)$ :

$p_e\left(z\right)=\frac{h\left(z\right)}{{\displaystyle \underset{D_r}{\int }h\left(z\right)\text{d}z}};$ (9)

where:

$h\left(z\right)\triangleq \exp \left\{-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\beta }_i{z}_i}-\frac{1}{2}{\displaystyle \underset{i,j=1}{\overset{TR}{\sum }}{\lambda }_{ij}{z}_i{z}_j-{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}{z}_i{z}_j{z}_k-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}{z}_i{z}_j{z}_k{z}_{\mathcal{l}}}}\right\};$ (10)

Re-writing Equations (1)-(4) in terms of the variable $z\triangleq r-r^e$ yields the following formal expressions for the known first four moments of the MaxEnt distribution $p_e\left(z\right)$ :

$0={\displaystyle \underset{D_e}{\int }{z}_i{p}_e\left(r\right)\text{d}}r,i=1,\cdots ,TR;$ (11)

$c_{ij}^e\triangleq {\displaystyle \underset{D_e}{\int }{z}_i{z}_j{p}_e\left(r\right)\text{d}}r;i,j=1,\cdots ,TR;$ (12)

$t_{ijk}^e\triangleq {\displaystyle \underset{D_e}{\int }{z}_i{z}_j{z}_k{p}_e\left(r\right)\text{d}}r;i,j,k=1,\cdots ,TR;$ (13)

$q_{ijk\mathcal{l}}^e\triangleq {\displaystyle \underset{D_e}{\int }{z}_i{z}_j{z}_k{z}_{\mathcal{l}}{p}_e\left(r\right)\text{d}}r;i,j,k,\mathcal{l}=1,\cdots ,TR.$ (14)

The expression of $p_e\left(z\right)$ can be evaluated in closed-form up to fifth-order in the variables $z_i$ by expanding in Taylor-series the third- and fourth-order terms in the expression of $h\left(z\right)$ , to obtain the following approximate expression:

$p_e\left(z\right)\cong \frac{f\left(z\right)\exp \left[-g\left(z\right)\right]}{Z\left(\beta \right)}$ , (15)

where the following definitions have been used:

$\begin{array}{l}g\left(z\right)\triangleq -{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\beta }_i{z}_i}-\frac{1}{2}{\displaystyle \underset{i,j=1}{\overset{TR}{\sum }}{\lambda }_{ij}{z}_i{z}_j}=-{\beta }^{†}z-\frac{1}{2}{z}^{†}\Lambda z;\\ \beta \triangleq {\left({\beta }_1,\cdots ,{\beta }_{TR}\right)}^{†};\Lambda \triangleq \left[\begin{array}{ccc}{\lambda }_{11}& ·& {\lambda }_{1,TR}\\ ·& ·& ·\\ {\lambda }_{TR,1}& ·& {\lambda }_{TR,TR}\end{array}\right];\end{array}$ (16)

$f\left(z\right)\triangleq 1-{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}{z}_i{z}_j{z}_k-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}{z}_i{z}_j{z}_k{z}_{\mathcal{l}};$ (17)

$Z\left(\beta \right)\triangleq {\displaystyle \underset{D_r}{\int }f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z}=F_0\left(\beta \right)-{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}{F}_3^{ijk}\left(\beta \right)-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}{F}_4^{ijk\mathcal{l}}\left(\beta \right);$ (18)

where:

$F_0\left(\beta \right)\triangleq {\displaystyle \underset{D_r}{\int }\exp \left\{-{\beta }^{†}z-\frac{1}{2}{z}^{†}\Lambda z\right\}\text{d}z};$ (19)

$F_3^{ijk}\left(\beta \right)\triangleq {\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k\exp \left\{-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\beta }_i{z}_i}-\frac{1}{2}{\displaystyle \underset{i,j=1}{\overset{TR}{\sum }}{\lambda }_{ij}{z}_i{z}_j}\right\}\text{d}z};$ (20)

$F_4^{ijk\mathcal{l}}\left(\beta \right)\triangleq {\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k{z}_{\mathcal{l}}\exp \left\{-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\beta }_i{z}_i}-\frac{1}{2}{\displaystyle \underset{i,j=1}{\overset{TR}{\sum }}{\lambda }_{ij}{z}_i{z}_j}\right\}\text{d}z}.$ (21)

The closed form expression of the integral shown in Equation (19) is well known:

$F_0\left(\beta \right)\triangleq {\displaystyle \underset{D_r}{\int }\exp \left\{-{\beta }^{†}z-\frac{1}{2}{z}^{†}\Lambda z\right\}\text{d}z}=K\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)$ , (22)

where:

$K\triangleq \frac{{\left(2\pi \right)}^{TR/2}}{\sqrt{Det\left(\Lambda \right)}};$ (23)

It follows from the definitions provided in Equations (19)-(21) that:

$F_3^{ijk}\left(\beta \right)=-\frac{{\partial }^3{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k};F_4^{ijk\mathcal{l}}\left(\beta \right)=\frac{{\partial }^4{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k\partial {\beta }_{\mathcal{l}}}.$ (24)

Inserting the results obtained in Equations (19)-(24) into Equation (18) yields the following expression:

$Z\left(\beta \right)=F_0\left(\beta \right)+{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}\frac{{\partial }^3{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k}-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}\frac{{\partial }^4{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k\partial {\beta }_{\mathcal{l}}}.$ (25)

The explicit expressions for the quantities ${\partial }^3{F}_0\left(\beta \right)/\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k$ and ${\partial }^4{F}_0\left(\beta \right)/\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k\partial {\beta }_{\mathcal{l}}$ are derived in Appendix A. Inserting into Equation (25) the expression obtained in Equation (19) for $F_0\left(\beta \right)$ together with the expressions obtained in Equations (64) and (66), in Appendix A, for the quantities ${\partial }^3{F}_0\left(\beta \right)/\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k$ and ${\partial }^4{F}_0\left(\beta \right)/\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k\partial {\beta }_{\mathcal{l}}$ , respectively, yields the following expression for the normalization integral $Z\left(\beta \right)$ :

$Z\left(\beta \right)=K\left\{1+{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}{G}_3^{ijk}\left(\beta \right)-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}{G}_4^{ijk\mathcal{l}}\left(\beta \right)\right\}\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right),$ (26)

where the quantities $G_3^{ijk}\left(\beta \right)$ and $G_4^{ijk\mathcal{l}}\left(\beta \right)$ are defined in Appendix A.

In terms of the approximate expression shown in Equation (15) for the distribution $p_e\left(z\right)$ , the moments defined by Equations (11)-(14) take on the following expressions:

$0=\frac{1}{Z\left(\beta \right)}{\displaystyle \underset{D_e}{\int }{z}_i}f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z=-\frac{1}{Z\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_i},i=1,\cdots ,TR;$ (27)

$c_{ij}^e=\frac{1}{Z\left(\beta \right)}{\displaystyle \underset{D_e}{\int }{z}_i}{z}_{j}f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z=\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j};i,j=1,\cdots ,TR;$ (28)

$\begin{array}{c}{t}_{ijk}^e=\frac{1}{Z\left(\beta \right)}{\displaystyle \underset{D_e}{\int }{z}_i}{z}_j{z}_{k}f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z\\ =-\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{3}Z\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k};i,j,k=1,\cdots ,TR;\end{array}$ (29)

$\begin{array}{c}{q}_{ijk\mathcal{l}}^e=\frac{1}{Z\left(\beta \right)}{\displaystyle \underset{D_e}{\int }{z}_i}{z}_j{z}_k{z}_{\mathcal{l}}f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z\\ =\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{4}Z\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k\partial {\beta }_{\mathcal{l}}};i,j,k,\mathcal{l}=1,\cdots ,TR.\end{array}$ (30)

The relation shown in Equation (27) can be expressed in terms of the normalization integral $Z\left(\beta \right)$ as follows:

$0=-\frac{1}{Z\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_i}=-\frac{\partial }{\partial {\beta }_i}\ln \left[Z\left(\beta \right)\right]$ . (31)

It follows from Equation (26) that:

$\ln \left[Z\left(\beta \right)\right]=\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta +\ln K+\ln \left\{1+{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}{G}_3^{ijk}\left(\beta \right)-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}{G}_4^{ijk\mathcal{l}}\left(\beta \right)\right\},$ (32)

It follows from Equations (31) and (32) that:

$\begin{array}{c}0=-\frac{\partial \ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_m}\\ =-{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{mb}{\beta }_b}-\frac{\left[{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}\frac{\partial G_3^{ijk}\left(\beta \right)}{\partial {\beta }_m}\right]-\left[{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}\frac{\partial G_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_m}}\right]}{1+{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}{G}_3^{ijk}\left(\beta \right)-{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}{G}_4^{ijk\mathcal{l}}\left(\beta \right)}\\ \cong -{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{mb}{\beta }_b}-\left[{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}\frac{\partial G_3^{ijk}\left(\beta \right)}{\partial {\beta }_m}\right]+\left[{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}\frac{\partial G_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_m}}\right]+O\left({\theta }_{ijk}^2\right).\end{array}$ (33)

The last approximate equality on the right-side of Equation (33) was obtained by expanding the respective denominator in a power series and neglecting the sixth- and higher-order terms.

It follows from the expressions obtained in Equations (68) and (71) of Appendix A that:

$\frac{\partial G_3^{ijk}\left(\beta \right)}{\partial {\beta }_m}=O\left({\Lambda }^{-2}\right);\frac{\partial G_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_m}=O\left({\Lambda }^{-3}\beta \right).$ (34)

It follows from Equations (33) and (34) that if the third-order correlations among the measured system responses are neglected by setting ${\theta }_{ijk}=0$ , $i,j,k=1,\cdots ,TR$ , then the solution of these equations becomes:

$\beta =0,\text{if}{\theta }_{ijk}=0,\text{for}i,j,k=1,\cdots ,TR.$ (35)

The solution of Equation (33) cannot be obtained analytically in closed form if the third-order correlations among the measured system responses are not neglected.

Taking the second derivative of $-\frac{\partial }{\partial {\beta }_i}\ln \left[Z\left(\beta \right)\right]$ with respect to

Differentiating the relation obtained in Equation (31) with respect to another Lagrange multiplier, ${\beta }_n$ , $n=1,\cdots ,TR$ , yields the following relation:

$\begin{array}{c}-\frac{{\partial }^2\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_n\partial {\beta }_m}=\frac{\partial }{\partial {\beta }_n}\left[-\frac{1}{Z\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}\right]\\ =\frac{1}{Z^2\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_n}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}-\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}\\ =-\frac{1}{Z\left(\beta \right)}{\displaystyle \underset{D_e}{\int }{z}_n{z}_m}f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z=-c_{mn}^e;m,n=1,\cdots ,TR.\end{array}$ (36)

The last equality on the right-side of Equation (36) has been obtained by using Equations (31) and (35).

On the other hand, differentiating the relation shown in Equation (33) with respect to ${\beta }_n$ , $n=1,\cdots ,TR$ , yields the following result:

$\begin{array}{c}-\frac{{\partial }^2\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_n\partial {\beta }_m}\cong -{\left({\Lambda }^{-1}\right)}_{mn}-{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}\frac{{\partial }^2{G}_3^{ijk}\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}+{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}\frac{{\partial }^2{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}}\\ \cong -{\left({\Lambda }^{-1}\right)}_{mn}.\end{array}$ (37)

The result obtained on the rightmost-side of Equation (37) follows by evaluating all terms at $\beta =0$ (which implies that third-order correlations were neglected) and by also neglecting the fourth-order correlations by setting ${\omega }_{ijk\mathcal{l}}=0$ . The results obtained in Equations (37) and (36) indicate that

${\left({\Lambda }^{-1}\right)}_{mn}=c_{mn}^e;{\Lambda }^{-1}=C^e;C^e\triangleq {\left(c_{mn}^e\right)}_{TR\times TR};m,n=1,\cdots ,TR,$ (38)

where $C^e\triangleq {\left(c_{mn}^e\right)}_{TR\times TR}$ denotes the known $TR\times TR$ -dimensional covariance matrix of the system responses. It follows from Equation (38) that:

$K={\left(2\pi \right)}^{TR/2}\sqrt{Det\left(C^e\right)}$ . (39)

In view of Equations (34) and (38), neglecting the third- and fourth-order system response correlations while obtaining the expressions shown in Equations (35) and (38) for the Lagrange multipliers (i.e., $\beta =0$ and ${\Lambda }^{-1}=C^e$ , respectively) is equivalent to neglecting double- and triple products of measured-system response correlations.

Differentiating the relation shown in Equation (37) with respect to another Lagrange multiplier, ${\beta }_{\mu }$ , $\mu =1,\cdots ,TR$ , yields the following result:

$-\frac{{\partial }^3\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}=-{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}\frac{{\partial }^3{G}_3^{ijk}\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}+{\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}\frac{{\partial }^3{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}}$ . (40)

On the other hand, differentiating the relation shown in Equation (36) with respect to ${\beta }_{\mu }$ and evaluating the resulting expression by using the results obtained in Equations (35) and (38) for the Lagrange multipliers $\beta =0$ and ${\Lambda }^{-1}=C^e$ yields the following result:

$\begin{array}{l}-{\left\{\frac{{\partial }^3\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0}={\left\{\frac{\partial }{\partial {\beta }_{\mu }}\left[\frac{1}{Z^2\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_n}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}-\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}\right]\right\}}_{\beta =0}\\ ={\left\{-\frac{2}{Z^3\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_{\mu }}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_n}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}+\frac{1}{Z^2\left(\beta \right)}\left[\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}+\frac{\partial Z\left(\beta \right)}{\partial {\beta }_n}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_m}\right]\right\}}_{\beta =0}\\ +{\left\{\frac{1}{Z^2\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_{\mu }}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}-\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{3}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0}=-{\left\{\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{3}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0}\\ ={\left\{\frac{1}{Z\left(\beta \right)}{\displaystyle \underset{D_e}{\int }{z}_{\mu }}{z}_n{z}_{m}f\left(z\right)\exp \left[-g\left(z\right)\right]\text{d}z\right\}}_{\beta =0}=t_{\mu nm}^e;\mu ,n,m=1,\cdots ,TR.\end{array}$ (41)

The last equality shown on the right-side of Equation (41) follows from the relation provided in Equation (29).

Evaluating the expression obtained in Equation (40) by using Equations (70) and (79) from Appendix B, in conjunction with the results obtained in Equations (35) and (38) for the Lagrange multipliers $\beta =0$ and ${\Lambda }^{-1}=C^e$ yields the following relation for determining the Lagrange multipliers ${\theta }_{ijk}$ :

$\begin{array}{l}{\displaystyle \underset{i,j,k=1}{\overset{TR}{\sum }}{\theta }_{ijk}}\left\{c_{im}^e\left[c_{jn}^e{c}_{k\mu }^e+c_{j\mu }^e{c}_{kn}^e\right]+c_{jm}^e\left[c_{in}^e{c}_{k\mu }^e+c_{i\mu }^e{c}_{kn}^e\right]+c_{km}^e\left[c_{in}^e{c}_{j\mu }^e+c_{i\mu }^e{c}_{jn}^e\right]\right\}\\ =-t_{\mu nm}^e;\mu ,n,m=1,\cdots ,TR.\end{array}$ (42)

Evidently, Equation (42) could be solved numerically, but cannot be solved analytically to obtain a closed-form solution in the general case. Furthermore, the triple cross-correlations $t_{\mu nm}^e$ , $\mu ,n,m=1,\cdots ,TR$ , are seldom available in practice. Usually, only the third-order self-correlations $t_a^e\triangleq t_{aaa}^e$ , for each system response $r_a^e$ , $a=1,\cdots ,TR$ , are available in practice for the measured system responses. In this case, Equation (42) will reduce to the following form:

${\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\tau }_{ab}{\theta }_b}=-t_a^e/6;{\tau }_{ab}\triangleq {\left(c_{ab}^e\right)}^3={\left(c_{ba}^e\right)}^3;{\theta }_b\triangleq {\theta }_{bbb};a,b=1,\cdots ,TR.$ (43)

Equation (43) can be solved explicitly by inverting the matrix $T\triangleq \left({\tau }_{ab}\right)$ to obtain:

$\theta =-T^{-1}{t}^e/6;\theta \triangleq {\left({\theta }_1,\cdots ,{\theta }_{TR}\right)}^{†};t^e\triangleq {\left(t_1^e,\cdots ,t_{TR}^e\right)}^{†};T\triangleq {\left({\tau }_{ab}\right)}_{TR\times TR}.$ (44)

In particular, if only the system response variances are available but the second-order correlations among the system responses are negligible or unavailable, then the result obtained in Equation (44) reduces to the following simple expression for determining the Lagrange multipliers ${\theta }_k$ , $k=1,\cdots ,TR$ :

${\theta }_k=-\frac{t_k^e}{6{\left(c_{kk}^e\right)}^3};\text{}\text{}\text{}\text{}\text{}\text{}\text{}\text{}k=1,\cdots ,TR.$ (45)

Differentiating the relation shown in Equation (40) with respect to another Lagrange multiplier, ${\beta }_{\nu }$ , $\nu =1,\cdots ,TR$ , yields the following result:

$-\frac{{\partial }^4\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}={\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}\frac{{\partial }^4{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}};\nu ,\mu ,n,m=1,\cdots ,TR$ . (46)

The expression of ${\partial }^4{G}_4^{ijk\mathcal{l}}\left(\beta \right)/\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m$ is known, as provided in Equation (84) in Appendix A.

On the other hand, differentiating the relation shown in Equation (41) with respect to ${\beta }_{\nu }$ and evaluating the resulting expression by using the results obtained in Equations (35) and (38) for the Lagrange multipliers $\beta =0$ and ${\Lambda }^{-1}=C^e$ yields the following expression:

$\begin{array}{c}-\frac{{\partial }^4\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}=\frac{\partial }{\partial {\beta }_{\nu }}\left\{-\frac{2}{Z^3\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_{\mu }}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_n}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}\right\}\\ +\frac{\partial }{\partial {\beta }_{\nu }}\left\{\frac{1}{Z^2\left(\beta \right)}\left[\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_m}+\frac{\partial Z\left(\beta \right)}{\partial {\beta }_n}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_m}\right]\right\}\\ +\frac{\partial }{\partial {\beta }_{\nu }}\left\{\frac{1}{Z^2\left(\beta \right)}\frac{\partial Z\left(\beta \right)}{\partial {\beta }_{\mu }}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}-\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{3}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}.\end{array}$ (47)

Performing the above differentiations and evaluating the resulting expression by using the results obtained in Equations (35) and (38) for the Lagrange multipliers $\beta =0$ and ${\Lambda }^{-1}=C^e$ yields the following relation:

$\begin{array}{l}-{\left\{\frac{{\partial }^4\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0}=-{\left\{\frac{1}{Z\left(\beta \right)}\frac{{\partial }^{4}Z\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0}\\ +{\left\{\frac{1}{Z^2\left(\beta \right)}\left[\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_m}+\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_n}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_m}+\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }}\frac{{\partial }^{2}Z\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}\right]\right\}}_{\beta =0}.\end{array}$ (48)

Using the expressions provided in Equations (28) and (30) to replace the expressions on the right-side of Equation (48) yields the following result:

$-{\left\{\frac{{\partial }^4\ln \left[Z\left(\beta \right)\right]}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0}=-q_{\nu \mu nm}^e+c_{\mu n}^e{c}_{\nu m}^e+c_{\nu n}^e{c}_{\mu m}^e+c_{\nu \mu }^e{c}_{nm}^e.$ (49)

It follows from the results obtained in Equations (49) and (46) that the following relation holds for $\nu ,\mu ,n,m=1,\cdots ,TR$ :

$-q_{\nu \mu nm}^e+c_{\mu n}^e{c}_{\nu m}^e+c_{\nu n}^e{c}_{\mu m}^e+c_{\nu \mu }^e{c}_{nm}^e={\displaystyle \underset{i,j,k,\mathcal{l}=1}{\overset{TR}{\sum }}{\omega }_{ijk\mathcal{l}}}{\left\{\frac{{\partial }^4{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}\right\}}_{\beta =0,{\Lambda }^{-1}=C^e}.$ (50)

The Lagrange multipliers ${\omega }_{ijk\mathcal{l}}$ for the fourth-order correlations of the measured system responses are to be obtained as the solution of Equation (50). This equation could be solved numerically but cannot be solved analytically to obtain closed-form solutions for these Lagrange multipliers. In addition, the general fourth-order correlations $q_{\nu \mu nm}^e$ are not expected to be available in practice; at most, the fourth-order self-correlations $q_a^e\triangleq q_{aaa}^e$ , for each system response $r_a^e$ , $a=1,\cdots ,TR$ , might be available in practice for the measured system responses. In this case, Equation (50) will reduce to the following form for the quantities ${\omega }_b\triangleq {\omega }_{bbbb}$ and $G_4^b\triangleq G_4^{bbbb}$ :

$-q_a^e+3{\left(c_{aa}^e\right)}^2={\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\omega }_b}{\left\{\frac{{\partial }^4{G}_4^b\left(\beta \right)}{{\left(\partial {\beta }_a\right)}^4}\right\}}_{\beta =0,{\Lambda }^{-1}=C^e}.$ (51)

Using the result presented in Equation (84) in Appendix B yields the following expression for the quantity ${\partial }^4{G}_4^b\left(\beta \right)/{\left(\partial {\beta }_a\right)}^4$ , for $\beta =0$ and ${\Lambda }^{-1}=C^e$ :

$\begin{array}{l}{\left\{\frac{{\partial }^4{G}_4^b\left(\beta \right)}{{\left(\partial {\beta }_a\right)}^4}\right\}}_{\beta =0,{\Lambda }^{-1}=C^e}=\{c_{\mathcal{l}m}^e{c}_{in}^e\left(c_{j\mu }^e{c}_{k\nu }^e+c_{k\mu }^e{c}_{j\nu }^e\right)+c_{\mathcal{l}m}^e{c}_{jn}^e\left(c_{i\mu }^e{c}_{k\nu }^e+c_{k\mu }^e{c}_{i\nu }^e\right)\\ +c_{\mathcal{l}m}^e{c}_{kn}^e\left(c_{i\mu }^e{c}_{j\nu }^e+c_{j\mu }^e{c}_{i\nu }^e\right)+c_{im}^e{c}_{\mathcal{l}n}^e\left(c_{j\mu }^e{c}_{k\nu }^e+c_{k\mu }^e{c}_{j\nu }^e\right)+c_{im}^e{c}_{jn}^e\left(c_{\mathcal{l}\mu }^e{c}_{k\nu }^e+c_{k\mu }^e{c}_{\mathcal{l}\nu }^e\right)\\ +c_{im}^e{c}_{kn}^e\left(c_{\mathcal{l}\mu }^e{c}_{j\nu }^e+c_{j\mu }^e{c}_{\mathcal{l}\nu }^e\right)+c_{jm}^e{c}_{\mathcal{l}n}^e\left(c_{i\mu }^e{c}_{k\nu }^e+c_{k\mu }^e{c}_{i\nu }^e\right)+c_{jm}^e{c}_{in}^e\left(c_{\mathcal{l}\mu }^e{c}_{k\nu }^e+c_{k\mu }^e{c}_{\mathcal{l}\nu }^e\right)\\ +c_{jm}^e{c}_{kn}^e\left(c_{\mathcal{l}\mu }^e{c}_{i\nu }^e+c_{i\mu }^e{c}_{\mathcal{l}\nu }^e\right)+c_{km}^e{c}_{\mathcal{l}n}^e\left(c_{i\mu }^e{c}_{j\nu }^e+c_{j\mu }^e{c}_{i\nu }^e\right)\\ {+c_{km}^e{c}_{in}^e\left(c_{\mathcal{l}\mu }^e{c}_{j\nu }^e+c_{j\mu }^e{c}_{\mathcal{l}\nu }^e\right)+c_{km}^e{c}_{jn}^e\left(c_{\mathcal{l}\mu }^e{c}_{i\nu }^e+c_{i\mu }^e{c}_{\mathcal{l}\nu }^e\right)\}}_{m=n=\mu =\nu =a}^{i=j=k=\mathcal{l}=b}=24{\left(c_{ba}^e\right)}^4.\end{array}$ (52)

Inserting the result obtained in Equation (52) into the right-side of Equation (51) yields the following vector-matrix equation for determining the Lagrange multipliers ${\omega }_b\triangleq {\omega }_{bbbb}$ , $b=1,\cdots ,TR$ :

${\displaystyle \underset{b=1}{\overset{TR}{\sum }}{p}_{ab}{\omega }_b}=\left[-q_a^e+3{\left(c_{aa}^e\right)}^2\right]/24;p_{ab}\triangleq {\left(c_{ba}^e\right)}^4={\left(c_{ab}^e\right)}^4;a=1,\cdots ,TR.$ (53)

The solution $\omega \triangleq {\left({\omega }_1,\cdots ,{\omega }_{TR}\right)}^{†}$ of Equation (53) is obtained in the following form:

$\begin{array}{l}\omega =P^{-1}s/24;P\triangleq {\left(p_{ab}\right)}_{TR\times TR};s\triangleq {\left(s_1,\cdots ,s_{TR}\right)}^{†};\\ s_a\triangleq 3{\left(c_{aa}^e\right)}^2-q_a^e;a=1,\cdots ,TR.\end{array}$ (54)

In particular, if the (second-order) correlations among the measured system responses are negligible or unavailable, then the result obtained in Equation (54) reduces to the following simple expression for determining the Lagrange multipliers ${\omega }_k$ , $k=1,\cdots ,TR$ :

${\omega }_k=\frac{3{\left(c_{kk}^e\right)}^2-q_k^e}{24{\left(c_{kk}^e\right)}^4};\text{}\text{}\text{}\text{}\text{}\text{}\text{}\text{}k=1,\cdots ,TR.$ (55)

The expressions of the Lagrange multipliers obtained in Equations (35), (38), (44) and (54) are not exact, but have been obtained in closed forms, so they can be used in Equations (9) and (10) to obtain the correspondingly approximate MaxEnt distribution, which will be denoted as $p_e^{ME}\left(r\right)$ and which has the following expression:

$p_e^{ME}\left(z\right)=\frac{h_e^{ME}\left(z\right)}{{\displaystyle \underset{D_r}{\int }{h}_e^{ME}\left(z\right)\text{d}z}};h_e^{ME}\left(z\right)\triangleq \exp \left\{-\frac{1}{2}{z}^{†}\Lambda z-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\theta }_i}{z}_i^3-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i}{z}_i^4\right\}.$ (56)

where the Lagrange multipliers $\Lambda$ , ${\theta }_i$ and ${\omega }_i$ , $i=1,\cdots ,TR$ , have the expressions obtained in Equations (38), (44) and (54), respectively. The approximations incurred during the course of obtaining the expression shown in Equation (56) for the fourth-order MaxEnt distribution $p_e^{ME}\left(r\right)$ of experimentally measured system responses are summarized below:

1) When obtaining the result $\beta =0$ in Equation (35) for the Lagrange multipliers for the expected/mean values of the system responses, the third-order correlations among the system responses were neglected. This approximation is confirmed, as shown in Appendix B, namely:

$r_k^{ME}\triangleq {\displaystyle \underset{D_e}{\int }{p}_e^{ME}\left(r\right)\text{d}}r=r_k^e,\text{if}{t}_{ijk}^e=0;i,j,k=1,\cdots ,TR.$ (57)

2) When obtaining the result ${\Lambda }^{-1}=C^e$ in Equation (38) for the Lagrange multipliers for the covariances of the system responses, the third- and fourth-order correlations among the system responses were neglected. This approximation is confirmed by using the expression of the fourth-order MaxEnt distribution $p_e^{ME}\left(r\right)$ given in Equation (56), the result shown in Equation (57) and the definition of the system responses’ covariances, as shown in Appendix C, to obtain the following expression:

$\begin{array}{l}{c}_{ij}^{ME}\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)p_e^e\left(r\right)\text{d}}r=c_{ij}^e;i,j=1,\cdots ,TR;\\ \text{if}{t}_{ijk}^e=0;q_{ijk}^e=0;i,j,k,\mathcal{l}=1,\cdots ,TR.\end{array}$ (58)

3) When obtaining the result $\theta =-T^{-1}{t}^e/\left(6K\right)$ in Equation (44) for the Lagrange multipliers for the triple-correlations of the system responses, only the self-triple-correlations among the system responses were considered. This approximation is confirmed, as shown in Appendix B, by using:

i) the expression of the approximate fourth-order MaxEnt distribution $p_e^{ME}\left(r\right)$ given in Equation (56);

ii) the result shown in Equation (57); and

iii) the definition of the self-correlations of third-order for the system responses.

Thus, as shown in Appendix B, the following expression is obtained for the triple-correlations $t_{ijk}^e$ ; $i,j,k=1,\cdots ,TR$ :

$t_{ijk}^{ME}\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)p_e^{ME}\left(r\right)\text{d}}r=t_{ijk}^e;i,j,k=1,\cdots ,TR.$ (59)

4) When obtaining the result $\omega =P^{-1}s/\left(24K\right)$ in Equation (54) for the Lagrange multipliers for the quadruple-correlations of the measured system responses, only the self-correlations of fourth order among the measured system responses were considered. This approximation is confirmed by using the expression of the approximate fourth-order MaxEnt distribution $p_e^{ME}\left(r\right)$ given in Equation (56), the result shown in Equation (57), and the definition of the self-correlations of third-order for the measured system responses to obtain, as shown in Appendix B, the following expression for the quadruple-correlations $q_{ijk\mathcal{l}}^e$ , $i,j,k,\mathcal{l}=1,\cdots ,TR$ :

$q_{ijk\mathcal{l}}^{ME}\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)\left(r_{\mathcal{l}}-r_{\mathcal{l}}^e\right)p_e^{ME}\left(r\right)\text{d}}r=q_{ijk\mathcal{l}}^e.$ (60)

In summary, the results obtained in Equations (57)-(60) indicate that the known/given means, covariances, triple and quadruple correlations for the system responses can all be used as values for the corresponding moments of the approximate fourth-order MaxEnt distribution $p_e^{ME}\left(r\right)$ given in Equation (56), since the approximations incurred when using this correspondence are at least of 2-orders higher than for the moment in question, namely:

1) The first-order moments of $p_e^{ME}\left(r\right)$ have third-order errors by comparison to the measured mean values.

2) The second-order moments of $p_e^{ME}\left(r\right)$ have fourth-order errors by comparison to the measured mean values.

3) The third-order moments of $p_e^{ME}\left(r\right)$ have fifth-order errors by comparison to the measured mean values.

4) The fourth-order moments of $p_e^{ME}\left(r\right)$ have sixth-order errors by comparison to the measured mean values.

5) Notably, if the triple- and the quadruple correlations are negligeable (or unavailable) then the MaxEnt distribution $p_e^{ME}\left(r\right)$ reduces to a multivariate Gaussian with mean $r^e$ and covariance matrix $C^e$ .

3. Discussion and Conclusions

This work has presented a novel closed-form expression for the fourth-order moments-constrained Maximum Entropy (MaxEnt) probability distribution, which was constructed from the known first four moments (means, covariances, skewness, kurtosis) of an otherwise unknown distribution of a high-dimensional multivariate uncertain quantity of interest, which was called a “system response”. This fourth-order MaxEnt distribution provides optimal compatibility of the available information while simultaneously ensuring minimal spurious information content, yielding an estimate of a probability density with the highest uncertainty among all densities satisfying the known moment constraints. This novel generic fourth-order MaxEnt distribution is of interest in its own right for applications in many areas, including solid-state physics, econometrics, statistical description of gas flows, weather and climate prediction. In the accompanying work [33] , this novel generic fourth-order MaxEnt distribution will be used to construct a novel fourth-order predictive modeling methodology aimed at obtaining “best-estimate results with reduced uncertainties” for the first four moments (mean values, covariance, skewness and kurtosis) of the optimally predicted distribution of model results and calibrated model parameters, by combining fourth-order experimental and computational information, including fourth (and higher) order sensitivities of computed model system responses to model parameters.

Appendix A: Auxiliary Computations for Constructing the Moment-Constrained Fourth-Order MaxEnt Distribution

Recall that the following closed-form expression of $F_0\left(\beta \right)$ was provided in Equation (19): $F_0\left(\beta \right)=K\exp \left({\beta }^{†}{\Lambda }^{-1}\beta /2\right)$ . Differentiating this expression with respect to a Lagrange multiplier ${\beta }_i$ yields the following relation:

$\frac{\partial F_0\left(\beta \right)}{\partial {\beta }_i}=K\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)$ . (61)

Differentiating the expressions in Equation (61) with respect to a Lagrange multiplier ${\beta }_j$ yields the following relations:

$\begin{array}{c}\frac{{\partial }^2{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j}=K\frac{\partial }{\partial {\beta }_j}\left\{\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\right]\right\}\\ =K{G}_2^{ij}\left(\beta \right)\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right);\end{array}$ (62)

where:

$G_2^{ij}\left(\beta \right)\triangleq {\left({\Lambda }^{-1}\right)}_{ij}+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right];$ (63)

Differentiating the expressions in Equation (62) with respect to a Lagrange multiplier ${\beta }_k$ yields the following relations:

$\begin{array}{l}\frac{{\partial }^3{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k}=K\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\frac{\partial }{\partial {\beta }_k}\left\{{\left({\Lambda }^{-1}\right)}_{ij}+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\right\}\\ +K\left\{{\left({\Lambda }^{-1}\right)}_{ij}+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\right\}\frac{\partial }{\partial {\beta }_k}\left[\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\right]\\ =K\left\{{\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]+{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\right\}\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\\ +K\left\{{\left({\Lambda }^{-1}\right)}_{ij}+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\right\}\\ \times \left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)=K{G}_3^{ijk}\left(\beta \right)\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right);\end{array}$ (64)

where:

$\begin{array}{l}{G}_3^{ijk}\left(\beta \right)\triangleq {\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]+{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\\ +{\left({\Lambda }^{-1}\right)}_{ij}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right];\end{array}$ (65)

Differentiating the expressions in Equation (64) with respect to a Lagrange multiplier ${\beta }_{\mathcal{l}}$ yields the following relations:

$\begin{array}{l}\frac{{\partial }^4{F}_0\left(\beta \right)}{\partial {\beta }_i\partial {\beta }_j\partial {\beta }_k\partial {\beta }_{\mathcal{l}}}=K\{{\left({\Lambda }^{-1}\right)}_{ij}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\\ +{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\end{array}$

$\begin{array}{l}\times \left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\}\frac{\partial }{\partial {\beta }_{\mathcal{l}}}\left[\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\right]+K\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\\ \times \frac{\partial }{\partial {\beta }_{\mathcal{l}}}\{{\left({\Lambda }^{-1}\right)}_{ij}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\\ +{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\}\end{array}$

$\begin{array}{l}=K\{{\left({\Lambda }^{-1}\right)}_{ij}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]+{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\\ +\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\}\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\\ +K\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right)\{{\left({\Lambda }^{-1}\right)}_{ij}{\left({\Lambda }^{-1}\right)}_{k\mathcal{l}}+{\left({\Lambda }^{-1}\right)}_{ik}{\left({\Lambda }^{-1}\right)}_{j\mathcal{l}}+{\left({\Lambda }^{-1}\right)}_{i\mathcal{l}}{\left({\Lambda }^{-1}\right)}_{jk}\\ +{\left({\Lambda }^{-1}\right)}_{i\mathcal{l}}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{j\mathcal{l}}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +{\left({\Lambda }^{-1}\right)}_{k\mathcal{l}}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\}=K{G}_4^{ijk\mathcal{l}}\left(\beta \right)\exp \left(\frac{1}{2}{\beta }^{†}{\Lambda }^{-1}\beta \right),\end{array}$ (66)

where:

$\begin{array}{c}{G}_4^{ijk\mathcal{l}}\left(\beta \right)\triangleq \left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\{{\left({\Lambda }^{-1}\right)}_{ij}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +{\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]+{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\}\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +{\left({\Lambda }^{-1}\right)}_{ij}{\left({\Lambda }^{-1}\right)}_{k\mathcal{l}}+{\left({\Lambda }^{-1}\right)}_{ik}{\left({\Lambda }^{-1}\right)}_{j\mathcal{l}}+{\left({\Lambda }^{-1}\right)}_{i\mathcal{l}}{\left({\Lambda }^{-1}\right)}_{jk}\\ +{\left({\Lambda }^{-1}\right)}_{i\mathcal{l}}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{j\mathcal{l}}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\\ \times \left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{k\mathcal{l}}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right].\end{array}$ (67)

The various derivatives of the functions $G_3^{ijk}\left(\beta \right)$ and $G_4^{ijk\mathcal{l}}\left(\beta \right)$ with respect to the Lagrange multipliers $\partial {\beta }_m$ , $m=1,\cdots ,TR$ , will also be used when evaluating the various derivatives of the normalization integral $Z\left(\beta \right)$ with respect to these Lagrange multipliers. The derivatives of the function $G_3^{ijk}\left(\beta \right)$ with respect to the Lagrange multipliers ${\beta }_m$ , $m=1,\cdots ,TR$ , are obtained by using Equation (65), which yields the following expressions:

$\begin{array}{c}\frac{\partial G_3^{ijk}\left(\beta \right)}{\partial {\beta }_m}={\left({\Lambda }^{-1}\right)}_{ik}{\left({\Lambda }^{-1}\right)}_{jm}+{\left({\Lambda }^{-1}\right)}_{jk}{\left({\Lambda }^{-1}\right)}_{im}+{\left({\Lambda }^{-1}\right)}_{ij}{\left({\Lambda }^{-1}\right)}_{km}\\ +{\left({\Lambda }^{-1}\right)}_{im}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{jm}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]{\left({\Lambda }^{-1}\right)}_{km}.\end{array}$ (68)

Differentiating the expression in Equation (68) with respect to a Lagrange multiplier ${\beta }_n$ yields the following relation:

$\begin{array}{c}\frac{{\partial }^2{G}_3^{ijk}\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}={\left({\Lambda }^{-1}\right)}_{im}\left\{{\left({\Lambda }^{-1}\right)}_{jn}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]{\left({\Lambda }^{-1}\right)}_{kn}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{jm}\left\{{\left({\Lambda }^{-1}\right)}_{in}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{kn}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{km}\left\{{\left({\Lambda }^{-1}\right)}_{in}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{jn}\right\}.\end{array}$ (69)

Differentiating the expression in Equation (69) with respect to a Lagrange multiplier ${\beta }_{\mu }$ yields the following relation:

$\begin{array}{c}\frac{{\partial }^3{G}_3^{ijk}\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}={\left({\Lambda }^{-1}\right)}_{im}\left\{{\left({\Lambda }^{-1}\right)}_{jn}{\left({\Lambda }^{-1}\right)}_{k\mu }+{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{kn}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{jm}\left\{{\left({\Lambda }^{-1}\right)}_{in}{\left({\Lambda }^{-1}\right)}_{k\mu }+{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{kn}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{km}\left\{{\left({\Lambda }^{-1}\right)}_{in}{\left({\Lambda }^{-1}\right)}_{j\mu }+{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{jn}\right\}.\end{array}$ (70)

The derivatives of the function $G_4^{ijk\mathcal{l}}\left(\beta \right)$ with respect to a Lagrange multipliers ${\beta }_m$ , $m=1,\cdots ,TR$ , are obtained by using Equation (67). Thus, the first derivative of the function $G_4^{ijk\mathcal{l}}\left(\beta \right)$ with respect to a Lagrange multipliers ${\beta }_m$ , $m=1,\cdots ,TR$ , has the following expression:

$\begin{array}{c}\frac{\partial G_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_m}={\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}\{{\left({\Lambda }^{-1}\right)}_{ij}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+{\left({\Lambda }^{-1}\right)}_{ik}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\\ +{\left({\Lambda }^{-1}\right)}_{jk}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\}+\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\{{\left({\Lambda }^{-1}\right)}_{ij}{\left({\Lambda }^{-1}\right)}_{km}\\ +{\left({\Lambda }^{-1}\right)}_{ik}{\left({\Lambda }^{-1}\right)}_{jm}+{\left({\Lambda }^{-1}\right)}_{jk}{\left({\Lambda }^{-1}\right)}_{im}\}\\ +{\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]{\left({\Lambda }^{-1}\right)}_{im}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\end{array}$

$\begin{array}{c}+\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{jm}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]{\left({\Lambda }^{-1}\right)}_{km}\\ +{\left({\Lambda }^{-1}\right)}_{i\mathcal{l}}\left\{{\left({\Lambda }^{-1}\right)}_{jm}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]{\left({\Lambda }^{-1}\right)}_{km}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{j\mathcal{l}}\left\{{\left({\Lambda }^{-1}\right)}_{im}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{km}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{k\mathcal{l}}\left\{{\left({\Lambda }^{-1}\right)}_{im}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]+\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{jm}\right\}.\end{array}$ (71)

Differentiating Equation (71) with respect to a Lagrange multiplier ${\beta }_n$ $n=1,\cdots ,TR$ , yields the following relation:

$\frac{{\partial }^2{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_n\partial {\beta }_m}=X_1+X_2+X_3+X_4+X_5+X_6$ , (72)

where:

$\begin{array}{c}{X}_1\triangleq {\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}\left\{{\left({\Lambda }^{-1}\right)}_{ij}{\left({\Lambda }^{-1}\right)}_{kn}+{\left({\Lambda }^{-1}\right)}_{ik}{\left({\Lambda }^{-1}\right)}_{jn}+{\left({\Lambda }^{-1}\right)}_{jk}{\left({\Lambda }^{-1}\right)}_{in}\right\}\\ +{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left\{{\left({\Lambda }^{-1}\right)}_{ij}{\left({\Lambda }^{-1}\right)}_{km}+{\left({\Lambda }^{-1}\right)}_{ik}{\left({\Lambda }^{-1}\right)}_{jm}+{\left({\Lambda }^{-1}\right)}_{jk}{\left({\Lambda }^{-1}\right)}_{im}\right\},\end{array}$ (73)

$\begin{array}{c}{X}_2\triangleq {\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}\{{\left({\Lambda }^{-1}\right)}_{in}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{jn}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]{\left({\Lambda }^{-1}\right)}_{kn}\},\end{array}$ (74)

$\begin{array}{c}{X}_3\triangleq {\left({\Lambda }^{-1}\right)}_{im}\{{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]{\left({\Lambda }^{-1}\right)}_{jn}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]{\left({\Lambda }^{-1}\right)}_{kn}\},\end{array}$ (75)

$\begin{array}{c}{X}_4\triangleq {\left({\Lambda }^{-1}\right)}_{jm}\{{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]{\left({\Lambda }^{-1}\right)}_{in}\left[{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{kn}\},\end{array}$ (76)

$\begin{array}{c}{X}_5\triangleq {\left({\Lambda }^{-1}\right)}_{km}\{{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]{\left({\Lambda }^{-1}\right)}_{in}\left[{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}\right]\\ +\left[{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]{\left({\Lambda }^{-1}\right)}_{jn}\},\end{array}$ (77)

$\begin{array}{c}{X}_6\triangleq {\left({\Lambda }^{-1}\right)}_{i\mathcal{l}}\left[{\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{kn}+{\left({\Lambda }^{-1}\right)}_{jn}{\left({\Lambda }^{-1}\right)}_{km}\right]\\ +{\left({\Lambda }^{-1}\right)}_{j\mathcal{l}}\left[{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{kn}+{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{km}\right]\\ +{\left({\Lambda }^{-1}\right)}_{k\mathcal{l}}\left[{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{jn}+{\left({\Lambda }^{-1}\right)}_{in}{\left({\Lambda }^{-1}\right)}_{jm}\right].\end{array}$ (78)

Differentiating Equation (72) with respect to a Lagrange multiplier ${\beta }_{\mu }$ $\mu =1,\cdots ,TR$ , yields the following relation:

$\frac{{\partial }^3{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}=Y_1+Y_2+Y_3+Y_4,$ (79)

where:

$\begin{array}{c}{Y}_1\triangleq {\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}{\left({\Lambda }^{-1}\right)}_{in}\left[{\left({\Lambda }^{-1}\right)}_{j\mu }{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c+{\left({\Lambda }^{-1}\right)}_{k\mu }{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}}\right]\\ +{\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}{\left({\Lambda }^{-1}\right)}_{jn}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}+{\left({\Lambda }^{-1}\right)}_{k\mu }{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\\ +{\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}{\left({\Lambda }^{-1}\right)}_{kn}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b+{\left({\Lambda }^{-1}\right)}_{j\mu }{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}}\right],\end{array}$ (80)

$\begin{array}{c}{Y}_2\triangleq {\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\left({\Lambda }^{-1}\right)}_{j\mu }{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c+{\left({\Lambda }^{-1}\right)}_{k\mu }{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}}\right]\\ +{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{jn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c}+{\left({\Lambda }^{-1}\right)}_{k\mu }{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\\ +{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{kn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}+{\left({\Lambda }^{-1}\right)}_{j\mu }{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right],\end{array}$ (81)

$\begin{array}{c}{Y}_3\triangleq {\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c+{\left({\Lambda }^{-1}\right)}_{k\mu }{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}}\right]\\ +{\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{in}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\displaystyle \underset{c=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{kc}{\beta }_c+{\left({\Lambda }^{-1}\right)}_{k\mu }{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}}\right]\\ +{\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{kn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}+{\left({\Lambda }^{-1}\right)}_{i\mu }{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right],\end{array}$ (82)

$\begin{array}{c}{Y}_4\triangleq {\left({\Lambda }^{-1}\right)}_{km}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}+{\left({\Lambda }^{-1}\right)}_{j\mu }{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}\right]\\ +{\left({\Lambda }^{-1}\right)}_{km}{\left({\Lambda }^{-1}\right)}_{in}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\displaystyle \underset{b=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{jb}{\beta }_b}+{\left({\Lambda }^{-1}\right)}_{j\mu }{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right]\\ +{\left({\Lambda }^{-1}\right)}_{km}{\left({\Lambda }^{-1}\right)}_{jn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{ia}{\beta }_a}+{\left({\Lambda }^{-1}\right)}_{i\mu }{\displaystyle \underset{d=1}{\overset{TR}{\sum }}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}d}{\beta }_d}\right].\end{array}$ (83)

Differentiating Equation (79) with respect to a Lagrange multiplier ${\beta }_{\nu }$ , $\nu =1,\cdots ,TR$ , yields the following relation:

$\frac{{\partial }^4{G}_4^{ijk\mathcal{l}}\left(\beta \right)}{\partial {\beta }_{\nu }\partial {\beta }_{\mu }\partial {\beta }_n\partial {\beta }_m}=W_1+W_2+W_3+W_4,$ (84)

where:

$\begin{array}{c}{W}_1\triangleq {\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}{\left({\Lambda }^{-1}\right)}_{in}\left[{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{k\nu }+{\left({\Lambda }^{-1}\right)}_{k\mu }{\left({\Lambda }^{-1}\right)}_{j\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}{\left({\Lambda }^{-1}\right)}_{jn}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{k\nu }+{\left({\Lambda }^{-1}\right)}_{k\mu }{\left({\Lambda }^{-1}\right)}_{i\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{\mathcal{l}m}{\left({\Lambda }^{-1}\right)}_{kn}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{j\nu }+{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{i\nu }\right],\end{array}$ (85)

$\begin{array}{c}{W}_2\triangleq {\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{k\nu }+{\left({\Lambda }^{-1}\right)}_{k\mu }{\left({\Lambda }^{-1}\right)}_{j\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{jn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\left({\Lambda }^{-1}\right)}_{k\nu }+{\left({\Lambda }^{-1}\right)}_{k\mu }{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{im}{\left({\Lambda }^{-1}\right)}_{kn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\left({\Lambda }^{-1}\right)}_{j\nu }+{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\nu }\right],\end{array}$ (86)

$\begin{array}{c}{W}_3\triangleq {\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{k\nu }+{\left({\Lambda }^{-1}\right)}_{k\mu }{\left({\Lambda }^{-1}\right)}_{i\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{in}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\left({\Lambda }^{-1}\right)}_{k\nu }+{\left({\Lambda }^{-1}\right)}_{k\mu }{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{jm}{\left({\Lambda }^{-1}\right)}_{kn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\left({\Lambda }^{-1}\right)}_{i\nu }+{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\nu }\right],\end{array}$ (87)

$\begin{array}{c}{W}_4\triangleq {\left({\Lambda }^{-1}\right)}_{km}{\left({\Lambda }^{-1}\right)}_{\mathcal{l}n}\left[{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{j\nu }+{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{i\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{km}{\left({\Lambda }^{-1}\right)}_{in}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\left({\Lambda }^{-1}\right)}_{j\nu }+{\left({\Lambda }^{-1}\right)}_{j\mu }{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\nu }\right]\\ +{\left({\Lambda }^{-1}\right)}_{km}{\left({\Lambda }^{-1}\right)}_{jn}\left[{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\mu }{\left({\Lambda }^{-1}\right)}_{i\nu }+{\left({\Lambda }^{-1}\right)}_{i\mu }{\left({\Lambda }^{-1}\right)}_{\mathcal{l}\nu }\right].\end{array}$ (88)

Appendix B: Approximations Inherent to the Fourth-Order Maximum Entropy Distribution

Recalling Equation (56), the expression of the approximate fourth-order MaxEnt probability distribution function $p_e^{ME}\left(r\right)$ for the responses has the following form:

$p_e^{ME}\left(z\right)=\frac{h_e^{ME}\left(z\right)}{{\displaystyle \underset{D_r}{\int }{h}_e^{ME}\left(z\right)\text{d}z}}\cong \frac{1}{Z_e^{ME}}{f}_e^{ME}\left(r\right)\exp \left[-g_e^{ME}\left(r\right)\right],$ (89)

where:

$g_e^{ME}\left(r\right)\triangleq \exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right];$ (90)

$f_e^{ME}\left(r\right)\triangleq 1-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\theta }_i}{\left(r_i-r_i^e\right)}^3-{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i}{\left(r_i-r_i^e\right)}^4;$ (91)

$Z_e^{ME}={\displaystyle \underset{D_r}{\int }{f}_e^{ME}\left(r\right)\exp \left[-g_e^{ME}\left(r\right)\right]\text{d}r}=K\left(1-{\omega }^{†}{g}_4\right)=K\left[1-3{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i{\left(c_{ii}^e\right)}^2}\right]$ . (92)

The MaxEnt probability distribution function $p_e^{ME}\left(r\right)$ is properly normalized, as shown below:

$\begin{array}{c}{\displaystyle \underset{D_r}{\int }{p}_e^{ME}\left(r\right)\text{d}r}=K^{-1}{\left[1-3{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i{\left(c_{ii}^e\right)}^2}\right]}^{-1}\{{\displaystyle \underset{D_r}{\int }\exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right]\text{d}r}\\ -{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\theta }_i}{\displaystyle \underset{D_r}{\int }{\left(r_i-r_i^e\right)}^3\exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right]\text{d}r}\\ -{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i}{\displaystyle \underset{D_r}{\int }{\left(r_i-r_i^e\right)}^4\exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right]\text{d}r}\}\\ =K^{-1}{\left[1-3{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i{\left(c_{ii}^e\right)}^2}\right]}^{-1}\left[K-0-3K{\displaystyle \underset{i=1}{\overset{TR}{\sum }}{\omega }_i{\left(c_{ii}^e\right)}^2}\right]=1.\end{array}$ (93)

The first-order moments $r_i^{ME}$ , $i=1,\cdots ,TR$ , of $p_e^{ME}\left(r\right)$ are given by the following expression:

$r_i^{ME}\triangleq {\displaystyle \underset{D_r}{\int }{r}_i{p}_e^{ME}\left(r\right)\text{d}r}$

$\begin{array}{l}=K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\{{\displaystyle \underset{D_r}{\int }{r}_k}\exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right]\text{d}r\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a}{\displaystyle \underset{D_r}{\int }{r}_i}{\left(r_a-r_a^e\right)}^3\exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right]\text{d}r\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a}{\displaystyle \underset{D_r}{\int }{r}_i{\left(r_a-r_a^e\right)}^4}\exp \left[-\frac{1}{2}{\left(r-r^e\right)}^{†}{\left(C^e\right)}^{-1}\left(r-r^e\right)\right]\text{d}r\}\end{array}$

$\begin{array}{l}=K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\{r_i^e{\displaystyle \underset{D_r}{\int }\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z}\\ -{\displaystyle \underset{D_r}{\int }{z}_i}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z-{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a}{r}_i^e{\displaystyle \underset{D_r}{\int }{z}_a^3}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_a^3}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z-{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a}{r}_i^e{\displaystyle \underset{D_r}{\int }{z}_a^4}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_a^4}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\}\end{array}$

$\begin{array}{l}=K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\\ \times \left\{K{r}_i^e-0-0-O\left[K{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a{\left(c_{aa}^e\right)}^2}\right]-3K{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{r}_i^e{\left(c_{aa}^e\right)}^2-0}\right\}\\ =K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}K\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]r_i^e\\ -K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\times K\times O\left[{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{t}_a^e{\left(c_{aa}^e\right)}^{-3}{\left(c_{aa}^e\right)}^2}\right]\\ =r_i^e+O\left(t_i^e/c_{ii}^e\right).\end{array}$ (94)

As indicated by the result obtained in Equation (94), if the third-order correlations $t_{ijk}^e$ among the measured responses are all neglected, then the MaxEnt moments $r_k^{ME}$ coincide with the measured moments, $r_k^e$ (i.e., $r_k^{ME}=r_k^e$ if $t_{ijk}^e=0$ ).

Within the approximation $r_k^{ME}=r_k^e$ , the second-order moments $c_{ij}^{ME}$ , $i,j=1,\cdots ,TR$ , of $p_e^{ME}\left(r\right)$ are given by the following expression:

$\begin{array}{c}{c}_{ij}^{ME}\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)p_e^{ME}\left(r\right)\text{d}}r\\ =K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\{{\displaystyle \underset{D_r}{\int }{z}_i{z}_j\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z}\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_a^3}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_a^4}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\}\\ =K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}K\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]\left\{c_{ij}^e-0-O\left({\omega }_a\right)\right\}\\ =c_{ij}^e;i,j=1,\cdots ,TR.\end{array}$ (95)

Within the approximation $r_k^{ME}=r_k^e$ , the third-order moments $t_{ijk}^{ME}$ , $i,j,k=1,\cdots ,TR$ , of $p_e^{ME}\left(r\right)$ are given by the following expression:

$\begin{array}{c}{t}_{ijk}^{ME}\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)p_e^{ME}\left(r\right)\text{d}}r\\ =K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\{{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z}\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k{z}_a^3}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k{z}_a^4}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\}\\ =t_{ijk}^e+HOT,\end{array}$ (96)

where the acronym “HOT” denotes “higher order terms.”

Within the approximation $r_k^{ME}=r_k^e$ , the fourth-order moments $q_{ijk}^e$ , $i,j,k,\mathcal{l}=1,\cdots ,TR$ , of $p_e^{ME}\left(r\right)$ are given by the following expression:

$\begin{array}{c}{q}_{ijk\mathcal{l}}^{ME}\triangleq {\displaystyle \underset{D_e}{\int }\left(r_i-r_i^e\right)\left(r_j-r_j^e\right)\left(r_k-r_k^e\right)\left(r_{\mathcal{l}}-r_{\mathcal{l}}^e\right)p_e^{ME}\left(r\right)\text{d}}r\\ =K^{-1}{\left[1-3{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a{\left(c_{aa}^e\right)}^2}\right]}^{-1}\{{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k{z}_{\mathcal{l}}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z}\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\theta }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k{z}_{\mathcal{l}}{z}_a^3}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\\ -{\displaystyle \underset{a=1}{\overset{TR}{\sum }}{\omega }_a}{\displaystyle \underset{D_r}{\int }{z}_i{z}_j{z}_k{z}_{\mathcal{l}}{z}_a^4}\exp \left[-\frac{1}{2}{z}^{†}{\left(C^e\right)}^{-1}z\right]\text{d}z\}\\ =q_{ijk\mathcal{l}}^e+HOT,\end{array}$ (97)

where the acronym “HOT” denotes “higher order terms”.

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

References

[1]	Humi, M., Wagschal, J.J. and Yeivin, Y. (1964) Multi-Group Constants from Integral Data. Proceedings of the 3rd International Conference on the Peaceful Uses of Atomic Energy, Geneva, 31 August-9 September 1964, 398 p.
[2]	Usachhev, L.N. (1964) Perturbation Theory for the Breeding Ratio and for Other Number Ratios Pertaining to Various Reactor Processes. Journal of Nuclear Energy. Parts A/B. Reactor Science and Technology, 18, 571-583. https://doi.org/10.1016/0368-3230(64)90142-9
[3]	Rowlands, J., et al. (1973) The Production and Performance of the Adjusted Cross-Section Set FGL5. Proceedings of the International Symposium on Physics of Fast Reactors, Tokyo, 16 October 1973.
[4]	Gandini, A. and Petilli, M. (1973) AMARA: A Code Using the Lagrange’s Multipliers Method for Nuclear Data Adjustment. CNEN-RI/FI(73)39, Comitato Nazionale Energia Nucleare, Rome.
[5]	Kuroi, H. and Mitani, H. (1975) Adjustment to Cross Section Data to fit Integral Experiments by Least Squares Method. Journal of Nuclear Science and Technology, 12, 663-680. https://doi.org/10.1080/18811248.1975.9733172
[6]	Dragt, J.B., Dekker, J.W.M., Gruppelaar, H. and Janssen, A.J. (1977) Methods of Adjustment and Error Evaluation of Neutron Capture Cross Sections; Application to Fission Product Nuclides. Nuclear Science and Engineering, 62, 117-129. https://doi.org/10.13182/NSE77-3
[7]	Lewis, J.M., Lakshmivarahan, S. and Dhall, S.K. (2006) Dynamic Data Assimilation: A Least Square Approach. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511526480
[8]	Lahoz, W., Khattatov, B. and Ménard, R. (2010) Data Assimilation: Making Sense of Observations. Springer, Berlin.
[9]	Cacuci, D.G., Navon, M.I. and Ionescu-Bujor, M. (2014) Computational Methods for Data Evaluation and Assimilation. Chapman & Hall/CRC, Boca Raton.
[10]	Cacuci, D.G. and Ionescu-Bujor, M. (2010) Best-Estimate Model Calibration and Prediction through Experimental Data Assimilation—I: Mathematical Framework. Nuclear Science and Engineering, 165, 18-44. https://doi.org/10.13182/NSE09-37B
[11]	Cacuci, D.G. and Ionescu-Bujor, M. (2010) Sensitivity and Uncertainty Analysis, Data Assimilation and Predictive Best-Estimate Model Calibration. In: Cacuci, D.G., Ed., Handbook of Nuclear Engineering, Springer, Boston, 1913-2051. https://doi.org/10.1007/978-0-387-98149-9_17
[12]	Jaynes, E.T. (1957) Information Theory and Statistical Mechanics. Physical Review Journals Archive, 106, 620-630. https://doi.org/10.1103/PhysRev.106.620
[13]	Cacuci, D.G. (2014) Predictive Modeling of Coupled Multi-Physics Systems: I. Theory. Annals of Nuclear Energy, 70, 266-278. https://doi.org/10.1016/j.anucene.2013.11.027
[14]	Cacuci, D.G. (2019) BERRU Predictive Modeling: Best-Estimate Results with Reduced Uncertainties. Springer, Berlin. https://doi.org/10.1007/978-3-662-58395-1
[15]	Cacuci, D.G. (2023) Second-Order MaxEnt Predictive Modelling Methodology. I: Deterministically Incorporated Computational Model (2nd-BERRU-PMD). American Journal of Computational Mathematics, 13, 236-266. https://doi.org/10.4236/ajcm.2023.132013
[16]	Cacuci, D.G. (2023) Second-Order MaxEnt Predictive Modelling Methodology. II: Probabilistically Incorporated Computational Model (2nd-BERRU-PMP). American Journal of Computational Mathematics, 13, 267-294. https://doi.org/10.4236/ajcm.2023.132014
[17]	Fang, R. and Cacuci, D.G. (2023) Second-Order MaxEnt Predictive Modelling Methodology. III: Illustrative Application to a Reactor Physics Benchmark. American Journal of Computational Mathematics, 13, 295-322. https://doi.org/10.4236/ajcm.2023.132015
[18]	Cacuci, D.G. and Fang, R. (2023) Demonstrative Application to an OECD/NEA Reactor Physics Benchmark of the 2nd-BERRU-PM Method—I: Nominal Computations Consistent with Measurements. Energies, 16, Article No. 5552. https://doi.org/10.3390/en16145552
[19]	Fang, R. and Cacuci, D.G. (2023) Demonstrative Application to an OECD/NEA Reactor Physics Benchmark of the 2nd-BERRU-PM Method—II: Nominal Computations Apparently Inconsistent with Measurements. Energies, 16, Article No. 5614. https://doi.org/10.3390/en16155614
[20]	Valentine, T.E. (2006) Polyethylene-Reflected Plutonium Metal Sphere Subcritical Noise Measurements. SUB-PU-METMIXED-001. In: International Handbook of Evaluated Criticality Safety Benchmark Experiments; NEA/NSC/DOC(95)03/I-IX; Organization for Economic Co-Operation and Development; Nuclear Energy Agency, Paris.
[21]	Cacuci, D.G. and Fang, R. (2023) The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology, Volume II: Overcoming the Curse of Dimensionality: Large-Scale Application. Springer, Cham. https://doi.org/10.1007/978-3-031-19635-5
[22]	Cacuci, D.G. (1981) Sensitivity Theory for Nonlinear Systems: I. Nonlinear Functional Analysis Approach. Journal of Mathematical Physics, 22, 2794-2802. https://doi.org/10.1063/1.525186
[23]	Cacuci, D.G. (1981) Sensitivity Theory for Nonlinear Systems: II. Extensions to Additional Classes of Responses. Journal of Mathematical Physics, 22, 2803-2812. https://doi.org/10.1063/1.524870
[24]	Cacuci, D.G. (2022) The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology, Volume I: Overcoming the Curse of Dimensionality: Linear Systems. Springer, Cham, 362 p. https://doi.org/10.1007/978-3-030-96364-4
[25]	Cacuci, D.G. (2023) The nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology, Volume III: Overcoming the Curse of Dimensionality: Nonlinear Systems. Springer, Cham, 369 p. https://doi.org/10.1007/978-3-031-22757-8
[26]	Bandyopadhyay, K., Bhattacharya, A., Biswas, P. and Drabold, D.A. (2005) Maximum Entropy and the Problem of Moments: A Stable Algorithm. Physical Review E, 71, Article ID: 057701. https://doi.org/10.1103/PhysRevE.71.057701
[27]	Wu, X. (2003) Calculation of Maximum Entropy Densities with Application to Income Distribution. Journal of Econometrics, 115, 347-354. https://doi.org/10.1016/S0304-4076(03)00114-3
[28]	Müller, I. and Ruggeri, T. (1993) Extended Thermodynamics. In: Truesdell, C., Ed., Springer Tracts in Natural Philosophy, Vol. 37, Springer, New York. https://doi.org/10.1007/978-1-4684-0447-0
[29]	Roulston, M.S. and Smith, L.A. (2002) Evaluating Probabilistic Forecasts Using Information Theory. Monthly Weather Review, 130, 1653-1660. https://doi.org/10.1175/1520-0493(2002)130<1653:EPFUIT>2.0.CO;2
[30]	Abramov, R., Majda, A. and Kleeman, R. (2005) Information Theory and Predictability for Low-Frequency Variability. Journal of the Atmospheric Sciences, 62, 65-87. https://doi.org/10.1175/JAS-3373.1
[31]	Haven, K., Majda, A. and Abramov R. (2005) Quantifying Predictability through Information Theory: Small Sample Estimation in a Non-Gaussian Framework. Journal of Computational Physics, 206, 334-362. https://doi.org/10.1016/j.jcp.2004.12.008
[32]	Abramov, R. (2009) The Multidimensional Moment-Constrained Maximum Entropy Problem: A BFGS Algorithm with Constraint Scaling. Journal of Computational Physics, 228, 96-108. https://doi.org/10.1016/j.jcp.2008.08.020
[33]	Cacuci, D.G. (2023) Fourth-Order Predictive Modelling: II. 4th-BERRU-PM Methodology for Combining Measurements with Computations to Obtain Best-Estimate Results with Reduced Uncertainties.
[34]	Shannon, C.E. (1948) A Mathematical Theory of Communication. The Bell System Technical Journal, 27, 379-423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x