Influence Functions for Risk and Performance Estimators

Shengyu Zhang, R. Douglas Martin^*, Anthony A. Christidis

University of Washington, Seattle, Washington, USA.

DOI: 10.4236/jmf.2021.111002                

Abstract

A new general method for computing standard errors of risk and performance estimators is developed. The method relies on the fact that the influence function of an estimator, the Gateaux derivative of the estimator functional in the direction of point mass distributions, may be used to represent the asymptotic variance of the estimator as the expected value of the squared influence function. The law of large numbers shows that the asymptotic variance of an estimator can be estimated as the time series average of the squared influence function, thereby yielding a very simple estimator standard error calculation that does not require knowledge of the asymptotic variance formula. We derive formulas for the influence functions of six risk estimators and seven performance estimators, thereby providing a convenient portfolio performance and risk management tool to easily compute standard errors for most risk and performance estimators of interest or practical importance. We conduct a simulation study to evaluate the quality of the standard errors and confidence interval error rates for the Sharpe ratio and downside Sharpe ratio estimators. Software implementations of our proposed method in the R packages RPEIF and RPESE are publicly available on CRAN.

Keywords

Influence Functions, Risk Estimator, Performance Estimator, Asymptotic Variance, Standard Errors

1. Introduction

Returns based risk estimators such as volatility, value-at-risk, and expected shortfall, and performance estimators such as the Sharpe ratio and Sortino ratio, have important roles in asset and portfolio risk assessment and management. Since such risk and performance estimators are based on observed returns, they are subject to estimation error that can be quite sensitive to non-normality of the returns distribution. Standard statistical practice dictates that the uncertainty in a risk and performance estimate should be quantified, minimally in terms of an estimator’s standard error (SE) and often preferably in terms of a confidence interval. Unfortunately, risk and performance estimators are often reported without a standard error, one reason for which is the following. The usual deterministic recipe for computing an estimator standard error is based evaluating the estimator’s asymptotic variance formula with sample estimates substituted for unknown parameter values in the formula, dividing that value by the sample size and taking the square root. The problem is that, except for the most popular estimators such as a value-at-risk or Sharpe ratio estimator, the asymptotic variance formulas are not well known or may not even exist in the quantitative finance literature. Furthermore, the bootstrap alternative to the standard errors and confidence intervals will often be unacceptable in portfolio risk and performance reporting because of their inherent randomness.

In this paper we overcome the above impediments to computing risk and performance estimator SE’s by introducing a new deterministic method of computing standard errors based on the use of the influence function (IF) borrowed from robust statistics. The new method allows one to use a risk or performance estimator’s IF formula to derive the estimator’s asymptotic variance expression, from which one can compute an estimator standard error via the usual recipe described above. However, we will show that an estimator IF formula allows one to easily compute an estimator standard error without having to derive the asymptotic variance formula. With that in mind, we derive the formulas of the IF’s for six risk estimators and seven performance estimators. Graphical display of the IF’s for all thirteen estimators gives a clear visual picture of the differences in the data sensitivity of the various estimators. The efficacy of the new method for computing estimator SE’s and corresponding confidence intervals is illustrated via Monte Carlo studies for the Sharpe ratio and downside Sharpe ratio performance estimators. An important feature of the new IF based SE method is that formulas are easily derived for other estimators not treated herein, using only the tools of basic calculus.

The influence function is a directional (Gateaux) derivative of the asymptotic function representation of an estimator that was introduced by Hampel [1] and developed further in the robust statistics research literature and used in applied statistics literature. A wide range of references to that literature is provided in robust statistics books by Hampel et al. [2] and Maronna et al. [3]. The first appearance of influence functions in quantitative finance research is, to our knowledge, in an appendix of Yamai and Yoshiba [4], where the authors use the influence function of a lightly trimmed expected shortfall estimator to derive the asymptotic variance formula of the estimator. Subsequently, Chapter 6 of Scherer and Martin [5] discusses influence functions for mean vectors and covariance matrices, and uses the results to derive influence function formulas for the weight vector, mean return and volatility of mean-variance optimal tangency portfolios. DeMiguel and Nogales [6] proposed improving on the stability and performance of minimum variance optimal portfolios using M-estimators and S-estimators, and derived the influence function formulas for these estimators. Cont et al. [7] computed influence functions for the value-at-risk (VaR) and the expected shortfall (ES) based on returns empirical distributions (referred to as the “historical” method), normal distributions and Laplace distributions. More recently Martin and Zhang [8] derived influence function formulas for both nonparametric and parametric expected shortfall (ES) for t-distributions, and showed that the influence function of the parametric ES has the undesirable feature that large profits are reflected as large risk.

Important related work on proving performance estimator asymptotic normality, and deriving asymptotic variance formulas, were provided by De Capitani [9] and De Capitani and Pasquazzi [10]. The first of these two papers derived asymptotic variance formulas for the Sortino and Omega ratios for both i.i.d. returns and stationary α-mixing time series that include GARCH(1, 1) returns. That paper also carried out extensive Monte Carlo finite-sample studies of the performance of the confidence intervals for such processes. The second paper focused on a general approach for establishing asymptotic normality and asymptotic variance expressions with serially dependent returns for a variety of performance ratio estimators that include the Sharpe ratio, VaR ratio and CVar ratio (called ESratio herein), which are among the estimators treated herein. It is interesting to note that this second paper briefly uses the term “influence function” for the result of applying a classical delta method, but does not refer to the Hampel influence function.

The remainder of the paper is organized as follows. Section 2 provides the definition and key properties of influence functions. Section 3 introduces the functional forms of six risk estimators that are treated in the quantitative finance literature, derives their nonparametric influence function formulas, discusses the evaluation of nuisance parameters, and provides graphical displays of their influence functions and discusses their different shapes; Section 4 does likewise for seven performance estimators. In Section 5 we describe how to use influence functions to conveniently compute the standard errors of risk and performance estimators without using the asymptotic variance expression. In that section we also show that for the standard deviation (volatility) and Sharpe ratio estimators, our simple method that does not require an asymptotic variance formula gives exactly the same result as when using the estimator asymptotic variance expression with sample estimates used in place of unknown parameter values. Section 6 uses Monte Carlo to study the effectiveness of using influence functions to compute finite sample standard errors, and associated confidence interval error rates, of risk and performance estimators. Section 7 briefly points to recent research results on generalizing the IF standard errors method for i.i.d. returns in the current paper, to deal with serially dependent returns. The Appendix provides supplementary material, including the derivation of IF based asymptotic variance formulas for all the risk and performance estimators treated in this paper, along with references to formal mathematical statistics derivations of most of the formulas.

2. Influence Functions Basics

Here we begin with a brief discussion of the representation of an estimator asymptotic value as a functional, followed by the definition of an influence function based on such representation. Then we state two key properties of influence functions.

2.1. Risk and Performance Estimator Functional Representation

Suppose one has a returns based risk or performance estimator ${\hat{\theta }}_n={\hat{\theta }}_n\left(r_1\mathrm{,}{r}_2\mathrm{,}\cdots \mathrm{,}{r}_n\right)$ where it is assumed throughout that the $r_i$ are identically distributed with common distribution function $F=F\left(r\right)$. All such estimators of practical interest converge in a probabilistic sense to a true but unknown parameter $\theta$ as the sample size n goes to infinity. Furthermore, since the value of $\theta$ is determined by the distribution function F, we have an estimator asymptotic functional representation $\theta \left(F\right)$.¹ For example, the sample mean estimator ${\hat{\mu }}_n$ has the functional representation

$\mu \left(F\right)={\displaystyle \int r\text{d}F\left(r\right)}$ (1)

where the range of integration here and throughout the paper is the entire real line, and the sample variance estimator ${\hat{\sigma }}_n^2$ has the functional representation:

${\sigma }^2\left(F\right)={\displaystyle \int {\left(r-\mu \left(F\right)\right)}^2}\text{d}F\left(r\right).$ (2)

The above functional representations are merely a mathematical way of representing the true but unknown values of the distribution mean and variance. For distributions that have a probability density $f\left(r\right)$ and $\text{d}F\left(r\right)=f\left(r\right)\text{d}r$, the integrals have their usual form as in introductory probability and statistics books. However, the following comments reveal the usefulness of the general forms of the integrals in the above expressions.

Given a functional representation $T\left(F\right)$ of an estimator, a nonparametric sample based estimator ${\hat{\theta }}_n$ is easily obtained by the “plug-in” principle of replacing the unknown returns distribution function F by the empirical distribution $F_n$ that has a jump of height 1/n at each of the observed returns values $r_1\mathrm{,}{r}_2\mathrm{,}\cdots \mathrm{,}{r}_n$ :

${\theta }_n=\theta \left(F_n\right)={\theta }_n\left(r_1\mathrm{,}{r}_2\mathrm{,}\cdots \mathrm{,}{r}_n\right)\mathrm{.}$ (3)

with such a substitution the integrals become summations with respect to discrete probabilities of 1/n assigned to each data value. For example, application of the plug-in principle to the above mean and variance functions results in the usual sample mean and sample variance estimators, respectively

${\hat{\mu }}_n=\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}{r}_t,{\hat{\sigma }}_n^2=\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}{\left(r_t-{\hat{\mu }}_n\right)}^2.$ (4)

Further details on functional representations may be found in Section 3.7 of [3].

2.2. Estimator Influence Function Definition

For a fixed distribution function $F\left(x\right)$, an influence function is based on use of the mixture distribution

$F_{\gamma }\left(x\right)=\left(1-\gamma \right)F\left(x\right)+\gamma {\delta }_r\left(x\right),0<\gamma <1/2$ (5)

where ${\delta }_r\left(x\right)$ is a point mass discrete distribution function with a jump of height one (also known as a “unit step function”) located at value r. The influence function of an estimator with functional form $T\left(F\right)$ is defined as

$IF\left(r;\hat{\theta },F\right)={\frac{\text{d}}{\text{d}\gamma }\theta \left(F_{\gamma }\right)|}_{\gamma =0}$ (6)

where we think of $\hat{\theta }$ as a representative of the entire sequence of estimators ${\hat{\theta }}_1\mathrm{,}{\hat{\theta }}_2\mathrm{,}{\hat{\theta }}_3\mathrm{,}\cdots$.

An influence function is a special directional derivative of the functional $T\left(F\right)$ in the direction of a point mass distributions ${\delta }_r$, evaluated at F.²

Applying the above formula to the sample mean functional (1) gives

$\begin{array}{c}IF\left(r;\hat{\mu },F\right)=\frac{\text{d}}{\text{d}\gamma }{\displaystyle \int x\text{d}{F}_{\gamma }\left(x\right)}\\ =\frac{\text{d}}{\text{d}\gamma }{\left[{\displaystyle \int x\text{d}F\left(x\right)}+\gamma {\displaystyle \int x\text{d}\left({\delta }_r\left(x\right)-F\left(x\right)\right)}\right]}_{\gamma =0}\\ =0+{\displaystyle \int x\text{d}{\delta }_r\left(x\right)}-{\displaystyle \int x\text{d}F\left(x\right)}\\ =r-\mu (\; F\; )\end{array}$

and for simplicity we write

$IF\left(r\mathrm{<mo>\; ;\; </mo>}\hat{\mu }\mathrm{,}F\right)=r-\mu$ (7)

where we keep in mind that $\mu =\mu \left(F\right)$.

Similarly, a straight forward application of 6 to the sample variance functional (2), details of which are provided at the beginning of Section 3, gives the sample variance influence function formula

$IF\left(r\mathrm{<mo>\; ;\; </mo>}\hat{\mu }\mathrm{<mo>\; ;\; </mo>}F\right)={\left(r-\mu \right)}^2-{\sigma }^2$ (8)

where $\mu =\mu \left(F\right)$ and ${\sigma }^2={\sigma }^2\left(F\right)$.

2.3. Two Key Influence Function Properties

The sample mean and sample variance influence function formulas (7) and (8) have the obvious property that their expected values are equal to zero. It is in fact a general property of an influence function $IF\left(r\mathrm{<mo>\; ;\; </mo>}\hat{\theta }\mathrm{,}F\right)$ that its expectation under the distribution F is equal to zero, namely

$E_F\left[IF\left(r;\hat{\theta },F\right)\right]=0$ (9)

and it is straightforward to verify that his condition holds for all the estimator influence functions derived in Sections 3 and 4.

A second general property of influence functions is that the finite sample estimator ${\theta }_n=\theta \left(F_n\right)$ obtained from the functional $\theta \left(F\right)$ has an influence function transformed returns representation

${\hat{\theta }}_n-\theta \left(F\right)=\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}IF\left(r_t;\hat{\theta },F\right)+\text{remainder}$ (10)

where the remainder goes to zero in a probabilistic sense rapidly relative to the size of the summation term as $n\to \infty$. Details on this property of influence functions, as well as the zero expectation property, can be found in Hampel [1], Hampel et al. [2], Fernholz [11] and Maronna et al. [3]. We come back to further discussion on use of the above representation in Section 5.

3. Risk Estimators Influence Functions

The risk estimators treated in this paper are listed in Table 1. We now describe the functional forms of the estimators and their finite-sample estimators, and derive the estimator influence functions. The derivations of the influence functions only require standard calculus rules for computing the derivative of a product, quotient, composition of two functions, and the derivative with respect to the argument and upper limit of an integral. For details on risk estimators see McNeil et al. [12], and for an early treatment of partial moments see Fishburn [13].

The derivation below of the influence function of the sample standard deviation is facilitated by first deriving the formula for the influence function of the sample variance given by (8) in Section 2. Plugging the expression (2) of the sample variance functional into the general influence function formula (6), we get

$IF\left(r\mathrm{<mo>\; ;\; </mo>}{\sigma }^2\mathrm{,}F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[{\displaystyle \int }{\left(x-\mu \left(F_{\gamma }\right)\right)}^2\text{d}{F}_{\gamma }\left(x\right)\right]}_{\gamma =0}$ (11)

$={\left[-2\frac{\text{d}\mu \left(F_{\gamma }\right)}{\text{d}\gamma }{\displaystyle \int }\left(x-\mu \left(F_{\gamma }\right)\right)\text{d}{F}_{\gamma }\left(x\right)+{\displaystyle \int }{\left(x-\mu \left(F_{\gamma }\right)\right)}^2\text{d}\left({\delta }_r\left(x\right)-F\left(x\right)\right)\right]}_{\gamma =0}$ (12)

where the first term above evaluated at $\gamma =0$. Splitting up the last two terms and evaluating each at $\gamma =0$ gives

Table 1. Risk estimator names and descriptions.

$IF\left(r\mathrm{<mo>\; ;\; </mo>}{\sigma }^2\mathrm{,}F\right)={\left(r-\mu \right)}^2-{\sigma }^2$ (13)

where $\mu =\mu \left(F\right)$ and ${\sigma }^2={\sigma }^2\left(F\right)$.

Standard Deviation (SD)

The standard deviation functional representation is

$\sigma \left(F\right)={\left({\sigma }^2\left(F\right)\right)}^{1/2}$ (14)

and its plug-in estimator is ${\hat{\sigma }}_n={\left({\hat{\sigma }}_n^2\right)}^{1/2}={\left(\frac{1}{n}{\displaystyle {\sum }_{t=1}^n}{\left(r_t-{\hat{\mu }}_n\right)}^2\right)}^{1/2}$ where ${\hat{\mu }}_n$ is the sample mean.

The influence function of the standard deviation estimator is

$IF\left(r\mathrm{<mo>\; ;\; </mo>}\sigma \mathrm{<mo>\; ;\; </mo>}F\right)={\frac{\text{d}}{\text{d}\gamma }{\left({\sigma }^2\left(F_{\gamma }\right)\right)}^{1/2}|}_{\gamma =0}$

and using the chain rule for the composition of functions gives

$\begin{array}{c}IF\left(r;\sigma ;F\right)=\frac{1}{2\cdot {\left({\sigma }^2\left(F_{\gamma }\right)\right)}^{1/2}}\cdot {\frac{\text{d}}{\text{d}\gamma }{\sigma }^2\left(F_{\gamma }\right)|}_{\gamma =0}\\ =\frac{1}{2\cdot \sigma \left(F\right)}\cdot IF\left(r;{\sigma }^2;F\right)\\ =\frac{1}{2\sigma }\left({\left(r-\mu \right)}^2-{\sigma }^2\right)\end{array}$ (15)

where $\mu =\mu \left(F\right)$, $\sigma =\sigma \left(F\right)$.

Semi-Standard Deviation (SemiSD)

The functional form of the semi-standard deviation is

$SemiSD\left(F\right)={\left({\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}{\left(x-\mu \left(F\right)\right)}^2\text{d}F\left(x\right)\right)}^{1/2}$ (16)

and the estimator is $\widehat{SemiSD}{}_n={\left(\frac{1}{n}{\displaystyle {\sum }_{r_t\le {\hat{\mu }}_n}}{\left(r_t-{\hat{\mu }}_n\right)}^2\right)}^{1/2}$.

The SemiSD functional is the square root of the semi-variance functional

$SemiSD\left(F\right)=S{V}^{1/2}(\; F\; )$

$SV\left(F\right)={\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}{\left(x-\mu \left(F\right)\right)}^2\text{d}F\left(x\right)\mathrm{.}$

So we first derive the semi-variance influence function

$\begin{array}{l}IF\left(r;SV,F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[{\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}{\left(x-\mu \left(F_{\gamma }\right)\right)}^2\text{d}{F}_{\gamma }\left(x\right)\right]}_{\gamma =0}\\ =-IF\left(r;\mu ,F\right)\cdot {\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}2\left(x-\mu \left(F\right)\right)\text{d}F\left(x\right)\\ +{\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}{\left(x-\mu \left(F\right)\right)}^2\text{d}\left[{\delta }_r\left(x\right)-F\left(x\right)\right]\\ =-2\left(r-\mu \right)\cdot {\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}2\left(x-\mu \left(F\right)\right)\text{d}F\left(x\right)+{\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}{\left(x-\mu \left(F\right)\right)}^2\text{d}{\delta }_r\left(x\right)\\ -{\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}{\left(x-\mu \left(F\right)\right)}^2\text{d}F\left(x\right)\\ =-2\left(r-\mu \right)\cdot {\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}\left(x-\mu \left(F\right)\right)\text{d}F\left(x\right)+{\left(r-\mu \right)}^{2}I\left(r\le \mu \right)-SemiS{D}^2\end{array}$

and then use the chain rule

$\begin{array}{l}IF\left(r;SemiSD;F\right)={\frac{\text{d}}{\text{d}\gamma }S{V}^{1/2}\left(F_{\gamma }\right)|}_{\gamma =0}\\ =\frac{-2\left(r-\mu \right)\cdot {\displaystyle {\int }_{-\infty }^{\mu \left(F\right)}}\left(x-\mu \left(F\right)\right)\text{d}F\left(x\right)+{\left(r-\mu \right)}^{2}I\left(r\le \mu \right)-SemiS{D}^2}{2\cdot SemiSD}\\ =\frac{{\left(r-\mu \right)}^2\cdot I\left(r\le \mu \right)-2\cdot SemiMean\cdot \left(r-\mu \right)-SemiS{D}^2}{2\cdot SemiSD}\end{array}$ (17)

where $\mu =\mu \left(F\right)$, $SemiSD=SemiSD\left(F\right)$, and $SemiMean=SemiMean\left(F\right)={\displaystyle {\int }_{-\infty }^{\mu }}\left(x-\mu \right)\text{d}F\left(x\right)$.

Lower Partial Moments (LPM1, LPM2)

The functional form of a lower partial moment of general order k, with a user specified threshold constant c, is

$LPMk\left(F\right)={\displaystyle {\int }_{-\infty }^c}{\left(c-x\right)}^k\text{d}F\left(x\right)$ (18)

and the estimator is ${\widehat{LPMk}}_n=\frac{1}{n}{\displaystyle {\sum }_{r_t\le c}}{\left(c-r_t\right)}^k$. This threshold c is often referred to as the minimum acceptable return (MAR).

The LPMk influence function is

$\begin{array}{c}IF\left(r;LPMk;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[LPMk\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ ={\displaystyle {\int }_{-\infty }^c}{\left(c-x\right)}^k\text{d}\left(\delta \left(x\right)-F\left(x\right)\right)\\ ={\left(c-r\right)}^{k}I\left(r\le c\right)-LPMk\end{array}$ (19)

where $LPMk=LPMk\left(F\right)$. Using $k=1$ and $k=2$ yields the influence functions of the lower partial momentsLPM1 and LPM2.

Expected Shortfall (ES)

The functional form of expected shortfall is

$E{S}_{\alpha }\left(F\right)=-\frac{1}{\alpha }{\displaystyle {\int }_{-\infty }^{q_{\alpha }\left(F\right)}}x\text{d}F\left(x\right)$ (20)

where the quantile functional $q_{\alpha }\left(F\right)$ is defined in (23) and the estimator is ${\widehat{ES}}_n=-\frac{1}{\lceil n\alpha \rceil }{\displaystyle {\sum }_{t=1}^{\lceil n\alpha \rceil }}{r}_{\left(t\right)}$, where $\lceil x\rceil$ is the smallest integer greater or equal to x.

By the chain rule and the rule for differentiation with respect to an integral upper limit, the influence function of ES is

$\begin{array}{l}IF\left(r;E{S}_{\alpha };F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[E{S}_{\alpha }\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{-1}{\alpha }\left[q_{\alpha }\cdot f\left(q_{\alpha }\right)\cdot IF\left(r;q_{\alpha };F\right)+{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}x\cdot \text{d}\left[{\delta }_r\left(x\right)-F\left(x\right)\right]\right]\\ =\frac{-1}{\alpha }\left[q_{\alpha }\cdot f\left(q_{\alpha }\right)\cdot \left(-1\right)\cdot IF\left(r;Va{R}_{\alpha };F\right)+r\cdot I\left(r\le q_{\alpha }\right)-{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}x\cdot \text{d}F\left(x\right)\right]\end{array}$

$\begin{array}{l}=\frac{-1}{\alpha }\left[q_{\alpha }\cdot f\left(q_{\alpha }\right)\cdot \left(-1\right)\cdot \frac{I\left(r\le q_{\alpha }\right)-\alpha }{f\left(q_{\alpha }\right)}+r\cdot I\left(r\le q_{\alpha }\right)+\alpha \cdot E{S}_{\alpha }\right]\\ =\frac{-1}{\alpha }\left[-q_{\alpha }\cdot \left(I\left(r\le q_{\alpha }\right)-\alpha \right)+r\cdot I\left(r\le q_{\alpha }\right)+\alpha \cdot E{S}_{\alpha }\right]\\ =\frac{-1}{\alpha }\left[q_{\alpha }\cdot \alpha +\left(r-q_{\alpha }\right)\cdot I\left(r\le q_{\alpha }\right)+\alpha \cdot E{S}_{\alpha }\right]\\ =-q_{\alpha }-E{S}_{\alpha }-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)\end{array}$ (21)

where $q_{\alpha }=q_{\alpha }\left(F\right)$ and $E{S}_{\alpha }=E{S}_{\alpha }\left(F\right)$.

Value-at-Risk (VaR)

The functional form of VaR is

$Va{R}_{\alpha }\left(F\right)=-q_{\alpha }\left(F\right)$ (22)

where the quantile functional $q_{\alpha }\left(F\right)$ is defined by the equation

$\alpha ={\displaystyle {\int }_{-\infty }^{q_{\alpha }\left(F\right)}}\text{d}F\left(x\right)$ (23)

and the VaR estimator is ${\widehat{VaR}}_n=-r_{\left(\lceil n\alpha \rceil \right)}$.

Under distribution $F_{\gamma }$ we have

$\alpha ={\displaystyle {\int }_{-\infty }^{q_{\alpha }\left(F_{\gamma }\right)}}\text{d}{F}_{\gamma }\left(x\right).$ (24)

Taking the derivative with respect to $\gamma$ on both sides of the above equation, and using the chain rule and the rule for the differentiation with respect to an integral upper limit, one obtains:

$0=\frac{\text{d}}{\text{d}\gamma }{\left[{\displaystyle {\int }_{-\infty }^{q_{\alpha }\left(F_{\gamma }\right)}}\text{d}{F}_{\gamma }\left(x\right)\right]}_{\gamma =0}$

$\begin{array}{c}0=IF\left(r;q_{\alpha },F\right)\cdot f\left(q_{\alpha }\right)+{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}\text{d}\left[\delta \left(x\right)-F\left(x\right)\right]\\ =-IF\left(r\mathrm{<mo>\; ;\; </mo>}Va{R}_{\alpha }\mathrm{<mo>\; ;\; </mo>}F\right)\cdot f\left(q_{\alpha }\right)+I\left(r\le q_{\alpha }\right)-\alpha \mathrm{.}\end{array}$

This results in the VaR influence function formula

$IF\left(r\mathrm{<mo>\; ;\; </mo>}Va{R}_{\alpha }\mathrm{,}F\right)=\frac{I\left(r\le q_{\alpha }\right)-\alpha }{f\left(q_{\alpha }\right)}$ (25)

where $q_{\alpha }=q_{\alpha }\left(F\right)$.

3.1. Risk Estimator Influence Functions Nuisance Parameters

Note that the various influence function formulas contain one or more nuisance parameters that need to be specified in order to compute influence function values for various values of a return r. For example, the IF of the sample standard deviation depends on the nuisance parameters $\mu =\mu \left(F\right)$ and $\sigma =\sigma \left(F\right)$, and the IF of the sample SemiSD depends upon $\mu =\mu \left(F\right)$, the semi-mean $SemiMean=SemiMean\left(F\right)$, and the semi-standard deviation $SSD=SSD\left(F\right)$. For purposes of the calculating and displaying the risk estimator influence functions to follow, we use nuisance parameter values based on the assumption of normally distributed returns with monthly mean return $\mu =1\%$, volatility $\sigma =5\%$, risk-free rate $r_f=0$, and assume tail probability $\alpha =0.10$ for quantiles, VaR and ES, estimators, and $c=0$ for lower partial moments and Sortino ratio with fixed threshold.

3.2. Shapes of Risk Estimators Influence Functions

It is instructive to plot and compare the shapes of the above risk estimator influence functions. Figure 1 displays those plots using the nuisance parameters computed as described above.

The first important thing that one notices in Figure 1 is that, unlike the other estimator influence functions, the SD and SemiSD estimator influence functions increase with increasing positive returns, quadratically so for the SD influence function, and linearly for the SemiSD influence function. In both cases this behavior is quite undesirable since all sufficiently large positive returns show up as risk contributors, the more so the more positive the return. On the other hand, the

Figure 1. Influence functions of six risk estimators with nuisance parameters as specified in Table 1.

influence function of all the risk estimators except VaR increase with increasingly negative returns for all sufficiently negative returns, as one would expect for a risk estimator. It is notable that the increasingly popular ES coherent risk estimator only increases linearly for losses, whereas the non-coherent risk estimators SemiSD and LPM2 increase quadratically. It is likely preferable to use a coherent shortfall type risk estimator that increases quadratically for negative returns, e.g., an expected quadratic shortfall (EQS) estimator.³

As for the small negative values of the SD and SemiSD influence functions for ranges of returns close to zero, these are due to returns “inliers” that represent negative risk for the SD and SemiSD. Finally, we note that the discontinuous and bounded natures of the VaR influence function are both undesirable features, the first because small changes in a return near the discontinuity can result in a large change in risk indication, and the second because increasingly large negative outliers do not indicate increasingly large risk.

Plotting the influence functions of the risk estimators in Figure 1 was done using the Risk and Performance Estimators Influence Functions (RPEIF) R package available at the CRAN link https://cran.r-project.org/web/packages/RPEIF/index.html, where a Reference annual and a Vignette are provided.

4. Performance Estimators Influence Functions

The performance estimators treated in this paper are listed in Table 2, and this section describes the functional forms of the estimators and their finite-sample estimators, and derives the estimator influence functions. As in Section 3, the derivations of the influence functions only require standard calculus rules.

For the downside Sharpe ratio see Ziemba [15], and for the Sortino ratio see Sortino and van-der-Meer [16], Sortino and Price [17] and Sortino and Forsey [18]. For the ESratio see Martin et al. [19], where it was called the STARR ratio. For the VaRratio see Favre and Galeano [20], for the RachevRatio see Biglova et al. [21] and Stoyan et al. [22], and for the Omega performance estimators see Keating and Shadwick [23].

Table 2. Performance estimator names and descriptions.

Sharpe Ratio (SR)

The functional representation of the Sharpe ratio is

$SR\left(F\right)=\frac{\mu \left(F\right)-r_f}{\sigma \left(F\right)}=\frac{{\mu }_e\left(F\right)}{\sigma \left(F\right)}$ (26)

and the estimator is ${\widehat{SR}}_n=\frac{{\hat{\mu }}_n-r_f}{{\hat{\sigma }}_n}=\frac{{\hat{\mu }}_{e,n}}{{\hat{\sigma }}_n}$ where ${\hat{\mu }}_n$ and ${\hat{\sigma }}_n$ are sample mean and sample standard deviation, respectively.

Noting that the influence function of ${\mu }_e\left(F\right)=\mu \left(F\right)-r_f$ is the same as the influence function of $\mu \left(F\right)$, and using the quotient rule for derivatives, the influence function for the Sharpe-Ratio is

$\begin{array}{c}IF\left(r;SR;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[SR\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{1}{\sigma }\cdot IF\left(r;\mu ;F\right)-\frac{{\mu }_e}{{\sigma }^2}\cdot IF\left(r;\sigma ;F\right)\\ =\frac{1}{\sigma }\left(r-\mu \right)-\frac{{\mu }_e}{{\sigma }^2}\cdot \frac{1}{2\sigma }\left({\left(r-\mu \right)}^2-{\sigma }^2\right)\\ =-\frac{SR}{2{\sigma }^2}{\left(r-\mu \right)}^2+\frac{1}{\sigma }\left(r-\mu \right)+\frac{SR}{2}\end{array}$ (27)

where ${\mu }_e={\mu }_e\left(F\right)$, $\sigma =\sigma \left(F\right)$ and $SR=SR\left(F\right)$.

Downside Sharpe Ratio (DSR)

Downside Sharpe Ratio is a short name for what Ziemba [15] called the “Symmetric Downside Risk Sharpe Ratio”, and the DSR functional is

$DSR\left(F\right)=\frac{\mu \left(F\right)-r_f}{\sqrt{2}SSD\left(F\right)}=\frac{{\mu }_e\left(F\right)}{\sqrt{2}SSD\left(F\right)}$ (28)

and the estimator is ${\widehat{DSR}}_n=\frac{{\hat{\mu }}_n-r_f}{\sqrt{2}{\widehat{SSD}}_n}=\frac{{\hat{\mu }}_n-r_f}{\sqrt{2}\sqrt{\frac{1}{n}{\displaystyle {\sum }_{r_t\le {\hat{\mu }}_n}}{\left(r_t-{\hat{\mu }}_n\right)}^2}}.$

The influence function for $\sqrt{2}DSR\left(F\right)$ is

$\begin{array}{l}IF\left(r;\sqrt{2}DSR;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[\sqrt{2}DSR\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{1}{SemiSD}\left(r-\mu \right)+{\mu }_e\left(F\right)\cdot \left(-1\right)\cdot \frac{1}{SemiS{D}^2}\cdot IF\left(r;SemiSD;F\right)\\ =\frac{1}{SemiSD}\left(r-\mu \right)-\frac{DSR}{SemiSD}\cdot IF\left(r;SemiSD;F\right)\\ =\frac{1}{SemiSD}\left(r-\mu \right)-\frac{DSR}{SemiSD}\cdot \frac{{\left(r-\mu \right)}^2\cdot I\left(r\le \mu \right)-2\cdot SemiMean\cdot \left(r-\mu \right)-SemiS{D}^2}{2\cdot SemiSD}\\ =-\frac{DSR\cdot I\left(r\le \mu \right)}{2\cdot SemiS{D}^2}{\left(r-\mu \right)}^2+\left(\frac{DSR\cdot SemiMean}{SemiS{D}^2}+\frac{1}{SemiSD}\right)\left(r-\mu \right)+\frac{DSR}{2}\end{array}$ (29)

where $\mu =\mu \left(F\right)$, $SemiMean=SemiMean\left(F\right)$, $SemiSD=SemiSD\left(F\right)$, and $DSR=DSR\left(F\right)$. So the influence function of $DSR\left(F\right)$ is given by the above expression divided by $\sqrt{2}$.

Sortino Ratio (SoR)

The Sortino ratio functional with threshold c is

$SoRc\left(F\right)=\frac{\mu \left(F\right)-c}{\sqrt{LPM{2}_c\left(F\right)}}=\frac{{\mu }_e\left(F\right)}{\sqrt{LPM{2}_c(\; F\; )}}$

and the estimator is ${\widehat{SoR}}_{c,n}=\frac{{\hat{\mu }}_n-c}{\sqrt{{\widehat{LPM2}}_n}}=\frac{{\hat{\mu }}_n-c}{\sqrt{\frac{1}{n}{\displaystyle {\sum }_{r_t\le c}}{\left(c-r_t\right)}^2}}$. Note that c is called MAR (“minimum acceptable return”) by Sortino.

The influence function for $SoRc\left(F\right)$ is

$\begin{array}{l}IF\left(r;SoRc;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[SoRc\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{1}{\sqrt{LPM{2}_c\left(F\right)}}\left(r-\mu \right)+{\mu }_e\left(F\right)\cdot \left(-\frac{1}{2}\right)\cdot \frac{1}{{\left(LPM{2}_c\right)}^{3/2}}\cdot IF\left(r;LPM{2}_c;F\right)\\ =\frac{1}{\sqrt{LPM{2}_c\left(F\right)}}\left(r-\mu \right)-\frac{1}{2}\cdot \frac{SoRc}{LPM{2}_c}\cdot \left({\left(c-r\right)}^{2}I\left(r\le c\right)-LPM{2}_c\right)\\ =\frac{-SoRc\cdot I\left(r\le c\right)}{2\cdot LPM{2}_c}{\left(r-c\right)}^2+\frac{1}{\sqrt{LPM{2}_c}}\left(r-\mu \right)+\frac{SoRc}{2}\end{array}$ (30)

where $\mu =\mu \left(F\right)$, $LPM{2}_c=LPM{2}_c\left(F\right)$ and $SoRc=SoRc\left(F\right)$.

Expected Shortfall Ratio (ESratio)

The expected shortfall ratio functional is

$ESratio\left(F\right)=\frac{\mu \left(F\right)-r_f}{E{S}_{\alpha }\left(F\right)}=\frac{{\mu }_e\left(F\right)}{E{S}_{\alpha }\left(F\right)}$ (31)

and the estimator is ${\widehat{ESratio}}_n=\frac{{\hat{\mu }}_n-r_f}{{\widehat{ES}}_n}=\frac{{\hat{\mu }}_n-r_f}{-\frac{1}{\lceil n\alpha \rceil }{\displaystyle {\sum }_{t=1}^{\lceil n\alpha \rceil }}{r}_{\left(t\right)}}$, where $\lceil x\rceil$ is the smallest integer greater or equal to x.

The influence function of expected shortfall ratio is derived as follows

$\begin{array}{c}IF\left(r;ESratio;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[ESratio\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{r-\mu }{E{S}_{\alpha }}-\frac{{\mu }_e\left(F\right)}{E{S}_{\alpha }^2}\cdot IF\left(r;E{S}_{\alpha },F\right)\\ =\frac{r-\mu }{E{S}_{\alpha }}-\frac{ESratio}{E{S}_{\alpha }}\left(-q_{\alpha }-E{S}_{\alpha }-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)\right)\end{array}$ (32)

where $\mu =\mu \left(F\right)$, $q_{\alpha }=q_{\alpha }\left(F\right)$, $E{S}_{\alpha }=E{S}_{\alpha }\left(F\right)$ and $ESratio=ESratio\left(F\right)$.

Value at Risk Ratio (VaRratio)

The functional form of the VaR ratio is

$VaRratio\left(F\right)=\frac{\mu \left(F\right)-r_f}{VaR\left(F\right)}=\frac{{\mu }_e\left(F\right)}{-q_{\alpha }\left(F\right)}$ (33)

and the estimator is ${\widehat{VaRratio}}_n=\frac{{\hat{\mu }}_n-r_f}{{\widehat{VaR}}_n}=\frac{{\hat{\mu }}_n-r_f}{-r_{\left(\lceil n\alpha \rceil \right)}}.$

The influence function of the VaR ratio is

$\begin{array}{c}IF\left(r;VaRratio;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[VaRratio\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{r-\mu }{-q_{\alpha }}-\frac{{\mu }_e\left(F\right)}{q_{\alpha }^2}\cdot IF\left(r;Va{R}_{\alpha }\right)\\ =-\frac{r-\mu }{q_{\alpha }}+\frac{VaRratio}{q_{\alpha }}\cdot \frac{I\left(r\le q_{\alpha }\right)-\alpha }{f\left(q_{\alpha }\right)}\end{array}$ (34)

where $\mu =\mu \left(F\right)$, $q_{\alpha }=q_{\alpha }\left(F\right)$ and $VaRratio=VaRratio\left(F\right)$.

Rachev Ratio (RachevRatio)

The functional form of the Rachev ratio is

$RachR\left(F\right)=\frac{E{G}_{\beta }}{E{S}_{\alpha }}=\frac{\frac{1}{\beta }\cdot {\displaystyle {\int }_{q_{1-\beta }}^{+\infty }}x\text{d}F\left(x\right)}{-\frac{1}{\alpha }\cdot {\displaystyle {\int }_{-\infty }^{q_{\alpha }}}x\text{d}F(\; x\; )}$

where $E{S}_{\alpha }=E{S}_{\alpha }\left(F\right)$ is the expected shortfall at level α, and $E{G}_{\beta }=E{G}_{\beta }\left(F\right)$ is the expected tail gain at upper β-quantile defined by the following equation

$E{G}_{\beta }=\frac{1}{\beta }\cdot {\displaystyle {\int }_{q_{1-\beta }}^{+\infty }}x\text{d}F\left(x\right).$

The Rachev ratio estimator is ${\widehat{RachR}}_n=\frac{{\widehat{EG}}_{\beta ,n}}{{\widehat{ES}}_{\alpha ,n}}=\frac{\frac{1}{\lceil n\beta \rceil }{\displaystyle {\sum }_{t=n-\lceil n\beta \rceil +1}^n}{r}_{\left(t\right)}}{-\frac{1}{\lceil n\alpha \rceil }{\displaystyle {\sum }_{t=1}^{\lceil n\alpha \rceil }}{r}_{\left(t\right)}}.$

The influence functions of $E{S}_{\alpha }\left(F\right)$ and $E{G}_{\beta }\left(F\right)$ are

$IF\left(r\mathrm{<mo>\; ;\; </mo>}E{S}_{\alpha }\mathrm{<mo>\; ;\; </mo>}F\right)=-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)-q_{\alpha }-E{S}_{\alpha }$

and

$IF\left(r\mathrm{<mo>\; ;\; </mo>}E{G}_{\beta }\mathrm{<mo>\; ;\; </mo>}F\right)=\frac{I\left(r\ge q_{1-\beta }\right)}{\beta }\cdot \left(r-q_{1-\beta }\right)+q_{1-\beta }-E{G}_{\beta }\mathrm{.}$

Using the above two influence functions gives the following formula for the influence function of the Rachev ratio:

$\begin{array}{c}IF\left(r\mathrm{<mo>\; ;\; </mo>}RachR\mathrm{<mo>\; ;\; </mo>}F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[RachR\left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{1}{E{S}_{\alpha }}\cdot IF\left(r;E{G}_{\beta };F\right)-\frac{E{G}_{\beta }}{E{S}_{\alpha }^2}\cdot IF\left(r;E{S}_{\alpha };F\right)\end{array}$

$\begin{array}{c}=\frac{1}{E{S}_{\alpha }}\left(\frac{I\left(r\ge q_{1-\beta }\right)}{\beta }\left(r-q_{1-\beta }\right)+q_{1-\beta }-E{G}_{\beta }\right)\\ -\frac{RachR}{E{S}_{\alpha }}\cdot \left(\frac{-I\left(r\le q_{\alpha }\right)}{\alpha }\left(r-q_{\alpha }\right)-q_{\alpha }-E{S}_{\alpha }\right)\end{array}$ (35)

where $\mu =\mu \left(F\right)$, $q_{\alpha }=q_{\alpha }\left(F\right)$, $q_{1-\beta }=q_{1-\beta }\left(F\right)$, $E{S}_{\alpha }=E{S}_{\alpha }\left(F\right)$, $E{G}_{\beta }=E{G}_{\beta }\left(F\right)$, and $RachR=RachR\left(F\right)$.

Omega Ratio (Omega)

The functional form of the Omega ratio is

$\Omega \left(F\right)=\frac{{\displaystyle {\int }_c^{+\infty }}\left(x-c\right)f\left(x\right)\text{d}x}{{\displaystyle {\int }_{-\infty }^c}\left(c-x\right)f\left(x\right)\text{d}x}=1+\frac{\mu -c}{LPM{1}_c(\; F\; )}$

where $LPM{1}_c\left(F\right)={\displaystyle {\int }_{-\infty }^c}\left(c-x\right)f\left(x\right)\text{d}x$ is the lower partial moments of order 1, and c is a user specified.

The Omega ratio estimator is ${\hat{\Omega }}_n=\frac{{\widehat{UPM1}}_{c,n}}{{\widehat{LPM1}}_{c,n}}=1+\frac{\frac{1}{n}{\displaystyle {\sum }_t}{r}_t-c}{\frac{1}{n}{\displaystyle {\sum }_{r_t\le c}}\left(c-r_t\right)}$.

The influence function of the Omega ratio is

$\begin{array}{c}IF\left(r;\Omega ;F\right)=\frac{\text{d}}{\text{d}\gamma }{\left[\Omega \left(F_{\gamma }\right)\right]}_{\gamma =0}\\ =\frac{1}{LPM{1}_c}\cdot \left(r-\mu \right)-\frac{\mu -c}{{\left(LPM{1}_c\right)}^2}\cdot IF\left(r;LPM{1}_c;F\right)\\ =\frac{1}{LPM{1}_c}\cdot \left(r-\mu \right)-\frac{\mu -c}{{\left(LPM{1}_c\right)}^2}\cdot \left(\left(c-r\right)\cdot I\left(r\le c\right)-LPM{1}_c\right)\\ =\frac{1}{LPM{1}_c}\cdot \left(r-c\right)+\frac{\mu -c}{{\left(LPM{1}_c\right)}^2}\cdot \left(r-c\right)\cdot I\left(r\le c\right)\end{array}$ (36)

Shapes of the Performance Estimators Influence Functions

Figure 2 displays the influence functions for the first six of the seven performance estimators in Table 2, using nuisance parameters computed as described in Section 3.1, and for the Rachev ratio using $\alpha =0.10$ and $\beta =0.10$. The influence function for the Omega ratio is similar to that of the Rachev ratio in that they both have two linearly increasing pieces with different slopes, but the Omega ratio does not have the flat spot that the Rachev ratio has, except for the special limiting case of the Rachev ratio where $q_{\alpha }=q_{1-\beta }$ where the flat spot disappears.

For the range of r values shown in Figure 2, all the influence functions are non-decreasing, and except for the Rachev ratio all are strictly increasing in r. Furthermore, the performance estimator influence function formulas show that these behaviors hold over the entire range of possible r values for all the performance estimator except for the Sharpe ratio, whose influence function is decreasing

Figure 2. influence functions of six performance estimators with nuisance parameters as specified in Table 2.

for all sufficiently large values of r. Thus, except for the Sharpe ratio, increasing values of r indicate increasing performance as one might expect of a performance estimator. While practitioners may argue that the large values of r required for the Sharpe ratio influence function to be decreasing are quite atypical, the lack of monotonicity of the Sharpe ratio influence function is none-the-less an unattractive feature of a performance estimator.

The influence functions shapes of the DSR, SoRc and ESratio estimators are strikingly similar, with all three being linear with positive slope for all sufficiently large return values, thereby indicating linearly increasing performance for such values. On the other hand, the DSR and Sortino ratio influence functions both decrease quadratically for increasingly large negative returns, while the ESratio decreases only linearly for all sufficiently large negative returns. It is likely preferable to use an expected quadratic shortfall (EQS) in the denominator an EQSratio performance estimator.

The VaR ratio influence function is linear except for the relatively small jump at the returns distribution quantile value for which the VaR influence has a jump discontinuity, and as such its shape is determined primarily by the sample mean estimator in the numerator of the VaR ratio. This feature of the VaR ratio makes it quite uninteresting as a performance estimator. Finally, the Rachev ratio influence function shows that all sufficiently large positive returns give rise to linearly increasing performance, and all sufficiently small negative returns give rise to linearly decreasing performance similar to that of the ES ratio. In contrast to the ES ratio, for which the linearly increasing influence for positive returns is due to the mean in the numerator, the more rapid linearly increasing slope of the Rachev ratio for positive returns is due to measuring the gain in performance with an upper tail expected value. This characteristic was considered to be an attractive one for portfolio optimization purposes by Z. Rachev, see Biglova et al. [21].

Plotting the influence functions of the performance estimators in Figure 2 was done using the Risk and Performance Estimators Influence Functions (RPEIF) R package available at the CRAN link https://cran.r-project.org/web/packages/RPEIF/index.html, where a Reference Manual and a Vignette are provided.

5. IF Based Standard Error Methods

The foundation of the influence function based standard error computation is the estimator influence function based series representation (10). The remainder term in that representation goes to zero as the sample size n goes to infinity, and one expects that the estimator representation

${\hat{\theta }}_n-\theta \left(F\right)=\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}IF\left(r_t;\hat{\theta },F\right)$ (37)

without remainder will be a good approximation for the sample sizes typically encountered in portfolio risk and performance analysis.⁴ It follows that the variance of ${\hat{\theta }}_n$ is approximated by

$\mathrm{var}\left({\hat{\theta }}_n\right)=\mathrm{var}\left[\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}IF\left(r_t;\hat{\theta },F\right)\right].$ (38)

when the returns $r_t$ are serially dependent, e.g., when the $r_t$ are a first order autoregression, or when the $r_t$ are uncorrelated but dependent as in the case of a GARCH(1, 1) model, one needs to account for the covariances between terms $IF\left(r_t\mathrm{<mo>\; ;\; </mo>}\hat{\theta }\mathrm{,}F\right)$ and $IF\left(r_u\mathrm{<mo>\; ;\; </mo>}\hat{\theta }\mathrm{,}F\right)$ for $t\ne u$ in computing $\text{var}\left({\hat{\theta }}_n\right)$. We remark further on this general case in Section 7, and concentrate on the idealized case where the returns are i.i.d.

For i.i.d. returns, the terms $IF\left(r_t\mathrm{<mo>\; ;\; </mo>}\hat{\theta }\mathrm{,}F\right)$ and $IF\left(r_u\mathrm{<mo>\; ;\; </mo>}\hat{\theta }\mathrm{,}F\right)$ in the above summation are independent for $t\ne u$, and in this case the above estimator variance expression reduces to

$\mathrm{var}\left({\hat{\theta }}_n\right)=\frac{1}{n}\mathrm{var}\left[I{F}^2\left(r_1;\hat{\theta },F\right)\right]=\frac{1}{n}E\left[I{F}^2\left(r_1;\hat{\theta },F\right)\right]$ (39)

where we have used the zero expectation property (9).

It is well-known in the robust statistics literature that the asymptotic variance $V\left(\theta \right)$ of a consistent estimator ${\hat{\theta }}_n$ is given by⁵

$V\left(\theta \right)=E\left[I{F}^2\left(r_1;\hat{\theta },F\right)\right].$ (40)

See for example Section 3.7 of [3]. Use of $V\left(\theta \right)$ in (40) gives the estimator finite sample variance approximation

$\mathrm{var}\left({\hat{\theta }}_n\right)=\frac{1}{n}V\left(\theta \right)$ (41)

and taking the square root, with the unknown $\theta$ replaced by its estimate ${\hat{\theta }}_n$ gives the estimator asymptotic variance based standard error (SE) formula

$S{E}_{avar}\left({\hat{\theta }}_n\right)={\left[\frac{1}{n}V\left({\hat{\theta }}_n\right)\right]}^{1/2}.$ (42)

5.1. IF Based Asymptotic Variance Formulas

Asymptotic variance formulas $V\left(\theta \right)$ have been derived in the theoretical econometric and quantitative finance literature for most, if not all, of the risk and performance estimators that one might want to use. On the other hand, asymptotic variance formulas of risk and performance estimators are easily derived using the IF based asymptotic variance formula (40). We illustrate this below for the case of the standard deviation and Sharpe ratio estimators, and provide derivations in Appendix A2 for the other estimators treated herein, along with literature references to asymptotic mathematical statistics derivations.

Standard Deviation Asymptotic Variance

The standard deviation influence function is given by, and so we immediately have

$V\left({\hat{\sigma }}_n\right)=E{\left[\frac{{\left(r-\mu \right)}^2-{\sigma }^2}{2\sigma }\right]}^2=\frac{E{\left(r-\mu \right)}^4-{\sigma }^4}{4{\sigma }^2}$ (43)

where $E{\left(r-\mu \right)}^4={\mu }_4$ is the fourth central moment.

Sharpe Ratio Asymptotic Variance

Using the Sharpe ratio influence function formula (27), we find the well-known result

$\begin{array}{c}V\left({\widehat{SR}}_n\right)=E{\left[-\frac{SR}{2{\sigma }^2}{\left(r-\mu \right)}^2+\frac{1}{\sigma }\left(r-\mu \right)+\frac{SR}{2}\right]}^2\\ =\frac{S{R}^2}{4{\sigma }^4}{\mu }_4+1+\frac{S{R}^2}{4}-2\frac{SR}{2{\sigma }^3}E{\left(r-\mu \right)}^3-\frac{S{R}^2}{2}\\ =1-k_{3}SR+\frac{k_4-1}{4}S{R}^2\end{array}$ (44)

where $k_3=E{\left(r-\mu \right)}^3/{\sigma }^3$ is the coefficient of skewness and $k_4={\mu }_4/{\sigma }^4$ is kurtosis.

Influence Functions Have a Built-In Delta Method

An important aspect of the influence function based method of obtaining an estimator asymptotic variance formula is that the standard delta methods used to obtain the asymptotic variance of a nonlinear transformation of an estimator or estimators is a “built-in” feature. For example, the usual classical method of obtaining the asymptotic variance of the standard deviation estimator is to first derive the sample variance asymptotic variance, which is the numerator expression in (43), and then use the delta method applied to the square root of the variance to obtain the divisor $4{\sigma }^2$ in (43). WithIF based method, the delta method is not needed. For the Sharpe ratio asymptotic variance, a more involved bivariate delta method, that requires the covariance matrix for the numerator and denominator estimates in the Sharpe ratio, has traditionally been used.⁶ But the result (44) is derived more simply and gives the same result as that obtained by the bivariate delta method. This aspect of the influence function approach for obtaining an asymptotic variance formula should not be totally surprising since both the influence function and the delta method involve a derivative linearization.

5.2. SE Computational Alternatives

The SE computational method (42) based on deriving an asymptotic variance expression using (40) is a common method of computing an estimator standard error. A very simple alternative to that approach that does not require use of the asymptotic variance formula is to use the sample mean estimate of (39) with the unknown risk or performance value and IF nuisance parameters replaced by their sample estimates. The resulting estimator direct IF based standard error formula is

$S{E}_{dirIF}\left({\hat{\theta }}_n\right)=\frac{1}{\sqrt{n}}\cdot {\left[\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}I{F}^2\left(r_t;{\hat{\theta }}_n\right)\right]}^{1/2}$ (45)

where the influence function argument F is dropped for notational convenience. For i.i.d. returns, the weak law of large numbers implies that the argument of the square root above is a consistent estimator of the estimator asymptotic variance (40) when $I{F}^2\left(r_t\mathrm{<mo>\; ;\; </mo>}{\hat{\theta }}_n\right)$ has a finite variance. Consequently, one expects the performance of the above SE computational method to be not much different than the method (42) for sample sizes commonly used for computing risk and performance estimators. In fact, it turns out that for some risk and performance estimators that the two SE methods result in exactly the same result. This is the case for the standard deviation (SD), Sharpe ratio (SR) and expected shortfall (ES) estimators, as we show below for SD and SR. A proof for ES is provided in Appendix A1.

Standard Deviation SE

Using the standard deviation asymptotic variance formula (43) in the standard error formula (42), we replace $\sigma$ with the sample volatility estimate ${\hat{\sigma }}_n$ (the square root of the sample variance estimate with divisor n) and replace ${\mu }_4$ with the plug-in estimate ${\hat{\mu }}_{4,n}=\frac{1}{n}{\displaystyle {\sum }_{t=1}^n}{\left(r_t-{\hat{\mu }}_n\right)}^4$ where ${\hat{\mu }}_n$ is the sample mean estimate. This results in the asymptotic variance based standard error formula

$S{E}_{avar}\left({\hat{\sigma }}_n\right)=\frac{1}{\sqrt{n}}\frac{{\left({\hat{\mu }}_{4,n}-{\hat{\sigma }}_n^4\right)}^{1/2}}{2{\hat{\sigma }}_n^2}.$ (46)

On the other hand, use of the standard deviation influence function formula (15) in the standard error formula (45) results in the direct IF based standard error formula

$\begin{array}{c}S{E}_{dirIF}\left({\hat{\sigma }}_n\right)=\frac{1}{\sqrt{n}}{\left[\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}{\left(\frac{{\left(r_t-{\hat{\mu }}_n\right)}^2-{\hat{\sigma }}_n^2}{2{\hat{\sigma }}_n}\right)}^2\right]}^{1/2}\\ =\frac{1}{\sqrt{n}}{\left[\frac{\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}\left({\left(r_t-{\hat{\mu }}_n\right)}^4-2{\hat{\sigma }}_n^2{\left(r_t-{\hat{\mu }}_n\right)}^2+{\hat{\sigma }}_n^4\right)}{4{\hat{\sigma }}_n^2}\right]}^{1/2}\\ =\frac{1}{\sqrt{n}}\frac{{\left({\hat{\mu }}_{4,n}-{\hat{\sigma }}_n^4\right)}^{1/2}}{2{\hat{\sigma }}_n^2}.\end{array}$ (47)

Thus for the standard deviation estimator, the asymptotic variance based the asymptotic variance method and the direct IF method give exactly the same SE formulas.

Sharpe Ratio SE

Using the Sharpe ratio asymptotic variance formula (42) in the standard error formula (44) and representing the unknown quantities by their sample estimates gives

$S{E}_{avar}\left({\widehat{SR}}_n\right)=\frac{1}{\sqrt{n}}{\left(1-{\hat{k}}_3\cdot {\widehat{SR}}_n+\frac{{\hat{k}}_4-1}{4}{\widehat{SR}}_n^2\right)}^2$ (48)

where the sample estimates above are ${\widehat{SR}}_n=\left({\hat{\mu }}_n-r_f\right)/{\hat{\sigma }}_n$, ${\hat{k}}_3={\left(n{\hat{\sigma }}_n^3\right)}^{-1}{\displaystyle {\sum }_{t=1}^n}{\left(r_i-{\hat{\mu }}_n\right)}^3$, and ${\hat{k}}_4={\left(n{\hat{\sigma }}_n^4\right)}^{-1}{\displaystyle {\sum }_{i=1}^n}{\left(r_i-{\hat{\mu }}_n\right)}^4$.

Ont the other hand use of the Sharpe ratio influence function formula (27) in the standard error formula (45), with unknown quantities replaced by their sample estimates, results in

$\begin{array}{c}n\cdot S{E}_{dirIF}^2\left({\widehat{SR}}_n\right)={\displaystyle \underset{t=1}{\overset{n}{\sum }}}{\left[-\frac{{\widehat{SR}}_n}{2{\hat{\sigma }}_n^2}{\left(r_t-{\hat{\mu }}_n\right)}^2+\frac{1}{{\hat{\sigma }}_n}\left(r_t-{\hat{\mu }}_n\right)+\frac{{\widehat{SR}}_n}{2}\right]}^2\\ =\frac{{\widehat{SR}}_n^2}{4{\hat{\sigma }}_n^4}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}{\left(r_t-{\hat{\mu }}_n\right)}^4+1+\frac{{\widehat{SR}}_n^2}{4}-\frac{{\widehat{SR}}_n}{{\hat{\sigma }}_n^3}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}{\left(r_t-{\hat{\mu }}_n\right)}^3\end{array}$

$\begin{array}{c}-\frac{{\widehat{SR}}_n^2}{2{\hat{\sigma }}_n^2}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}{\left(r_t-{\hat{\mu }}_n\right)}^2+0\\ =1-{\hat{k}}_3\cdot {\widehat{SR}}_n+\frac{{\hat{k}}_4-1}{4}{\widehat{SR}}_n^2.\end{array}$ (49)

Thus $S{E}_{avar}\left({\widehat{SR}}_n\right)$ and $S{E}_{dirIF}\left({\widehat{SR}}_n\right)$ are identical expressions. Note that we use a divisor of n for all sample based estimators, and if for example one insisted on using a divisor of $n-1$ for the sample variance/standard deviation, then the above equivalence of methods would not hold. But in that case it is easy to show that the difference between the two methods is $\left(1-\frac{1}{2}{\widehat{SR}}^2\right)/n$.

In Appendix A1 it is shown that for the expected shortfall (ES) estimator, the $S{E}_{avar}$ and $S{E}_{dirIF}$ are identical, and note that confirming that the same is true of the value-at-risk (VaR) estimator is quite easy. It is not clear at present whether or not the “avar” and “dirIF” methods are identical for any risk or performance estimator, but it seems unlikely. However, if $I{F}^2\left(r_1\mathrm{<mo>\; ;\; </mo>}\hat{\theta }\mathrm{,}F\right)$ has an absolute moment of order $p\ge 2$, then $\frac{1}{n}{\displaystyle {\sum }_{t=1}^n}I{F}^2\left(r_t;\hat{\theta }\right)$ converges to $V\left(\theta \right)=E\left[I{F}^2\left(r_1;\hat{\theta },F\right)\right]$ at a rate $n^{-p/2}$, as was shown by Brillinger [25]. Thus one expects the difference between these two quantities to be small for sample sizes of 100 or greater for the case the reasonable case where $p=2$, and even smaller when $p>2$. For example, in the case of the Sharpe ratio we will have $p=2$ if the returns have a finite absolute fourth moment.

Given the simplicity of the direct IF method of computing a risk or performance estimator standard error, which does not make use of an asymptotic variance formula, one may ask why bother with those formulas? One reason is that those formulas provide an understanding of how the values of risk and performance estimates, the distribution parameters, and the risk manager’s choice of discretionary parameters (e.g., tail probabilities and threshold of lower partial moments, etc.) influence the asymptotic variance, and hence computed SE values based on notional returns distribution parameter values. For example, in the case of the Sharpe ratio SE formula (48), an analyst can plug in notional values for the unknown Sharpe ratio, skewness and excess kurtosis.

However, for computing standard errors of returns based risk and performance estimators, use of the direct IF formula (45) has the potential to become standard practice. This is the method that is implemented as a user option for the special case of i.i.d. returns in the Risk and Performance Estimator Standard Errors (RPESE) R package available at https://cran.r-project.org/web/packages/RPESE/index.html, where a Reference Manual and Vignette are provided.

6. Performance of IF Based Standard Error Method

In order to get a sense of the accuracy of risk and performance estimator standard errors, and ensuing confidence interval error rates, computed using the formula (45), we carried out Monte Carlo simulation studies for the SD, SemiSD, SR and DSR estimators. The standard errors labeled $S{E}_{EIF}$ in following discussion are computed with the RPESE R package for computing standard errors of risk and performance estimators using the estimator influence function (EIF) method. We focus here on the SR and DSR, and provide simulation results for the SD and SemiSD estimators in Appendix A3. Our Monte Carlo study compares the expected values of the computed standard errors with “true” standard errors obtained by direct simulation, and reports: 1) the absolute and percent bias of the computed standard errors, and 2) the error rates of 95% confidence intervals based on the standard errors. The details of the simulations are as follows for the case of Sharpe ratio standard errors at sample sizes $N=60,120,240$, for normal and t-distributed returns and with Sharpe ratios $SR=0.2$ and $SR=0.5$.

The simulation process is as follows for the case of normally distributed returns:

(1) Simulate a sample of size N from the return distribution $F\sim N\left(\mu \mathrm{,}{\sigma }^2\right)$ with $\mu =0.01$, with $\sigma =0.05$ for $SR=0.2$, and $\sigma =0.02$ for $SR=0.5$.

(2) Estimate the sample mean $\hat{\mu }$ and sample standard deviation $\hat{\sigma }$, and compute the Sharpe Ratio estimate $\widehat{SR}=\frac{\hat{\mu }-r_f}{\hat{\sigma }}$ with risk free rate $r_f=0$.

(3) Repeat (1) and (2) $M=30000$ times, resulting in Sharpe ratio estimates ${\widehat{SR}}_1\mathrm{,}{\widehat{SR}}_2\mathrm{,}\cdots \mathrm{,}{\widehat{SR}}_M$, and calculate the “true” standard error $S{E}_{MC}$ of the Sharpe ratio as the sample standard deviation of those M estimates.

(4) For each of the M simulated returns of length N we compute the standard error of Sharpe Ratio based on the formula 45, thereby obtaining the computed standard errors $S{E}_1\mathrm{,}S{E}_2\mathrm{,}\cdots \mathrm{,}S{E}_M$, and then calculate the mean $S{E}_{EIF}$ of those M results as the performance of our standard error method.

(5) Calculate the SE method bias $SEBias=S{E}_{EIF}-S{E}_{MC}$, and the SE method percent bias $100\times SEBias/S{E}_{MC}$.

(6) Based on a normal distribution approximation for the ${\widehat{SR}}_i,i=1,2,\cdots ,M$, calculate a nominal 5% error rate confidence interval for each ${\widehat{SR}}_i,i=1,2,\cdots ,M$ as $\left({\widehat{SR}}_i-t_{\alpha /2\mathrm{,}n-1}S{E}_i\mathrm{,}{\widehat{SR}}_i+t_{\alpha /2\mathrm{,}n-1}S{E}_i\right)$, where $t_{1-\alpha /2\mathrm{,}n-1}$ is the $\left(1-\alpha /2\right)$ -th quantile of the t-distribution with $n-1$ degrees of freedom, and calculate the error rate as the fraction of the M replicates for which the replicate confidence interval does not contain the true Sharpe ratio SR.

We repeat the above Monte Carlo simulation for the same three sample sizes using t-distributed returns with degrees of freedom (dof) equal to 5, which is a fairly fat-tailed distribution, $\mu =0.01$, and t distribution scale parameter s is chosen such that the Sharpe ratio of the t-distributed of returns are 0.2 and 0.5 as in the normal distribution case.

The results are displayed in Table 3. It is evident that the MC and EIF SE’s decrease and their Pct. Bias values get smaller as the sample size increases, and the error rates decrease toward their nominal 5% value as the sample sizes increase. The SE’s and error rates depend only very slightly on the SR value for

Table 3. Monte Carlo simulation study of Sharpe ratio standard error estimate.

normal distributions, but more so for thet-distributions. Except for sample size 60, the SE’s are less than half the value of the SR and sometimes considerably less than one half at the larger sample sizes and the larger SR value. As such, the SE’s are quite serviceable except at sample size 60.

It is clear that the Pct Bias and Error Rate values are finite sample effects, with the negative bias in the SE’s and positive bias in the error rates both decreasing with increasing sample size. While the error rates for the normal distribution are overall acceptable and quite acceptable at sample sizes 120 and 240, it will be desirable to implement a bias correction method for the SE’s and error rates for t-distributions. This is a topic for future research.

The Monte Carlo simulation for the Downside Sharpe ratio (DSR), the standard deviation (SD) and the semi-standard deviation (SemiSD) are carried out in a similar manner as for the Sharpe ratio, and the results are shown in Table 4 for the DSR. The pattern of the Pct Bias and Error Rate values for the DSR are quite similar to those for the SR, and the more negative Pct Bias values and the correspondingly higher error rates for the DSR estimate relative to the SR estimate are attributable to the effective sample size differences, i.e., the DSR has roughly half the sample size of the SR and correspondingly the Pct Bias values for the DSR are about $\sqrt{2}$ larger than those for the SR. While the error rates for the DSR for sample sizes 120 and 240 for the normal distribution are pretty acceptable, the error rates for the normal distribution at sample size 60 and the t-distribution at all three sample size are in need of some method of finite sample bias correction to the standard error estimates. This is a topic for further research.

The Monte Carlo results for the SD and SemiSD are shown in TableA1 and TableA2 in Appendix A3.

Table 4. Monte Carlo simulation study of downside Sharpe ratio standard error estimate.

The code to replicate the simulation results for the SR, DSR, SD and SemiSD estimators is available at https://github.com/AnthonyChristidis/InfluenceFunctions_JMF_Simulation.

7. Serially Dependent Returns

In the more general case where the $r_t,t=1,2,\cdots ,n$ are serially correlated, the IF transformed returns time series $IF\left(r_t\mathrm{<mo>\; ;\; </mo>}T\mathrm{,}F\right)$ will generally have serial correlation that needs to be accounted for when computing the variance on the right-hand-side of (38). Spectral analysis theory, extensively used in science and engineering, shows that the variance of the sum of the values of a serially correlated stationary time series is given by the spectral density of the time series at zero frequency. Thus, estimating the variance (38) with serially correlated returns may be accomplished by estimating the spectral density at zero frequency of the IF transformed returns time series $IF\left(r_t\mathrm{<mo>\; ;\; </mo>}T\mathrm{,}F\right)$. Chen and Martin [26] used this approach to compute approximate standard errors of risk and performance estimators, and showed by application examples and Monte Carlo studies that the method works well for several risk and performance estimators. The CM method is implemented in the “Risk and Performance Estimator Standard Errors” R package RPESE authored by Christidis [27].

8. Concluding Comments

We have introduced a new general method for computing standard errors of risk and performance estimators that is simple to implement and does not require an estimator’s asymptotic variance formula. The method only requires a formula for the influence function (IF) of a risk or performance estimator, from which a standard error is computed with simple arithmetic operations on the time series of influence function transformed returns. We have derived the influence function formulas for this purpose for a variety of risk estimators and performance ratio estimators, including the most well-known estimators frequently used in practice as well as a few that are less frequently used. A considerable benefit of the simple IF method of computing an estimator’s standard error is that, if a portfolio or risk manager discovers or invents a new risk or performance estimator for which they want to compute a standard error, they can easily derive the formula for the estimator’s influence function using only the rules of basic calculus.

Although our IF based SE computational method does not require an asymptotic variance formula, such formulas are none-the-less useful in understanding how the values of nuisance parameters, e.g., the values of the Sharpe ratio, skewness and kurtosis in the case of the Sharpe ratio, affect the accuracy of the standard errors (whether computed with our simple method, or via an asymptotic variance formula). Fortunately, an estimator’s asymptotic variance formula is easily computed as the expected value of the squared IF, Appendix A2 derives these formulas for each of the risk and performance estimators considered. An important convenience that arises with an IF derivation of an asymptotic variance formula is that it obviates the need for using the classic delta method, which is an especially welcome simplification in the case of performance ratios for which the bivariate delta method requires derivation of a two-by-two covariance matrix.

Our Monte Carlo simulation studies of the Sharpe ratio and downside Sharpe ratio estimator standard errors and associated confidence interval error rates demonstrate the efficacy of the method, as well as the need to develop finite-sample bias correction formulas for the confidence interval error rates for some sample sizes and some distributions. This is a topic for future study.

Appendix

Appendix A1 contains a proof of the equivalence between IF based standard errors and a direct approach for the standard error of the expected shortfall. Appendix A2 contains the derivation of asymptotic variance formulas using influence functions for the risk and performance estimators discussed in Sections 3 and 4 respectively. Appendix A3 contains a simulation study of IF based standard errors for the standard deviation and the semi-standard deviation.

A1. Equivalence of Expected Shortfall Standard Error Formulas

The influence function of expected shortfall, as given by (21), is

$IF\left(r\mathrm{<mo>\; ;\; </mo>}E{S}_{\alpha }\mathrm{<mo>\; ;\; </mo>}F\right)=-q_{\alpha }-E{S}_{\alpha }-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)$ (50)

and so the asymptotic variance of an expected shortfall estimator is:

$Va{r}_{\infty }\left(E{S}_n\right)=E\left[{\left(-q_{\alpha }-E{S}_{\alpha }-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)\right)}^2\right]$ (51)

$=\frac{1}{{\alpha }^2}{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}{r}^2\text{d}F\left(r\right)+\left(\frac{1}{\alpha }-1\right)\cdot q_{\alpha }^2-\left(2-\frac{2}{\alpha }\right)\cdot E{S}_{\alpha }\cdot q_{\alpha }-E{S}_{\alpha }^2.$ (52)

Let $r_{\left(1\right)}\le r_{\left(2\right)}\le \cdots \le r_{\left(n\right)}$ be the ordered values of the observed returns, and let $\lceil x\rceil$ be the smallest integer greater or equal to x. Then the sample estimate of the lower α-quantile and expected shortfall are:

${\hat{q}}_a=r_{\left(\lceil na\rceil \right)}$ (53)

$\widehat{E{S}_a}=-\frac{1}{\lceil na\rceil }{\displaystyle \underset{i=1}{\overset{\lceil na\rceil }{\sum }}}{r}_{\left(i\right)}.$ (54)

The standard error formula for Expected Shortfall is:

$\widehat{SE}\left(E{S}_n\right)=\frac{1}{\sqrt{n}}\cdot {\left[\frac{1}{n{\alpha }^2}{\displaystyle \underset{i=1}{\overset{\lceil n\alpha \rceil }{\sum }}}{r}_{\left(i\right)}^2+\left(\frac{1}{\alpha }-1\right)\cdot {\hat{q}}_{\alpha }^2-\left(2-\frac{2}{\alpha }\right)\cdot \widehat{E{S}_{\alpha }}\cdot {\hat{q}}_{\alpha }-{\widehat{E{S}_{\alpha }}}^2\right]}^{1/2}$ (55)

where the unknown α-quantile, expected shortfall are replaced by their sample estimates.

It’s trivial to show that the finite sample standard error of the Expected Shortfall from (55) is exactly identical to that from (45).

Plugging in the ordered value and re-arranging, the sample average of squared influence function is:

$\begin{array}{c}\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}I{F}^2\left(r_i;E{S}_{\alpha };F\right)=\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left(q_{\alpha }+E{S}_{\alpha }\right)}^2+\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{I\left(r_i\le q_{\alpha }\right)}{{\alpha }^2}\cdot {\left(r_i-q_{\alpha }\right)}^2+2\left(q_{\alpha }+E{S}_{\alpha }\right)\cdot \frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{I\left(r_i\le q_{\alpha }\right)}{\alpha }\cdot \left(r_i-q_{\alpha }\right)\\ ={\left(q_{\alpha }+E{S}_{\alpha }\right)}^2+\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n\alpha }{\sum }}}\frac{1}{{\alpha }^2}\cdot {\left(r_{\left(i\right)}-q_{\alpha }\right)}^2+2\left(q_{\alpha }+E{S}_{\alpha }\right)\cdot \frac{1}{n}{\displaystyle \underset{i=1}{\overset{n\alpha }{\sum }}}\frac{1}{\alpha }\cdot \left(r_{\left(i\right)}-q_{\alpha }\right)\\ ={\left(q_{\alpha }+E{S}_{\alpha }\right)}^2+\frac{1}{n{\alpha }^2}{\displaystyle \underset{i=1}{\overset{n\alpha }{\sum }}}{r}_{\left(i\right)}^2+\frac{2q_{\alpha }}{\alpha }\cdot E{S}_{\alpha }+\frac{1}{\alpha }{q}_{\alpha }^2+2\left(q_{\alpha }+E{S}_{\alpha }\right)\cdot \left(-E{S}_{\alpha }-q_{\alpha }\right)\\ =\frac{1}{n{\alpha }^2}{\displaystyle \underset{i=1}{\overset{n\alpha }{\sum }}}{r}_{\left(i\right)}^2+\left(\frac{1}{\alpha }-1\right)q_{\alpha }^2+\left(\frac{2}{\alpha }-2\right)E{S}_{\alpha }\cdot q_{\alpha }-E{S}_{\alpha }^2.\end{array}$ (56)

Replace $q_{\alpha }$ and $E{S}_{\alpha }$ with ${\hat{q}}_{\alpha }$ and $\widehat{E{S}_{\alpha }}$ in the above equation, we get standard error formula based on the squared empirical influence function:

$\begin{array}{c}\widehat{SE}\left(E{S}_{\alpha }\right)=\frac{1}{\sqrt{n}}\cdot {\left[\frac{1}{n}{\displaystyle \underset{t=1}{\overset{n}{\sum }}}I{F}^2\left(r_i;E{S}_{\alpha };F\right)\right]}^{1/2}\\ =\frac{1}{n}{\left[\frac{1}{n{\alpha }^2}{\displaystyle \underset{i=1}{\overset{n\alpha }{\sum }}}{r}_{\left(i\right)}^2+\left(\frac{1}{\alpha }-1\right){\hat{q}}_{\alpha }^2+\left(\frac{2}{\alpha }-2\right)\widehat{E{S}_{\alpha }}\cdot {\hat{q}}_{\alpha }-{\widehat{E{S}_{\alpha }}}^2\right]}^{1/2}.\end{array}$ (57)

Note the equation in (57) is identical to Equation (55).

A2. Asymptotic Variance of Risk and Performance Estimators

Here we use influence function based asymptotic variance formula (40) to derive the asymptotic variance expressions for the risk and performance estimators of Sections (3) and (4), and provide references for standard mathematical statistics literature proofs of the formulas.

Standard Deviation (SD)

The expression for the standard deviation asymptotic variance was derived in Section 5.1, and is included here for completeness:

$V\left({\hat{\sigma }}_n\right)=\frac{E{\left(r-\mu \right)}^4-{\sigma }^4}{4{\sigma }^2}$

where $E{\left(r-\mu \right)}^4={\mu }_4$ is the fourth central moment. This result may also be found in a number of mathematical statistics textbooks

Semi-Standard Deviation (SSD)

Using the Semi-Standard deviation influence function formula (17), the asymptotic variance of Semi-SD is

$\begin{array}{c}V\left(\widehat{SemiSD}\right)=E{\left[\frac{{\left(r-\mu \right)}^2\cdot I\left(r\le \mu \right)-2\cdot SemiMean\cdot \left(r-\mu \right)-SemiS{D}^2}{2\cdot SemiSD}\right]}^2\\ =\frac{{\mu }_{4-}+4SemiMea{n}^2\cdot {\sigma }^2-4SemiMean\cdot {\mu }_{3-}-SemiS{D}^4}{4SemiS{D}^2}\end{array}$

where

${\mu }_{3-}={\displaystyle {\int }_{-\infty }^{\mu }}{\left(x-\mu \right)}^3\text{d}F\left(x\right)\text{and}{\mu }_{4-}={\displaystyle {\int }_{-\infty }^{\mu }}{\left(x-\mu \right)}^4\text{d}F\left(x\right)\mathrm{.}$

See De Capitani [9] for a formal asymptotic normality derivation.

Lower Partial Moment (LPM1, LPM2)

Using the LPM influence function formula (19), the asymptotic variance of the LPM of order k is

$\begin{array}{c}V\left(\widehat{LP{M}_k}\right)=E{\left[{\left(c-r\right)}^{k}I\left(r\le c\right)-LP{M}_k\right]}^2\\ =E\left[{\left(c-r\right)}^{2k}I\left(r\le c\right)-2{\left(c-r\right)}^{k}I\left(r\le c\right)\cdot LP{M}_k+LP{M}_k^2\right]\\ =LP{M}_{2k}-LP{M}_k^2.\end{array}$

For the special case of k= 1, $V\left(\widehat{LP{M}_1}\right)=LP{M}_2-LP{M}_1^2$.

We have found no published asymptotic normal distribution derivation, but it should be quite straightforward.

Expected Shortfall (ES)

Using the ES influence function formula (21), the asymptotic variance of ES is

$\begin{array}{c}V\left(\widehat{ES}\right)=E{\left[-q_{\alpha }-E{S}_{\alpha }-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)\right]}^2\\ =\frac{1}{{\alpha }^2}\cdot {\displaystyle {\int }_{-\infty }^{q_{\alpha }}}{r}^2\text{d}F\left(r\right)+\left(\frac{1}{\alpha }-1\right)\cdot q_{\alpha }^2-\left(2-\frac{2}{\alpha }\right)\cdot E{S}_{\alpha }\cdot q_{\alpha }-E{S}_{\alpha }^2.\end{array}$

For standard mathematical statistics derivation see Stoyanov and Rachev [28].

Value-at-Risk (VaR)

Using the VaR influence function formula (25), the asymptotic variance of VaR is

$V\left(\widehat{Va{R}_a}\right)=E{\left[\frac{I\left(r\le q_{\alpha }\right)-\alpha }{f\left(q_{\alpha }\right)}\right]}^2=\frac{\left(1-\alpha \right)\alpha }{{\left[f\left(q_{\alpha }\right)\right]}^2}.$

We note that VaR is sample quantile, i.e., a probability $\alpha$ order statistics, and as such the above asymptotic variance formula is also available in many mathematical statistics textbooks.

Sharpe Ratio (SR)

The expression for the Sharpe ratio asymptotic variance was derived in Section 5.1, and is included here for completeness

$V\left({\widehat{SR}}_n\right)=1-k_{3}SR+\frac{k_4-1}{4}S{R}^2$

where $k_3=E{\left(r-\mu \right)}^3/{\sigma }^3$ is the coefficient of skewness and $k_4={\mu }_4/{\sigma }^4$ is kurtosis. For standard derivation see Mertens [24].

Downside Sharpe Ratio (DSR)

Using the DSR influence function formula (29), the asymptotic variance of DSR is

$\begin{array}{l}V\left(\widehat{DSR}\right)=E[-\frac{DSR\cdot I\left(r\le \mu \right)}{2\sqrt{2}\cdot SemiS{D}^2}{\left(r-\mu \right)}^2+(\frac{DSR\cdot SemiMean}{\sqrt{2}SemiS{D}^2}\\ +{\frac{1}{\sqrt{2}SemiSD})\left(r-\mu \right)+\frac{DSR}{2\sqrt{2}}]}^2\\ =E[\frac{DS{R}^2\cdot I\left(r\le \mu \right)}{8\cdot SemiS{D}^4}{\left(r-\mu \right)}^4+(\frac{DSR\cdot SemiMean}{\sqrt{2}SemiS{D}^2}\\ +{\frac{1}{\sqrt{2}SemiSD})}^2{\left(r-\mu \right)}^2+\frac{DS{R}^2}{8}]+E[-\frac{DSR\cdot I\left(r\le \mu \right)}{\sqrt{2}\cdot SemiS{D}^2}\\ \cdot (\frac{DSR\cdot SemiMean}{\sqrt{2}SemiS{D}^2}+\frac{1}{\sqrt{2}SemiSD}){\left(r-\mu \right)}^3-\frac{DS{R}^2\cdot I\left(r\le \mu \right)}{4SemiS{D}^2}{\left(r-\mu \right)}^2]\end{array}$

$\begin{array}{l}+E\left[\left(\frac{DSR\cdot SemiMean}{\sqrt{2}SemiS{D}^2}+\frac{1}{\sqrt{2}SemiSD}\right)\frac{DSR}{\sqrt{2}}\left(r-\mu \right)\right]\\ =\frac{DS{R}^2}{8\cdot SemiS{D}^4}LPM{4}_{\mu }+{\left(\frac{DSR\cdot SemiMean}{\sqrt{2}SemiS{D}^2}+\frac{1}{\sqrt{2}SemiSD}\right)}^2{\sigma }^2+\frac{DS{R}^2}{8}\end{array}$

$\begin{array}{l}-\frac{DSR}{\sqrt{2}\cdot SemiS{D}^2}\left(\frac{DSR\cdot SemiMean}{\sqrt{2}SemiS{D}^2}+\frac{1}{\sqrt{2}SemiSD}\right)LPM{3}_{\mu }-\frac{DS{R}^2}{4}\\ =\frac{{\sigma }^2}{2SemiS{D}^2}-\left(\frac{2SemiMean\cdot {\sigma }^2-LPM{3}_{\mu }}{2SemiS{D}^3}\right)DSR\\ +\left(\frac{LPM{4}_{\mu }+4SemiMea{n}^2\cdot {\sigma }^2-4SemiMean\cdot SemiSD\cdot LPM{3}_{\mu }}{8\cdot SemiS{D}^4}-\frac{1}{8}\right)DS{R}^2.\end{array}$

We have not found a formal asymptotic normal distribution derivation in the published literature.

Sortino Ratio (SoRc)

Using the Sortino ratio influence function formula (30), the asymptotic variance of Sortino ratio is

$\begin{array}{l}V\left(\widehat{SoRc}\right)=E{\left[\frac{-SoRc\cdot I\left(r\le c\right)}{2\cdot LPM{2}_c}{\left(r-c\right)}^2+\frac{1}{\sqrt{LPM{2}_c}}\left(r-\mu \right)+\frac{SoRc}{2}\right]}^2\\ =E\left[\frac{So{R}^{2}c\cdot I\left(r\le c\right)}{4\cdot {\left(LPM{2}_c\right)}^2}{\left(r-c\right)}^4+\frac{1}{LPM{2}_c}{\left(r-\mu \right)}^2+\frac{So{R}^{2}c}{4}\right]\\ +E[-\frac{SoRc\cdot I\left(r\le c\right)}{{\left(LPM{2}_c\right)}^{3/2}}{\left(r-c\right)}^2\left(r-\mu \right)\\ -\frac{So{R}^{2}c\cdot I\left(r\le c\right)}{2LPM{2}_c}{\left(r-c\right)}^2+\frac{SoRc}{\sqrt{LPM{2}_c}}\left(r-\mu \right)]\end{array}$

$\begin{array}{l}=\frac{So{R}^{2}c}{4\cdot {\left(LPM{2}_c\right)}^2}LPM{4}_c+\frac{{\sigma }^2}{LPM{2}_c}+\frac{So{R}^{2}c}{4}\\ +E\left[-\frac{SoRc\cdot I\left(r\le c\right)}{{\left(LPM{2}_c\right)}^{3/2}}{\left(r-c\right)}^2\left(r-c+c-\mu \right)\right]-\frac{So{R}^{2}c}{2}\\ =\frac{So{R}^{2}c}{4\cdot {\left(LPM{2}_c\right)}^2}LPM{4}_c+\frac{{\sigma }^2}{LPM{2}_c}-\frac{So{R}^{2}c}{4}\\ -\frac{SoRc}{{\left(LPM{2}_c\right)}^{3/2}}LPM{3}_c-\frac{SoRc\cdot \left(c-\mu \right)}{{\left(LPM{2}_c\right)}^{3/2}}LPM{2}_c\\ =\frac{{\sigma }^2}{LPM{2}_c}-\left(\frac{LPM{3}_c-\left(\mu -c\right)LPM{2}_c}{{\left(LPM{2}_c\right)}^{3/2}}\right)SoRc+\left(\frac{LPM{4}_c}{4\cdot {\left(LPM{2}_c\right)}^2}-\frac{1}{4}\right)So{R}^{2}c.\end{array}$

For a published derivation see De Capitani [9], where $\mu =E\left(Y\right)$ and Y is equal to our $r-c$.

ES Ratio (ESratio)

Using the ES Ratio influence function formula (32), the asymptotic variance of ES ratio is

$\begin{array}{c}V\left(\widehat{ESratio}\right)=E{\left[\frac{r-\mu }{E{S}_{\alpha }}-\frac{ESratio}{E{S}_{\alpha }}\left(-q_{\alpha }-E{S}_{\alpha }-\frac{I\left(r\le q_{\alpha }\right)}{\alpha }\cdot \left(r-q_{\alpha }\right)\right)\right]}^2\\ =\frac{{\sigma }^2}{E{S}_{\alpha }^2}-\frac{2ESratio}{E{S}_{\alpha }^2}\cdot A+\frac{ESrati{o}^2}{E{S}_{\alpha }^2}\cdot B\end{array}$

where

$\begin{array}{l}A=\frac{-1}{a}{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}\left(r-\mu \right)\left(r-q_{\alpha }\right)\text{d}F\left(r\right)\\ \text{and}B=\frac{1}{a^2}{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}{r}^2\text{d}F\left(r\right)+\left(\frac{1}{\alpha }-1\right)q_{\alpha }^2-\left(2-\frac{2}{\alpha }\right)E{S}_{\alpha }\cdot q_{\alpha }-E{S}_{\alpha }^2.\end{array}$

For a standard asymptotic normality derivation see Capitani and Pasquazzi [10].

VaR Ratio (VaRratio)

Using the VaR Ratio influence function formula (34), we find the asymptotic variance of VaR ratio is

$\begin{array}{c}V\left(\widehat{VaRratio}\right)=E{\left[-\frac{r-\mu }{q_{\alpha }}+\frac{VaRratio}{q_{\alpha }}\cdot \frac{I\left(r\le q_{\alpha }\right)-\alpha }{f\left(q_{\alpha }\right)}\right]}^2\\ =\frac{{\sigma }^2}{q_{\alpha }^2}+\frac{VaRrati{o}^2}{q_{\alpha }^2}\frac{\left(1-\alpha \right)\alpha }{{\left[f\left(q_{\alpha }\right)\right]}^2}-\frac{2VaRratio}{q_{\alpha }^2\cdot f\left(q_{\alpha }\right)}\left[-\alpha \left(E{S}_{\alpha }+\mu \right)\right].\end{array}$

For a formal asymptotic normality derivation see Capitani and Pasquazzi [10].

Rachev Ratio (RachevRatio)

Using the Rachev Ratio influence function formula (35), the asymptotic variance of Rachev Ratio is

$\begin{array}{c}V\left(\widehat{RachR}\right)=E[\frac{1}{E{S}_{\alpha }}\left(\frac{I\left(r\ge q_{1-\beta }\right)}{\beta }\left(r-q_{1-\beta }\right)+q_{1-\beta }-E{G}_{\beta }\right)\\ -{\frac{RachR}{E{S}_{\alpha }}\cdot \left(\frac{-I\left(r\le q_{\alpha }\right)}{\alpha }\left(r-q_{\alpha }\right)-q_{\alpha }-E{S}_{\alpha }\right)]}^2\\ =\frac{1}{E{S}_{\alpha }^2}\cdot V_1+\frac{E{G}_{\beta }^2}{E{S}_{\alpha }^4}\cdot V_2-\frac{2E{G}_{\beta }}{E{S}_{\alpha }^3}\cdot V_3\end{array}$

where

$V_1=\frac{1}{b^2}{\displaystyle {\int }_{q_{1-\beta }}^{+\infty }}{r}^2\text{d}F\left(r\right)+\left(\frac{1}{\beta }-1\right)q_{1-\beta }^2+\left(2-\frac{2}{\beta }\right)E{G}_{\beta }\cdot q_{1-\beta }-E{G}_{\beta }^2$

$V_2=\frac{1}{{\alpha }^2}{\displaystyle {\int }_{-\infty }^{q_{\alpha }}}{r}^2\text{d}F\left(r\right)+\left(\frac{1}{\alpha }-1\right)q_{\alpha }^2-\left(2-\frac{2}{\alpha }\right)E{S}_{\alpha }\cdot q_{\alpha }-E{S}_{\alpha }^2$

$V_3=-\left(q_{1-\beta }-E{G}_{\beta }\right)\left(-q_{\alpha }-E{S}_{\alpha }\right).$

We could not find standard asymptotic normality derivation of the above result in the literature.

Omega Ratio (Omega)

Using the Omega ratio influence function formula 36, the asymptotic variance of Omega function is

$V\left(\hat{\Omega }\right)=E{\left[\frac{r-c}{LPM{1}_c}+\frac{\left(\mu -c\right)\cdot \left(r-c\right)\cdot I\left(r\le c\right)}{{\left(LPM{1}_c\right)}^2}\right]}^2$

$\begin{array}{l}=E\left[\frac{{\left(r-c\right)}^2}{{\left(LPM{1}_c\right)}^2}+\frac{2\left(\mu -c\right)\cdot {\left(r-c\right)}^2\cdot I\left(r\le c\right)}{{\left(LPM{1}_c\right)}^3}+\frac{{\left(\mu -c\right)}^2\cdot {\left(r-c\right)}^2\cdot I\left(r\le c\right)}{{\left(LPM{1}_c\right)}^4}\right]\\ =\frac{{\sigma }^2+{\left(\mu -c\right)}^2}{{\left(LPM{1}_c\right)}^2}+\frac{2\left(\mu -c\right)\cdot LPM{2}_c}{{\left(LPM{1}_c\right)}^3}+\frac{{\left(\mu -c\right)}^2\cdot LPM{2}_c}{{\left(LPM{1}_c\right)}^4}.\end{array}$

Then with $\Omega =1+\omega$ where $\omega =\left(\mu -c\right)/LPM{1}_c$, the above formula becomes

$V\left(\hat{\Omega }\right)=\frac{{\sigma }^2}{{\left(LPM{1}_c\right)}^2}+\frac{2LPM{2}_c}{{\left(LPM{1}_c\right)}^2}\omega +\left(\frac{{\left(LPM{1}_c\right)}^2+LPM{2}_c}{{\left(LPM{1}_c\right)}^2}\right){\omega }^2.$

For an asymptotic derivation see De Capitani [9].

A3. Monte Carlo Simulation Results for SD and SemiSD Estimators

The Monte Carlo simulation results below for the standard deviation (SD) and semi-standard deviation (SemiSD) estimators was carried out in a similar manner as for the Sharpe ratio (SR) and downside Sharpe ratio (DSR) estimator in Section 6, and the results are displayed in TableA1 and TableA2. What is perhaps a little surprising is the larger finite sample Pct Bias and Error Rate values for the SD and SemiSD estimators, as compared with the SR and DSR estimators in Section 6. The fourth central moment appearing in the asymptotic variance of the SD and SemiSD estimators (see Appendix A2) is likely a cause of this. Some research is needed on methods to correct the finite sample biases evident in TableA1 and TableA2.

Table A1. Monte Carlo simulation study of standard deviation standard error estimate.

Table A2. Monte Carlo simulation study of semi-standard deviation standard error estimate.

NOTES

¹The term functional refers to a function whose domain is an infinite dimensional space, e.g., the space of all distribution functions, or all distribution functions for which the mean exists, etc.

²This kind of derivative is called a Gateaux derivative.

³This is the SMCR special case of the class of higher-moment coherent risk (HMCR) measures studied by Krokhmal [14].

⁴A justification for dropping the remainder term in (10) can be found in Section 3.7 of [3].

⁵Asymptotic normality of ${\hat{\theta }}_n$ is the condition that $\sqrt{n}\left({\hat{\theta }}_n-\theta \right)\stackrel{d}{\to }N\left(\mathrm{0,}V\left(\theta \right)\right)$ where the $\stackrel{d}{\to }$ means convergence in distribution, i.e., the sequence of distribution functions of the left hand side converge to a normal distribution function with mean zero and variance $V\left(\theta \right)$.

⁶See for example, Mertens [24].

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Hampel, F.R. (1974) The Influence Curve and Its Role in Robust Estimation. Journal of the American Statistical Association, 69, 383-393. https://doi.org/10.1080/01621459.1974.10482962
[2]	Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J. and Stahel, W.A. (1986) Robust Statistics: The Approach Based on Influence Functions. Wiley, Hoboken.
[3]	Maronna, R.A., Martin, R.D., Yohai, V.J. and Salibian-Barrera, M. (2019) Robust Statistics: Theory and Methods (with R). Wiley, Hoboken.
[4]	Yamai, Y. and Yoshiba, T. (2002) Comparative Analyses of Expected Shortfall and Value-at-Risk: Their Estimation Error, Decomposition, and Optimization. Monetary and Economic Studies, 87-121.
[5]	Scherer, B. and Martin, R.D. (2005) Introduction to Modern Portfolio Optimization with NUOPT and S-PLUS. Springer, New York. https://doi.org/10.1007/978-0-387-27586-4
[6]	De Miguel, V. and Nogales, F.J. (2009) Portfolio Selection with Robust Estimation. Operations Research, 57, iv-799. https://doi.org/10.1287/opre.1080.0566
[7]	Cont, R., Deguest, R. and Scandolo, G. (2010) Robustness and Sensitivity Analysis of Risk Measurement Procedures. Quantitative Finance, 10, 593-606. https://doi.org/10.1080/14697681003685597
[8]	Martin, R.D. and Zhang, S.Y. (2019) Nonparametric Versus Parametric Expected Shortfall. Journal of Risk, 21, 1-41. https://ssrn.com/abstract=2747179 https://doi.org/10.21314/JOR.2019.416
[9]	De Capitani, L. (2014) Interval Estimation for the Sortino Ratio and the Omega Ratio. Communications in Statistics, 43, 1385-1429. https://doi.org/10.1080/03610918.2012.722808
[10]	De Capitani, L. and Pasquazzi, L. (2015) Inference for Performance Measures for Financial Assets. METRON, 73, 73-98. https://doi.org/10.1007/s40300-014-0055-y
[11]	Fernholz, L.T. (1983) Von Mises Calculus for Statistical Functionals. Vol. 19, Springer, New York. https://doi.org/10.1007/978-1-4612-5604-5
[12]	McNeil, A.J., Frey, R. and Embrechts, P. (2015) Quantitative Risk Management: Concepts, Techniques and Tools-Revised Edition. Princeton University Press, Princeton.
[13]	Fishburn, P. (1977) Mean-Risk Analysis with Risk Associate with Below-Target Returns. The American Economic Review, 67, 116-126.
[14]	Krokhmal, P.A. (2007) Higher Moment Coherent Risk Measures. Quantitative Finance, 7, 373-387. https://doi.org/10.1080/14697680701458307
[15]	Ziemba, W. (2005) The Symmetric Downside-Risk Sharpe Ratio. Journal of Portfolio Management, 32, 108-122. https://doi.org/10.3905/jpm.2005.599515
[16]	Sortino, F.A. and van der Meer, R.A.H. (1991) Downside Risk. Journal of Portfolio Management, 17, 27-31. https://doi.org/10.3905/jpm.1991.409343
[17]	Sortino, F.A. and Price, L.N. (1994) Performance Measurement in a Downside Risk Framework. Journal of Investing, 3, 59-64. https://doi.org/10.3905/joi.3.3.59
[18]	Sortino, F.A. and Forsey, H.J. (1996 On the Use and Misuse of Downside Risk. Journal of Portfolio Management, 22, 35-42. https://doi.org/10.3905/jpm.1996.35
[19]	Martin, R.D., Rachev, S.Z. and Siboulet, F. (2003) Phi-alpha Optimal Portfolios and Extreme Risk Management. The Best of Wilmott 1: Incorporating the Quantitative Finance Review, 1, 70-83.
[20]	Favre, L. and Galeano, J. (2002) Mean-Modified Value-at-Risk Optimization with Hedge Funds. The Journal of Alternative Investment, 5, 21-25. https://doi.org/10.3905/jai.2002.319052
[21]	Biglova, A., Ortobelli, S., Rachev, Z. and Stoyano, S. (2004) Different Approaches to Risk Estimation in Portfolio Theory. Journal of Portfolio Management, 31, 103-112. https://doi.org/10.3905/jpm.2004.443328
[22]	Stoyanov, S.V., Rachev, S.T. and Fabozzi, F.J. (2007) Optimal financial Portfolios. Applied Mathematical Finance, 14, 401-436. https://doi.org/10.1080/13504860701255292
[23]	Keating, C. and Shadwick, W.F. (2002) A Universal Performance Measure. Journal of Performance Measurement, 6, 59-84.
[24]	Mertens, E. (2002) Comments on the Correct Variance of Estimated Sharpe Ratios in Lo (2002, FAJ) When Returns Are IID. http://www.elmarmertens.org/
[25]	Brillinger, D.R. (1962) A Note on the Rate of Convergence of a Mean. Biometrika 49, 574-576. https://doi.org/10.1093/biomet/49.3-4.574
[26]	Chen, X. and Martin, R.D. (2021) Standard Errors of Risk and Performance Measure Estimators for Serially Correlated Returns. Journal of Risk, 23, 1-41. https://doi.org/10.21314/JOR.2020.446
[27]	Christidis, A.A. and Chen, X. (2019) RPESE R Package Manual and Vignette. https://cran.r-project.org/web/packages/RPESE
[28]	Stoyanov, S.V. and Rachev, S.T. (2008) Asymptotic Distribution of the Sample Average Value-at-Risk. Journal of Computational Analysis and Applications, 3, 443-461.