Stochastic Maximum Principle for Optimal Advertising Models with Delay and Non-Convex Control Spaces

Giuseppina Guatteri¹, Federica Masiero²

¹Dipartimento di Matematica, Politecnico di Milano via Bonardi, Milano, Italia.
²Dipartimento di Matematica e Applicazioni, Università di Milano-Bicocca via Cozzi, Milano, Italia.

DOI: 10.4236/apm.2024.146025                

Abstract

In this paper we study optimal advertising problems that model the introduction of a new product into the market in the presence of carryover effects of the advertisement and with memory effects in the level of goodwill. In particular, we let the dynamics of the product goodwill to depend on the past, and also on past advertising efforts. We treat the problem by means of the stochastic Pontryagin maximum principle, that here is considered for a class of problems where in the state equation either the state or the control depend on the past. Moreover the control acts on the martingale term and the space of controls U can be chosen to be non-convex but now the space of controls U can be chosen to be non-convex. The maximum principle is thus formulated using a first-order adjoint Backward Stochastic Differential Equations (BSDEs), which can be explicitly computed due to the specific characteristics of the model, and a second-order adjoint relation.

Keywords

Stochastic Optimal Control, Delay Equations, Advertisement Models, Stochastic Maximum Principle

1. Introduction

We consider a stochastic model for problems of optimal advertising under uncertainty, so we have to study a stochastic version of advertising models. We start from the stochastic model introduced in [1] (see also [2] and [3] ): we consider carryover effects of the advertising, which in the model reads as delay in the control, and with memory effects of the level of goodwill, which in the model reads as delay in the state. We refer also to [4] for optimal advertising problems with memory effects in the level of goodwill.

In our model the delay in the effect of advertising affects the martingale term of the state equation, namely we consider, following [1] , the controlled stochastic differential equation in $ℝ$ with pointwise delay in the state and in the control

$\{\begin{array}{l}\text{d}x\left(t\right)=\left[b_{0}u\left(t\right)-a_{0}x\left(t\right)-a_{d}x\left(t-d\right)\right]\text{d}t+{\sigma }_{1}x\left(t\right)\text{d}{W}_t^1+{\sigma }_{2}u\left(t-d\right)\text{d}{W}_t^2,\\ x\left(\tau \right)=x_0\left(\tau \right),\tau \in \left[-d,0\right].\end{array}$ (1.1)

Following the usual notations, see e.g. [1] , x is the level of goodwill, $a_0$ and $a_d$ are factors related to the image deterioration in case of no advertising, $b_0$ is a constant representing an advertising effectiveness factor. The diffusion term ${\sigma }_{1}x\left(t\right)\text{d}{W}_t^1$ represents for the word of mouth communication, with ${\sigma }_1\ge 0$ the so-called “advertising volatility”; while the second diffusion term keeps track of the delayed effect of the advertising effort u, and the constant ${\sigma }_2\ge 0$ is the effectiveness volatility of the communication. Besides, $x_0\left(0\right)$ is the initial goodwill level, while $x_0\left(\tau \right),\tau \in \left[-d,0\right)$ is the history of the goodwill level before the advertising campaign is started.

The functional to maximize, over all controls in $\mathcal{U}$ , is the following profit, defined on finite horizon:

$\bar{J}\left(t\mathrm{,}x\mathrm{,}u\right)=\mathbb{E}{\displaystyle {\int }_t^T}\left(-c\left(u\left(s\right)\right)+l\left(x\left(s\right)\right)\right)\text{d}s+\mathbb{E}r\left(x\left(T\right)\right)\mathrm{,}$ (1.2)

where c is the cost of advertising, l is the current reward, and r is the reward from the final goodwill. Our purpose is to derive a maximum principle for such a problem with non convex control space U, extending the results already present in the literature for the convex case, see e.g. [5] , where the stochastic maximum principle for control problems with pointwise delay in the state and in the control is studied, see also [6] and [7] where more general models are treated but the convexity of the control space U is still required. We underline the fact that stochastic control problems with diffusion depending on the control are difficult to treat; concerning problems related to advertising we mention the recent paper [8] , where the author solves the problem by means of the dynamic programming approach, differently from here the case of pointwise delay cannot be treated, moreover it is taken into account a diffusion depending linearly on the control, but not on the delayed control. We stress that one novelty of this paper is to handle the new difficulty coming form the non convexity of U: also in the non delayed case the non convexity of U makes the approach based on stochastic maximum principle more complicated. First we need to utilize the spike variation of the control and introduce the second variation to handle the control acting on the martingale term. We will therefore follow [9] to introduce this additional step. Thanks to the specific characteristics of the state equation, we can derive a quasi-explicit form for both the first adjoint, which results in an anticipating backward equation, see [10] , and the second adjoint, which is written using the optimal cost and a simple auxiliary process. Note that the specific case we address is not considered in [11] and does not completely fall within the hypotheses of [9] , as we also consider the delay the control appearing in the martingale term.

The structure of the paper is the following: in section 2 we present the notation, the assumptions and the control problem. Section 3 is devoted to collect the results on the first variation of the optimal state and also on the second variation, which we have to study since the space of controls is not convex. Section 4 concerns the first and second adjoint relations necessary to formulate the stochastic maximum principle, which is stated and proved section 4.3.

2. Stochastic Control Problem for Delay Equations Appearing in Advertising Models

2.1. Assumptions and Preliminaries

Let $W\left(t\right)$ be a 2 dimensional brownian motion defined on a complete probability space $\left(\Omega \mathrm{,}F\mathrm{,}\mathbb{P}\right)$ and ${\left({\mathcal{F}}_t\right)}_{t\ge 0}$ the natural filtration generated by W, augmented in the usual way. We fix a finite time horizon $T>0$ and we set, for every $q\ge 1$ and for every $0\le u\le v\le T$ , the following spaces:

$L_{\mathcal{F}}^0\left(\Omega \times \left[u\mathrm{,}v\right]\mathrm{<mo>\; ;\; </mo>}{ℝ}^k\right)=\mathrm{<mo>\; :\; </mo>}\left\{X\mathrm{<mo>\; :\; </mo>}\Omega \times \left[u\mathrm{,}v\right]\to {ℝ}^k\mathrm{,}\left({\mathcal{F}}_t\right)\text{-progressive}\text{measurable}\right\}$ (2.1)

$\begin{array}{l}{L}_{\mathcal{F}}^q\left(\Omega \times \left[u\mathrm{,}v\right]\mathrm{<mo>\; ;\; </mo>}{ℝ}^k\right)\\ =\mathrm{<mo>\; :\; </mo>}\left\{X\mathrm{<mo>\; :\; </mo>}\Omega \times \left[u\mathrm{,}v\right]\to {ℝ}^k\mathrm{,}\left({\mathcal{F}}_t\right)\text{-progr}\text{.meas}\text{.,}{\left(\mathbb{E}{\displaystyle {\int }_u^v}{\left|X\left(t\right)\right|}^q\text{d}t\right)}^{1/q}<\infty \right\}\end{array}$ (2.2)

$\begin{array}{l}{L}_{\mathcal{F}}^q\left(\Omega \mathrm{<mo>\; ;\; </mo>}C\left(\left[u\mathrm{,}v\right]\mathrm{<mo>\; ;\; </mo>}{ℝ}^k\right)\right)\\ =\mathrm{<mo>\; :\; </mo>}\left\{X\mathrm{<mo>\; :\; </mo>}\Omega \times \left[u\mathrm{,}v\right]\to {ℝ}^k\mathrm{,}\left({\mathcal{F}}_t\right)\text{-progr}\text{.meas}\text{.,}{\left(\mathbb{E}\underset{t\in \left[u\mathrm{,}v\right]}{\sup }{\left|X\left(t\right)\right|}^q\right)}^{1/q}<\infty \right\}\end{array}$ (2.3)

2.2. Formulation of the Problem

We recall the state equation we are interested in:

$\{\begin{array}{l}\text{d}x\left(t\right)=\left[b_{0}u\left(t\right)-a_{0}x\left(t\right)-a_{d}x\left(t-d\right)\right]\text{d}t+{\sigma }_{1}x\left(t\right)\text{d}{W}_t^1+{\sigma }_{2}u\left(t-d\right)\text{d}{W}_t^2,\\ x\left(\tau \right)=x_0\left(\tau \right),\tau \in \left[-d,0\right),x\left(0\right)=x_0\left(0\right).\end{array}$ (2.4)

We assume the following on the coefficients:

Hypothesis 2.1

(i) $a_0\mathrm{,}{b}_0\mathrm{,}{\sigma }_1\mathrm{,}{\sigma }_2\in ℝ$ ;

(ii) the control strategy u, the advertisement in this case, belongs to the space

$\mathcal{U}\mathrm{<mo>\; :\; </mo>}=\left\{z\in L_{\mathcal{F}}^0\left(\Omega \times \left[\mathrm{0,}T\right]\mathrm{,}ℝ\right)\mathrm{<mo>\; :\; </mo>}z\left(t\right)\in U\mathrm{,}\mathbb{P}-a\mathrm{.}s\mathrm{.}\right\}$

where U is a bounded subset of $ℝ$ possibly non convex, in particular U can be a bounded subset of $\mathbb{N}$ and this represents the realistic situation in which the advertisement is multiple of a given quantity;

(iii) $d>0$ is the delay with which the past of the state affects the system;

We recall that the purpose is to maximize, over all controls in $\mathcal{U}$ , the following profit, on finite horizon:

$\bar{J}\left(t\mathrm{,}x\mathrm{,}u\right)=\mathbb{E}{\displaystyle {\int }_t^T}\left(-c\left(u\left(s\right)\right)+l\left(x\left(s\right)\right)\right)\text{d}s+\mathbb{E}r\left(x\left(T\right)\right)\mathrm{,}$ (2.5)

Hypothesis 2.2 We assume the following:

(i) according to the literature, see e.g. [12] , $c\mathrm{<mo>\; :\; </mo>}U\to ℝ$ is non linear, convex and locally lipschitz;

(ii) $l\mathrm{<mo>\; :\; </mo>}\left[\mathrm{0,}T\right]\times ℝ\to ℝ$ represents the current reward, it is twice differentiable with at most linear growth;

(iii) $r\mathrm{<mo>\; :\; </mo>}ℝ\to ℝ$ represents the foreseen reward from the final goodwill and it is twice differentiable and strictly increasing, with bounded second derivatives.

From now on, since we recall and apply the results in [9] where a mini-mization problem is considered, we focus on the problem (equivalent to the original one of maximizing the profit $\bar{J}$ given in (2.5) since $J=-\bar{J}$ ) of minimizing, over all admissible control in U, the cost functional

$J\left(t\mathrm{,}x\mathrm{,}u\right)=\mathbb{E}{\displaystyle {\int }_t^T}\left(c\left(u\left(s\right)\right)-l\left(x\left(s\right)\right)\right)\text{d}s-\mathbb{E}r\left(x\left(T\right)\right)\mathrm{,}$ (2.6)

We are going to formulate necessary conditions for optimality.

3. First and Second Order Variations of the Optimal State Equation

Since U is not necessarily convex we use the spike variation method, see [13] , see also [9] for the case of delay equation.

Let $\left(u\mathrm{,}x\right)$ be an optimal couple: that is u is an optimal control and x its corresponding optimal trajectory, that is solution to Equation (2.4). The spike variation works as follows: for $\epsilon >0$ we consider an interval $V_{\epsilon }\subset \left[\mathrm{0,}T\right]$ , with $m\left(V_{\epsilon }\right)=\epsilon$ , where m is the Lebesgue measure. Let $v\in U$ we set

$u^{\epsilon }\left(t\right)=\{\begin{array}{ll}u\left(t\right)\hfill & t\in \left[\mathrm{0,}T\right]\backslash V_{\epsilon }\hfill \\ v\hfill & t\in V_{\epsilon }\hfill \end{array}$ (3.1)

We are going to derive a maximum principle for the control problem with state Equation (2.4) and cost functional (2.6) in the case of non convex space of controls. We will denote by $x^{\epsilon }$ the solution of (2.4) corresponding to $u^{\epsilon }$ and $\delta u$ the spike variation of the control, i.e. $\delta u\left(t\right):=\left(u\left(t\right)-v\right)I_{V_{\epsilon }}\left(t\right)$ .

We can write the equation for the first order variation of the state:

$\{\begin{array}{l}\text{d}{y}^{\epsilon }\left(t\right)=\left[-a_0{y}^{\epsilon }\left(t\right)-a_d{y}^{\epsilon }\left(t-d\right)\right]\text{d}t+{\sigma }_1{y}^{\epsilon }\left(t\right)\text{d}{W}_t^1+{\sigma }_2\delta u\left(t-d\right)\text{d}{W}_t^2,t\in \left[0,T\right],\\ y^{\epsilon }\left(0\right)=0,y^{\epsilon }\left(\tau \right)=0,-d\le \tau <0.\end{array}$

(3.2)

where $\delta u^{\epsilon }\left(t-d\right)=\left(u\left(t-d\right)-v\right)I_{V_{\epsilon }}\left(t-d\right)$ and the second variation is, for $t\in \left[\mathrm{0,}T\right]$ ,

$\{\begin{array}{l}\text{d}{z}^{\epsilon }\left(t\right)=\left[-a_0{z}^{\epsilon }\left(t\right)-a_d{z}^{\epsilon }\left(t-d\right)\right]\text{d}t+b_0\delta u\left(t\right)\text{d}t+{\sigma }_1{z}^{\epsilon }\left(t\right)\text{d}{W}_t^1\\ z^{\epsilon }\left(0\right)=0,z^{\epsilon }\left(\tau \right)=0,-d\le \tau <0.\end{array}$ (3.3)

It is well known that such equations are well posed in $L_{\mathcal{F}}^2\left(\Omega \mathrm{<mo>\; ;\; </mo>}C\left(\left[\mathrm{0,}T\right]\mathrm{<mo>\; ;\; </mo>}ℝ\right)\right)$ (see e.g. [14] for a general theory on stochastic delay equations).

The following asymptotic behaviors hold, see ([9, Theorem 3.3]) for the delayed system and the classic result in ([15, Theorem 4.4]),

$\mathbb{E}\underset{t\in \left[\mathrm{0,}T\right]}{\sup }{\left|y^{\epsilon }\left(t\right)\right|}^2=O\left(\epsilon \right)\mathrm{,}\mathbb{E}\underset{t\in \left[\mathrm{0,}T\right]}{\sup }{\left|z^{\epsilon }\left(t\right)\right|}^2=O\left(\epsilon \right)\mathrm{,}$ (3.4)

$\mathbb{E}\underset{t\in \left[\mathrm{0,}T\right]}{\sup }{\left|x^{\epsilon }\left(t\right)-x\left(t\right)-y^{\epsilon }\left(t\right)\right|}^2=O\left({\epsilon }^2\right)\mathrm{,}$ (3.5)

$\mathbb{E}\underset{t\in \left[\mathrm{0,}T\right]}{\sup }{\left|x^{\epsilon }\left(t\right)-x\left(t\right)-y^{\epsilon }\left(t\right)-z^{\epsilon }\left(t\right)\right|}^2=o\left({\epsilon }^2\right)\mathrm{,}$ (3.6)

Moreover, $\forall p\ge 1$ ,

$\mathbb{E}\underset{t\in \left[-d,T\right]}{\sup }{\left|y^{\epsilon }\left(t\right)\right|}^p<+\infty ,\mathbb{E}\underset{t\in \left[-d,T\right]}{\sup }{\left|z^{\epsilon }\left(t\right)\right|}^p<+\infty ,$ (3.7)

4. Maximum Principle for the Stochastic Delay Equation

In this Section, first we formulate the first order adjoint equation in 4.1, then we pass to the second order adjoint in 4.2, and finally we formulate the stochastic maximum principle in 4.3.

4.1. First Order Duality Relation

Following [9] we define the first order adjoint equation of (3.2), that is the anticipated backward equation

$\{\begin{array}{l}p\left(t\right)=r_x\left(x\left(T\right)\right)+{\displaystyle {\int }_t^T}{l}_x\left(x\left(s\right)\right)\text{d}s-a_0{\displaystyle {\int }_t^T}p\left(s\right)\text{d}s-a_d{\displaystyle {\int }_t^T}{\mathbb{E}}^{F_s}p\left(s+d\right)\text{d}s\\ +{\sigma }_1{\displaystyle {\int }_t^T}{\mathbb{E}}^{F_s}{q}^1\left(s+d\right)\text{d}s+{\displaystyle {\int }_t^T}{q}^1\left(s\right)\text{d}{W}_s^1+{\displaystyle {\int }_t^T}{q}^2\left(s\right)\text{d}{W}_s^2\\ p\left(T-\tau \right)=0,\text{for}\text{all}\tau \in \left[-d,0\right),\\ q^1\left(T-\tau \right)=q^2\left(T-\tau \right)=0\text{a}\text{.e}\text{.}\tau \in \left[-d,0\right).\end{array}$ (4.1)

that admits a unique solution $\left(p\mathrm{,}q\right)\in L_{\mathcal{F}}^2\left(\Omega \mathrm{<mo>\; ;\; </mo>}C\left(\left[\mathrm{0,}T\right]\mathrm{<mo>\; ;\; </mo>}ℝ\right)\right)\times L_{\mathcal{F}}^2\left(\Omega \times \left[\mathrm{0,}T\right]\mathrm{<mo>\; ;\; </mo>}{ℝ}^2\right)$ , see e.g. [10] where we write $q=\left(q^1\mathrm{,}{q}^2\right)$ .

Since Equation (4.1) is linear, in view of the future times conditions we have an explicit, recursive, formulation for the solution $\left(p\mathrm{,}q\right)$ :

Proposition 4.1 Assume Hypotheses 2.1 and 2.2. Define for every

$k=0,\cdots ,\left[\frac{T}{d}\right]$ ,

$p^k\left(t\right):=p_{{|}_{\left[\left(T-\left(k+1\right)d\right)\vee 0,\left(T-kd\right)\right]}}(\; t\; )$

and

$q^{k,1}\left(t\right):=q_{{|}_{\left[\left(T-\left(k+1\right)d\right)\vee 0,\left(T-kd\right)\right]}}^1\left(t\right),q^{k,2}\left(t\right):=q_{{|}_{\left[\left(T-\left(k+1\right)d\right)\vee 0,\left(T-kd\right)\right]}}^2\left(t\right).$

Then $\left(p^k\mathrm{,}{q}^k\right)$ solve:

$\{\begin{array}{l}{p}^0\left(t\right)={\text{e}}^{-a_0\left(T-t\right)}{\mathbb{E}}^{{\mathcal{F}}_t}\left(r_x\left(x\left(T\right)\right)\right)+{\mathbb{E}}^{{\mathcal{F}}_t}{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{l}_x\left(x\left(s\right)\right)\text{d}s,t\in \left[T-d,T\right],\\ q^{0,1}\left(t\right)={\text{e}}^{-a_0\left(T-t\right)}{L}_1\left(t\right)-{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{K}_1\left(s,t\right)\text{d}s,\text{for}\text{a}\text{.e}\text{.}t\in \left[T-d,T\right],\\ q^{0,2}\left(t\right)={\text{e}}^{-a_0\left(T-t\right)}{L}_2\left(t\right)-{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{K}_2\left(s,t\right)\text{d}s,\text{for}\text{a}\text{.e}\text{.}t\in \left[T-d,T\right].\\ p^k\left(t\right)={\text{e}}^{-a_0\left(T-kd-t\right)}{p}^{k-1}\left(T-kd\right)-a_d{\displaystyle {\int }_t^{T-kd}}{\text{e}}^{-a_0\left(s-t\right)}{\mathbb{E}}^{F_s}{p}^{k-1}\left(s+d\right)\text{d}s\\ +{\sigma }_1{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{\mathbb{E}}^{F_s}{q}^{k-1,2}\left(s+d\right)\text{d}s+{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{q}^{k,1}\left(s\right)\text{d}{W}_s^1\\ +{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{q}^{k,2}\left(s\right)\text{d}{W}_s^2,t\in \left[\left(T-\left(k+1\right)d\right)\vee 0,\left(T-kd\right)\right],k:1,\cdots ,\left[\frac{T}{d}\right]\end{array}$ (4.2)

where $L_1$ and $L_2$ are given by:

${\mathbb{E}}^{F_t}\left(r_x\left(X\left(T\right)\right)\right)=\mathbb{E}\left(r_x\left(X\left(T\right)\right)\right)+{\displaystyle {\int }_t^T}{L}_1\left(s\right)\text{d}{W}^1\left(s\right)+{\displaystyle {\int }_t^T}{L}_2\left(s\right)\text{d}{W}^1\left(s\right).$ (4.3)

and $K_1$ and $K_2$ are given by:

${\mathbb{E}}^{F_t}\left(l_x\left(s\right)\right)=\mathbb{E}\left(l_x\left(s\right)\right)+{\displaystyle {\int }_{T-d}^t}{K}_1\left(s,\tau \right)\text{d}{W}^1\left(\tau \right)+{\displaystyle {\int }_{T-d}^t}{K}_2\left(s,\tau \right)\text{d}{W}^2\left(\tau \right).$ (4.4)

Proof. Notice that, to avoid trivialities, we are taking $d<T$ .

Let us consider the first order adjoint Equation (4.1), (, , for $t\in \left(T-d,T\right]$ : in view of the fact that $p\left(t\right)=q^i\left(t\right)=0$ , $i=1,2$ for a.e. $t\in \left[T,T+d\right]$ , for $t\in \left[T-d,T\right]$ it can be rewritten as a linear BSDE

$\{\begin{array}{l}p\left(t\right)=r_x\left(x\left(T\right)\right)+{\displaystyle {\int }_t^T}{l}_x\left(x\left(s\right)\right)\text{d}s-a_0{\displaystyle {\int }_t^T}p\left(s\right)\text{d}s+{\displaystyle {\int }_t^T}{q}^1\left(s\right)\text{d}{W}_s^1+{\displaystyle {\int }_t^T}{q}^2\left(s\right)\text{d}{W}_s^2\\ p\left(T-\tau \right)=0,\text{for}\text{all}\tau \in \left[-d,0\right),\\ q^1\left(T-\tau \right)=q^2\left(T-\tau \right)=0\text{a}\text{.e}.\tau \in \left[-d,0\right),\end{array}$

(4.5)

and its solution for $t\in \left[T-d,T\right]$ , is given by

$\begin{array}{c}{p}^0\left(t\right)={\text{e}}^{-a_0\left(T-t\right)}{r}_x\left(x\left(T\right)\right)+{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{l}_x\left(x\left(s\right)\right)\text{d}s\\ +{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{q}^{0,1}\left(s\right)\text{d}{W}_s^1+{\displaystyle {\int }_t^T}{\text{e}}^{-a_0\left(s-t\right)}{q}^{0,2}\left(s\right)\text{d}{W}_s^2\end{array}$ (4.6)

Formula (4.2) is just the rearrangement of the variation of constant formula, while (4.3) and (4.4) is an application of the Martingale Representation Theorem, see also [16] . □

Using (3.2), (3.2) and (4.1), we deduce the first duality relation:

Proposition 4.2 Assume 2.1 and 2.2 then:

$\begin{array}{c}\mathbb{E}p\left(T\right)y^{\epsilon }\left(T\right)=\mathbb{E}{r}_x\left(x\left(T\right)\right)y^{\epsilon }\left(T\right)\\ =-\mathbb{E}{\displaystyle {\int }_0^T}\left[l_x\left(x\left(s\right)\right)y^{\epsilon }\left(s\right)+q^2\left(s\right)\delta u\left(s-d\right)\right]\text{d}s\end{array}$ (4.7)

$\begin{array}{c}\mathbb{E}p\left(T\right)z^{\epsilon }\left(T\right)=\mathbb{E}{r}_x\left(x\left(T\right)\right)z^{\epsilon }\left(T\right)\\ =-\mathbb{E}{\displaystyle {\int }_0^T}{l}_x\left(x\left(s\right)\right)z^{\epsilon }\left(s\right)\text{d}s+\mathbb{E}{\displaystyle {\int }_0^T}p\left(s\right)\delta u\left(s\right)\text{d}s\end{array}$ (4.8)

Moreover the cost can be written as follows:

$\begin{array}{c}J\left(u\right)-J\left(u^{\epsilon }\right)=\mathbb{E}{\displaystyle {\int }_0^T}\left(-\delta l\left(t\right)+b_{0}p\left(t\right)\delta u\left(t\right)-{\sigma }_2{q}^2\left(t\right)\delta u\left(t-d\right)\right)\text{d}t\\ +\frac{1}{2}\mathbb{E}{r}_{xx}\left(x\left(T\right)\right){\left(y^{\epsilon }\left(T\right)\right)}^2+\frac{1}{2}\mathbb{E}{\displaystyle {\int }_0^T}{l}_{xx}\left(t,x\left(t\right)\right){\left(y^{\epsilon }\left(t\right)\right)}^2\text{d}t\end{array}$ (4.9)

Proof. See ( [9] , Propositions 4.1 and 4.2). □

4.2. Second Order Duality Relation

Following ( [9] , Theorem 4.9]) we introduce the process P, called $P_{\mathrm{0,0}}$ in [9] , that in this simplified case can be explicitely defined by means of a auxiliary process ${\tilde{y}}^{s,1}$ (here s indicates the initial time and 1 is the initial condition) that solves the following equation

$\{\begin{array}{l}\text{d}{\tilde{y}}^{s,1}\left(t\right)=\left(-a_0{\tilde{y}}^{s,1}\left(t\right)-a_d{\tilde{y}}^{s,1}\left(t-d\right)\right)\text{d}t+{\sigma }_1{\tilde{y}}^{s,1}\left(t\right)\text{d}W\left(t\right),t\in \left[s,T\right],\\ {\tilde{y}}^{s,1}\left(s\right)=1,y\left(\tau \right)=0,-d\le \tau <s.\end{array}$ (4.10)

By means of the solution process ${\tilde{y}}^{s,1}\in L_{\mathcal{F}}^2\left(\Omega \mathrm{<mo>\; ;\; </mo>}C\left(\left[\mathrm{0,}T\right]\mathrm{<mo>\; ;\; </mo>}ℝ\right)\right)$ , the process P can be now defined by

$P\left(s\right)={\mathbb{E}}^{{\mathcal{F}}_s}{r}_{xx}\left(x\left(T\right)\right){\tilde{y}}^{s\mathrm{,1}}{\left(T\right)}^2+{\mathbb{E}}^{{\mathcal{F}}_s}{\displaystyle {\int }_s^T}{l}_{xx}\left(t\right){\tilde{y}}^{s\mathrm{,1}}{\left(t\right)}^2\text{d}t\mathrm{.}$ (4.11)

Also in this case, the solution ${\tilde{y}}^{\mathrm{1,}s}$ of Equation (4.10) can be explicitly defined by recursion:

Proposition 4.3 Assume Hypotheses 2.1 and 2.2. Define

${\tilde{y}}^k\left(t\right)\mathrm{<mo>\; :\; </mo>}={\tilde{y}}_{{\mathrm{<mo>\; |\; </mo>}}_{\left[\left(s+kd\right),\left(s+\left(k+1\right)d\right)\wedge T\right]}}^{s\mathrm{,1}}\left(t\right)$ , for every $k\mathrm{<mo>\; :\; </mo>0,}\cdots \mathrm{,}\left[\frac{T}{d}\right]$

Then ${\tilde{y}}^k$ solves

$\{\begin{array}{l}{\tilde{y}}^0\left(t\right)={\text{e}}^{a_0-\frac{{\sigma }_1^2}{2}\left(t-s\right)+{\sigma }_1\left(W^1\left(t\right)-W^1\left(s\right)\right)},t\in \left[s,\left(s+d\right)\wedge T\right]\\ {\tilde{y}}^k\left(t\right)={\text{e}}^{-a_0\left(t-\left(s+kd\right)\right)}{\tilde{y}}^{k-1}\left(s+kd\right)-a_d{\displaystyle {\int }_{s+kd}^t}{\text{e}}^{-a_0\left(t-r\right)}{\tilde{y}}^{k-1}\left(r-d\right)\text{d}s\\ +{\sigma }_1{\displaystyle {\int }_{s+kd}^t}{\text{e}}^{a_0\left(t-r\right)}{\tilde{y}}^k\left(s\right)\text{d}{W}_s^1,t\in \left[\left(s+kd\right),\left(s+\left(k+1\right)d\right)\wedge T\right],k=1,\cdots ,\left[\frac{T}{d}\right]\end{array}$ (4.12)

Proof. Thanks to the initial condition in (25), we have that for every $t\in \left[s,s+d\wedge T\right]$ :

$\{\begin{array}{l}\text{d}{\tilde{y}}^{s,1}\left(t\right)=-a_0{\tilde{y}}^{s,1}\left(t\right)\text{d}t+{\sigma }_1{\tilde{y}}^{s,1}\left(t\right)\text{d}{W}_t^1\hfill \\ {\tilde{y}}^{s,1}\left(s\right)=1,\hfill \end{array}$ (4.13)

Therefore ${\tilde{y}}^0\left(t\right)={\tilde{y}}_{{|}_{\left[s,\left(s+d\right)\wedge T\right]}}^{s,1}\left(t\right)={\text{e}}^{a_0-\frac{{\sigma }_1^2}{2}\left(t-s\right)+{\sigma }_1\left(W^1\left(t\right)-W^1\left(s\right)\right)}$ . Then the general case for $t>s+d$ follows by the application of the constant variation formula on any interval $\left[\left(s+kd\right),\left(s+\left(k+1\right)d\right)\wedge T\right]$ . □

The expansion of the cost then becomes:

Proposition 4.4 Assume 2.1 and 2.2. Then following expansion holds true:

$\begin{array}{c}J\left(u\right)-J\left(u^{\epsilon }\right)=\mathbb{E}{\displaystyle {\int }_0^T}\left(-\delta c\left(s\right)+b_{0}p\left(s\right)\delta u\left(s\right)-{\sigma }_2{q}^2\left(s\right)\delta u\left(s-d\right)\right)\left(t\right)\text{d}s\\ +\frac{1}{2}\mathbb{E}{\displaystyle {\int }_0^T}{\sigma }_2^2{\left(\delta u\left(s-d\right)\right)}^{2}P\left(s\right)\text{d}s+o\left(\epsilon \right)\end{array}$ (4.14)

Proof. The proof is in [9] [Proposition 4.9, Theorem 4.10, Theorem 4.11] and is a consequence of an approximation procedure based on the regularization of the dirac measure ${\delta }_d$ and the well known reformulation of the delayed problem in infinite dimensions. □

4.3. The Stochastic Maximum Principle

We are now in the position to state our main result.

Theorem 4.5 Under assumptions 2.1 and 2.2, any optimal couple $\left(x\mathrm{,}u\right)$ satisfies the following variational inequality:

$\begin{array}{l}{b}_0\left(u\left(t\right)-v\right)p\left(t\right)-{\sigma }_2\left(u\left(t\right)-v\right){\mathbb{E}}^{F_t}{q}^2\left(t+d\right)-\left(c\left(u\left(t\right)\right)-c\left(v\right)\right)\\ +\frac{1}{2}{\sigma }_2{\left(u\left(t\right)-v\right)}^2{\mathbb{E}}^{F_t}P\left(t+d\right)\le 0,\forall v\in U,a.e.\mathbb{P}-a.s.\end{array}$ (4.15)

Where $\left(p\mathrm{,}q\right)\in L_{\mathcal{F}}^2\left(\Omega \mathrm{<mo>\; ;\; </mo>}C\left(\left[\mathrm{0,}T\right]\mathrm{<mo>\; ;\; </mo>}ℝ\right)\right)\times L_{\mathcal{F}}^2\left(\Omega \times \left[\mathrm{0,}T\right]\mathrm{<mo>\; ;\; </mo>}{ℝ}^2\right)$ , is the solution of the first order adjoint equation (16), and P is the second adjoint process given by (4.11).

Proof. Let $t\in \left[\mathrm{0,}T\right]$ , and $V_{\epsilon }=\left[t\mathrm{,}t+\epsilon \right]$ , then

$\begin{array}{c}J\left(u\right)-J\left(u^{\epsilon }\right)=\mathbb{E}{\displaystyle {\int }_0^T}\left(-\delta c\left(s\right)+b_{0}p\left(s\right)\delta u\left(s\right)-{\sigma }_2{q}^2\left(s\right)\delta u\left(s-d\right)\right)\left(t\right)\text{d}s\\ +\frac{1}{2}\mathbb{E}{\displaystyle {\int }_0^T}{\sigma }_2^2{\left(\delta u\left(s-d\right)\right)}^{2}P\left(s\right)\text{d}s\\ =\mathbb{E}{\displaystyle {\int }_0^T}(-\left[\left(c\left(u\left(s\right)\right)-v\right)+b_{0}p\left(s\right)\left(u\left(s\right)-v\right)\right]I_{V_{\epsilon }}\left(s\right)\\ -{\sigma }_2{q}^2\left(s\right)\left(u\left(s-d\right)-v\right)I_{V_{\epsilon }}\left(s-d\right))\text{d}s\\ +\frac{1}{2}\mathbb{E}{\displaystyle {\int }_0^T}{\sigma }_2^2{\left(u\left(s-d\right)-v\right)}^{2}P\left(s\right)I_{V_{\epsilon }}\left(s-d\right)\text{d}s\end{array}$

$\begin{array}{c}=-\mathbb{E}{\displaystyle {\int }_t^{t+\epsilon }}\left(c\left(u\left(s\right)-v\right)I_{V_{\epsilon }}\left(s\right)-b_{0}p\left(s\right)\left(u\left(s\right)-v\right)\right)\text{d}s\\ -\mathbb{E}{\displaystyle {\int }_t^{t+\epsilon }}{\mathbb{E}}^{F_t}{\sigma }_2{q}^2\left(s^{\prime }+d\right)\left(u\left(s^{\prime }\right)-v\right)\text{d}s\\ +\frac{1}{2}\mathbb{E}{\displaystyle {\int }_t^{t+\epsilon }}{\mathbb{E}}^{F_t}{\sigma }_2^2{\left(u\left(s^{\prime }\right)-v\right)}^{2}P\left(s^{\prime }+d\right)\text{d}{s}^{\prime }.\end{array}$ (4.16)

Letting $\epsilon$ tends to 0, by standard arguments we deduce (4.15).

Acknowledgements

The authors are grateful to the GNAMPA (Gruppo Nazionale per l’Analisi Matematica, la Probabilità e le loro Applicazioni) for the financial support.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Grosset, L. and Viscolani, B. (2004) Advertising for a New Product Introduction: A Stochastic Approach. Top, 12, 149-167. https://doi.org/10.1007/bf02578929
[2]	Gozzi, F. and Marinelli, C. (2006) Stochastic Optimal Control of Delay Equations Arising in Advertising Models. Stochastic Partial Differential Equations and Applications, VII, Chapman & Hall/CRC, Boca Raton, 133-148.
[3]	Gozzi, F., Marinelli, C. and Savin, S. (2009) On Controlled Linear Diffusions with Delay in a Model of Optimal Advertising under Uncertainty with Memory Effects. Journal of Optimization Theory and Applications, 142, 291-321. https://doi.org/10.1007/s10957-009-9524-5
[4]	Hartl, R.F. (1984) Optimal Dynamic Advertising Policies for Hereditary Processes. Journal of Optimization Theory and Applications, 43, 51-72. https://doi.org/10.1007/bf00934746
[5]	Chen, L. and Wu, Z. (2010) Maximum Principle for the Stochastic Optimal Control Problem with Delay and Application. Automatica, 46, 1074-1080. https://doi.org/10.1016/j.automatica.2010.03.005
[6]	Guatteri, G. and Masiero, F. (2021) Stochastic Maximum Principle for Problems with Delay with Dependence on the Past through General Measures. Mathematical Control & Related Fields, 11, 829-855. https://doi.org/10.3934/mcrf.2020048
[7]	Hu, Y. and Peng, S. (1996) Maximum Principle for Optimal Control of Stochastic System of Functional Type. Stochastic Analysis and Applications, 14, 283-301. https://doi.org/10.1080/07362999608809440.
[8]	de Feo, F. (2023) Stochastic Optimal Control Problems with Delays in the State and in the Control via Viscosity Solutions and an Economical Application. arXiv: 2308.14506.
[9]	Guatteri, G. and Masiero, F. (2023) Stochastic Maximum Principle for Equations with Delay: Going to Infinite Dimensions to Solve the Non-Convex Case. arXiv: 2306.07422.
[10]	Peng, S. and Yang, Z. (2009) Anticipated Backward Stochastic Differential Equations. The Annals of Probability, 37, 877-902. https://doi.org/10.1214/08-aop423
[11]	Meng, W., Shi, J., Wang, T. and Zhang, J.F. (2023) A General Maximum Principle for Optimal Control of Stochastic Differential Delay Systems. arXiv: 2302.03339.
[12]	Tapiero, C.S. (1978) Optimum Advertising and Goodwill under Uncertainty. Operations Research, 26, 450-463. https://doi.org/10.1287/opre.26.3.450
[13]	Peng, S. (1990) A General Stochastic Maximum Principle for Optimal Control Problems. SIAM Journal on Control and Optimization, 28, 966-979. https://doi.org/10.1137/0328054
[14]	Mohammed, S.-E.A. (1998) Stochastic Differential Systems with Memory: Theory, Examples and Applications. In: Decreusefond, L., Øksendal, B., Gjerde, J. and Üstünel, A.S., Eds., Stochastic Analysis and Related Topics VI, Birkhäuser Boston, 1-77. https://doi.org/10.1007/978-1-4612-2022-0_1
[15]	Yong, J. and Zhou, X.Y. (1999) Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer-Verlag, New York.
[16]	Hu, Y. and Peng, S. (1991) Adapted Solution of a Backward Semilinear Stochastic Evolution Equation. Stochastic Analysis and Applications, 9, 445-459. https://doi.org/10.1080/07362999108809250