A New Unified Path to Smoothing Nonsmooth Exact Penalty Function for the Constrained Optimization

Bingzhuang Liu

School of Mathematics and Statistics, Shandong University of Technology, Zibo, China.

DOI: 10.4236/ojop.2021.103005                

Abstract

We propose a new unified path to approximately smoothing the nonsmooth exact penalty function in this paper. Based on the new smooth penalty function, we give a penalty algorithm to solve the constrained optimization problem, and discuss the convergence of the algorithm under mild conditions.

Keywords

Penalty Function, Constrained Optimization, Smoothing Method, Optimal Solution

1. Introduction

We consider the following constrained optimization problem

(P) $\begin{array}{ll}\min \hfill & f\left(x\right)\hfill \\ \text{s}\text{.t}\text{.}\hfill & g_j\left(x\right)\le \mathrm{0,}j=\mathrm{1,2,}\cdots \mathrm{,}m\mathrm{,}\hfill \end{array}$ (1)

where $f,g_j:{\Re }^n\to \Re ,j=1,2,\cdots ,m$ are continuously differentiable functions. This model has important applications in many fields such as industry, engineering, and computational science. There are many optimization methods to solve this kind of problem, and the penalty function method is one of the most important methods. In the penalty function method, the original constraint conditions are reflected to the new objective function by constructing penalty function, and then the original constrained optimization problem is transformed into a series of unconstrained optimization problems.

In the many penalty functions that have been proposed, the exact penalty function is often discussed, such as the $l_1$ penalty function and the $l_p$ penalty function.

The classical $l_1$ penalty function (Zangwill [1] ) is given as

$L_1\left(x,\beta \right)=f\left(x\right)+\beta {\displaystyle \underset{j=1}{\overset{m}{\sum }}}\max \left\{g_j\left(x\right),0\right\},$ (2)

where $\beta >0$ is a penalty parameter. The $l_p$ penalty function is given as

$L_p\left(x,\beta \right)=f\left(x\right)+\beta {\left[{\displaystyle \underset{j=1}{\overset{m}{\sum }}}\max \left\{g_j\left(x\right),0\right\}\right]}^p,$ (3)

where $\beta >0$ is a penalty parameter and $0<p<1$. But these exact penalty functions are often nonsmooth, which hampers the use of fast convergent algorithms such as the conjugate gradient method, the Newton method, and the quasi-Newton method. Many scholars have proposed smooth approximations to the classical exact penalty functions, which can be found in the references ( [2] - [14] ), and different penalty algorithms have been given to solve different optimization problems. In [10] [11] and [14], smooth approximations to $l_1$ penalty function were proposed for nonlinear inequality constrained optimization problems. Different smoothing penalty functions were also proposed in [13] to solve the global optimization problems. To solve the problem (P), [7] proposed two smooth approximations to the exact penalty function

$L_{\frac{1}{2}}\left(x,\beta \right)=f\left(x\right)+\beta {\displaystyle \underset{j=1}{\overset{m}{\sum }}}\sqrt{g_j^{+}\left(x\right)}.$

In [6] and [12], some smoothing techniques for the above exact penalty function were also given.

Smoothed penalty methods can also be applied to solve the optimization problems with large scale such as the network-structured problems and the minimax problems in [3], and the traffic flow network models in [8].

[5] gave a family of smoothing penalty functions to the $l_1$ penalty function and established a simple penalty algorithm.

In this paper, a new unified smooth approximation path to the $l_p$ penalty function is proposed for the problem (P). On the basis of the proposed smoothing penalty functions, a new approximate algorithm is established, and the convergence of the algorithm is discussed under appropriate conditions.

Remark 1 we assume in this paper that

$\underset{x\in {\Re }^n}{\inf }{f}_0\left(x\right)>0.$ (4)

The above assumption is common since if it is not satisfied, then we can take the place of $f_0\left(x\right)$ by ${\text{e}}^{f_0\left(x\right)}+1$.

2. Approximately Smoothing Exact Penalty Functions

For the $l_p$ penalty function (3), we give a new family of smooth approximation in this section as follows,

$L_p\left(x,\beta ,r\right)=f\left(x\right)+{\left[r{\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(\frac{{\beta }^{\frac{1}{p}}{g}_j\left(x\right)}{r}\right)\right]}^p,$ (5)

where $r>0$ is a parameter and the function $\psi \mathrm{<mo>\; :\; </mo>}\Re \to {\Re }_{+}$ is a continuously differentiable function, and for any $t\in \Re$, $\psi \left(t\right)\ge 0$.

Here we assume the function $\psi$ satisfies the following properties:

(a1) $\psi (\cdot )$ is monotonically increasing, and ${\psi }^{\prime }\left(0\right)>0$ ;

(a2) $\underset{t\to +\infty }{\lim }\frac{\psi \left(t\right)}{t}=1$.

It is easy to show that the following functions are all examples of the function $\psi \left(t\right)$.

${\psi }_1\left(t\right)=\{\begin{array}{ll}2{\text{e}}^t,\hfill & \text{if}t<0;\hfill \\ t+\log \left(1+t\right)+2,\hfill & \text{if}t\ge 0;\hfill \end{array}$

${\psi }_2\left(t\right)=\{\begin{array}{ll}0,\hfill & \text{if}t<0;\hfill \\ \frac{2}{3}{t}^2,\hfill & \text{if}0\le t\le 1;\hfill \\ t-\frac{1}{3}{\text{e}}^{1-t},\hfill & \text{if}t>1;\hfill \end{array}$

${\psi }_3\left(t\right)=\log \left(1+{\text{e}}^t\right)\mathrm{<mo>\; ;\; </mo>}$

${\psi }_4\left(t\right)=\frac{\sqrt{t^2+4}+t}{2};$

${\psi }_5\left(t\right)=\{\begin{array}{ll}\frac{1}{2}{\text{e}}^t,\hfill & \text{if}t\le 0;\hfill \\ \frac{1}{2}{\text{e}}^{-t}+t,\hfill & \text{if}t>0;\hfill \end{array}$

${\psi }_6\left(t\right)=\{\begin{array}{ll}{\text{e}}^t+1,\hfill & \text{if}t\le 0;\hfill \\ t+2,\hfill & \text{if}t>0;\hfill \end{array}$

${\psi }_7\left(t\right)=\{\begin{array}{ll}0,\hfill & \text{if}t<-1;\hfill \\ \frac{{\left(t+1\right)}^2}{4},\hfill & \text{if}-1\le t\le 1;\hfill \\ t,\hfill & \text{if}t>1.\hfill \end{array}$

From (a1) and (a2), it is easy to know that

$\underset{r\to 0^{+}}{\lim }r\psi \left(\frac{t}{r}\right)=t^{+},$

where $t^{+}=\max \left\{0,\mathrm{<mtext>\; \hspace{0.17em}\; </mtext>}t\right\}$.

It follows that

$L_p\left(x\mathrm{,}\beta \mathrm{,}r\right)=f\left(x\right)+{\left[r{\displaystyle \underset{j\mathrm{<mo>\; =\; </mo>1}}{\overset{m}{\sum }}}\psi \left(\frac{{\beta }^{\frac{1}{p}}{g}_j\left(x\right)}{r}\right)\right]}^p\to f\left(x\right)+\beta {\left[{\displaystyle \underset{j\mathrm{<mo>\; =\; </mo>1}}{\overset{m}{\sum }}}{g}_j^{+}\left(x\right)\right]}^p\mathrm{,}\left(r\to 0^{+}\right)\mathrm{.}$

We move on to the properties of the function $\psi (\cdot )$ and look at the following proposition.

Proposition 2.1 If $\psi (\cdot )$ satisfies the properties (a1) and (a2), then for any $u\in {\Re }^m$, $\sigma \left(u\right)={\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(u_j\right)$ satisfies the following properties.

(b1) For any real number $\epsilon >0$, there exists a positive real number ${\eta }_{\epsilon }>0$ such that

$\underset{c_k\to +\infty }{\lim \inf }\underset{\Vert u^{+}\Vert \ge \epsilon }{\inf }\frac{\sigma \left(c_{k}u\right)}{c_k}\ge {\eta }_{\epsilon },$

where $u_j^{+}=\max \left\{0,\mathrm{<mtext>\; \hspace{0.17em}\; </mtext>}{u}_j\right\}$ and $u^{+}={\left(u_1^{+}\mathrm{,}{u}_2^{+}\mathrm{,}\cdots \mathrm{,}{u}_m^{+}\right)}^{\text{T}}$.

(b2) For $c_k\to +\infty \left(k\to \infty \right)$, there exist ${\epsilon }_k\to 0^{+}\left(k\to \infty \right)$ such that

$\underset{k\to \infty }{\lim \sup }\underset{\Vert u^{+}\Vert \le {\epsilon }_k}{\sup }\frac{\sigma \left(c_{k}u\right)}{c_k}=0.$

(b3) There exists a constant ${\sigma }_0$ such that

$\sigma \left(u\right)\ge {\sigma }_0\mathrm{,\; <mtext>\; \hspace{0.17em}\; </mtext>\; <mtext>\; \hspace{0.17em}\; </mtext>}\text{forany}\mathrm{<mtext>\; \hspace{0.17em}\; </mtext>}u\in {\Re }^m\mathrm{.}$

(b4) There exists a constant ${\sigma }_1$ such that

$\sigma \left(u\right)\le {\sigma }_1\mathrm{,\; <mtext>\; \hspace{0.17em}\; </mtext>\; <mtext>\; \hspace{0.17em}\; </mtext>}\text{forany}\mathrm{<mtext>\; \hspace{0.17em}\; </mtext>}u\le 0.$

Proof. We first show that $\sigma \left(u\right)={\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(u_j\right)$ satisfies the property (b1).

For $\Vert u^{+}\Vert \ge \epsilon$, there exists a $j_0$ such that $u_j\ge \frac{\epsilon }{\sqrt{m}}$. Otherwise, if $\forall j$, $u_j<\frac{\epsilon }{\sqrt{m}}$, then $\Vert u^{+}\Vert =\sqrt{{\displaystyle {\sum }_{j=1}^m}{\left(u_j^{+}\right)}^2}<\epsilon$. Since $\psi (\cdot )$ is a monotonically increasing positive function, we have that

$\begin{array}{c}\underset{\Vert u^{+}\Vert \ge \epsilon }{\inf }\frac{\sigma \left(c_{k}u\right)}{c_k}=\underset{\Vert u^{+}\Vert \ge \epsilon }{\inf }\frac{1}{c_k}{\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(c_k{u}_j\right)\\ \ge \underset{u_{j_0}\ge \frac{\epsilon }{\sqrt{m}}}{\inf }\frac{1}{c_k}\psi \left(c_k{u}_{j_0}\right)\\ =\frac{1}{c_k}\psi \left(c_k\frac{\epsilon }{\sqrt{m}}\right),\end{array}$

where the inequality is got by the positiveness of $\psi (\cdot )$, and the last equality is got by that $\psi (\cdot )$ is increasing.

Again by the property (a2) of $\psi (\cdot )$, we obtain that

$\begin{array}{c}\underset{c_k\to +\infty }{\lim \inf }\underset{\Vert u^{+}\Vert \ge \epsilon }{\inf }\frac{\sigma \left(c_{k}u\right)}{c_k}\ge \underset{c_k\to +\infty }{\lim \inf }\frac{1}{c_k}\psi \left(c_k\frac{\epsilon }{\sqrt{m}}\right)\\ =\underset{c_k\to +\infty }{\lim \inf }\frac{\sqrt{m}}{c_k\epsilon }\psi \left(c_k\frac{\epsilon }{\sqrt{m}}\right)\frac{\epsilon }{\sqrt{m}}\\ =\frac{\epsilon }{\sqrt{m}}.\end{array}$

Let ${\eta }_{\epsilon }=\frac{\epsilon }{\sqrt{m}}$, then we prove that $\sigma \left(u\right)$ satisfies the property (b1).

We now show that $\sigma \left(u\right)={\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(u_j\right)$ satisfies the property (b2).

For $c_k\to +\infty \left(k\to \infty \right)$, set ${\epsilon }_k=\frac{1}{c_k}$, and ${\sigma }_1=m\psi \left(1\right)$. Since $\psi (\cdot )$ is increasing, we have

$\begin{array}{c}\underset{\Vert u^{+}\Vert \le {\epsilon }_k}{\sup }\sigma \left(c_{k}u\right)=\underset{\Vert u^{+}\Vert \le {\epsilon }_k}{\sup }{\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(c_k{u}_j\right)\\ \le \underset{u_j\le {\epsilon }_k,j=1,\cdots ,m}{\sup }{\displaystyle \underset{j=1}{\overset{m}{\sum }}}\psi \left(c_k{u}_j\right)\\ =m\psi \left(c_k{\epsilon }_k\right)\\ =m\psi \left(1\right).\end{array}$

Since $\psi (\cdot )\ge 0$, $\sigma \left(u\right)$ satisfies the property (b2).

Since $\psi (\cdot )\ge 0$ and $\psi (\cdot )$ is increasing, we can easily get the properties (b3) and (b4). $\square$

3. Smooth Penalty Algorithm and Its Convergence

We propose an algorithm based on the penalty function $L_p\left(x\mathrm{,}\beta \mathrm{,}r\right)$ and discuss its global convergence.

Algorithm 3.1 Step 0. Let ${\beta }_0=1$, $r_0=1$, ${\omega }_0=1$, and set $k:=0$.

Step 1. Find an

$x^k\in \arg \underset{x\in {\Re }^n}{\min }{L}_p\left(x\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)\mathrm{,}$ (6)

or $x^k$ satisfies the following inequality

$L_p\left(x^k\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)\le \underset{x\in {\Re }^n}{\inf }{L}_p\left(x\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)+{\omega }_k\mathrm{.}$ (7)

Step 2. Let

$r_{k+1}=\{\begin{array}{ll}\frac{1}{2}{r}_k\mathrm{,}\hfill & \text{if}0\le \Vert g^{+}\left(x^k\right)\Vert \le r_k\mathrm{<mo>\; ;\; </mo>}\hfill \\ r_k\mathrm{,}\hfill & \text{otherwise}\mathrm{.}\hfill \end{array}$

${\beta }_{k+1}=\{\begin{array}{ll}{\beta }_k\mathrm{,}\hfill & \text{if}\Vert g^{+}\left(x^k\right)\Vert =\mathrm{0\; <mo>\; ;\; </mo>}\hfill \\ 2{\beta }_k\mathrm{,}\hfill & \text{otherwise}\mathrm{.}\hfill \end{array}$

Step 3. Set ${\omega }_{k+1}=\frac{1}{2}{\omega }_k$, $k:=k+1$, and return to Step 1.

Now we study the global convergence of the algorithm. For an $\epsilon \ge 0$, we define the relaxed feasible set of the problem (P) by

${\Omega }_{\epsilon }=\left\{x\in {\Re }^n\mathrm{<mo>\; |\; </mo>}{g}_j\left(x\right)\le \epsilon \mathrm{,1,}\cdots \mathrm{,}m\right\}\mathrm{.}$

Thus ${\Omega }_0$ can denote the feasible set of (P). In this paper we always suppose that ${\Omega }_0\ne \varnothing$. We denote the optimal solution set of (P) by ${\Omega }_0^{\mathrm{<mo>\; *\; </mo>}}$.

The perturbation function of (P) is defined as

${\theta }_f\left(\epsilon \right)=\underset{x\in {\Omega }_{\epsilon }}{\inf }f\left(x\right)\mathrm{.}$

Then the optimal value of (P) is

${\theta }_f\left(0\right)=\underset{x\in {\Omega }_0}{\inf }f\left(x\right).$

It can be easily showed that ${\theta }_f\left(\epsilon \right)$ is upper semi-continuous at $\epsilon =0$. Thus the continuity of ${\theta }_f\left(\epsilon \right)$ at $\epsilon =0$ is equivalent to the lower semi-continuity of ${\theta }_f\left(\epsilon \right)$ at $\epsilon =0$. Set

$F_{\epsilon }=\left\{x\in {\Re }^n|f\left(x\right)\le {\theta }_f\left(0\right)+\epsilon \right\}$

and

$S_k\left(\epsilon \right)=\left\{x\in {\Re }^n\mathrm{<mo>\; |\; </mo>}L\left(x\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)\le \underset{z\in {\Re }^n}{\inf }L\left(z\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)+\epsilon \right\}\mathrm{.}$

Now we give the following lemma.

Lemma 3.1 The sequence $\left\{r_k\right\}$ generated by Algorithm 3.1 converges to 0.

Proof. Assume to the contrary that $\left\{r_k\right\}$ does not converge to 0, then by that $\left\{r_k\right\}$ decreases monotonically, there exists a $k_0$ such that $\forall k\ge k_0$, $r_k=r_{k_0}$. It follows from Step 2 of Algorithm 3.1 that $\forall k\ge k_0$, $r_k=r_{k_0}$, $\Vert g^{+}\left(x^k\right)\Vert >r_k=r_{k_0}$, and $\underset{k\to \infty }{\lim }{\beta }_k=+\infty$.

Let $\bar{x}\in {\Omega }_0$, then $g\left(\bar{x}\right)\le 0$. By Proposition 2.1, we know that $\forall k\ge k_0$,

$\begin{array}{c}{L}_p\left(x^k,{\beta }_k,r_{k_0}\right)\le \underset{x\in {\Re }^n}{\inf }{L}_p\left(x,{\beta }_k,r_k\right)+{\omega }_k\\ \le L_p\left(\bar{x},{\beta }_k,r_{k_0}\right)+{\omega }_k\\ =f\left(\bar{x}\right)+{\left[r_{k_0}\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(\bar{x}\right)}{r_{k_0}}\right)\right]}^p+{\omega }_k\\ \le f\left(\bar{x}\right)+{\left[r_{k_0}{\sigma }_1\right]}^p+{\omega }_k.\end{array}$ (8)

where ${\sigma }_1$ is given by Proposition 2.1.

For sufficiently large $k\ge k_0$, by Proposition 2.1, we have that

$\begin{array}{c}{L}_p\left(x^k,{\beta }_k,r_{k_0}\right)=f\left(x^k\right)+{\left[r_{k_0}\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(x^k\right)}{r_{k_0}}\right)\right]}^p\\ \ge {\left[r_{k_0}\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(x^k\right)}{r_{k_0}}\right)\right]}^p\\ \ge {\beta }_k\underset{\Vert g^{+}\left(x\right)\Vert >r_{k_0}}{\inf }{\left[\frac{r_{k_0}}{{\beta }_k^{\frac{1}{p}}}\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(x\right)}{r_{k_0}}\right)\right]}^p\\ \ge \frac{1}{2}{\beta }_k{\eta }_{r_{k_0}}^p.\end{array}$

The last inequality can be got by the property (b1) of Proposition 2.1, where $c_k=\frac{{\beta }_k^{\frac{1}{p}}}{r_{k_0}}$. By that $\underset{k\to \infty }{\lim }{\beta }_k=+\infty$, the right side of the above last inequality goes to $\infty$, which contradicts with (8). So $\left\{r_k\right\}$ generated by Algorithm 2.1 converges to 0. $\square$

Lemma 3.2 $\forall \epsilon >0$, for all sufficiently large k, it holds that $S_k\left(\epsilon \right)\subseteq {\Omega }_{\epsilon }$.

Proof. Assume to the contrary that there exists an ${\epsilon }_0>0$ and a subsequence $K\subset N$ such that $\forall k\in K$, $\exists z^k\in S_k\left({\epsilon }_0\right)$, but $z^k\notin {\Omega }_{{\epsilon }_0}$. Then there exists a subsequence $K_0\subseteq K$ and an index $j_0\in \left\{\mathrm{1,2,}\cdots \mathrm{,}m\right\}$ such that $\forall k\in K_0$,

$g_{j_0}\left(z^k\right)>{\epsilon }_0.$ (9)

Then by Lemma 3.1, we have for sufficiently large $k\in K_0$ that $\Vert g^{+}\left(z^k\right)\Vert >{\epsilon }_0\ge r_k$. Then it follows from Step 2 of Algorithm 3.1 that $\underset{k\to \infty }{\lim }{\beta }_k=+\infty$.

By (9) and the property (b1) of Proposition 2.1, for sufficiently large k, we have that

$\begin{array}{c}\underset{x\in {\Re }^n}{\inf }{L}_p\left(x,{\beta }_k,r_k\right)+{\epsilon }_0\ge L_p\left(z^k,{\beta }_k,r_k\right)\\ =f\left(z^k\right)+{\left[r_k\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(z^k\right)}{r_k}\right)\right]}^p\\ \ge {\beta }_k\underset{\Vert g^{+}\left(x\right)\Vert \ge {\epsilon }_0}{\inf }{\left[\frac{r_{k_0}}{{\beta }_k^{\frac{1}{p}}}\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(x\right)}{r_{k_0}}\right)\right]}^p\\ \ge \frac{1}{2}{\beta }_k{\eta }_{{\epsilon }_0}^p.\end{array}$ (10)

Then by $\underset{k\to \infty }{\lim }{\beta }_k=+\infty$, the right side of the last inequality of (10) goes to $\infty$.

Let $\bar{x}\in {\Omega }_0$, then $g\left(\bar{x}\right)\le 0$. By Proposition 2.1, we know that $\forall k\ge k_0$,

$\begin{array}{c}\underset{x\in {\Re }^n}{\inf }{L}_p\left(x\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)+{\epsilon }_0\le L_p\left(\bar{x}\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)+{\epsilon }_0\\ =f\left(\bar{x}\right)+{\left[r_k\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(\bar{x}\right)}{r_k}\right)\right]}^p+{\epsilon }_0\\ \le f\left(\bar{x}\right)+{\left[r_k{\sigma }_1\right]}^p+{\epsilon }_0\mathrm{,}\end{array}$ (11)

which contradicts with (10). $\square$

Theorem 3.1 (Perturbation Theorem) Assume that $\left\{x^k\right\}$ is a sequence generated by Algorithm 3.1, then it holds that

1) $\underset{k\to \infty }{\lim }f\left(x^k\right)=\underset{\epsilon \to 0^{+}}{\lim }{\theta }_f\left(\epsilon \right)\mathrm{<mo>\; ;\; </mo>}$

2) $\underset{k\to \infty }{\lim }{L}_p\left(x^k\mathrm{,}{\beta }_k\mathrm{,}{r}_k\right)=\underset{\epsilon \to 0^{+}}{\lim }{\theta }_f\left(\epsilon \right)\mathrm{<mo>\; ;\; </mo>}$

3) $\underset{k\to \infty }{\lim }{\left[r_k\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(x^k\right)}{r_k}\right)\right]}^p=0.$

Proof. Since the perturbation function ${\theta }_f\left(\epsilon \right)$ is monotonically decreasing on $\epsilon >0$, and $\forall k\in K$, ${\theta }_f\left(\epsilon \right)\le {\theta }_f\left(0\right)$, it holds that $\underset{\epsilon \to 0^{+}}{\lim }{\theta }_f\left(\epsilon \right)$ exists and is finite. By Proposition 2.1, for $\frac{1}{r_k}\to +\infty \left(k\to \infty \right)$, $\exists {\epsilon }_k\to 0^{+}$, such that

$\underset{k\to \infty }{\lim \sup }\underset{\Vert u^{+}\Vert \le {\epsilon }_k}{\sup }{r}_k\sigma \left(\frac{u}{r_k}\right)=0.$ (12)

Choose an ${{\epsilon }^{\prime }}_k>0$ such that ${\beta }_k^{\frac{1}{p}}{{\epsilon }^{\prime }}_k\le {\epsilon }_k$, and set ${\bar{\epsilon }}_k=\frac{{{\epsilon }^{\prime }}_k}{\sqrt{m}}$, then we have

$\underset{k\to \infty }{\lim }{\theta }_f\left({\bar{\epsilon }}_k\right)=\underset{\epsilon \to 0^{+}}{\lim }{\theta }_f\left(\epsilon \right)\mathrm{.}$ (13)

Then we again choose ${\delta }_k>0$ and ${\delta }_k\to 0\left(k\to \infty \right)$. By the definition of infimum, for each k, there exists a $z^k\in {\Omega }_{{\bar{\epsilon }}_k}$ such that

$f\left(z^k\right)\le {\theta }_f\left({\bar{\epsilon }}_k\right)+{\delta }_k\mathrm{.}$

Since $z^k\in {\Omega }_{{\bar{\epsilon }}_k}$, we have that $g_j\left(z^k\right)\le {\bar{\epsilon }}_k=\frac{{{\epsilon }^{\prime }}_k}{\sqrt{m}},i=1,\cdots ,m$, then we can obtain that

${\beta }_k\Vert g^{+}\left(z^k\right)\Vert \le {\beta }_k{{\epsilon }^{\prime }}_k\le {\epsilon }_k\mathrm{.}$ (14)

On the other side, for any $\epsilon >0$, by the proof of Lemma 3.2, we have for all sufficiently large k that

$x^k\in {\Omega }_{\epsilon }\mathrm{.}$ (15)

Thus, for any $\epsilon >0$, by the property (b3) and Proposition 2.1, we have that

$\begin{array}{c}{\theta }_f\left(\epsilon \right)\le f\left(x^k\right)\\ \le f\left(x^k\right)+{\left[r_k\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(x^k\right)}{r_k}\right)\right]}^p-{\left[r_k{\sigma }_0\right]}^p\\ =L_p\left(x^k,{\beta }_k,r_k\right)-{\left[r_k{\sigma }_0\right]}^p\\ \le \underset{x\in {\Re }^n}{\inf }{L}_p\left(x,{\beta }_k,r_k\right)+{\omega }_k-{\left[r_k{\sigma }_0\right]}^p\\ \le f\left(z^k\right)+{\left[r_k\sigma \left(\frac{{\beta }_k^{\frac{1}{p}}g\left(z^k\right)}{r_k}\right)\right]}^p+{\omega }_k-{\left[r_k{\sigma }_0\right]}^p\\ \le {\theta }_f\left({\bar{\epsilon }}_k\right)+{\delta }_k+{\left[r_k\underset{\Vert u^{+}\Vert \le {\epsilon }_k}{\sup }\sigma \left(\frac{u}{r_k}\right)\right]}^p+{\omega }_k-{\left[r_k{\sigma }_0\right]}^p.\end{array}$

Let $k\to \infty$, and put two sides of the above inequality to the limit, we obtain that 1)-3) hold. $\square$

Theorem 3.2 Assume that $\left\{x^k\right\}$ is a sequence generated by Algorithm 3.1, then every accumulation point of $\left\{x^k\right\}$ is an optimal solution of the problem (P)

Proof. By Lemma 3.2, for sufficiently large k, we have

$x^k\in {\Omega }_{\epsilon }\mathrm{.}$ (16)

Suppose that $x^{\mathrm{<mo>\; *\; </mo>}}$ is an accumulation point of $\left\{x^k\right\}$, by the continuity of $g_j\left(i=1,\cdots ,m\right)$ and (16), we know that $x^{\mathrm{<mo>\; *\; </mo>}}\in {\Omega }_{\epsilon }$. Again by the arbitrariness of $\epsilon$, we have that $x^{\mathrm{<mo>\; *\; </mo>}}\in {\Omega }_0$.

By the Perturbation Theorem, we obtain that $f\left(x^{*}\right)=\underset{k\to \infty }{\lim }f\left(x^k\right)=\underset{\epsilon \to 0^{+}}{\lim }{\theta }_f\left(\epsilon \right)\le {\theta }_f\left(0\right)$. $\square$

4. Conclusions

In this paper, we propose a uniform path of smooth approximation for the classical nonsmooth penalty function. Our model contains some of the existing models. In addition, we also give a class of relaxed smooth penalty algorithm, and prove the convergence of the algorithm under some weak conditions.

In the future work, we will use the model and algorithm of this paper to carry out numerical experiments and compare with some existing methods. We also consider applying the model and algorithm in this paper to the study of power market equilibrium optimization problem.

Fund

This research is supported by National Natural Science Foundation of China (11771255, 11801325), Young Innovation Teams of Shandong Province (2019KJI013) and the Natural Science Foundations of Shandong Province (ZR2015AL011).

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

References

[1]	Zangwill, W.I. (1967) Nonlinear Programming via Penalty Function. Management Science, 13, 334-358. https://doi.org/10.1287/mnsc.13.5.344
[2]	Bental, A. and Teboulle, M. (1989) A Smoothing Technique for Non-Differentiable Optimization Problems. Lecture Notes in Mathematics, Springer Verlag, Berlin, Vol. 1405, 1-11. https://doi.org/10.1007/BFb0083582
[3]	Pinar, M.C. and Zenios, S.A. (1994) On Smoothing Exact Penalty Functions for Convex Constarained Optimization. SIAM Journal on Optimization, 4, 486-511. https://doi.org/10.1137/0804027
[4]	Auslender, A., Cominetti, R. and Haddou, M. (1997) Asymptotic Analysis for Penalty and Barrier Methods in Convex and Linear Programming. Mathematics of Operational Research, 22, 43-62. https://doi.org/10.1287/moor.22.1.43
[5]	Gonzaga, C.C. and Castillo, R.A. (2003) A Nonlinear Programming Algorithm Based on Non-Coercive Penalty Functions. Mathematical Programming, 96, 87-101. https://doi.org/10.1007/s10107-002-0332-z
[6]	Wu, Z.Y., Bai, F.S., Yang, X.Q. and Zhang, L.S. (2004) An Exact Lower Order Penalty Function and Its Smoothing in Nonlinear Programming. Optimization, 53, 51-68. https://doi.org/10.1080/02331930410001662199
[7]	Meng, Z.Q., Dang, C.Y. and Yang, X.Q. (2006) On the Smoothing of the Square-Root Exact Penalty Function for Inequality Constrained Optimization. Computational Optimization and Applications, 35, 375-398. https://doi.org/10.1007/s10589-006-8720-6
[8]	Herty, M., Klar, A., Singh, A.K. and Spellucci, P. (2007) Smoothed Penalty Algorithms for Optimization of Nonlinear Models. Computational Optimization and Applications, 37, 157-176. https://doi.org/10.1007/s10589-007-9011-6
[9]	Di Pillo, G., Lucidi, S. and Rinaldi, F. (2012) An Approach to Constrained Global Optimization Based on Exact Penalty Functions. Journal of Global Optimization, 54, 251-260. https://doi.org/10.1007/s10898-010-9582-0
[10]	Lian, S.J. (2012) Smoothing Approximation to l1 Exact Penalty Function for Inequality Constrained Optimization. Applied Mathematics and Computation, 219, 3113-3121. https://doi.org/10.1016/j.amc.2012.09.042
[11]	Xu, X.S., Meng, Z.Q., Sun, J.W., Huang, L.G. and Shen, R. (2013) A Second-Order Smooth Penalty Function Algorithm for Constarained Optimization Problems. Com-putational Optimization and Application, 55, 155-172. https://doi.org/10.1007/s10589-012-9504-9
[12]	Lian, S.J. and Duan, Y.Q. (2016) Smoothing of the Lower-Order Exact Penalty Function for Inequality Constrained Optimization. Journal of Inequalities and Applications, 185, 1-12. https://doi.org/10.1186/s13660-016-1126-9
[13]	Wu, Z.Y., Lee, H.W.J., Bai, F.S. and Zhang, L.S. (2017) Quadratic Smoothing Approximation to l1 Exact Penalty Function in Global Optimization. Journal of Industrial and Management Optimization, 1, 533-547. https://doi.org/10.3934/jimo.2005.1.533
[14]	Liu, B.Z. (2019) A Smoothing Penalty Function Method for the Constrained Optimization Problem. Open Journal of Optimization, 8, 113-126. https://doi.org/10.4236/ojop.2019.84010