A Heterogeneous Information Fusion Method for Maritime Radar and AIS Based on D-S Evidence Theory

Chao Wu^1,2,3,4, Qing Wu^1,2,3, Feng Ma^2,3*, Shuwu Wang^2,3,5

¹School of Transportation and Logistics Engineering, Wuhan University of Technology, Wuhan, China.
²State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan, China.
³Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan, China.
⁴School of Electronic and Information, Yangtze University, Jingzhou, China.
⁵School of Energy and Power Engineering, Wuhan University of Technology, Wuhan, China.

DOI: 10.4236/eng.2023.1512058                

Abstract

Maritime radar and automatic identification systems (AIS), which are essential auxiliary equipment for navigation safety in the shipping industry, have played significant roles in maritime safety supervision. However, in practical applications, the information obtained by a single device is limited, and it is necessary to integrate the information of maritime radar and AIS messages to achieve better recognition effects. In this study, the D-S evidence theory is used to fusion the two kinds of heterogeneous information: maritime radar images and AIS messages. Firstly, the radar image and AIS message are processed to get the targets of interest in the same coordinate system. Then, the coordinate position and heading of targets are chosen as the indicators for judging target similarity. Finally, a piece of D-S evidence theory based on the information fusion method is proposed to match the radar target and the AIS target of the same ship. Particularly, the effectiveness of the proposed method has been validated and evaluated through several experiments, which proves that such a method is practical in maritime safety supervision.

Keywords

D-S Evidence Theory, Heterogeneous Information Fusion, Radar Image, AIS Message

1. Introduction

Ship navigation safety has always been the most important problem in the shipping industry. In order to ensure the safety of navigation, the navigation environment information of ships needs to be mastered as comprehensively and in real-time as possible. Traffic information, natural environment information and channel information together constitute navigation environment information of ships, in which traffic information includes the position, heading and speed of ships and other factors. Compared with natural environment information and channel information, the randomness and rapidity of traffic information change are the strongest, which also greatly increases the difficulty of accurately obtaining real-time traffic information. On the other hand, the ever-changing traffic information is the primary concern for navigators and channel supervision departments.

Ship Automatic Identification System (AIS) is applied to maritime safety and communication between ship-shore and ship-ship. It is a digital navigation aid system and equipment integrating network technology, modern communication technology, computer technology and electronic information display technology. The use of the AIS system has been required by the International Maritime Organization (IMO) since December 31, 2004. The regulation requires AIS to be fitted aboard all ships of 300 gross tonnages and upwards engaged on international voyages, cargo ships of 500 gross tonnages and upwards not engaged on international voyages and all passenger ships irrespective of size [1] . AIS system is composed of communication controllers such as a VHF communicator, GPS locator, shipborne display and sensor, which can automatically exchange important information such as ship position, speed, heading, ship name and call sign [2] . When the AIS installed on the ship sends out the information, it also receives the information of other ships within the VHF coverage, thereby realizing automatic response.

In addition, as an open data transmission system, it can be connected with radar, ARPA, ECDIS, VTS and other terminal equipment and the Internet to form a maritime traffic management and surveillance network, which can effectively reduce ship collision accidents. However, from the effect of practical application, the performance of AIS is not unsatisfactory. First of all, the communication channels of aviation and navigation basically work in the VHF frequency band, but the reduction of communication quality due to the influence of environmental factors is the most fatal weakness of VHF. When ships sail in inland rivers, high-quality AIS communication is often not guaranteed due to the complex terrain along inland rivers and the influence of human activities. AIS message loss and errors in parsing often occur. Secondly, in theory, AIS needs to keep working from the beginning of launching to the end of scrapping. However, in practical applications, AIS does not turn on from time to time, and even some small boats sailing in inland rivers do not have AIS installed at all. These situations invisibly increase the difficulty of obtaining accurate traffic information. In addition, according to the design requirements, the update rate of AIS information will vary with the change of ship speed and heading. However, the statistical results obtained from practical use show that the update rate of AIS information carried by ships during inland navigation is between 10 - 15 seconds, or even longer. Based on the above situation, the actual use effect of AIS is often poor. If you want to simply use AIS to obtain real-time traffic information, it will increase the safety risk of ship navigation.

Navigation radar is a major milestone in the development of navigation technology. Navigation radar plays an irreplaceable role in ship navigation and channel management. In busy or complicated navigation channels, shore-based radars are usually erected along the coast according to the actual situation to ensure that channel supervisors can grasp the situation in the channel in real time. When the navigation radar is working, it receives the electromagnetic echo of the detected object and uses the plane position display of distance azimuth polar coordinates to display the distance and azimuth of the detected object. However, in actual use, the radar image may be distorted due to the influence of the performance of the radar itself and the reflection characteristics of the detected object. It is also possible that the radar image is not ideal or even impossible to image because of the interference of rain and snow clutter, sea wave clutter, co-frequency clutter, etc. Or it may be blocked by some obstacles to form fan-shaped shadows on the screen [3] . The above factors will greatly reduce the working effect of marine radar.

AIS data and radar data have essential differences in data structure, but the two different types of data can reflect the actual traffic information of ships in the channel. The fusion of AIS and radar data can greatly reduce the disadvantages of single AIS or radar data, and obtain more accurate traffic information in the channel, so as to ensure the navigation safety of ships.

In this study, a fusion method of inland waterway AIS data and marine radar data is proposed. The data used in this study are AIS data and radar image data collected in the same time period. This method is the application of two heterogeneous data fusion of AIS system and marine radar image. The remainder of this article is organized as follows. In section “Literature review”, the application of data fusion based on D-S evidence theory is introduced. Section “The proposed approach” presents a heterogeneous data fusion methodology. In the section “Case study”, a case study is conducted to demonstrate and validate the proposed methodology. The conclusions of this study are presented in section “Conclusion”.

2. Literature Review

The concept of data fusion originated in the 1970s, and multi-sensor data fusion technology emerged in the 1980s as a research hotspot that continues to this day. The Gulf War in 1991 served as a proving ground for numerous research findings and practical systems related to data fusion technology, which achieved exceptional results. Multi-sensor data fusion technology should not be considered as a general signal processing technique, as it involves the integration of information from multiple sensors and is distinct from single or multi-sensor monitoring and measurement. Multi-sensor data fusion is actually a process of making higher-level integrated decision based on the measurement results of multiple sensors. Multi-sensor data fusion can comprehensively analyze local data provided by multiple sensors of the same or different types, eliminate possible redundancy and contradiction among multiple sensors, reduce their uncertainty, and finally obtain consistent interpretation and description of the measured object, so that the system can obtain more adequate and accurate information [4] . Nowadays, by fusing data and related information from multiple sensors, it can realize more accurate judgment than a single sensor. It is also widely used in environmental monitoring [5] , robotics [6] and medical technology [7] in non-military fields.

Evidence theory [8] was first put forward by Dempster in 1967 and further developed by his student Shafer in 1976. Evidence theory is an inexact reasoning theory, which belongs to the category of artificial intelligence. As an uncertain reasoning method, it has the ability to directly express “uncertainty” and “don’t know”. The combination rules of evidence theory play an important role in target recognition, medical diagnosis and many other aspects that need to comprehensively consider the uncertain information from multiple sources and obtain problem results. In the fields of target recognition and medical diagnosis, the combination rules of evidence theory play a crucial role in comprehensively considering uncertain information from multiple sources and deriving accurate problem solutions.

In the aspect of transformer fault diagnosis, Yue et al. [9] used the improved D-S evidence theory to combine three models: multi-class parallel vector machine, multi-class support vector machine and reverse transfer neural network, and obtained more accurate fault diagnosis results by fusing the algorithm results. Similarly, in terms of fault diagnosis, Ji et al. [10] proposed an intelligent fault diagnosis method based on D-S evidence theory for a variety of faults of hydraulic valves. Firstly, the signal segment containing fault information is selected to construct the sample set. Then, the sample set is simultaneously input into a single classifier including long-term and short-term memory network (LSTM), convolutional neural network (CNN) and random forest (RF). These intelligent classification methods summarize the fault characteristics through spontaneous learning, and reveal the probability of each type of fault separately. All probabilities are constructed as basic probability distribution functions, and the basic probability distribution functions are further calculated in the process of information fusion according to D-S evidence theory. Finally, the fault type is identified according to the fusion result. In terms of deciphering the functions of non-coding genomic regions, Zhang et al. [11] used four sequence derived feature representation methods, namely kmer, reverse complement kmers, mismatch spectrum and pseudo dinucleotide combination to encode the sequence based on the D-S evidence theory method. Then four sub classifiers based on support vector machine are constructed by using these sequence features. Finally, D-S evidence theory is applied to fuse the outputs of these four basic learners to get the final result. In the aspect of emergency decision-making, Li et al. [12] put forward some new PLTS algorithms on the basis of D-S evidence theory in order to avoid terrible consequences caused by minor mistakes, and put forward a probabilistic linguistic weighted average operator (DS-PLWA) based on D-S evidence theory, which can keep the PLTS form and avoid information loss. Wang et al. [13] proposed a new weighted evidence combination method based on the distance between evidence and entropy function. Firstly, the weight of each piece of evidence is determined by calculating the distance between the evidence. Then, the entropy function is used to correct the weight obtained in the first step. The results of numerical examples show that this method can deal with highly contradictory evidence effectively and has good convergence performance. Kushwah et al. [14] proposed a multi-sensor fusion method based on temporal evidence theory for indoor activity recognition. This fusion method adopts the incremental conflict resolution strategy under the framework of D-S evidence theory. After introducing the time information, the proposed framework is applied to the activity detection in smart home.

3. The Proposed Approach

Based on D-S evidence theory, the real-time acquired AIS data and radar image data are fused to obtain more accurate channel traffic information. Firstly, the AIS message is analyzed and its coordinates are converted. Targets in the radar image are identified and the coordinates of the preliminarily determined targets are converted. Then, two different types of data are fused and preprocessed. The preprocessing process includes screening the AIS targets contained in the radar images and screening the targets contained in the radar images based on the AIS target time. After the fusion preprocessing is completed, the index selection of the fusion target will be carried out. Finally, the mass function and fusion algorithm are set and the results are obtained. The flow chart of this algorithm is shown in Figure 1.

3.1. Radar Image Processing

3.1.1. Radar Target Recognition Based on Image

At present, most maritime radar echo signals are generally displayed in the form of light spots, and the detected targets are usually represented by grayscale images or pseudo-color images. However, due to the influence of multipath effect of electromagnetic waves and the background noise, a large number of light spots will also be generated on the radar screen. These false targets are mixed with real ship targets, which will greatly increase the difficulty of accurately identifying traffic information in the channel.

In ship navigation and channel supervision, it is of great significance to accurately identify the traffic information in the channel. Although the traditional manual recognition of target attributes in radar images can ensure high accuracy, it costs a lot of time and energy of radar operators. Especially for the highly crowded inland waterways, a large number of radar spots of different sizes and

Figure 1. Block diagram of algorithm principle.

shapes will cover the whole radar screen. This situation will consume the energy of radar operators to a great extent, and manual recognition will reduce the recognition efficiency and accuracy.

With the help of intelligent algorithm, the work intensity of manual recognition can be reduced while the recognition efficiency is improved. Through the research on the radar echo spots, it can be found that there are many differences between the spot of the real target and the spot of the false target in the radar image. Experienced radar operators can effectively distinguish real targets from false targets in 10 consecutive frames of radar images, which is mainly based on the displacement and shape of radar spot. Generally, in the time continuous radar image, the motion state of the ship spot under the navigation state is relatively stable, and the displacement and movement direction of the spot can be found regularly. The noise spots in the radar image or the spots of other targets usually drift in a small range. Therefore, the displacement of radar spot in continuous frame time can be used as one of the characteristics to distinguish ship target and false target, that is, the spot displacement is equal to the pixel value of target motion in continuous time.

In addition, in terms of appearance, there are also obvious differences between the radar spot and the noise spot of the ship target, that is, the radar spot of the ship is slender than the noise spot. The difference in appearance between the two spots is shown in Figure 2. The left image is the noise spot, and the right image is the ship spot. This feature can be used to distinguish real ships from false targets. This feature can be expressed by slender length, that is, the ratio of the spot size to its circumscribed circle area.

3.1.2. Image Coordinate Conversion

Digital images are represented by arrays or matrices with pixels as the basic unit. Thus, the pixel coordinate system of the image can be constructed, in which the origin of the coordinate system is the vertex of the upper left corner of the image, and the abscissa and ordinate correspond to the number of columns and rows in the image array respectively. The image pixel coordinate system is shown in Figure 3.

When the detection range of the radar is set, the center of the obtained radar image is the location of the radar, and all targets in a plane circle with the range as the radius will be displayed in the image. However, radar image is a special form of real environment information. The difference between the size of the object displayed in the radar image and its actual size is determined by the radar detection range and the pixel size of the image. Therefore, in order to make the pixel scale have physical meaning, that is, to calculate the real length of each pixel

Figure 2. Comparison between noise spot and ship spot.

Figure 3. Schematic diagram of image pixel coordinate system.

based on the radar detection range, it is necessary to establish an image physical coordinate system based on the image pixel coordinate system [15] . As can be seen from the figure, the origin of the physical coordinate system is the center of the image plane, that is, the point O₁(U₁, V₁). The conversion relationship between the image pixel coordinate system and the image physical coordinate system is shown in Formula (1).

$\{\begin{array}{l}u=\frac{x}{dx}+u_1\\ v=\frac{y}{dy}+v_1\end{array}$ (1)

where, dx represents the size of each pixel on the horizontal axis and dy represents the size of each pixel on the vertical axis. Write Formula 1 in matrix form, as shown in Formula (2).

$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{dx}& 0& u_1\\ 0& \frac{1}{dy}& v_1\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]$ (2)

Generally, in the process of converting the image physical coordinate system to the world coordinate system, it also needs to undergo the conversion of the camera coordinate system. The principle of the conversion process is shown in Figure 4.

In the figure, x-y plane is the image plane. Point O is the optical center of the camera. X_c axis is parallel to the x axis of the image coordinate system. Y_c axis is

Figure 4. Schematic diagram of image physical coordinate system converted to world coordinate system.

parallel to the y axis of the image coordinate system. Z_c axis is the optical axis of the camera. Z_c axis is perpendicular to the image plane. f is the focal length, which is equal to the length of OO₁. The conversion relationship between the physical image coordinate system and the camera coordinate system is shown in Formula (3).

$\{\begin{array}{l}x=f\frac{X_c}{Z_c}\\ y=f\frac{Y_c}{Z_c}\end{array}$ (3)

Write Formula (3) in matrix form, as shown in Formula (4).

$Z_c\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]$ (4)

The world coordinate system is the O_w-X_wY_wZ_w coordinate system shown in the figure. The conversion relationship between camera coordinate system and world coordinate system is shown in Formula (5).

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]=\left[\begin{array}{cc}R& t\\ O^{\text{T}}& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ X\\ 1\end{array}\right]=L_w\left[\begin{array}{c}X\\ Y\\ X\\ 1\end{array}\right]$ (5)

where, R is the rotation matrix, which is the product of the three axial rotation matrices of x, y, z. T is the translation vector, which is the translation distance in the three axial directions. L_w is a 4 × 4 matrix composed of rotation and translation.

From the above derivation, the conversion relationship between the image pixel coordinate system and the world coordinate system can be obtained, as shown in Formula (6).

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{dx}& 0& u_1\\ 0& \frac{1}{dy}& v_1\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ O^{\text{T}}& 1\end{array}\right]\left[\begin{array}{c}X\\ Y\\ X\\ 1\end{array}\right]$ (6)

3.2. AIS Message Processing

3.2.1. AIS Message Parsing

AIS has played a great role in identifying ships, helping track targets, simplifying and promoting information exchange, providing auxiliary information for collision avoidance and reducing oral mandatory ship reports. In the specific application, AIS adopts the globally unique ship coding system, that is, MMSI code is used as the identification means. From the beginning of construction to the final scrapping and disintegration, each ship only uses a globally unique MMSI code.

The water mobile communication service identification code used in AIS message, i.e. MMSI code, is a nine-digit code sent by the ship’s wireless communication system on its wireless channel, which can uniquely identify all kinds of stations and group call stations. The actual AIS message format is shown in Table 1. Generally speaking, AIS message is usually composed of header, total number of statements required to transmit message, statement number, message identification code, channel, ship information, number of filling bits and inspection code.

In the AIS message format shown in the figure, the seventh item represents the encapsulated message. In its expression, the first character describes the type of the message. The format of message will also vary according to different types. In AIS information, the commonly used message types are shown in Table 2.

The ship trajectory in the vector space, which contains the MMSI number of the ship, the timestamp, geographical position in the form of longitude and latitude

Table 1. AIS message format.

Table 2. The common message types of AIS.

points of reference (LONG, LAT), speed (SOG), course (C), heading (H), etc [16] . When parsing the AIS message, the contents of the first six items and the eighth item can be directly extracted without any conversion. The content contained in the seventh item is the difficulty in the whole AIS message parsing process. The reason is that, on the one hand, different types of messages may contain the same information, and on the other hand, the positions of the same information in the messages may be different. It can be found from Table 1 that the message IDs 1, 2, 3, 18, and 19 contain the longitude and latitude information of the ship. However, the latitude and longitude information are located in the 62nd to 117th bits of the message in the message IDs 1, 2, and 3, and the latitude and longitude information is located in the 58th to 113th bits in the message IDs 18 and 19.

The parsing of AIS message is mainly the operation of Bit, that is, the information in the message needs to be shifted and spliced several times in the parsing process. In this study, when using the PC to parse the AIS message, the characters are first converted into 6Bit ASCII codes, then the converted 6Bit codes are spliced to form a new character string, and finally the corresponding information is extracted by installing Bits according to the format of the messages. In the process of extracting information, it is necessary to constantly shift and splice.

3.2.2. AIS Coordinate Conversion

After the completion of AIS data parsing, GPS data is one of the most important information. The process of converting the radar image coordinate system into a rectangular coordinate system has been discussed above. In order to perform data fusion, it is necessary to perform coordinate conversion on the longitude and latitude information of the ship, and the acquired GPS data of the ship is also converted into the same standard rectangular coordinate system data.

Gauss-Kruger projection or equiangular transverse elliptic cylinder projection, referred to as Gauss projection [17] , is a kind of orthographic projection between ellipsoid and plane of the earth. The basic idea of the projection method is that an elliptic cylinder is horizontally sleeved outside the ellipsoid of the earth and tangent to a meridian (central meridian or axis meridian). The central axis of the elliptic cylinder passes through the center of the ellipsoid, and the areas within a certain range of longitude difference on both sides of the central meridian are projected onto the elliptical cylinder, and then the cylinder is expanded into a projection surface. Its schematic diagram is shown in Figure 5.

Gauss projection can be divided into 6-degree band and 3-degree band. Starting from the meridian of 0-degree band, the Gauss projection c divides the band from west to east every 6 degrees of longitude difference and it is numbered as 1, 2, 3...

Assuming that the band number is expressed by n and the longitude of the central meridian is expressed by L₀, the relationship between n and L₀ can be expressed by Formula (7).

Figure 5. Schematic diagram of Gauss projection.

$L_0=6n-3$ (7)

The 3-degree band of Gauss projection is formed on the basis of the 6-degree band. Part of the central meridian in the projection of the 3-degree band coincides with the central meridian of the 6-degree band, and another part coincides with the dividing meridian of the 6-degree band. Assuming that the band number is expressed by n’ and the longitude of the central meridian is expressed by L, the relationship between n’ and L can be expressed by Formula (8).

$L=3n^{\prime }$ (8)

Generally, topographic maps with a scale greater than 1:10,000 adopt 3-degree band projection, and topographic maps with a scale of 1:25,000 to 1:500,000 adopt 6-degree band projection.

The Gauss projection is calculated as shown in Formula (9).

$\{\begin{array}{l}x=X+\frac{N}{2{{\rho }^{″}}^2}\sin B\cos B{l^{″}}^2+\frac{N}{24{{\rho }^{″}}^4}\sin B{\cos }^{3}B\left(5-t^2+9{\mu }^2\right){l^{″}}^4\\ y=\frac{N}{{\rho }^{″}}\cos B{l}^{″}+\frac{N}{6{{\rho }^{″}}^3}{\cos }^{3}B\left(1-t^2+{\mu }^2\right){l^{″}}^3+\frac{N}{120{\rho }^5}{\cos }^{5}B\left(5-18t^2+t^4\right){l^{″}}^5\end{array}$ (9)

After changing Formula (9), Formula (10) can be obtained, i.e.

$\{\begin{array}{l}x=X+\frac{N}{2}t{\cos }^{2}B{l}^2+\frac{N}{24}t\left(5-t^2+9{\mu }^2+4{\mu }^4\right){\cos }^{4}B{l}^4+\frac{N}{720}t\left(61-58t^2+t^4\right){\cos }^{6}B{l}^6\\ y=N\cos Bl+\frac{N}{6}\left(1-t^2+{\mu }^2\right){\cos }^{3}B{l}^3+\frac{N}{120}\left(5-18t^2+t^4+14{\mu }^2-58{\mu }^2{t}^2\right){\cos }^{5}B{l}^5\end{array}$ (10)

$\pi =3.1415926535897932$

${\rho }^{\circ }=180÷\pi$

${\rho }^{\prime }=180\times 60÷\pi$

${\rho }^{″}=180\times 60\times 60÷\pi$

$\mu =e^{\prime }\cos B$ , $e^{\prime }$ is the second eccentricity of the ellipse, $e^{\prime }=\frac{\sqrt{a^2-b^2}}{b}$ , $t=\tan B$ ,

$l=\frac{{\left(L-L_0\right)}^{\prime \text{}\prime }}{{\rho }^{″}}$ , L₀ is the central meridian longitude.

$X={\displaystyle {\int }_0^{B}M\text{d}B}$ , X is the arc length of the meridian measured from the equator.

$M=a\left(1-e^2\right){\left(1-e^2{\sin }^{2}B\right)}^{-\frac{3}{2}}$ , M is the radius of meridian curvature

$N=a{\left(1-e^2{\sin }^{2}B\right)}^{-\frac{1}{2}}$ , N is the curvature radius of the unitary ring.

$e=\frac{\sqrt{a^2-b^2}}{a}$ e is the first eccentricity of the ellipse.

3.3. Radar Target and AIS Target Fusion Preprocessing

Under normal circumstances, the shore-based surveillance radar set near the important flight segment can monitor all the conditions in the channel in real time and record it during operation. Radar images of all passing ships within the supervised segment can be fully recorded. At the same time, the AIS data of the passing ships during this time period can also be completely collected. However, radar images and AIS data are two completely different types of information. In order to verify the fusion effect of these two types of information, it is necessary to perform necessary preprocessing on the premise of unifying the two information coordinate system standards.

First, the starting and ending intervals of the recorded radar images should be selected, that is, the radar images obtained during a certain working period of the shore-based surveillance radar should be selected. When the working time period of the radar is determined, the AIS data of all passing ships in this time period are searched from the collected AIS data.

However, the working frequency of two different types of information is a problem that cannot be ignored. Generally, the working frequency of shore-based surveillance radar is 24 frames/min, that is, one frame of radar image is acquired in 2.5 s. The emission frequency of ship-borne AIS terminal is closely related to the ship’s speed. Class A does not exceed 2 s once, and Class B does not exceed 5 s once. The specific definitions are shown in Table 3.

It can be seen that the radar images are sampled at equal time intervals, and the acquisition time of AIS data is generally not synchronized with the acquisition time of radar images. The AIS target data will appear between two consecutive radar images with a high probability. In view of this situation, it is necessary to judge whether the AIS data at a certain time match the previous radar image or the next radar image. This situation is shown in Figure 6. As can be seen from the figure, in order to obtain a better matching effect, the interval between the time point of the occurrence of the AIS data and the time point of the occurrence of the radar image should be minimized.

3.4. Index Selection of Fusion Target

In terms of multivariate evidence fusion, it is the key to extract elements that

Table 3. AIS emission frequency.

Figure 6. Schematic diagram of radar image target matching with AIS target.

express the same meaning contained in the respective information for two completely different types of information: radar images and AIS data. The extraction of elements with the same meaning in different types of information can be done from the following aspects.

On the premise that the radar image coordinate system and AIS data coordinate system are unified, the radar image target and AIS target after fusion preprocessing have their own coordinate distribution in the same coordinate system. If the coordinate position of a target in the radar image in the coordinate system is the same or similar to that of a target in AIS data, it can be considered as the same target. The coordinate position is represented by the horizontal and vertical parameters, so the vertical and horizontal coordinate parameters of the target in the radar image and the vertical and horizontal coordinate parameters (x, y) of the target in the AIS data can be used as two judgment bases.

In addition, in a unified coordinate system, the heading angle of the target in the radar image and the heading angle of the target in the AIS data can be calculated. The heading angle θ is also an important basis for judging whether the two targets are the same ship.

Due to the good periodicity of radar images, the moving speed of a target can be calculated according to the displacement and time interval of a target in continuous images. Although the actually acquired AIS data has irregular acquisition time, the moving speed of a certain target can still be calculated. Therefore, the moving speed of the target in the two types of information is also an important element in the fusion process.

3.5. Fusion Method Based on DS Evidence Theory

D-S evidence theory can treat and analyze propositions by transforming them into mathematical sets. Different from the probability theory, which only considers a single element, the set of evidence theory generally contains multiple elements. It is precisely because of its fuzzy characteristics that it can better express the uncertainty of proposition.

Based on the D-S evidence theory, the two types of information of radar image target and AIS target are fused. From the content of the previous section, the recognition framework can be divided into three cases: matching, mismatching, and indeterminate. Radar image and AIS data can provide the target’s coordinate parameters, heading angle, speed and other fusion indexes, that is (x, y, θ, v), and each indicator has its Basic Probability Assignment (BPA). The mass function can be obtained by combining the probability judgments provided by radar images and AIS data. The situation is shown in Table 4.

AIS target ( $x_i,y_i,{\theta }_i,v_i$ )

ARPA target ( $x_j,y_j,{\theta }_j,j$ )

$\Theta =\left\{\text{match},\text{notmatch},\text{uncertain}\right\}$

$m_x=f_x\left(x_i-x_j\right)$ , $m_y=f_y\left(y_i-y_j\right)$ , $m_{\theta }=f_{\theta }\left({\theta }_i-{\theta }_j\right)$ , $m_v=f_v\left(v_i-v_j\right)$

Table 4. Three recognition situations.

After obtaining the above evidence, Dempster synthesis rule is used for evidence synthesis. The process is as follows.

Recognition framework $\Theta =\left\{\varnothing ,a,b,X\right\}$ , $X=\left\{a,b\right\}$

$\{\begin{array}{l}m(\varnothing )=0\\ \left(m_1\oplus m_2\right)\left(A\right)=\frac{1}{1-K}{\displaystyle {\sum }_{B\cap C=A\ne \varnothing }{m}_1\left(B\right)m_2\left(C\right)}\\ K={\displaystyle {\sum }_{B\cap C=\varnothing }{m}_1\left(B\right)m_2\left(C\right)}\end{array}$ (11)

$K=a_{11}{a}_{22}+a_{21}{a}_{12}$ (12)

$m_{12}\left(A\right)=\frac{1}{1-K}\left(a_{11}{a}_{12}+a_{11}{a}_{32}+a_{31}{a}_{12}\right)$ (13)

$m_{12}\left(B\right)=\frac{1}{1-K}\left(a_{21}{a}_{22}+a_{21}{a}_{32}+a_{31}{a}_{22}\right)$ (14)

$m_{12}\left(A,B\right)=\frac{1}{1-K}{a}_{31}{a}_{32}$ (15)

After calculation, the final result of D-S fusion is shown in Table 5.

4. Case Study

To validate the proposed approach, a field test was conducted at the Zhujiajian Passenger Transport Center Terminal, Zhoushan City, Zhejiang Province, China.

4.1. Experimental Platform and Experimental Process

The experimental water area and experimental platform are shown in Figure 7. The experimental data collection point is located in Zhoushan City, Zhejiang Province. The radar is broadband 4G^TM and X band, and the radardetection range is 4 nm. The distance from the AIS shore-based base station to the shore-based radar is 0.4 Km. The data collection time is 60 min.

4.2. Experimental Process and Results

Step 1. Unified coordinate system of radar image target and AIS data.

In this study, although GPS information of suspected ship targets and some ships in navigation can be obtained by processing radar images and AIS data separately in the early stage, the coordinate system of targets in radar images is pixel coordinate system, while GPS in AIS represents geographic coordinate system. The targets in these two different coordinate systems are not comparable at all. In order to fuse two different types of information, it must be unified into the

Table 5. The final result of D-S fusion.

Figure 7. Satellite image of water area for experimental data collection.

Figure 8. The radar image target and AIS data after unifying the coordinate system.

same coordinate system. According to the method mentioned above, the target in the radar image and the GPS coordinates in the AIS data are transformed into coordinate systems respectively. Only the two different types of information displayed in the unified coordinate system can meet the preliminary conditions for fusion. The radar image target and AIS data after unifying the coordinate system are shown in Figure 8. Blue represents AIS data and black represents radar image target.

Step 2. The fusion of radar image target and AIS target.

After preprocessing, the radar image target and AIS target can complete the preliminary matching. In the unified coordinate system, the coordinate position relationship of the two types of targets, the heading angle and speed of the ships can be obtained. These parameters can be regarded as the basic probability distribution. After obtaining the mass function, the final fusion result can be calculated by using Dempster synthesis rule to synthesize evidence. Figure 9(a)-(j) show a group of partial results after fusion. It can be seen from the figure that SN

Figure 9. Schematic diagram of fusion results.

is the radar image target, represented by a black circle, MMSI is the AIS target, represented by a red diamond, the red circle represents the fusion result, and P represents the calculated probability after fusion.

The fusion algorithm proposed in this study can effectively fuse radar images and AIS data, and the effect is good.

5. Conclusions

Maritime radar and AIS, as important aids to navigation safety in the shipping industry, have played a great role in maritime safety supervision. However, in practical application, the radar output image may be distorted, unable to image or form fan-shaped shadow due to some special reasons, which will seriously affect the working effect of maritime radar. Besides, in the process of AIS information processing, the content presented in each received message is a reflection of the state of the ship at a certain moment. These data may reflect the real state of the ship, or abnormal changes may occur due to the problems of positioning equipment or software. In addition, many factors such as topography, meteorology and external electromagnetic environment will attenuate AIS information, thus reducing its reliability. In order to further ensure the navigation safety of ships, it can be considered to fuse the relevant indexes in the two types of information, namely, maritime radar output images and AIS data, so as to increase the effective identification of targets in the navigation channel.

In this study, the D-S evidence theory is used to extract relevant indexes for fusion based on the two types of heterogeneous information: maritime radar output images and AIS data. Firstly, the two kinds of heterogeneous information are unified into the same coordinate system, and the new recognition results are obtained by fusing the target’s coordinate position, heading angle, speed and other indexes. Through the test of radar images and AIS data collected in the field, this method is proved to be effective and the results are ideal.

Acknowledgements

This work is financially supported by the Funds for the National Key R&D Program of China (Grant No. 2021YFB1600400), National Natural Science Foundation of China under Grant No. 52171352 and 52201415, Key R&D Program of Zhejiang Province Grant 2021C03015.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Zhang, M., Conti, F., Le Sourne, H., Vassalos, D., Kujala, P. and Lindroth, D., et al. (2021) A Method for the Direct Assessment of Ship Collision Damage and Flooding Risk in Real Conditions. Ocean Engineering, 237, Article No. 109605. https://doi.org/10.1016/j.oceaneng.2021.109605
[2]	Zhu, S., Levinson, D.M., Liu, H., et al. (2010) The Traffic and Behavioral Effects of the I-35W Mississippi River Bridge Collapse. Transportation Research Part A: Policy and Practice, 44, 771-784.
[3]	Ma, F., Wu, Q., Yan, X., et al. (2015) Classification of Automatic Radar Plotting Aid targets Based on Improved Fuzzy C-Means. Transportation Research Part C: Emerging Technologies, 51, 180-195. https://doi.org/10.1016/j.trc.2014.12.001
[4]	Wan, C., Zhao, Y., Zhang, D. and Fan, L. (2023). A System Dynamics-Based Approach for Risk Analysis of Waterway Transportation in a Mixed Traffic Environment. Maritime Policy & Management, 1-23. https://doi.org/10.1080/03088839.2023.2224328
[5]	Wu, H. (2021) Efficient Environmental Monitoring System Based on Data Fusion and Predictive Prediction. In: Xu, Z., Parizi, R.M., Loyola-González, O., Zhang, X., Eds. Cyber Security Intelligence and Analytics. CSIA 2021. Springer, Cham. https://doi.org/10.1007/978-3-030-70042-3_36
[6]	Che, H. (2021) Multi-Sensor Data Fusion Method Based on ARIMA-LightGBM for AGV Positioning. 2021 5th International Conference on Robotics and Automation Sciences (ICRAS), Wuhan, 11-13 June 2021, 272-276. https://doi.org/10.1109/ICRAS52289.2021.9476452
[7]	Zhou, X. and Wu, Y. (2017) Research on Application of Data Fusion Methods in Medical Case-based Retrieval. Electronic Science and Technology, 30, 45-48.
[8]	Fan, X. and Zuo, M.J. (2006) Fault Diagnosis of Machines Based on D-S Evidence Theory. Part 1: D-S Evidence Theory and Its Improvement. Pattern Recognition Letters, 27, 366-376. https://doi.org/10.1016/j.patrec.2005.08.025
[9]	Zheng, Q., Minzhong, Y. and Zhang, Y. (2000) Synthetic Diagnostic Method for Insulation Fault of Oil-Immersed Power Transformer. Proceedings of the 6th International Conference on Properties and Applications of Dielectric Materials, Xi’an, 21-26 June 2000, 872-875.
[10]	Ji, X., Ren, Y., Tang, H., et al. (2020) An Intelligent Fault Diagnosis Approach Based on Dempster-Shafer Theory for Hydraulic Valves. Measurement, 165, Article No. 108129. https://doi.org/10.1016/j.measurement.2020.108129
[11]	Zhang, S., Lin, J., Su, L. and Zhou, Z. (2019) pDHS-DSET: Prediction of DNase I Hypersensitive Sites in Plant Genome Using DS Evidence Theory. Analytical Biochemistry, 564-565, 54-63. https://doi.org/10.1016/j.ab.2018.10.018
[12]	Li, P. and Wei, C. (2019) An Emergency Decision-Making Method Based on D-S Evidence Theory for Probabilistic Linguistic Term Sets. International Journal of Disaster Risk Reduction, 37, Article No. 101178. https://doi.org/10.1016/j.ijdrr.2019.101178
[13]	Wang, J., Xiao, F., et al. (2016) Weighted Evidence Combination Based on Distance of Evidence and Entropy Function. International Journal of Distributed Sensor Networks, 12. https://doi.org/10.1177/155014773218784
[14]	Kushwah, A., Kumar, S., and Hegde, R.M. (2014) Multi-Sensor Data Fusion Methods for Indoor Activity Recognition Using Temporal Evidence Theory. Pervasive and Mobile Computing, 21, 19-29. https://doi.org/10.1016/j.pmcj.2014.10.009
[15]	Zhou, X., Wang, Y. and Lu, X. (2021) Approach for ISAR Imaging of Near-Field Targets Based on Coordinate Conversion and Image Interpolation. Journal of Systems Engineering and Electronics, 32, 425-436. https://doi.org/10.23919/JSEE.2021.000036
[16]	Zhang, M., Kujala, P., Musharraf, M., et al. (2023) A Machine Learning Method for the Prediction of Ship Motion Trajectories in Real Operational Conditions. Ocean Engineering, 283, Article No. 114905. https://doi.org/10.1016/j.oceaneng.2023.114905
[17]	Wang, H. and Gao, Z. (2019) Back-Calculating Pattern Area Error on Ellipsoidal Surface Using Gaussian Projection Map. Beijing Surveying and Mapping, 33, 713-718.