A New Image Stabilization Model for Vehicle Navigation


When a video camera is mounted on a vehicle’s frame, it experiences the same ride as a passenger and is subject to vertical displacement as the vehicle hits bumps on the road. This results in a captured video that may be difficult to watch because the bumps are transferred to the recorded video. This paper presents a new image stabilization model for vehicle navigation that can remove the effect of vertical vehicular motion due to road bumps. It uses a wheel sensor that monitors the wheel’s reaction with respect to road disturbances prior to the vehicle’s suspension system. This model employs an inexpensive sensor and control circuitry. The vehicle’s suspension system, bumpy road, and the compensation control system are modeled analytically. Experimental results show that the proposed model works suc-cessfully. It can eliminate 10 cm of drift and results in only 1 cm disturbance at the onset and the end of bumps.

Share and Cite:

F. Shih and A. Stone, "A New Image Stabilization Model for Vehicle Navigation," Positioning, Vol. 1 No. 1, 2010, pp. 8-17. doi: 10.4236/pos.2010.11002.

1. Introduction

Vehicular imaging systems are common in police vehicles, trucking and transportation systems, rail cars, and buses. Such systems are typically mounted on the vehicle’s frame or a component of the vehicle attached to the frame (e.g. a dashboard), inheriting the ride from the vehicle’s suspension system which protects the frame and passengers. The inspection of videos captured from vehicles is typically nauseating because of the constant jittery motion that is transferred from road perturbations, dampened by the vehicles suspension, and then transferred to the camera. To remedy this problem, three primary methods are usually employed: feature registration, electro-mechanical stabilization, and optical stabilization. Each approach has own advantages and disadvantages that are briefly introduced below.

A common method of image stabilization is feature registration. Brooks [1], Liang et al. [2], and Broggi et al. [3] used feature extraction to lock on a portion of an image in a frame and then employed correction transforms to attempt alignment of the locked target in subsequent frames. This approach is quite computationally intensive, so it limits its real-time applicability. A typical approach identifies a natural horizon or in some cases the lines on a road as a reference to adjust subsequent frames. Since each frame needs to be thoroughly analyzed to identify the reference object and in some cases to determine what to do when there are gaps or missing references, much processing is required for each frame.

A second approach is electro-mechanical stabilization, which employs inertial systems or gyroscopes to detect variations in movement and make corrections to the lens group or imaging plane [4]. Recent advances based on Micro-Electro-Mechanical Systems (MEMS) technology allow gyroscopes to be integrated into Application-Specific Integrated Circuits (ASICs) [5]. The gyroscope measures displacements that are sent to an electronic image stabilization system to perform corrections. The inertial changes can be measured accurately and filtered out, and the appropriate compensation can be made to adjust the image frame. As the price for integrated gyros continues to fall, this approach becomes more favorable nowadays.

Finally, the optical stabilization approach manages a group of lenses in the imaging equipment to compensate for vibration or slow moving disturbances [6-20]. It does not act as quickly as the frame registration or electro-mechanical solutions due to its mechanical compensation.

This paper describes a new image stabilization model for vehicle navigation that can remove the effect of vertical vehicular motion due to road bumps. It uses a wheel sensor that monitors the wheel’s reaction with respect to road disturbances prior to the vehicle’s suspension system. This model employs an inexpensive sensor and control circuitry. The paper is organized as follows. We present the analytical model of the baseline system in Section 2. The analytical model of the electronically stabilized system and experimental results are described in Section 3. Conclusions are drawn in Section 4.

2. Analytical Model of the Baseline System

In this section, we describe a vehicle with an ordinary camera mounted to the vehicle’s frame, which demonstrates a baseline of a typical image viewing experience and the problems occurred. Figure 1 shows a model of a baseline vehicle without electronic camera stabilization. The sensor is in a front wheel assembly, and the camera may be mounted at the head of the vehicle. The schema shows a logical view of the road, wheel, suspension system, and camera assembly. The wheel is modeled as a spring (), and the suspension system is modeled as a spring () and damper () [11]. The imaging subsystem has a rigid mount to the vehicle’s frame, and thus experiences the same response to road conditions as the vehicle frame does. Collectively, this model of springs and dampers allows one to create a control system model that can describe the behavior of the vehicle’s response to varying road conditions. The force of the ground is labeled as fg(t). The displacements of the road, wheel, and vehicle frame are denoted as Yr, Yw, and Yv, respectively. Each of these displacements is assumed to be 0- valued at the initial condition of t0.

In order to determine the vehicle’s response, the parameters of the model are required. The vehicle data used for the springs and damper in this paper are adapted from [11] as an example of real parameters. These model parameters are itemized in Table 1, where and denote the spring rates of the suspension and tire respectively, anddenote the sprung and unsprung masses of the vehicle and wheel respectively, anddenotes the damper rate of the suspension spring. The suspension parameters are well known by vehicle designers. By using the logical model and representative data, we can create an analytical model.

2.1. Analytical Model

Let, and denote the vertical displacements of the road, wheel, and vehicle from their initial reference positions, respectively. When the vehicle is at rest, these values are assumed to be 0. We can derive the motion equations using Newton’s second law as:



Note that the single and double dot notations denote first and second derivatives with respect to time,

Table 1. Model parameters.

Figure 1. Model of a baseline vehicle without electronic camera stabilization.

respectively. The first derivative of any displacement is its velocity, while the second derivative is its acceleration. These equations can be mapped to the socalled state-space form for this particular problem. The following state-space representation is modified from [12] and is implemented in MATLAB™. Let the variable u denote the input vector, and let y denote the output vector. We have

, (3)




2.2 Behavior of the Uncompensated System

The vehicle imaging system will share the same ride with the driver. It is subject to the vehicle’s response of the suspension system that can be modeled as a low-pass filter of the bumps in the road as smoothed out by the suspension system. This is referred to as an open-loop system because there is no feedback to the imaging system. For simplicity, we model a bump in the road as a rectangular pulse, which is a 10 cm high disturbance that the vehicle rolls over for approximately 1 second. The vehicle response to this bump and the effect on the image recording are analyzed.

We multiply the B matrix by 0.1 since B is the coefficient for the unit step function u in Equation (3). The step response can be obtained using the MATLAB™ step( ) function. To obtain the response to the whole bump (entering and leaving), the MATLAB™ lsim( ) function from the Control System Toolbox is used. In this case, the resulting response of a 1 second wide bump starting at 0.2 seconds is shown in Figure 2.

The graph in Figure 2 shows the bump (dotted line), the response of the wheel (), and the response of the vehicular frame () which is the representative of the camera’s response to the bump over time. The y-axis is the displacement in meters. It can be seen that the wheel response is quick and abrupt. The vehicular frame benefits from the additional spring and damper system of the car’s shock absorbers. It shows a smoother, more delayed response and produces the overshoot and undershoot of the frame. It actually ascends higher than the bump and

Figure 2. Response of a 1 second wide bump starting at 0.2 seconds.

then dips back down to a negative displacement for a short while. Note that different parameters for the springs and the damper (shock absorber) will result in different responses of the car.

2.3. Image Viewing Experience of the Open-Loop System

We apply the aforementioned vehicle frame response to the imaging plane using the displacements of the curve as offsets from a representative image. Figure 3 shows the reference scene for the effect of vehicle bounce. It is observed that the white line appears on the road and it is smooth and linear. This is visibly distorted by the bump.

Next, the response of a 1 second wide bump starting at 0.2 seconds, , shown in Figure 2, is mapped to the image of the viewer’s experience. This is done by sweeping through the image left-to-right and a vertical row at a time. Each vertical row represents a time interval. Each vertical row of pixels is viewed as a point in time or a frame of a video sequence. By taking the signal and scale it to displace the row of image pixels over time, the effect of a video is created.

From the vehicle’s perspective, a comfortable ride for the user and a safe ride for the vehicle are paramount. From the camera’s perspective, we wish to completely flatten out this response, so the view is level. From Figure 4, the representative image viewed on the display of an unstabilized imaging system results in a view that has both large disturbances and a lasting effect because of the damping response of the suspension. It is the effect to be eliminated with electronic image stabilization.

Figure 3. The reference scene for the effect of vehicle bounce.

3. Analytical Model of the Electronically Stabilized System

In this section, we present the analytical model of the electronically stabilized system. Some suspension systems are characterized as an active suspension, in which the damper Cs is an active element whose stiffness is adjusted by an active control system [10]. The closedloop nature of the system is resulted from feedback and control in the electrical form, feeding the imaging subsystem. Thus, the closed-loop control is for the image display platform, not for the vehicle’s ride. This closedloop attribute allows the imaging subsystem to monitor the response of the wheel on the road and make adjustments that coincide with the vehicle’s frame which holds the imaging subsystem [13-16].

Figure 1 is modified by a sensor device attached to the vehicle’s wheel assembly as shown in Figure 4. The sensor relays an analog electrical voltage signal that

Conflicts of Interest

The authors declare no conflicts of interest.


[1] A. Brooks, “Real-Time Digital Image Stabilization,” Electrical Engineering Department, Northwestern University, Evanston, Illinois, 2003.
[2] Y. Liang, H. Tyan, S. Chang, H. Liao and S. Chen, “Video Stabilization for a Camcorder Mounted a Moving Vehicle,” IEEE Transactions on Vehicular Technology, Vol. 53, No. 6, 2004, pp. 1636-1648.
[3] A. Broggi, P. Grisleri, T. Graf and M. Meinecke, “A Software Video Stabilization System for Automotive Oriented Applications,” Proceedings of IEEE Conference on Vehicular Technology, Dallas, TX, 2005, pp. 2760-2764.
[4] F. Panerai, G. Metta and G. Sandini, “Adaptive Image Stabilization: A Need for Vision-Based Active Robotic Agents,” Proceedings of Conference on Simulation of Adaptive Behavior, Paris, France, 2000.
[5] D. Sachs, S. Nasiri and D. Goehl, “Image Stabilization Technology Overview,” InvenSense Whitepaper, 2006.
[6] K. Sato, S. Ishizuka, A. Nikami and M. Sato, “Control Techniques for Optical Image Stabilizing System,” IEEE Transactions on Consumer Electronics, Vol. 39, No. 3, 1993, pp. 461-466.
[7] J. S. Jin, Z. Zhu and G. Xu, “A Stable Vision System for Moving Vehicles,” IEEE Transactions on Intelligent Transportation Systems, Vol. 1, No. 1, 2000, pp. 32-39.
[8] Z. Zhu, “Full View Spatio-Temporal Visual Navigation—Imaging, Modeling and Representation of Real Scenes,” China Higher Education Press, China, 2001.
[9] B. Chan and C. Sandu, “A Ray Tracing Approach to Simulation and Evaluation of a Real-Time Quarter Car Model with Semi-Active Suspension System Using Matlab,” Proceedings of Design Engineering Technical Conference, Chicago, Illinois, 2003, pp. 2129-2134.
[10] N. Nise, “Control Systems Engineering,” 4th Edition, Wiley, Hoboken, New Jersey, 2004.
[11] M. Khajavi and V. Abdollahi, “Comparison between Optimized Passive Vehicle Suspension System and Semi Active Fuzzy Logic Controlled Suspension System Regarding Ride and Handling,” Transactions on Engineering, Computing and Technology, Vol. 19, 2007, pp. 57-61.
[12] W. C. Messner, D. Tilbury and W. Messner, “Control Tutorials for MATLAB and Simulink: A Web-Based Approach,” Prentice Hall, Englewood Cliffs, New Jersey, 1998.
[13] C. Morimoto and R. Chellappa, “Fast Electronic Digital Image Stabilization for Off-Road Navigation,” Real-Time Imaging, Vol. 2, No. 5, 1996, pp. 285-296.
[14] Z. Duric, and A. Rosenfeld, “Image sequence Stabilization in Real Time,” Real-Time Imaging, Vol. 2, No. 5, 1996, pp. 271-284.
[15] M. A. Turk, D. G. Morgenthaler, K. D. Gremban and M. Marra, “VITS—A Vision System for Autonomous Land Vehicle Navigation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 10, No. 3, 1988, pp. 342-361.
[16] S. B. Balakirsky and R. Chellappa, “Performance Characterization of Image Stabilization Algorithms,” Proceedings of International Conference of Image Processing, Lausanne, Switzerland, 1996, pp. 413-416.
[17] Z. Durica, R. Goldenbergc, E. Rivlina and A. Rosenfeld, “Estimating Relative Vehicle Motions in Traffic Scenes,” Pattern Recognition, Vol. 35, No. 6, 2002, pp. 1339-1353.
[18] A. Cretual and F. Chaumette, “Dynamic Stabilization of a Pan and Tilt Camera for Submarine Image Visualization,” Computer Vision and Image Understanding, Vol. 79, No. 1, 2000, pp. 47-65.
[19] Z. Duric, E. Rivlin and A. Rosenfeld, “Qualitative Description of Camera Motion from Histograms of Normal Flow,” Proceedings of International Conference of Pattern Recognition, Barcelona, Spain, Vol. 3, 2000, pp. 194-198.
[20] S. Sugimoto, H. Tateda, H. Takahashi and M. Okutomi, “Obstacle Detection Using Millimeter-Wave Radar and its Visualization on Image Sequence,” Proceedings of International Conference of Pattern Recognition, Cambridge, UK, Vol. 3, 2004, pp. 342-345.

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.