Hierarchical Image Segmentation Using a Combined Geometrical and Feature Based Approach

Melissa Cote, Parvaneh Saeedi

School of Engineering Science, Simon Fraser University, Burnaby, Canada.
School of Engineering Science, Simon Fraser University, Burnaby, CanadaSchool of Engineering Science, Simon Fraser University, Burnaby, Canada.

DOI: 10.4236/jdaip.2014.24014                

Abstract

This paper presents a fully automatic segmentation algorithm based on geometrical and local attributes of color images. This method incorporates a hierarchical assessment scheme into any general segmentation algorithm for which the segmentation sensitivity can be changed through parameters. The parameters are varied to create different segmentation levels in the hierarchy. The algorithm examines the consistency of segments based on local features and their relationships with each other, and selects segments at different levels to generate a final segmentation. This adaptive parameter variation scheme provides an automatic way to set segmentation sensitivity parameters locally according to each region's characteristics instead of the entire image. The algorithm does not require any training dataset. The geometrical attributes can be defined by a shape prior for specific applications, i.e. targeting objects of interest, or by one or more general constraint(s) such as boundaries between regions for non-specific applications. Using mean shift as the general segmentation algorithm, we show that our hierarchical approach generates segments that satisfy geometrical properties while conforming with local properties. In the case of using a shape prior, the algorithm can cope with partial occlusions. Evaluation is carried out on the Berkeley Segmentation Dataset and Benchmark (BSDS300) (general natural images) and on geo-spatial images (with specific shapes of interest). The F-measure for our proposed algorithm, i.e. the harmonic mean between precision and recall rates, is 64.2% on BSDS300, outperforming the same segmentation algorithm in its standard non-hierarchical variant.

Keywords

Image Segmentation, Adaptive Color Analysis, Shape Analysis, Prior Model, Image Processing, Split-and-Merge Segmentation, Perceptual Grouping

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Chen, H., Chien, W. and Wang, S. (2004) Contrast-Based Color Image Segmentation. IEEE Signal Processing Letters, 11, 641-644. http://dx.doi.org/10.1109/LSP.2004.830116
[2]	Felzenszwalb, P. and Huttenlocher, D. (2004) Efficient Graph-Based Image Segmentation. International Journal of Computer Vision, 59, 167-181. http://dx.doi.org/10.1023/B:VISI.0000022288.19776.77
[3]	Lo, E.H., Pickering, M.R., Frater, M.R. and Arnold, J.F. (2011) Image Segmentation from Scale and Rotation Invariant Texture Features from the Double Dyadic Dual-Tree Complex Wavelet Transform. Image and Vision Computing, 29, 15-28. http://dx.doi.org/10.1016/j.imavis.2010.08.004
[4]	Tan, K.S. and Isa N.A.M. (2011) Color Image Segmentation Using Histogram Thresholding—Fuzzy C-Means Hybrid Approach. Pattern Recognition, 44, 1-15. http://dx.doi.org/10.1016/j.patcog.2010.07.013
[5]	Cootes, T.F., Taylor, C.J., Cooper, D.H. and Graham, J. (1995) Active Shape Models—Their Training and Application. Computer Vision and Image Understanding, 61, 38-59. http://dx.doi.org/10.1006/cviu.1995.1004
[6]	Staib, L.H. and Duncan, J.S. (1992) Boundary Finding with Parametrically Deformable Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 1061-1075. http://dx.doi.org/10.1109/34.166621
[7]	Wang, Y. and Staib, L.H. (1998) Boundary Finding with Correspondence Using Statistical Shape Models. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Santa Barbara, 23-25 June 1998, 338-345.
[8]	Fussenegger, M., Roth, P.M., Bischof, H. and Pinz, A. (2006) On-Line, Incremental Learning of a Robust Active Shape Model. DAGM-Symposium, 122-131.
[9]	Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, W. and Willsky, A. (2001) Model-Based Curve Evolution Techniques for Image Segmentation. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, 8-14 December 2001, I-463-I-468.
[10]	Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, W. and Willsky, A. (2003) A Shape-Based Approach to the Segmentation of Medical Imagery Using Level Sets. IEEE Transactions on Medical Imaging, 22, 137- 154. http://dx.doi.org/10.1109/TMI.2002.808355
[11]	Foulonneau, A., Charbonnier, P. and Heitz, F. (2006) Affine-Invariant Geometric Shape Priors for Region-Based Active Contours. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 1352-1357. http://dx.doi.org/10.1109/TPAMI.2006.154
[12]	Chen, Y., Tagare, H., Thiruvenkadam, S., Huang, F., Wilson, D., Gopinath, K.S., Briggs, R.W. and Geiser, E. (2002) Using Shape Priors in Geometric Active Contours in a Variational Framework. International Journal of Computer Vision, 50, 315-328. http://dx.doi.org/10.1023/A:1020878408985
[13]	Leventon, M.E., Grimson, W.E.L. and Faugeras, O. (2000) Statistical Shape Influence in Geodesic Active Contours. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, 13-15 June 2000, 316-323.
[14]	Rousson, M. and Paragios, N. (2008) Prior Knowledge, Level Set Representations & Visual Grouping. International Journal of Computer Vision, 76, 231-243. http://dx.doi.org/10.1007/s11263-007-0054-z
[15]	Riklin-Raviv, T., Kiryati, N. and Sochen, N. (2007) Prior-Based Segmentation and Shape Registration in the Presence of Perspective Distortion. International Journal of Computer Vision, 72, 309-328. http://dx.doi.org/10.1007/s11263-006-9042-y
[16]	Chan, T.F. and Vese, L.A. (2001) Active Contours without Edges. IEEE Transactions on Image Processing, 10, 266-277. http://dx.doi.org/10.1109/83.902291
[17]	Sharon, E., Galun, M., Sharon, D., Basri, R. and Brandt, A. (2006) Hierarchy and Adaptivity in Segmenting Visual Scenes. Nature, 442, 810-813. http://dx.doi.org/10.1038/nature04977
[18]	Paris, S. and Durand, F. (2007) A Topological Approach to Hierarchical Segmentation Using Mean Shift. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, 17-22 June 2007, 1-8.
[19]	Comaniciu, D. and Meer, P. (2002) Mean Shift: A Robust Approach toward Feature Space Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603-619. http://dx.doi.org/10.1109/34.1000236
[20]	Alpert, S., Galun, M., Brandt, A. and Basri, R. (2012) Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34, 315-327. http://dx.doi.org/10.1109/TPAMI.2011.130
[21]	Corso, J.J., Yuille, A. and Tu, Z. (2008) Graph-Shifts: Natural Image Labeling by Dynamic Hierarchical Computing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, 23-28 June 2008, 1-8.
[22]	Bezdek, J.C. (1981) Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York. http://dx.doi.org/10.1007/978-1-4757-0450-1
[23]	Duda, R., Hart, P. and Stork, D. (2001) Pattern Classification. John Wiley & Sons, New York.
[24]	Hu, M.K. (1962) Visual Pattern Recognition by Moment Invariants. IRE Transactions on Information Theory, 8, 179- 187. http://dx.doi.org/10.1109/TIT.1962.1057692
[25]	Martin, D.R., Fowlkes, C., Tal, D. and Malik, J. (2001) A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. Tech. Rep. UCB/CSD-01-1133, EECS Department, University of California, Berkeley.
[26]	Felzenszwalb, P. and McAllester, D. (2006) A Min-Cover Approach for Finding Salient Curves. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop, New York, 17-22 June 2006, 185.
[27]	Martin, D.R., Fowlkes, C.C. and Malik, J. (2004) Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 530-549. http://dx.doi.org/10.1109/TPAMI.2004.1273918
[28]	Maire, M., Arbeláez, P., Fowlkes, C. and Malik, J. (2008) Using Contours to Detect and Localize Junctions in Natural Images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, 23-28 June 2008, 1-8.
[29]	Agouris, P., Doucette, P. and Stefanidis, A. (2004) Automation and Digital Photogrammetric Workstations. In: Manual of Photogrammetry, 5th Edition, American Society for Photogrammetry and Remote Sensing, Bethesda, 949-981.
[30]	McKeown, D., Bulwinkle, T., Cochran, S., Harvey, W., McGlone, C. and Shufelt, J. (2000) Performance Evaluation for Automatic Feature Extraction. International Archives of Photogrammetry and Remote Sensing, 33, 379-394.
[31]	Ruther, H., Martine, H.M. and Mtalo, E.G. (2002) Application of Snakes and Dynamic Programming Optimization Technique in Modeling of Buildings in Informal Settlement Areas. ISPRS Journal of Photogrammetry and Remote Sensing, 56, 269-282. http://dx.doi.org/10.1016/S0924-2716(02)00062-X
[32]	Peng, J. and Liu, Y.C. (2005) Model and Context-Driven Building Extraction in Dense Urban Aerial Images. International Journal of Remote Sensing, 26, 1289-1307. http://dx.doi.org/10.1080/01431160512331326675