Hausdorff Dimension of Multi-Layer Neural Networks

DOI: 10.4236/apm.2013.39A1002   PDF   HTML   XML   2,649 Downloads   4,448 Views  

Abstract

This elucidation investigates the Hausdorff dimension of the output space of multi-layer neural networks. When the factor map from the covering space of the output space to the output space has a synchronizing word, the Hausdorff dimension of the output space relates to its topological entropy. This clarifies the geometrical structure of the output space in more details.

Share and Cite:

J. Ban and C. Chang, "Hausdorff Dimension of Multi-Layer Neural Networks," Advances in Pure Mathematics, Vol. 3 No. 9A, 2013, pp. 9-14. doi: 10.4236/apm.2013.39A1002.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] K. Hornik, M. Stinchcombe and H. White, “Multilayer Feedforward Networks Are Universal Approximators,” Neural Networks, Vol. 2, No. 5, 1989, pp. 359-366.
http://dx.doi.org/10.1016/0893-6080(89)90020-8
[2] B. Widrow and M. Lehr, “30 Years of Adaptive Neural Networks: Perceptron Madaline, and Backpropagation,” Proceedings of the IEEE, Vol. 78, No. 9, 1990, pp. 1415-1442. http://dx.doi.org/10.1109/5.58323
[3] Y. A. Alsultanny and M. M. Aqul, “Pattern Recognition Using Multilayer Neural-Genetic Algorithm,” Neurocomputing, Vol. 51, 2003, pp. 237-247.
http://dx.doi.org/10.1016/S0925-2312(02)00619-7
[4] B. Widrow, “Layered Neural Nets for Pattern Recognition,” IEEE Transactions on Acoustics, Speech, and Signal Processing, Vol. 36, No. 7, 1962, pp. 1109-1118.
http://dx.doi.org/10.1109/29.1638
[5] J. J. Hopeld and D. W. Tank, “Neural Computation of Decisions in Optimization Problems,” Biological Cybernetics, Vol. 52, No. 3, 1985, pp. 141-152.
[6] C. Peterson and B. Soderberg, “A New Method for Mapping Optimization Problems onto Neural Network,” International Journal of Neural Systems, Vol. 1, 1989, pp. 3-22. http://dx.doi.org/10.1142/S0129065789000414
[7] J.-C. Ban and C.-H. Chang, “The Learning Problem of Multi-Layer Neural Networks,” Neural Networks, Vol. 46, 2013, pp. 116-123.
http://dx.doi.org/10.1016/j.neunet.2013.05.006
[8] J.-C. Ban, C.-H. Chang and S.-S. Lin, “The Structure of Multi-Layer Cellular Neural Networks,” Journal of Differential Equations, Vol. 252, No. 8, 2012, pp. 4563-4597.
http://dx.doi.org/10.1016/j.jde.2012.01.006
[9] J.-C. Ban, C.-H. Chang, S.-S. Lin and Y.-H. Lin, “Spatial Complexity in Multi-Layer Cellular Neural Networks,” Journal of Differential Equations, Vol. 246, No. 2, 2009, pp. 552-580. http://dx.doi.org/10.1016/j.jde.2008.05.004
[10] S.-N. Chow and J. Mallet-Paret, “Pattern Formation and Spatial Chaos in Lattice Dynamical Systems: I and II,” IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, Vol. 42, No. 10, 1995, pp. 746-756. http://dx.doi.org/10.1109/81.473583
[11] K. Falconer, “Fractal Geometry: Mathematical Foundations and Application,” 2nd Edition, John Wilet & Sons, New York, London, Sydney, 2003.
http://dx.doi.org/10.1002/0470013850
[12] Y. Pesin, “Dimension Theory in Dynamical Systems: Contemporary Views and Application,” The University of Chicago Press, Chicago, 1997.
http://dx.doi.org/10.7208/chicago/9780226662237.001.0001
[13] J. Juang and S.-S. Lin, “Cellular Neural Networks: Mosaic Pattern and Spatial Chaos,” SIAM Journal on Applied Mathematics, Vol. 60, No. 3, 2000, pp. 891-915.
http://dx.doi.org/10.1137/S0036139997323607
[14] D. Lind and B. Marcus, “An Introduction to Symbolic Dynamics and Coding,” Cambridge University Press, Cambridge, 1995.
http://dx.doi.org/10.1017/CBO9780511626302
[15] B. Kitchens, “Symbolic Dynamics. One-Sided, Two-Sided and Countable State Markov Shifts,” Springer-Verlag, New York, 1998.
[16] J.-C. Ban and C.-H. Chang, “On the Structure of Multi-Layer Cellular Neural Networks. Part II: The Complexity between Two Layers,” Submitted, 2012.

  
comments powered by Disqus

Copyright © 2020 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.