Author(s)

D. Wang^1*, X. Luo², F. Yuan¹, N. Podhorszki³

¹Environmental Science Division, Oak Ridge National Laboratory, Oak Ridge, TN, USA.
²Department of Computer Science and Electric Engineering, University of Tennessee, Knoxville, TN, USA.
³Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN, USA.

This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e., automated code analysis and instrumentation), in-situ infrastructure (i.e., ADIOS) and big data analysis engines (i.e., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. The in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.

In-Situ Data Analysis, Source Code Analysis, Data Staging, ADIOS, Earth System Model, Machine Learning, SciKit-Learn, E3SM

Wang, D. , Luo, X. , Yuan, F. and Podhorszki, N. (2017) A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure. Journal of Computer and Communications, 5, 76-85. doi: 10.4236/jcc.2017.514007.