Prototyping an Analytic Tool for Accessing and Extracting Information from Big Model Data

Application research studies about wind energy, solar energy, extreme weathers and economic analysis, as well as air quality and human health requires statistical analysis of long term data at higher resolutions.    Recently, NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) has released climate reanalysis data from Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2), generated by NASA Global Modeling and Assimilation office (GMAO) by using the Goddard Earth Observing System Data Assimilation System Version 5 (GEOS-5).   The MEERRA-2 contains over 35 years, beginning 1980, global hourly or 3-hourly meteorological parameters, aerosol, and chemistry data, such as near surface wind, wind profile, air temperature, relative humidity, surface irradiance, aerosols, ozone, and CO. etc. The data spatial resolution is 0.5^o x 0.625^o (lat x lon).  It is a challenge to download the parameters of the long time period at interested locations due to large data volume and existing data file structures, even with the existing powerful subsetting service.   For example, extracting the time series of one parameter at a single data point for the entire model period from the existing service may take over 10 hours.  GES DISC is exploring methods on improving the accessing of the big model data.  One work is trying to reconstruct the data files from one-day per file to multiple days or months aggregated files, then make them available in the interoperable data services, such as OPeNDAP, and THREDDS for allowing data be accessed directly by applications.  This presentation will show the results of optimized time aggregation for varies data access cases.     The reconstructed data has improved the performance of long time series data accessing dramatically, which enables some basic analysis on-the-fly, such as daily mean, min/max, as well as extract information, such as extreme events.