6.4 MOS Parse: Library for Converting MOS Datasets to Machine Learning Formats

Wednesday, 15 January 2020: 11:30 AM
157AB (Boston Convention and Exhibition Center)
Hannah Aizenman, The Graduate Center, CUNY, Rego Park, NY; and O. Lucero, T. Schiminovich, and M. Grossberg

Model Output Statistics (MOS) Guidance can be downloaded from the National Weather Service in a text format that is easy for humans to read, but difficult for programs to interpret, making batch and large scale machine learning analysis difficult. To facilitate such analysis we reformatted model output to a nearly universal file format readable by nearly all data analysis programs and programming languages: spreadsheets. We developed a python script that does this reformatting, validated it using the Iowa Mesonet dataset, and used the script to convert model output files from the years 2000 to 2019. We then created a Python library that allows other users to make this conversion without needing to write their own programs, allowing them to perform their own analysis faster. We demonstrate the use of our library by comparing model output data to observed data, retrieved from the Global Historical Climatology Network. As a collection of csvs is also not practical as the data size increases, we then pushed the data into a PostGIS table, on top of which we layered a GraphQL interface to make this data more discoverable and queryable.
- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner