85th AMS Annual Meeting

Thursday, 13 January 2005: 4:30 PM
NuSDaS - meteorological database for operational weather service
Eizi Toyoda, University of Wisconsin, Madison, WI
Poster PDF (411.5 kB)
NuSDaS (Numerical Prediction Standard Dataset System) is a database system of Japan Meteorological Agency (JMA) that has been used in operational weather service since the main supercomputer of JMA was replaced with UNIX-based HITACHI SR8000 in 2001. Use of NuSDaS is mandatory in the JMA operational suites in order to standardize data format, data access subroutine, and terms of data structure.

Since it is not general-purpose database, data structure and interface of NuSDaS are highly specialized for meteorological data, especially those of numerical weather prediction. Database is made with records that are numerical data on a two-dimensional grid. A record is identified by following identifiers:

  • data type name,
  • reference time,
  • member of ensemble forecast if appropriate,
  • valid time,
  • level name, and
  • parameter name.
Records are stored in files in a data directory for a data type. Separated data files are used for different reference time and, by optional configuration, member or valid time, so that merging of similar datasets can be done efficiently.

To help wider activities in research and development for operation, following improvements have been made recently:

  • port to little-endian computers,
  • port to computers with different Fortran-C linkage,
  • interface to Ruby, an object-oriented language, and
  • implementation of data management tools.

Current efforts are made for two major topics: improvement of performance and support for distributed database.

The former one is essential. Higher reading/writing efficiency has to be achieved for future operational suites in future computers. That will require change of file format and direct use of system call. Limitations due to 2 gigabyte file size limit should also be abolished when the file format is changed.

The latter one will have relevance to wider range of users. In order to facilitate data access from remote computers, a data transfer protocol using HTTP was developed. Under this protocol, the data server has functionalities of automatic data format conversion and traffic control. The server can relay a request for data to another server that has the data requested, and then the servers negotiate about which host convert the data format using content negotiation framework defined in HTTP. This data server has been employed experimentally in several operations.

Supplementary URL: