Accelerating the Transition of Satellite Precipitation Retrievals from Research to Operations Using Modern Software Engineering and MLOps Best Practices

Tomorrow.io provides global precipitation products for governmental, commercial, and individual clients. In 2024, in collaboration with academic partners, Tomorrow.io operationalized a cutting-edge machine learning framework for multi-sensor satellite precipitation retrievals. This research-to-operations initiative aimed to facilitate low-latency, real-time global precipitation retrievals from various geostationary and low-earth orbit satellites. The primary goal was to establish a methodology that leverages Tomorrow.io’s Microwave Sounder (TMS) constellation at inference time, shortly after the first satellites were launched in August 2024. Unlike geostationary visible and infrared sensors, microwave sounders are sensitive to atmospheric attenuation and scattering caused by precipitation, allowing their brightness temperature readings to significantly enhance global satellite precipitation estimates.

This presentation will explore the rapid operationalization of the research framework within a year, covering the complete project lifecycle—from concept to validation and production. Remarkably, the system was deployed into production concurrently with the launch of the first two sounders, enabling a seamless integration of Tomorrow.io’s satellite data into real-time operations. The swift development and deployment of the ML precipitation retrieval framework were bolstered by Azure Cloud, which provided immediate access to GPUs for both training and inference, greatly expediting the prototyping and optimization of the data pipeline. The heritage of the TMS sensor from NASA’s TROPICS mission provided over three years of training data from multiple satellites to leverage for at-launch inference. Additionally, the implementation of robust unit and integration testing ensures code quality and efficient resource management, namely memory and CPU usage.

Leveraging established development patterns from Tomorrow.io’s existing product pipelines enabled the transition from prototype to production in mere months. This process utilized event-driven job scaling within Kubernetes and streamlined CI/CD practices. An automated real-time validation system further ensures the continuous maintenance of product quality.

This academic-industry collaboration to develop and transition a state-of-the-art satellite precipitation retrieval framework into a consumer-ready product highlights a significant acceleration of the research-to-operations timeline. The effective integration of Tomorrow.io’s satellite constellation data into the company’s offerings demonstrates the benefits of such collaborations.