AI for Earth: Building an Extensible, Distributed API Pipeline Platform for Artificial Intelligence

After building models, researchers and scientists are faced with a daunting problem: they must decide how to deploy their model, integrate it with required data sources, secure it, disseminate results, and keep the model current. The myriad of data-sources, whether they be on-premises, cloud based data sources, or IoT device exhaust, further exacerbates the problem. The Microsoft AI for Earth Team has built a generic API Platform, tuned for long-running inference, which gives scientists a way to host their models and dynamically scale on load. The model is hosted within an API, which gives the researcher the ability to grant keyed access globally and permits the API to be used within composite pipelines. Finally, through AI for Earth libraries, a scientist can consume data from traditional databases, cloud-based storage, IoT devices, and streaming technologies. The AI for Earth API Platform is an open source solution for the model consumption and resource sharing problems that have maintained artificial barriers to the advancement of research and science for decades.