The NESDIS Common Cloud Framework (NCCF): NCCF Operational Performance Data Pipeline Processing demonstration from data ingest to data distribution utilizing the NCCF Kibana Dashboard to illustrate the “data flow”.

This presentation will demonstrate and describe the NCCF Operational Performance dashboard to showcase how data moves through the NCCF Data Pipeline in order to generate products and distribute them to NESDIS’s customers while meeting low latency requirements. Data comes from Data Providers via multiple methods and protocols which are ingressed through NCCF Secure Ingest. Products are generated using NCCF Product Generation which operates Cloud Containerized Algorithm Packages (CCAPs); also known as algorithms and distributed to Data Consumers. Currently, NCCF is ingesting 28 Million + files (154TB +) from data providers and producing 886 products from 97 algorithms all in the course of a single month. In addition, NCCF is currently supporting 28+ satellites which consistently ingest data. The NCCF operates an ElasticSearch Kibana dashboard environment to visualize mission critical system metrics such as ingress, egress, real time job monitoring to provide all relevant stakeholders with the information they need.