Presented in this work is a suite of QA/QC processes to validate and augment solar irradiance data with novel focus placed on scalability. Because of varying levels of station maintenance and curation, the underlying foundation of QA/QC should be scalable. A subset of analysis can be initiated in real-time, in-situ during acquisition on a data logger, as well as more completely, on a local machine or in the cloud.
At a high level, the analysis aims to identify and annotate both quantitative and qualitative features in solar data, utilizing up to four commonly acquired components: plane of array (POA), global horizontal (GHI), direct normal (DNI), and diffuse horizontal (DHI) irradiance. The QA/QC suite consists of analytic processes that can be systematically and modularly applied based on data availability. QA/QC begins with fundamental sanity checks to ensure spatial time alignment, phase shifts, and the detection of outliers and missing data. Empirical validations can then be performed through limit and ratio testing across one or more of the components. Additionally, the suite includes algorithms to detect qualitative features such as symmetry, shading, dew/frost, and soiling. To enhance analysis by providing guidance through further comparison, the framework also provides tools to generate additional referential irradiance data sets of varying complexity via clear-sky and all-sky models. Following QA/QC, to maximize insights in the case of limited data/missing components, decompositional and transpositional models are also available to generate additional referential data. To enhance interpretability, the resulting features and issues are annotated. This is vital for managing large networks and the significant volumes of data they produce, making human oversight more feasible in the final phase of QA/QC.
In terms of implementation, conducting QA/QC on a local machine or cloud allows for the incorporation of supplementary satellite data (e.g., atmospheric constituents) in generating additional datasets, such as clear and all-sky referential models (i.e., REST2 or FARMS respectively). However, since the required parameters on a logger are typically unavailable, this referential modeling was limited to clear-sky conditions and required further refinement to ensure its viability. Notably, considerable research was dedicated to developing two robust clear-sky detection algorithms capable of operating in real-time: 1D filtering (which considers smoothness, rate of change, and variance) and 2D temporal inference through day-to-day changes. To minimize parametric overhead, clear-sky reference modeling relied on the Haurwitz model, which requires only the zenith angle. This model was statistically adjusted based on clear-sky points to enhance its accuracy in fitting the data. Despite its limited input, the adjusted model closely approximates the commonly used REST2 model.
This study set out to achieve two main objectives: primarily, to create a comprehensive framework for solar irradiance QA/QC, and secondly, to examine its portability for data loggers. In this context, it is promising that a functional subset can be implemented on a data logger, offering the benefit of real-time reactivity, which enables issues and faults to be identified directly at the acquisition level, while also conveniently reducing the need for downstream processing.
- Indicates paper has been withdrawn from meeting
- Indicates an Award Winner