Potential Flaws in the Evaluation Design of Weather Modification Studies Using Radar

Evaluating the impact of planned weather modification experiments remains one of our biggest problems. The challenge is to detect a significant increase in surface rainfall against the background of natural variability, which can be significant. The observed natural variability is a function of the chosen experimental unit. Weather modification experiments used different units, including rain days, target and control areas, convective storm turrets, and isolated convective cells. Those gave rise to a variety of experimental designs to compare seeded and non-seeded storms. The smaller experimental used exhibits less natural variability and experiments focus on those had generally more statistically significant results. The South African Rainfall Enhancement project, between the late 1970s and 2000, was one of the first to tackle this problem using weather radar. Tracking software was developed to identify convective storm entities. The convective cells were used as experimental units in a double-blind, randomized control experiment. Statistical significant results were obtained that were subsequently independently reevaluate several times. Studies concluded a statistically significant increase in storm lifetimes as well as ensuing rainfall enhancement. The link between the microphysical intervention and the dynamic response has been the topic of many efforts to explain. The software developed into the open source suite referred to as Titan, the Thunderstorm Identification and Tracking algorithms, and more recently the LIDAR Radar open source environment (LROSE), which is still widely used for weather modification operations and evaluation. Studies following on the original design, including Mexico, a semi-operational experiment in South Africa, Mexico, USA, UAE, Australia, and Saudi Arabia built on this design and produced similar results using Titan. This paper explores aspects that have not been used to explain the results obtained by these experiments. It reviews and compares experimental designs of the above mentioned projects. Radar reflectivity and doppler velocity for ones season of convective summer rainfall from South Africa, the United Arab Emirates, Saudi Arabia, the USA, and Thailand are used. Multiple seasons were available for the South African radar. The latest LROSE/TITAN tracking algorithm was run using a threshold reflectivity of 30dBZ, matching the original design of the South African experiment. It is shown that the probability density function of storm lifetimes is typically positively skewed, log-normal, and a function of geography and season. The threshold reflectivity, as well as other storm tracking parameters, has significant impacts on this probability density function. The skewness is worse for dryer years or geographies. This relation is used to explain the observed dynamic effect of hygroscopic seeding experiments in the traditional design. It can also explain similar results in experiments that used similar, but slightly modified designs, like the operational South African, Mexican, the Australian, and UAE campaigns. This highlights uncontrolled variables that might have lead to the perceived positive results in the South African Rainfall Enhancement project. It also alleviates some of the tension between comparisons of the field campaigns with modeling studies. The paper concludes with suggested changes to the original experimental design that can help to control for this flaw in the future.