by Woo-Jin Lee1
The three main operational components of daily weather production systems are real-time observing and data collection, routine global exchange of data and information and the systematic operational processing of data to produce meteorological analyses, numerical weather predictions (NWP) and weather forecasts and warnings. Thus, these three components – the Global Observing System (GOS), the WMO Information System (WIS), and the Global Data-processing and Forecasting System (GDPFS) – have formed the backbone of the World Weather Watch (WWW) System.
These systems require substantial investments in order to serve the public need for targeted weather and climate information to mitigate disaster risk and save lives and property. Are the resulting services worth the investment? Can meteorological service be valued in order to gauge the return on investments? The Korean Meteorological Administration (KMA), having heavily invested in its data processing and forecasting system (DPFS), made the valuing of its services part of its mandate. The investments: modern technological infrastructure and scientific expertise in various fields, including telecommunication, automated systems for acquiring observations, data management facilities – storage, retrieval, processing and manipulation, quality control – an automated NWP production scheduling and control system; NWP computing and engineering, and a sophisticated data integration and visualization system. What is the value of the resulting weather and climate services in the daily lives of the population?
The relative importance of weather information in daily lives depends on the nature of one’s activities, climatic regimes and the degree of a country’s socio-economic development. Every National Meteorological Service also operates in its own unique national environment – economic conditions, public perception of weather and climate, importance relative to wide-ranging government priorities, decision-making processes for government funding, social interest or belief in prevention and mitigation versus post-disaster recovery, etc. – however, the implications of the KMA study remain pertinent to all. The valuation exercise demonstrated that the level of services that KMA provides, thanks to its investments in DPFS, is highly valued and a benefit to the society it serves.
Data processing and forecasting investments
DPFS refers to the complex technical skills and technological array required for NWP across all time scales, including data assimilation, integration of numerical models of the atmosphere, post-processing of observational and forecast data, interpretation of model outputs, and the production of weather forecasts and warnings. KMA has invested in the development of its DPFS for several decades, and has evolved its system through the different stages of technical infrastructure, advancing from graphical interpretation of NWP outputs and the application of binary model outputs, to running a regional NWP model with lateral boundary conditions obtained from other centres, to operating a regional model with a data assimilation system for real-time input of observations to attain the most advanced stage of running a global model with four-dimension variational (4D-Var) data assimilation.
KMA concentrated strategically on two early breakthrough areas. First, the automation of telecommunications that became the foundation of its data-processing system, then the acquisition of a first supercomputer system, installed in 1999. This acquisition reflected national support for the installation and application of a state-of-the-science limited area numerical simulation model (LAM) of the atmosphere at a supercomputing facility, and led the way to the establishment of an atmospheric modelling group, which in turn led to additional funding support for software development and applications to meet many national needs. Maintaining these elements to the present, KMA sustained its DPFS assets by adopting technological advancements and by investing in the young scientists attracted to the innovative technological environment at KMA. This resulted in more relevant operational products and services to meet the ever-growing need for meteorological and environmental predictions.
Social Environment
As stated earlier, the value of weather and climate products and services to a society depends on the socio-economic and environmental context. KMA serves a population living in an area that is prone to natural hazards. Korea is affected by the various severe weather phenomena of mid-latitude nations such as heavy rain and snow storms, tropical cyclones, Asian yellow sand and dust storms, severe thunderstorms with violent winds, hail and lightning, and temperature extremes. During the summer time, there are heavy rains over the Korean Peninsula due to tropical storms. The rugged orography of 70 per cent of the country’s land area and the unique geographical environment with influences from the Asian Continent confronting those from the Pacific Ocean add complexity to the nation’s mid-latitude climate, and the challenges for protecting its people and economic activities from meteorological stresses and extremes.
The heavily populated limited metropolitan and industrialized areas, as well as the combined demographic and social environment, challenge weather prediction systems and KMA forecasters.
Technical performance
The valuing of DPFS should be viewed from two perspectives: assessment of technical performance and its relation to economic terms. The standard verification measures under the WMO Commission for Basic Systems (CBS), as well as other measures, can be effectively used for assessing performance. The conversion of verification results into monetary terms requires further translation, including assumptions, to create valued information that links the outputs of production; basically that products and services must be linked to their consumption in society.
The attributes of weather forecasts include accuracy or reliability, consistency, precision and forecast lead-time – each of which measures aspects of technical performance. Making a connection of performance with the operational costs, one can deduce the value of the investment in DPFS and the relative value associated among different attributes. Verifying accuracy is the most common objective measure, and is universally shared among WMO numerical weather prediction centres.
The benefit of DPFS in terms of accuracy is most evident from the recent performance at the European Centre for Medium-Range Weather Forecasts (ECMWF). The root mean square error (RMSE) at 500 hPa of the ECMWF global model shows improvement through the last several decades, especially over the southern hemisphere and in the medium-range. Similar trends with a higher rate of improvement with time can be found within the NWP performance at KMA. The current RMSE for five-day forecasts is comparable to that for three-day forecasts 10 years ago. Predictability has been extended two days further in time over the last 10 years.
The increased accuracy is partly due to the research and development on satellite data assimilation and physical parameterization, and partly to the rapid technology transfer from advanced meteorological centres. The frequent upgrade of supercomputers over the last decade also contributed to the improvement of model performance. Advancement in high resolution modelling contributed to the effectiveness of heavy precipitation warnings along with the progress in nowcasting tools, radar data application techniques, and multi-model ensemble prediction techniques.
The improvement in forecasting tropical cyclone tracks, including the increase in lead-time, has significantly contributed to the effectiveness of forecasts and early warnings for anticipated heavy precipitation up to five days in advance. The advancement of DPFS and increased density of the observation network also enabled new services for the public benefit. For example, village scale forecasts across a 5 km mesh became possible when the resolution and accuracy of regional models reached acceptable levels of quality. Currently, KMA provides hourly updates of village scale forecasts based on human interpretations of the numerical prediction database for ten weather elements including temperature, precipitation, wind, and relative humidity.
The social benefit of DPFS investments can be monitored by user satisfaction surveys, an indirect measure of the value of the DPFS investment. KMA conducts such public surveys twice a year. User satisfaction indices, derived from surveys during 2007-2011, range from 60 to 80 per cent for short-term forecasts and warnings, and showed slightly lower levels of satisfaction for weekly forecasts. The satisfaction index has gradually increased in the last five years by 11.8 points for short-range forecasts, by 20.6 points for medium-range, and by 12.3 points for warnings. In addition, more than two thirds of the population use 1-3 day forecasts for daily decision-making.
An example of graphic forecast with 5km x 5km spatial resolution on the KMA web page (www.digital.go.kr)
Value of forecasts and warnings
KMA provides daily forecasts and warnings to Korean citizens and overseas visitors to protect lives and properties from adverse weather and climate, as a national weather service is normally expected to do. This includes weekly and extended range forecasts for temperature, wind, and precipitation, and various warnings for heavy rain or snow, strong wind, severe thunderstorms, storm surges, and tropical cyclones.
Based on survey results and economic studies conducted by KMA, an average Korean household is willing to pay 20 424 KRW per year for weather and climate services, which amounts to a total of 3 589 billion KRW (or 3.3 billion USD) annually. This is equivalent to 145 per cent of the annual budget of the KMA. The potential value of additional improvements in public forecast and warning systems exceeds 45 per cent of the KMA annual budget. Thus, from the public service point of view alone, approximately 1 119 billion KRW (or 1.1 billion USD) represents the value attained from investing in the forecasting infrastructure at the KMA. It is to be noted that the value estimated is conservative, as it does not include the additional value from enhanced services provided by the commercial sector.
The value of quality assured forecast and warning production grossly divides into direct and indirect benefits. Timely, accurate warnings directly contribute to mitigating the loss of life, property, infrastructure and livelihoods from adverse weather and climate events, which in turn generate additional indirect benefits for the commercial sector. But in most cases, these indirect benefits cannot easily be converted into monetary terms.
Physical protection measures, such as drainage and dike systems, are considered to offer short-term returns while measures to reduce vulnerability, such as an improved understanding of risks, early warning systems and insurance programmes, provide much longer term returns. According to the study of Hallegatte (2012), large returns of up to 36 billion USD can be realized from an investment of 1 billion USD in early warning systems. The potential benefit over the cost of investment ranges from a factor of 4 to 36.
There are many measurable attributes to a forecast that are often trade-offs – investing in improving one attribute means loss in another. For example, there is a trade-off between forecast lead-time and the reliability of warnings. Increasing the advanced delivery of warnings affords more time for action, but leads to an increase in the frequency of warnings that do not materialize, or the “false alarm rate.” The reverse situation is true for the reliable warnings, i.e., lower false alarm rates are achieved with shorter advanced warnings. A 6-hour lead-time for flood warnings results in a 35 per cent reduction of damages, whereas a 12-hour lead-time provides a 60 per cent reduction of damages (Schroter et al., 2008). Therein lies the complexity of assessing the value of forecasts and warnings, relative to the several key attributes of the forecasts.
It is interesting to note that observation has shown that developing countries only harness part of the full potential benefit from early warning systems. Low-income countries only gain 10 per cent of the maximum benefit, while the upper middle-income countries gain about 50 per cent (Hallegatte, 2012). When national security, disaster risk reduction and sustainable development are high priority policy areas, there is value to be gained from investing in early warning and forecast systems.
Damage reduction as a function of lead-time, reproduced from Schroter et al. (2008).
Valuing the investment in major Numerical Weather Prediction
The focus now turns to the cost-benefit analysis for major software developments in order to demonstrate how to value DPFS. From 2011 to 2019, KMA will develop its next generation NWP model, while maintaining the high quality of forecasts through continuous updates of its Unified Model initially acquired from the UK Met Office. This project requires additional human and financial resources. The Korea Development Institute (KDI) has estimated the monetary value of the project in terms of the unit increment of warning lead-time. The procedure for the estimates is presented schematically in Figure 3.7. For the purposes of estimating value, the RMSE measure of the 500hPa geopotential height forecasts for day-5 is used. Three experimental pathways and a control pathway are defined by the rate of improvement in terms of RMSE, starting at the initial RMSE of 2008.
The constant rate of improvement of 2.6 per cent is assumed for the experimental pathways with the model development project, which is based on the average of RMSEs among developed centres with self-developed models (KDI, 2010). Different initial RMSEs in 2008 lead to different evolution of RMSEs in the experimental pathways. In contrast, the slower constant rate of improvement of 1.4 per cent is assumed for the control or “reference” pathway, which is based on the past experience in the absence of a self-developed model at KMA. The prime interest is to compare the control pathway starting from the initial RMSE of the UK Met Office (considered among the best global NWP centres in 2008), with the reasonable experimental pathway starting from the initial RMSE value of the 5th ranked centre, and the averaged RMSE values of the 5th and 7th ranked centres in 2008.
The gain in terms of the RMSE differences between the control pathway and the experimental pathways can be converted into monetary values. The relationship between investment on NWP software development and its effect on the lead-time of heavy rain warnings during the summer months is evaluated from the past record at KMA to determine the reduction of economic losses due to earlier warnings. The cost benefit ratio for the project is 1.1 between the present value of total benefits of 79 billion KRW (or 71 million USD) and investment cost of 72 billion KRW over the time period from 2011-2029, with the future monetary values discounted to the present time at the rate of 5.5 per cent per year.
Root mean squared errors (RMSE) for day-5 forecasts of geopotential heights at 500hPa estimated for the period of model development project (2010-2019), and through 2029.
In addition to the NWP software development project, the annual investment on forecast system tools requires 7 billion KRW, which is expected to increase the benefit to be equivalent to 600 billion KRW. The assumption made is that a 1 minute increase in warning lead-time is equivalent to a reduction of 0.05 per cent in damage (KMA, 2003) and a 1 per cent improvement of forecast accuracy leads to a 2 per cent reduction of damage (Park, 2002). The establishment of KMA’s typhoon warning center in a new building with experts and support staff of 20 professionals and related operational costs and research funding is expected to yield cost avoidance of 240 billion KRW – 13 times the annual operational costs. It is assumed that a 5 per cent reduction of damages can be realized by improving the early warning system (Korea Institute of S&T Evaluation and Planning, 2005).
Fund raising
It is critical to securing the support of policy-makers, including law-makers and funding agencies, in the important task of promoting the value of forecasts and warnings as well as other meteorological services. KMA actively involves government leaders in key agenda items to encourage collateral support among different government agencies and to create the widest possible market for value-added meteorological products and services (Kim and Renee, 2005). The Korean government’s initiative on green growth supports the implementation of an early warning system for disaster risk reduction, and extending the application of climate information for energy conservation and renewable energy production. The mass media is helpful in drawing the attention of budget authorities and the national assembly to the need for investment in DPFS, especially during the recovery periods from disastrous impacts of severe weather and climate events.
As a result of promotion and fund raising, meteorological research funds in Korea have increased more than ten-fold since 1999. Funding has doubled every five years, which, coincidentally, matches the upgrade cycles of supercomputers for NWP applications. As a result, research on meteorological observations, forecasting, climate and marine meteorology has significantly expanded.
The flourishing success of Korea’s data processing and forecasting system provides a solid foundation for the advancement of other meteorological components. It has resulted in an increase of funding for applied research areas using models and supercomputers, comprehensive climate change studies, applications in environmental and air quality modelling, water management, energy conservation and geo-engineering.
While acknowledging the successful developments and growth at KMA as a consequence of attributing societal value to products and services that result from investments in DPFS, one caution should be given to those who want to follow the same path: be careful not to over-state the potential benefits. Having oversold the benefits when justifying the purchase of a supercomputer, for several years KMA had to deal with public criticism because the weather forecasts associated with the investment in the supercomputer failed to satisfy the public expectation for 100 per cent accuracy in forecasts, whether in reality or in their perception of the forecast quality.
Summary
The KMA case study demonstrates the value of investments in DPFS in two areas: technical achievements and in economic terms. KMA’s aim is to share its experiences with WMO Members seeking financial support from governments or funding agencies to further develop their own DPFS. KMA, in partnership with the Korea International Cooperation Agency, is ready to collaborate with DPFS centres in developing countries to share their knowledge and expertise.
The WMO Global Data-processing and Forecasting System comprises the data-processing and forecasting systems of all Members and will need to evolve in a way that permits the sharing of forecast data and information to complement individual national data-processing and forecasting systems in to meet national multi-national needs.
References
Hallegatte, S., 2012: Early warning weather systems have very real benefits. http://blogs.worldbank.org/developmenttalk
KMA, 2003: Optimization and effective operation of KMA supercomputer, 202 pp.
Korea Development Institute, 2010: Evaluation report for next-generation numerical weather prediction model development project (in Korean). 199 pp.
Korea Institute of S&T Evaluation and Planning, 2005: A study for implementation of typhoon center and long term research planning (in Korean), 161 pp.
Park, K.-H., 2002: The analysis of economic effects of KISTI supercomputing center. Hankyoung University paper series, 34, 37-45.
Schroter, K., M. Otrowski. C. Velasco, H.P. Nachtnebel, B. Kahl, M. Beyene, C. Rubin, and M. Gocht, 2008: Effectiveness and efficiency of early warning systems for flash-floods (EWASE). First CRUE ERA-NET Common Call – Effectiveness and efficiency of non-structural flood risk management measures, 137 pp.
_________
