by Kim Whitehall1,3, Chris Mattmann1,2,4, Duane Waliser1,2, Jinwon Kim2, Cameron Goodale1, Andrew Hart1, Paul Ramirez1, Paul Zimdars1, Dan Crichton1, Gregory Jenkins3, Colin Jones5, Ghassam Asrar6, Bruce Hewitson7
State, national and international climate assessment reports are important to provide a scientific basis for the understanding and assessment of impacts of climate variability and change on economic sectors such as agriculture and food, water resources, energy and transport. A core element of these assessment reports is future climate projections based on models that not only provide a foretelling of physical indicators of future climate but also indirectly provide information on societal impacts of such changes. Thus they are a key source of science-based information for addressing climate policy, adaptation issues, mitigation planning and risk management strategies.
These quantitative climate projections are usually based on ensembles of multi-component, coupled dynamical global and regional climate models. It is of utmost importance to bring as much observational scrutiny as possible to their simulations of past and present climates, to build confidence in their future projections. While systematic multi-model experimentation and evaluation have been undertaken in many facets for global-scale assessments during the past two decades, for example by the Intergovernmental Panel on Climate Change (IPCC), the development and application of infrastructure for regional-scale models has been somewhat limited due to lack of international coordination. The World Climate Research Programme (WCRP) established the Coordinated Regional Downscaling Experiment (CORDEX; Giorgi et al. 2009, Jones et al. 2011) to facilitate such coordination for regional climate downscaling. One important requirement of the CORDEX framework is to improve access to existing long-term climate quality observations for the evaluation of regional climate projections. A schematic depiction of the roles of models and observations, and the connections between global and regional models, for informing decision-making related to climate impacts is provided in Figure 1.
 |
Figure 1 – Schematic where observations (red) and global and regional models (black) play key roles in quantifying climate impacts and developing climate projections; a process typically carried out from left to right, with the goal of a thoroughly informed decision-making process on the far right. The blue box highlights the focus area of this article |
To meet this CORDEX objective and the high priority regional need, the Jet Propulsion Laboratory, California Institute of Technology (JPL), and their Joint Institute for Regional Earth System Science and Engineering (JIFRESSE) with the University of California, Los Angeles (UCLA), have developed the Regional Climate Model Evaluation System (RCMES). This is one of several regional evaluation activities that WCRP is promoting that targets 10 continental-size focus regions, for example, Africa, S. Asia, Central America, Arctic, Australia. In parallel, the WCRP in partnership with other sister programmes and organizations is establishing a network of regional experts to carry out such evaluations and ultimately assist with interpretation of the scientifically based climate assessments for decision-makers in economic sectors of interest to their region.
The goal of this paper is to introduce RCMES to the broader climate modelling and decision-support communities, recognizing that the Global Framework for Climate Services (GFCS) offers a major opportunity for greater use of RCMES.
Meeting Climate Modelling Needs
Success in remote sensing and climate modelling has led to massive amounts of data being generated at numerous climate/weather observation and modelling centers around the world. These datasets are of various formats, resolutions and coverage. As an example, the total data volume of Coupled Model Intercomparison Project 3 (CMIP3), which supported the IPCC Fourth Assessment Report: Climate Change 2007 (AR4), was 10’s of terabytes, while Coupled Model Intercomparison Project 5 (CMIP5) is expected to be a few petabytes – that is orders of magnitude larger in size and complexity. Similarly, the amount of satellite data, as evidenced by the successful launches of the international Earth Observing System (EOS), grew by nearly two orders of magnitude from 1997 to 2005. Thus, archiving, distributing and processing of these large datasets whilst maintaining data integrity and expedient workflow capabilities have become a major challenge for climate scientists and other users.
Historically, the global climate community has openly shared datasets and the methodologies for handling and analyzing data with each other via numerous portals such as NASA’s Distributed Active Archive Centers (DAACs) for satellite data or the University Corporation for Atmospheric Research (UCAR) Climate Data Guide for many modelling and in-situ based data sets.
Climate scientists have tackled some of the aforementioned challenges by developing systematic frameworks for model experimentation and comparison. Iconic examples are the Atmospheric Model Intercomparison Project (AMIP; Gates et al. 1999) from the early 1990s, followed by the Coupled Model Intercomparison Project (CMIP; Covey et al. 2003). The latter served as the modelling basis for IPCC assessment reports under the auspices of the WCRP. In fact, the success of the early CMIPs demanded a more tailored and high capacity Information Technology (IT) solution to address archiving and distribution needs, particularly starting with CMIP3. The Earth System Grid Federation (ESGF; Williams et al. 2009), sponsored primarily by the US Department of Energy, was adopted and further developed to address these challenges. Having met the terabyte needs of CMIP3, and now in a more distributed and capable fashion, the petabyte needs of CMIP5, it has been deemed a significant success.
Beyond the need for systematic frameworks for model experimentation, and IT capabilities to carry out global climate model intercomparisons and evaluations, is the need for some community standards for evaluation metrics. Having such standard, and community-accepted, metrics are crucial to gauge and track model development as well as provide overall non-biased performance skill measures for the models. While standardized skill metrics have been adopted and utilized by the weather forecast communities for some time, they have only recently been promoted within the climate modelling community. Notable among them is the Madden-Julian Oscillation Working Group (e.g., Waliser et al., 2009), and the Climate Metrics Panel jointly sponsored by the Working Group on Coupled Modelling (WGCM) and Working Group on Numerical Experimentation (WGNE) to define a set of multi-variate, multi-process metrics for climate models (e.g. Gleckler et al. 2008). A primary target for the application of such metrics is the CMIPs.
New Focus on Regional Climate Needs
The resources, capabilities and frameworks that have come together to service the global modelling community bring it closer to its goals of improving the fidelity of climate models and the projections resulting from these models. Thanks to these, users will be provided with better assessments of climate impacts, vulnerabilities and risks to major global economic sectors. However, in response to an urgent request by the participating countries in the UN Framework Convention on Climate Change (UNFCCC) for regional, seasonal to decadal climate information for decision-makers, there are major efforts under way by international research coordination programmes such as the WCRP.
This is most evident in the additions of the decadal and regional climate prediction/projection in the CMIP5 experimental design and in the CORDEX framework. CORDEX provides a common framework through which systematic regional downscaling experiments can follow from the CMIP5 decadal or century scale climate projections. This coordinated framework contributes greatly to the considerable challenge, and multi-step process, of carrying out well-prepared climate projections and translating the resulting information into a format understandable to end-users/decision-makers.
The progress in facilitating the developing, archiving and access to model data has not eliminated the formidable computational resources needed for evaluation and analysis of such data, which typically also involve equally capable resources for the observations and their databases. Given the much higher spatial resolution required, regional model output archiving and dissemination are as demanding as global models, despite the savings from their limited area of coverage. The Earth System Grid Federation has identified this as a major challenge to address in its future activities. Although its nascent capabilities are being applied to the regional problem, there are still key gaps in delivering model-based climate change/projection information to the end-user:
- Having observations and model output in close enough proximity in form/format and accessibility to provide a feasible means for robust model evaluation capabilities from which credible future climate projections/assessments will result, and from which model improvements are expected to emerge;
- As with the global models, there is a need for a robust set of standard model evaluation metrics;
- Acknowledging that the regional problem is strongly tied to decision support issues and questions at a local level – these capabilities need to be available at local levels and not only accessible to the top tier institutes, universities and laboratories. This is an important concern because many of the nations/communities that do not have the institutional capacity to produce or handle large datasets are among the most susceptible to climate variability and change (IPCC, 2007).
The RCMES development effort (see Acknowledgements for support) derives from combined knowledge and expertise in climate science, regional modelling, remote sensing, and a number of information technology areas, including archiving, processing, distributing, and visualizing massive datasets. The overarching objective of RCMES is to provide a science-based, climate information system that is well grounded in state-of-the-art technology capabilities and tools and is usable by a diverse regional community on affordable workstations, with capabilities to compare climate models output with high-quality observation datasets, using existing and/or user-developed computational tools via user-friendly interfaces. This will relieve regional users from major difficulties in climate research and applications by alleviating the need for expensive computational equipment and the computational experts to administer such a system. These are:
- Addressing data format and metadata heterogeneities foremost in observation datasets, but also model datasets;
- Identifying and making available state-of-the-art and most well accepted climate observations, including what are often perceived to be as less tractable satellite datasets;
- Overcoming spatial and temporal differences between datasets through established transformation and re-gridding methods;
- Expediting data access especially during multiple user projects;
- Eliminating the need for large computational resources for archiving and analyzing observation and model datasets.
Such capabilities will greatly facilitate regional community efforts around the globe that are in sore need of information on regional climate change that will aid in their planning and implementing adaptation strategies optimized for their region.
The Regional Climate Model Evaluation System
RCMES is a platform that simplifies the process of jointly using observational and/or model datasets to support regional assessments of climate variability, change and impacts. RCMES provides remote access to available satellite, re-analysis and in-situ datasets stored with similar formats within a single repository, whilst offering basic statistical metrics, data manipulation and visualization capabilities to make their intercomparison easier and user-friendly for scientists and other users. Through these capabilities, RCMES expedites the model evaluation process for climate researchers, by permitting them to spend less time finding, transferring, converting and manipulating data, thus allowing more time for evaluation and analysis. Based on its advanced software architecture (see box), RCMES reduces the institutional computer resources required for climate model evaluation and analysis, facilitates larger community participation in climate research and impact assessments, and increases use of these capabilities especially by scientists and users from developing nations/regions.
At present, RCMES is being utilized for model evaluations associated with the WCRP’s CORDEX programme and the US National Climate Assessment (NCA). CORDEX-Africa and CORDEX-N. America (aka NARCCAP, used for US NCA) have been the first tangible uses of RCMES, with collaborations and users now rapidly developing for CORDEX-S. Asia, CORDEX-E. Asia and CORDEX-Arctic.
IT View of RCMES
RCMES consists of two components, the Regional Climate Model Evaluation Database (RCMED) and the Regional Climate Model Evaluation Toolkit (RCMET) as illustrated in Figure 1. These two components may be viewed as the data server (RCMED), which is installed at a data center that users interact with remotely, and the client component (RCMET), which is installed on the users’ local system and optionally becomes customized by the user.
RCMED is a large, scalable database of observation datasets residing at JPL. It utilizes cloud-computing software, which facilitate efficient data storage and expedite data access. The data in RCMED has been extracted and transformed from the original formats to a common format. Users may query the data in RCMED through a web-service layer based on URLs from their system. The query returns data to their systems that are stored in a compressed format for subsequent use should a similar query be made. Much of the architecture was designed to benefit users with modest Internet speeds. The table opposite provides an excerpt of the datasets currently stored in RCMED.
The RCMET is a collection of Python functions. These functions provide users with input/output, conversion, analysis and services including: loading user-defined data from other sources, re-gridding datasets to spatial and temporal homogeneity, spatial and temporal decomposition of datasets, as well as dataset analyses and visualization. Encapsulated in RCMET are basic common statistical metrics for analysis such as means, bias, standard deviation, and correlations. Visualization of these metrics is also available.
Users are allowed to personalize RCMES for their purposes with regards to workflow, defining subroutines for increased functionality and defining various visualization options. The RCMET component is also able to service a diverse computer-literate community through a web-based graphical user interface whereby the Python functions are available via Bottle services (a Python web Framework) or accessed via command line and scripting mechanisms.
The RCMES team is working with JPL and NASA to make RCMET (and RCMES) open source software, thus users are encouraged to create and share their personalized functions with the community, using the open source software for NASA as a guide.
Datasets available in RCMED
Figure 2 – Example of a bias analysis using RCMES for annual-mean daily maximum temperature from 1989 to 2006. The observation dataset is CRU surface station analysis (left) and the model data are from the climate hindcast using the Second-generation Regional Climate Model (RACMO2) of Royal Dutch Meteorological Institute (KNMI) driven by the large-scale forcing data derived from the ERA-Interim reanalysis (right).
Targeting Climate and Social Scientists
RCMES objectives include a commitment to providing a tool that can be applied without pre-defined approaches. Thus far, communities of users have utilized RCMES in two very distinct ways. Firstly as a data management and model evaluation tool, for example in regional projects involving CORDEX, such as CORDEX Africa and NARCCAP - the N. American component of CORDEX. The RCMES team is also participating in three more CORDEX regions including the Arctic, South Asia, and East Asia. To date this has included attending the organizational workshops and developing collaborations and RCMES improvements that would further enhance its utilities for the given CORDEX region. Regional RCMES users report that the key advantages of using RCMES are the accessibility to various observation datasets, the ease of inserting their own data, availability of built-in re-gridding functions for datasets (local or remote), and general ease of use.
Secondly, RCMES has been used as a metric and visualization tool in science-based environmental assessment projects. The users of RCMES in this regard report key advantages in the ability to access multiple observation datasets, visualization of data, basic model evaluation metrics and ease of use. Through the Climate and Development Knowledge Network (CDKN) project, we are exploring the use and integration of RCMES with the University of Cape Town, South Africa, and its Climate Information Portal (CIP). RCMES has also been introduced to groups such as Leadership for Environment and Development (LEAD) in Africa, as a means of aiding assessments.
The Way Forward
The next steps for RCMES involve incorporating: (1) additional observation datasets – those suited for model evaluation as well as those relevant to new decision-support systems; (2) additional methods for quantifying statistically the robustness of the analyses results; (3) functions and visualization capabilities that are commonly used in climate and environmental Geographical Information System applications; and (4) promoting its greater use in other regional activities that will emerge as a part of the Global Framework for Climate Services. Arrangements are also underway make RCMES available through the Earth System Grid Federation.
The RCMES team looks forward to continued and enhanced engagement with CORDEX and, where possible, to providing climate analysis resources and even a training platform to the Global Framework for Climate Services through the WMO network of National Meteorological and Hydrological Services (NMHSs). Through wider collaboration, and the team’s efforts to make RCMES open source software, capabilities will grow.
(Please visit http://rcmes.jpl.nasa.gov for capability updates, collaborative activities, and application outcomes. We welcome your suggestions and feedback.)
Acknowledgements
Support provided by NASA’s Earth Sciences Division, NASA NCA (ID: 11-NCA11-0028), AIST (ID: AIST-QRS-12-0002) and the Applied Sciences Program via the American Recovery and Reinvestment Act (ARRA), and the National Science Foundation ExArch program (ID: 1125798), a component of the G8 initiative. Valuable contributions to the RCMES activity by way of collaboration comes from the WCRP Coordinated Regional Climate Downscaling Experiment (CORDEX), the North American Regional Climate Change Assessment Program (NARCCAP), the Climate & Development Knowledge Network (CDKN), and the University of Cape Town, and PCMDI/DOE through support of the obs4MIPs activity.
References
Covey, C. et al., 2003: An Overview of Results from the Coupled Model Intercomparison Project (CMIP). Global and Planetary Change, 37, 103-133.
Gates, W.L. et al., 1999: An Overview of the Results of the Atmospheric Model Intercomparison Project (AMIP I). Bulletin of the American Meteorological Society, 80, 29–55.
Giorgi, F., C. Jones and G. R. Asrar, 2009: Addressing climate information needs at the regional level: the CORDEX framework. WMO Bulletin, 58, 3, 175 – 183.
Gleckler, P. J., K. E. Taylor, and C. Doutriaux, 2008: Performance metrics for climate models, Journal of Geophysical Research-Atmospheres, 113(D6), D06104.
Hart, A., C. Goodale, C. Mattmann, P. Zimdars, D. Crichton, P. Lean, J. Kim, and D. Waliser. A Cloud-Enabled Regional Climate Model Evaluation System. In Proceedings of the ICSE 2011 Workshop on Software Engineering for Cloud Computing - SECLOUD, Honolulu, HI, May 22, 2011.
IPCC, 2007: Climate Change 2007: The Physical Science Basis, edited by S. Solomon, D. Qin, M. Manning, Z. Chen, M. Marquis, K.B. Averyt, M. Tignor, H.L. Miller. Cambridge University Press, Cambridge, United Kingdom.
Jones C., F. Giorgi and G. Asrar, 2011: The Coordinated Regional Downscaling Experiment: CORDEX An international downscaling link to CMIP5. CLIVAR Exchanges No.56, Vol. 16, 34-40
Waliser, D. et al., 2009: MJO Simulation Diagnostics, Journal of Climate, 22(11), 3006-3030.
Williams, D.N. et al., 2009: The Earth System Grid: Enabling Access to Multi-Model Climate Simulation Data. Bulletin of the American Meteorological Society, 90, 195-205.
____________
1 Jet Propulsion Laboratory, California Institute of Technology
2 UCLA Joint Institute for Regional Earth System Science Engineering
4 University of Southern California
5 Swedish Meteorological and Hydrological Institute
6 World Climate Research Program, WMO
