Data rescue and archives

Data rescue and archives

Data repositories and archives play a critical role as the source for the observational data used in the study of weather and climate. After over two centuries of recording observations on physical media and the last 20 years on digital media, these records are at risk.

All across the globe, countries hold valuable weather records – on aging paper, microfilm and outdated digital media – that are at risk. These observations could bolster the climate record and allow a greater understanding of climate. Countries must identify, appraise, archive and migrate these records to new technologies to make them available for research. The conversion of observation data to common electronic formats is vital to preserving and enhancing the global climate record.

Creating homogeneous, complete data sets from disparate collections is a fundamental challenge facing the climate research community. This is best achieved by leveraging partnerships to share resources and areas of expertise. The goal is to bring more information into the hands of scientists who seek to understand the global climate system in order to anticipate changes to the climate.

The U.S. Climate Database Modernization Program (CDMP) converted more than fifty million observational records to digital image formats such as PDF, TIFF and JPEG and supported the keying of hundreds of millions of weather elements into digital datasets. As a result , hourly observations for long-running stations in the U.S. are available in image formats as well as keyed data files, and the U.S. has a dense coverage of data that dates to the 19th century. - Rescue, Archival and Stewardship of Weather Records and DataWMO Bulletin Vol. 64(1), 2015

Data Rescue Projects

The recovery of climate records from paper and obsolete electronic media is an ongoing task for WMO, National Meteorological and Hydrological Services and other institutions. Updating media storage is always challenging as technology evolves rapidly. At the national level, considerable efforts continue to recover and capture historical climate records and to make them available in database systems.

INDARE

Recovery projects have recently been launched or completed in Africa, Asia, the Caribbean, the Mediterranean, Indian Ocean rim countries, Small Island Developing States of the Pacific,  and in South America. In addition to imaging and digitizing massive volumes of historical climate records,  these projects permit the production and update of climatological standard normals in the World Weather Records as well as the update of climate indices and many other products useful for application in agriculture, water resources, health and disaster risk reduction.

The INdian Ocean DAta REscue (INDARE) project covers Indian Ocean rim countries and islands: Kenya, Madagascar, Maldives, Mauritius, Mozambique, Myanmar, Pakistan, Seychelles, Sri Lanka, Tanzania and Yemen. INDARE activities contribute to four broad areas: modernization of the database, capacity development, climate information generation and support implementation of the Global Framework for Climate Services (GFCS). The WMO-IOC Joint Technical Commission for Oceanography and Marine Meteorology (JCOMM), the International Atmospheric Circulation Reconstructions over the Earth (ACRE) Initiative, the International Environmental Data Rescue Organization (IEDRO), the Met Office (UK), the Climate Change Centre (Spain) and the Indian Ocean Commission (COI) are partnering in the project. The aim is to address climate data recovery in this important, yet poorly studied and understood, region.

 

IDARE Portal

International Data Rescue Initiative

The International Data Rescue Initiative (I-DARE), launched by WMO in May 2017, provides guidance on data rescue and climate data management specifications; an international portal as a single global infrastructure for providing information on rescued and to-be rescued archives; and resource plans that include reference on key activities and funding requirements  at country level, in the regions and globally

World Data Centres

Each of the six World Data Centres of the WMO Global Atmosphere Watch (GAW) is responsible for archiving one or more atmospheric measurement parameters or measurement types. They are operated and maintained by individual host institutions. They collect, document and archive atmospheric measurements and the associated metadata from measurement stations worldwide and make these data freely available to the scientific community. In some cases, World Data Centres also provide additional products including data analyses, maps of data distributions, and data summaries.