2011 Workshop:DBs in CEDAR
Using, Administering, and Submitting Data to the VSTO and Madrigal Databases within the CEDAR Community
Location, Date/Time and Duration
Altitudes: IT - Latitudes: global - Inst/Model: radar - Other: data from all MLT/IT instruments plus models
Format of the Workshop
short presentations plus hands-on and discussion about future plans
Requested Specific Days
Special technology requests
internet access and possible need for extra laptops for hands-on by participants
The VSTO and Madrigal database systems within the CEDAR community are trying to merge so that data in either system are available to the other system, with both access modalities retained. Both systems are also thinking about newer formats so this is a good time to bring these plans to the community to get their feedback.
The Madrigal and CEDAR databases have been around a long time, but have undergone a number of recent developments. Madrigal was originally developed at MIT Haystack Observatory (http://madrigal.haystack.mit.edu/madrigal/) to allow access to upper atmospheric realtime or archived data, but for the next release Jicamarca Radio Observatory is a co-developer. Madrigal is a local database that automatically shares metadata, allowing each site to act like a virtual observatory. The CEDAR DB (http://cedarweb.hao.ucar.edu) was developed at NCAR for incoherent scatter radar data as well as optical and other data, and stores all data locally, allowing central archiving. Both the CEDAR DB and Madrigal presently share the CEDAR database format based on 16 bit integers. Madrigal was installed at NCAR in 2009 (http://madrigal.hao.ucar.edu), with the capability of automatically downloading and archiving data from all other Madrigal sites, making it unnecessary to submit data separately to Madrigal and the CEDAR DB. Madrigal delivers data as ASCII flat files or in any variant of the CEDAR database format via a web interface or via application program interfaces (API's) in python, Matlab, or IDL. The CEDAR data are delivered via the on-line Virtual Solar Terrestrial Observatory (VSTO) system. The data can be retrieved in ASCII files, in netCDFv3 files, and in binary form using the openDAP protocol.
This workshop will feature discussions of future directions in community database design and database formats. Madrigal is exploring HDFv5 while VSTO is exploring netCDFv4 (.nc) with the goal of ingesting and serving NCAR model output (TIE-GCM, TIME-GCM, WACCM) in their native netCDF format. Both formats should be able to be supported especially since netCDFv4 uses HDFv5 as a storage layer. Our goal is to merge the Madrigal and CEDAR systems while allowing access via Madrigal or VSTO. We also plan to showcase examples of how new instruments' data are put into either system, and accessed via either system, or across systems. Attendees are encouraged to discuss ways in which the utility of these systems can be enhanced.
- Highlights of the CEDAR Database - Barbara Emery (HAO/NCAR)
- What's new with the Madrigal database? - Bill Rideout (MIT Haystack Observatory
- How to use the Madrigal database for atmospheric science - Bill Rideout (MIT Haystack Observatory (67 page tutorial on using Madrigal)
This is where the final summary workshop report will be.
Upload presentation and link to it here. Links to other resources.
- Add links to your presentations here, including agendas, that are uploaded above. Please add bullets to separate talks. See further information on how to upload a file and link to it.