2011 Workshop:Modeling Challenge
CEDAR-GEM Modeling Challenge
Location, Date/Time and Duration
1) Climatology on Tuesday June 28, 2011, Convention Center O'Keefe/Milagro, 0130-0330 PM
2-3) GEM-CEDAR Challenge, Metrics, Planning, etc on Thursday June 30, 2001, Eldorado Anasazi, 10-12AM and 0130-0330PM
Conveners (First is CEDAR convener, Second is GEM convener, the rest are co-conveners)
Format of the Workshop
Short Presentations and Discussions
Justification for the workshop
The joint CEDAR/GEM workshop is the best opportunity to facilitate collaboration between Ionosphere/Thermosphere and Magnetosphere communities in order to better understand key properties of the coupling of the magnetosphere and ionosphere/thermosphere.
Requested Specific Days
Two sessions focused on series of events less than 3 days each should follow one after another (Contacts: M. Kuznetsova (GEM), J. Shim (CEDAR)). One session focused on time period longer than 3 months (Climatology projects) can be scheduled at another time during the workshop (Contacts: B. Emery (CEDAR), T. Guild (GEM).
Special technology requests
Modelers, data providers and users of space weather models are invited to participate in the CEDAR-GEM Modeling Challenge Workshop (Tuesday, June 28 1:30 pm - 3:30 pm, and Thursday, June 30 10 am - 3:30 pm) during the 2011 Joint CEDAR-GEM Workshop in Santa Fe. Details of the Challenge and instructions on how to prepare and submit model output time series and can be found at CCMC, GEM and CEDAR Web sites:
The CEDAR-GEM Challenge is built upon GEM GGCM and CEDAR ETI Challenges. During the Workshop, GEM and CEDAR communities will share the experiences and lessons learned from the first rounds of the Challenges, address topics of common interest and analyze the effects of geospace model coupling on metrics results.
Both CEDAR and GEM communities have recognized that due to the maturity and increasing complexity of state-of-the-art space weather models, there is a great need for a systematic and quantitative evaluation of different modeling approaches. During the last two years both GEM and CEDAR communities addressed this need by organizing and implementing comprehensive, community-wide efforts to test model predictions against observations. In the summer of 2008 the GEM GGCM Metrics and Validation Focus Group initiated a series of metrics studies (aka GGCM Modeling Challenge) focusing on the inner magnetospheric dynamics and ground magnetic field perturbations. A year later the CEDAR community initiated the IT modeling challenge called CEDAR Electrodynamics Thermosphere Ionosphere (ETI) Challenge. The goal of the two Challenges is to evaluate the current state of the space physics modeling capability, to facilitate interaction between research and operation communities in developing metrics for space weather models, to address the differences between various modeling approaches, to track model improvements over time, to facilitate collaboration among modelers, data providers and research communities, and provide feedback for further model improvement. The Community Coordinating Modeling Center (CCMC) is supporting both Challenges and maintaining a web site with interactive access to model output archive and observational data used for metrics studies. In anticipation of the joint GEM-CEDAR Workshop five time intervals were included in lists of events addressed by both GEM and CEDAR Modeling Challenges:
- December 14, 2006 12:00 UT - December 16, 00:00 UT (06348-349)
- August 31, 2001 00:00 UT - September 1, 00:00 UT (01243)
- August 31, 2005 10:00 UT - September 1, 12:00 UT (05243-244)
- May 15, 2005 00:00 UT - May 15, 2005, 20:00 UT (05135)
- July 9, 2005 00:00 UT - July 12, 2005, 00:00 UT (05190-192)
- (a 6th event was DELETED): October 29th, 2003 06:00 UT - October 30th, 06:00 UT (03302-303)
Through collaboration between CEDAR and GEM Communities, by analyzing simulation results for ionosphere/thermosphere and magnetosphere models for the same set of events, we can analyze the effects of the geospace environment on the ionosphere. Many of the magnetospheric models are coupled to ionosphere-thermosphere models, so that the metrics can be conducted for both coupled and uncoupled simulations on both models.
Modelers are invited to submit simulation results for the 5 events listed above before June 10th 2011 through the interactive submission interface http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/. Model output file formats for each physical parameter and measurement availability table with satellite trajectories and ground stations locations can be found at http://hickam.ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/fileformat.php, http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/measurements.php, http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/measurements-mag
Submissions of coupled magnetosphere-ionosphere models are especially encouraged. To study the effect of different drivers, ionosphere/thermosphere modelers are requested to perform a series of simulations for each event with different models for the ionosphere potential pattern: 1) Weimer 2005 using 15-min averages of the IMF input parameters lagged -5 to -20 min provided by the NCAR and the CCMC; 2) AMIE provided by ASTRA (Geoff Crowley); 3) Global magnetosphere models provided by the CCMC. Please contact Barbara Emery or Masha Kuznetsova if you need assistance to get ionospheric potentials from AMIE and/or global magnetosphere models. List of physical parameters to be used for metrics studies:
Ionosphere/Thermosphere models or coupled model components:
- Vertical and horizontal drifts at Jicamarca (VperpN and VperpE)
- Neutral density at CHAMP orbit (Nden)
- Electron density at CHAMP orbit (Eden)
- NmF2 from LEO satellites (CHAMP and COSMIC) and ISRs
- HmF2 from LEO satellites (CHAMP and COSMIC) and ISRs
- Temperature Tn and neutral winds obtained by Fabry-Perot Spectrometer at 250 km (Arrival Heights, Antarctica; Resolute Bay, Canada)
- Ne, Te, Ti at 300 km (Millstone Hill, Sondrestrom, EISCAT, Svalbard ISRs).
- Ion vertical velocity at Sondrestrom ISR
- Some plots of the ground-based data sets suggested are in this pdf.
Geospace models or coupled model components:
- Magnetic field at geosynchronous orbit
- Ground magnetic perturbations
- Dst index
- Auroral oval position (high latitude boundary)
- Auroral oval position (low latitude boundary)
Parameters along DMSP tracks:
- Poynting flux (Joule heating) into ionosphere along DMSP tracks
- Plasma Velocity (Vx - along track, Vy cross track, Vz - vertical)
Additional time series in support of simulation results analysis:
- Cross polar cap potential (northern and southern hemisphere)
- Joule heating (or Poynting flux) integrated over each hemisphere in GW.
We propose the following topics to be addressed during 3 sessions that include presentations and discussions:
Agenda Thursday June 30 (4 hours) (Contacts: M. Kuznetsova (GEM), J. Shim (CEDAR))
10am - noon
- (~1 h) Review of the first round of GEM and CEDAR Challenges results. Simulations results, observational data and reports on metrics studies for the first round of Challenges are available at the CCMC website. Comments and discussions are welcomed. Results of the first round of Challenges will be used as a benchmark for further studies.
- Introduction (M. Kuznetsova)
- From first GEM metrics studies to operational geospace model selection. Lessons learned and recent advances (A. Pulkkinen).
- Highlights of the CEDAR ETI Challenge results and recent advances (J-S. Shim)
- (~1 h) Effects of IT/geospace models coupling on metrics results.
- Poynting flux into the ionosphere (D. Knipp, L. Rastaetter)
- Comparison of different approaches to introducing geomagnetic activity effects into ionosphere models (A. Ridley, N. Maruyama)
1:30pm - 3:30pm
- (~30 h) Dst index Challenge summary (L. Rastaetter, D. Welling)
- (~1h) Challenges of the model-data comparison and how to address them
- Uncertainty analysis of model outputs (R. Schunk)
- How to define the equatorward boundary of the auroral oval from simulations (Y. Zheng)
- What metrics to apply and how to calculate skill score for global (e.g., 2D time dependent) observational data. TEC metrics studies planning (B. Emery, L. Goncharenko, A. Coster)
- 2011 Fall AGU Poster by Shim et al. on December 2006 storm TEC will use 5 deg glat, 5 deg glon/20 min UT bins of TEC using only 5 longitudes chosen from MIT and JPL analyses of land GPS TEC (may include also COSMIC and Jason as available in the longitude sectors)
- (~30m) General discussion on GEM-CEDAR Modeling Challenges. Planning of future activities
Session on Climatology Projects (2 hours Tues June 28 130-330 PM)
Contacts: B. Emery (CEDAR), T. Guild (GEM).
Ionosphere-Thermosphere (IT, 1h)
The Climatology session will focus on long time intervals of 3 months or longer, where seasonal variations (~3 months) are of particular interest in the Ionosphere-Thermosphere (IT) community. The year of ISR observations (March 2007 – March 2008) was selected for the Climatology Challenge at the first CEDAR ETI Challenge Workshop in the summer of 2009. Simulation results for this time period for all IT models residing at CCMC will be available before the workshop. Another tme interval of interest is the year 2005 that is adopted for the Dst metrics study. Analysis of long time intervals is also of interest to the GEM inner magnetosphere community for ring current and radiation belt studies. The CCMC is running global magnetosphere MHD models in realtime. The simulations results are archived and can be used for the climatology project. In support of this joint GEM-CEDAR climatology project CCMC is developing on-line long timelines analysis tool. Additional data and modeling results from the inner magnetosphere include climatological ring current model runs, data-based empirical models of inner magentospheric plasma, and long-term average properties in the plasma sheet.
Possible useful data sets for data comparisons in the IT system are neutral density estimates at perigee from selected satellites analyzed by Bruce Bowman that are available for many years. Ionospheric climatology sets include hmF2 and NmF2 from many ground stations, as well as global medians from COSMIC binned by Alan Burns over 8-month solar minimum solstices (e.g. Oct06-Feb07 and Oct07-Feb08). Initial climatological studies focused on the 2005 or 2007-2008 periods will be shown, as well as discussions of suitable variables to check against.
- 8min - Liying Qian and Stan Solomon: Model-Data Comparison of Thermospheric Neutral Density
- 8min - Mariangel Fedrizzi and CTIPe NOAA team: CHAMP/CTIPe Neutral Density Comparisons from 2005 to 2009
- 8min - Wenbin Wang, Alan Burns, and the NCAR TIE-GCM team: The NCAR TIE-GCM: Model Description, Development, and Validation, or COSMIC NmF2 and hmF2 climatology
- 8min - Ludger Scherliess, Rod Heelis, and USU GAIM team: Assimilation of COSMIC DATA into GAIM-Physics-Based Data Assimilation Model (GAIM-FP): IT climatology at low and mid-latitudes during solar minimum
- 5min - Bela Fejer: Equatorial Vertical Plasma Drifts from ROCSAT During Low Solar Flux Periods (not given, but presentation is included)
- 5min - Shunrong Zhang and John Holt: Ionospheric Climatology from Incoherent Scatter Radar (ISR) Observations: A Quick Overview
- 8min - Ja Soon Shim: Long time line analysis tool (using CTIPe and TIEGCM ionosphere year-long runs as an example)
- 7min - General discussion of what data sets to use for a climatology Challenge next year
Suggested Data Sets for IT Climatology The median (not average) data and model results should be plotted to see baseline shifts. The delta differences should be plotted to see better trends, etc. Storm times and subsequent negative and positive storm phases should be removed in studies.
- Neutral Density
- John Emmert's (NRL) global daily density at 250, 300, 550 km
- Bruce Bowman's density at individual satellite perigees
- CHAMP in-situ latitude medians(?) (2 versions have 10% differences) relative variations
- GRACE in-situ orbital medians
- Low-latitude Viz
- Te, Ti, Ne, Vi
- ISR during IPY (Mar07-Mar08)
- ISRIM - models for several ISR stations by Shunrong Zhang (MIT)
- NmF2 and hmF2
- COSMIC 4-month seasonal medians from 0.5-4 hour, or even from 9-15LT (6h) for dayside and nightside
- TEC for 2011 Fall AGU Poster by Emery et al. will use the IPY period (March 2007 to March 2008) ~2-month medians over 5 deg glat and 15 deg glon/1 hr UT bins for 5 longitudes (American, European, Japanese sectors) same as for the Dec06 storm period.
- GPS land (~2000 stations in 2007 in MIT analysis from Anthea Coster in madrigal DB, may have JPL analysis)
- COSMIC (starts late 2006, USU analysis from Ludger Scherliess, and possibly NCAR analysis from Alan Burns)
- Jason (ocean, since Topex is 1992-2003)
Agreed Upon Longitude Slices for TEC or NmF2 Data As of September 12, 2011, we agreed upon 8 5-deg geographic longitude slices for any possible 36 5-deg latitude bins from -90 to +90 (e.g. -90 to -85) along these slices for a possible total of 288 lat/lon bins. The longitude slices are (from 0-360):
- 200-205 (-160 to -155)
- 250-255 (-110 to -105)
- 285-290 (-75 to -70)
- 345-350 (-15 to -10)
For the 06348-349 (Dec 14-15, 2006) storm, median 15-min values will be found from the data sets (e.g. 0000-0014:59:59) for GPS TEC. For COSMIC hmF2 and NmF2, the longitude slices are expanded +/5 degrees on either side for 15-deg longitude bins for better statistics.
The experimenters will provide the median value, the number of observations making up this median value, and the standard deviation (of the mean) of these observations to represent an 'error bar'. Weighted TEC means can also be provided from MIT where the error bar is composed of 3 major parts: 1) 24-h UT bias for each location, 2) number of points in each bin, and 3) elevation angles for converting slant TEC to vertical TEC. Missing values will show zero number of observations in the bin with some negative (unrealistic) number for the median value (probably -999). The missing standard deviation is also negative. Probably median TECs with standard deviations more than 4 TECU will not be used. Other periods may have an obvious 24-h bias problem which will not be used.
The climatology study was revised from 4 months 07305-08060 (Nov 1 2007 to Feb 29 2008) to +/-30 days around December 21 solstice (07355) to avoid the change-over from winter to summer in late February, and to also avoid any stratospheric warming effects. There were 4 stratwarms in 2008 in the Northern Hemisphere in Jan-Feb, where the first stratwarm maximized on January 23 in the stratosphere, with later effects in the ionosphere. The new proposed period, which could be changed, is Nov 21 2007 to Jan 20 2008 (07325-08020) or 61 days.
However, the data will be the same for the climatology as for the December 2006 run except instead of 15 min median values, we use 1-h median values (e.g. 0000-0059:59:59). The other change is to expand the COSMIC longitudes to 25 degrees (+/-10 degrees from the longitudes above). Medians of the 61 days will be used probably omitting bins with large standard deviations, or 1 data point.
Results of the Fall AGU (see below) suggested the data should be divided into low Kp (<=1 and Vsw<=450 km/s for slow-speed periods) and moderate Kp (>=2 and Vsw>=5 for HSS periods) since the global neutral densities and TEC showed peaks during the HSS periods. There are 25 days in each period where:
- 25 HSS days: 07325-330, 345-346, 351-357, 370-373 (08005-8), 379-384 (08014-19)
- 25 Low Kp days: 07332-343, 349-350, 358-360, 362-364, 366-369 (08001-4), 376 (08011)
Further Climatology Presentations
1) Mini-GEM at 2011 Fall AGU in San Francisco Sunday December 4
The climatology presentation by Emery et al. had some errors (LT/UT mix-up in IRI for NmF2 and hmF2) so a revised version of this talk and AGU poster is below.
- Revised SA41B-1856: Systematic Climatology Assessment of Ionosphere/Thermosphere Models During November 2007 to January 2008 by Emery et al.
2) 13th International Symposium on Equatorial Aeronomy (ISEA13) in Paracas, Peru, March 12-16, 2012.
- Revised talk on "Low and Mid-Latitude Climatology Assessment of Ionosphere/Thermosphere Models During Solar Minimum" by Emery et al.
Ring Current, Radiation Belts, and the Plasma Sheet (1h)
- Bob Weigel: Gleaning physical insight from impulse response functions with the example of the missing semiannual variation in the 'am' geomagnetic index. (not given, but presentation is included)
- Katie Garcia (BU) Magnetopause Validation (Based on Garcia and Hughes, Finding the LFM Magnetopause, a Statistical Perspective, JGR, 2007)
- Mike Liemohn and Roxanne Katus (U MI) Initial Results from the Continuous HEIDI Simulations (of the ring current)
- Lutz Rastaetter and Hyesook Lee (CCMC) Data Archived from CCMC Real-Time Simulations
- Paul O'Brien (Aerospace Corp) What we can learn about the radiation belts from simulating a "climatological" interval. (not given, but presentation is included)
- an open discussion of climatological data sets and reanalyses one can use to drive long model runs. (15 minutes)
This is where the final summary workshop report will be.
Upload presentation and link to it here. Links to other resources.
- Add links to your presentations here, including agendas, that are uploaded above. Please add bullets to separate talks. See further information on how to upload a file and link to it.
- http://cedarweb.hao.ucar.edu/wiki/index.php/2009_Workshop:CEDAR_ETI_Challenge 2009 CEDAR Workshop
- http://cedarweb.hao.ucar.edu/wiki/index.php/2010_Workshop:CEDAR_ETI_Challenge 2010 CEDAR Workshop
- This document for the 2011 Joint CEDAR-GEM Workshop
- See above for the 2011 Mini-GEM Workshop before the 2011 Fall AGU