2011 Workshop:Modeling Challenge

From CedarWiki
Jump to: navigation, search

CEDAR-GEM Modeling Challenge

Location, Date/Time and Duration

6 hours

1) Climatology on Tuesday June 28, 2011, Convention Center O'Keefe/Milagro, 0130-0330 PM

2-3) GEM-CEDAR Challenge, Metrics, Planning, etc on Thursday June 30, 2001, Eldorado Anasazi, 10-12AM and 0130-0330PM

Conveners (First is CEDAR convener, Second is GEM convener, the rest are co-conveners)

Ja Soon ShimMasha Kuznetsova
Barbara Emery
Aaron Ridley
Delores Knipp
Tim Fuller-Rowell
Timothy B Guild
Jan Sojka
Naomi Maruyama
Geoff Crowley

Format of the Workshop

Short Presentations and Discussions

Estimated attendance


Justification for the workshop

The joint CEDAR/GEM workshop is the best opportunity to facilitate collaboration between Ionosphere/Thermosphere and Magnetosphere communities in order to better understand key properties of the coupling of the magnetosphere and ionosphere/thermosphere.

Requested Specific Days

Two sessions focused on series of events less than 3 days each should follow one after another (Contacts: M. Kuznetsova (GEM), J. Shim (CEDAR)). One session focused on time period longer than 3 months (Climatology projects) can be scheduled at another time during the workshop (Contacts: B. Emery (CEDAR), T. Guild (GEM).

Special technology requests


Modelers, data providers and users of space weather models are invited to participate in the CEDAR-GEM Modeling Challenge Workshop (Tuesday, June 28 1:30 pm - 3:30 pm, and Thursday, June 30 10 am - 3:30 pm) during the 2011 Joint CEDAR-GEM Workshop in Santa Fe. Details of the Challenge and instructions on how to prepare and submit model output time series and can be found at CCMC, GEM and CEDAR Web sites:

http://cedarweb.hao.ucar.edu/wiki/index.php/2011_Workshop:Modeling_Challenge and http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/

The CEDAR-GEM Challenge is built upon GEM GGCM and CEDAR ETI Challenges. During the Workshop, GEM and CEDAR communities will share the experiences and lessons learned from the first rounds of the Challenges, address topics of common interest and analyze the effects of geospace model coupling on metrics results.

Both CEDAR and GEM communities have recognized that due to the maturity and increasing complexity of state-of-the-art space weather models, there is a great need for a systematic and quantitative evaluation of different modeling approaches. During the last two years both GEM and CEDAR communities addressed this need by organizing and implementing comprehensive, community-wide efforts to test model predictions against observations. In the summer of 2008 the GEM GGCM Metrics and Validation Focus Group initiated a series of metrics studies (aka GGCM Modeling Challenge) focusing on the inner magnetospheric dynamics and ground magnetic field perturbations. A year later the CEDAR community initiated the IT modeling challenge called CEDAR Electrodynamics Thermosphere Ionosphere (ETI) Challenge. The goal of the two Challenges is to evaluate the current state of the space physics modeling capability, to facilitate interaction between research and operation communities in developing metrics for space weather models, to address the differences between various modeling approaches, to track model improvements over time, to facilitate collaboration among modelers, data providers and research communities, and provide feedback for further model improvement. The Community Coordinating Modeling Center (CCMC) is supporting both Challenges and maintaining a web site with interactive access to model output archive and observational data used for metrics studies. In anticipation of the joint GEM-CEDAR Workshop five time intervals were included in lists of events addressed by both GEM and CEDAR Modeling Challenges:

  • December 14, 2006 12:00 UT - December 16, 00:00 UT (06348-349)
  • August 31, 2001 00:00 UT - September 1, 00:00 UT (01243)
  • August 31, 2005 10:00 UT - September 1, 12:00 UT (05243-244)
  • May 15, 2005 00:00 UT - May 15, 2005, 20:00 UT (05135)
  • July 9, 2005 00:00 UT - July 12, 2005, 00:00 UT (05190-192)
  • (a 6th event was DELETED): October 29th, 2003 06:00 UT - October 30th, 06:00 UT (03302-303)

Through collaboration between CEDAR and GEM Communities, by analyzing simulation results for ionosphere/thermosphere and magnetosphere models for the same set of events, we can analyze the effects of the geospace environment on the ionosphere. Many of the magnetospheric models are coupled to ionosphere-thermosphere models, so that the metrics can be conducted for both coupled and uncoupled simulations on both models.

Modelers are invited to submit simulation results for the 5 events listed above before June 10th 2011 through the interactive submission interface http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/. Model output file formats for each physical parameter and measurement availability table with satellite trajectories and ground stations locations can be found at http://hickam.ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/fileformat.php, http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/measurements.php, http://ccmc.gsfc.nasa.gov/challenges/GEM-CEDAR/measurements-mag

Submissions of coupled magnetosphere-ionosphere models are especially encouraged. To study the effect of different drivers, ionosphere/thermosphere modelers are requested to perform a series of simulations for each event with different models for the ionosphere potential pattern: 1) Weimer 2005 using 15-min averages of the IMF input parameters lagged -5 to -20 min provided by the NCAR and the CCMC; 2) AMIE provided by ASTRA (Geoff Crowley); 3) Global magnetosphere models provided by the CCMC. Please contact Barbara Emery or Masha Kuznetsova if you need assistance to get ionospheric potentials from AMIE and/or global magnetosphere models. List of physical parameters to be used for metrics studies:

Ionosphere/Thermosphere models or coupled model components:

  • Vertical and horizontal drifts at Jicamarca (VperpN and VperpE)
  • Neutral density at CHAMP orbit (Nden)
  • Electron density at CHAMP orbit (Eden)
  • NmF2 from LEO satellites (CHAMP and COSMIC) and ISRs
  • HmF2 from LEO satellites (CHAMP and COSMIC) and ISRs
  • Temperature Tn and neutral winds obtained by Fabry-Perot Spectrometer at 250 km (Arrival Heights, Antarctica; Resolute Bay, Canada)
  • Ne, Te, Ti at 300 km (Millstone Hill, Sondrestrom, EISCAT, Svalbard ISRs).
  • Ion vertical velocity at Sondrestrom ISR
  • Some plots of the ground-based data sets suggested are in this pdf.

Geospace models or coupled model components:

  • Magnetic field at geosynchronous orbit
  • Ground magnetic perturbations
  • Dst index
  • Auroral oval position (high latitude boundary)
  • Auroral oval position (low latitude boundary)

Parameters along DMSP tracks:

  • Poynting flux (Joule heating) into ionosphere along DMSP tracks
  • Plasma Velocity (Vx - along track, Vy cross track, Vz - vertical)

Additional time series in support of simulation results analysis:

  • Cross polar cap potential (northern and southern hemisphere)
  • Joule heating (or Poynting flux) integrated over each hemisphere in GW.

We propose the following topics to be addressed during 3 sessions that include presentations and discussions:

Agenda Thursday June 30 (4 hours) (Contacts: M. Kuznetsova (GEM), J. Shim (CEDAR))

10am - noon

  1. (~1 h) Review of the first round of GEM and CEDAR Challenges results. Simulations results, observational data and reports on metrics studies for the first round of Challenges are available at the CCMC website. Comments and discussions are welcomed. Results of the first round of Challenges will be used as a benchmark for further studies.
    1. Introduction (M. Kuznetsova)
    2. From first GEM metrics studies to operational geospace model selection. Lessons learned and recent advances (A. Pulkkinen).
    3. Highlights of the CEDAR ETI Challenge results and recent advances (J-S. Shim)
  2. (~1 h) Effects of IT/geospace models coupling on metrics results.
    1. Poynting flux into the ionosphere (D. Knipp, L. Rastaetter)
    2. Comparison of different approaches to introducing geomagnetic activity effects into ionosphere models (A. Ridley, N. Maruyama)

1:30pm - 3:30pm

  1. (~30 h) Dst index Challenge summary (L. Rastaetter, D. Welling)
  2. (~1h) Challenges of the model-data comparison and how to address them
    1. Uncertainty analysis of model outputs (R. Schunk)
    2. How to define the equatorward boundary of the auroral oval from simulations (Y. Zheng)
    3. What metrics to apply and how to calculate skill score for global (e.g., 2D time dependent) observational data. TEC metrics studies planning (B. Emery, L. Goncharenko, A. Coster)
      1. 2011 Fall AGU Poster by Shim et al. on December 2006 storm TEC will use 5 deg glat, 5 deg glon/20 min UT bins of TEC using only 5 longitudes chosen from MIT and JPL analyses of land GPS TEC (may include also COSMIC and Jason as available in the longitude sectors)
  3. (~30m) General discussion on GEM-CEDAR Modeling Challenges. Planning of future activities

Session on Climatology Projects (2 hours Tues June 28 130-330 PM)

Contacts: B. Emery (CEDAR), T. Guild (GEM).

Ionosphere-Thermosphere (IT, 1h)

The Climatology session will focus on long time intervals of 3 months or longer, where seasonal variations (~3 months) are of particular interest in the Ionosphere-Thermosphere (IT) community. The year of ISR observations (March 2007 – March 2008) was selected for the Climatology Challenge at the first CEDAR ETI Challenge Workshop in the summer of 2009. Simulation results for this time period for all IT models residing at CCMC will be available before the workshop. Another tme interval of interest is the year 2005 that is adopted for the Dst metrics study. Analysis of long time intervals is also of interest to the GEM inner magnetosphere community for ring current and radiation belt studies. The CCMC is running global magnetosphere MHD models in realtime. The simulations results are archived and can be used for the climatology project. In support of this joint GEM-CEDAR climatology project CCMC is developing on-line long timelines analysis tool. Additional data and modeling results from the inner magnetosphere include climatological ring current model runs, data-based empirical models of inner magentospheric plasma, and long-term average properties in the plasma sheet.

Possible useful data sets for data comparisons in the IT system are neutral density estimates at perigee from selected satellites analyzed by Bruce Bowman that are available for many years. Ionospheric climatology sets include hmF2 and NmF2 from many ground stations, as well as global medians from COSMIC binned by Alan Burns over 8-month solar minimum solstices (e.g. Oct06-Feb07 and Oct07-Feb08). Initial climatological studies focused on the 2005 or 2007-2008 periods will be shown, as well as discussions of suitable variables to check against.


Suggested Data Sets for IT Climatology The median (not average) data and model results should be plotted to see baseline shifts. The delta differences should be plotted to see better trends, etc. Storm times and subsequent negative and positive storm phases should be removed in studies.

  • Neutral Density
    • John Emmert's (NRL) global daily density at 250, 300, 550 km
    • Bruce Bowman's density at individual satellite perigees
    • CHAMP in-situ latitude medians(?) (2 versions have 10% differences) relative variations
    • GRACE in-situ orbital medians
  • Low-latitude Viz
    • Rocsat
  • Te, Ti, Ne, Vi
    • ISR during IPY (Mar07-Mar08)
    • ISRIM - models for several ISR stations by Shunrong Zhang (MIT)
  • NmF2 and hmF2
    • COSMIC 4-month seasonal medians from 0.5-4 hour, or even from 9-15LT (6h) for dayside and nightside
  • TEC for 2011 Fall AGU Poster by Emery et al. will use the IPY period (March 2007 to March 2008) ~2-month medians over 5 deg glat and 15 deg glon/1 hr UT bins for 5 longitudes (American, European, Japanese sectors) same as for the Dec06 storm period.
    • GPS land (~2000 stations in 2007 in MIT analysis from Anthea Coster in madrigal DB, may have JPL analysis)
    • COSMIC (starts late 2006, USU analysis from Ludger Scherliess, and possibly NCAR analysis from Alan Burns)
    • Jason (ocean, since Topex is 1992-2003)

Agreed Upon Longitude Slices for TEC or NmF2 Data As of September 12, 2011, we agreed upon 8 5-deg geographic longitude slices for any possible 36 5-deg latitude bins from -90 to +90 (e.g. -90 to -85) along these slices for a possible total of 288 lat/lon bins. The longitude slices are (from 0-360):

  1. 25-30
  2. 90-95
  3. 140-145
  4. 175-180
  5. 200-205 (-160 to -155)
  6. 250-255 (-110 to -105)
  7. 285-290 (-75 to -70)
  8. 345-350 (-15 to -10)

For the 06348-349 (Dec 14-15, 2006) storm, median 15-min values will be found from the data sets (e.g. 0000-0014:59:59) for GPS TEC. For COSMIC hmF2 and NmF2, the longitude slices are expanded +/5 degrees on either side for 15-deg longitude bins for better statistics.

The experimenters will provide the median value, the number of observations making up this median value, and the standard deviation (of the mean) of these observations to represent an 'error bar'. Weighted TEC means can also be provided from MIT where the error bar is composed of 3 major parts: 1) 24-h UT bias for each location, 2) number of points in each bin, and 3) elevation angles for converting slant TEC to vertical TEC. Missing values will show zero number of observations in the bin with some negative (unrealistic) number for the median value (probably -999). The missing standard deviation is also negative. Probably median TECs with standard deviations more than 4 TECU will not be used. Other periods may have an obvious 24-h bias problem which will not be used.

The climatology study was revised from 4 months 07305-08060 (Nov 1 2007 to Feb 29 2008) to +/-30 days around December 21 solstice (07355) to avoid the change-over from winter to summer in late February, and to also avoid any stratospheric warming effects. There were 4 stratwarms in 2008 in the Northern Hemisphere in Jan-Feb, where the first stratwarm maximized on January 23 in the stratosphere, with later effects in the ionosphere. The new proposed period, which could be changed, is Nov 21 2007 to Jan 20 2008 (07325-08020) or 61 days.

However, the data will be the same for the climatology as for the December 2006 run except instead of 15 min median values, we use 1-h median values (e.g. 0000-0059:59:59). The other change is to expand the COSMIC longitudes to 25 degrees (+/-10 degrees from the longitudes above). Medians of the 61 days will be used probably omitting bins with large standard deviations, or 1 data point.

Results of the Fall AGU (see below) suggested the data should be divided into low Kp (<=1 and Vsw<=450 km/s for slow-speed periods) and moderate Kp (>=2 and Vsw>=5 for HSS periods) since the global neutral densities and TEC showed peaks during the HSS periods. There are 25 days in each period where:

  1. 25 HSS days: 07325-330, 345-346, 351-357, 370-373 (08005-8), 379-384 (08014-19)
  2. 25 Low Kp days: 07332-343, 349-350, 358-360, 362-364, 366-369 (08001-4), 376 (08011)

Further Climatology Presentations

1) Mini-GEM at 2011 Fall AGU in San Francisco Sunday December 4

The climatology presentation by Emery et al. had some errors (LT/UT mix-up in IRI for NmF2 and hmF2) so a revised version of this talk and AGU poster is below.

2) 13th International Symposium on Equatorial Aeronomy (ISEA13) in Paracas, Peru, March 12-16, 2012.

Ring Current, Radiation Belts, and the Plasma Sheet (1h)


  • Katie Garcia (BU) Magnetopause Validation (Based on Garcia and Hughes, Finding the LFM Magnetopause, a Statistical Perspective, JGR, 2007)
  • an open discussion of climatological data sets and reanalyses one can use to drive long model runs. (15 minutes)

Workshop Summary

This is where the final summary workshop report will be.

Presentation Resources

Upload presentation and link to it here. Links to other resources.

Upload Files Here

  • Add links to your presentations here, including agendas, that are uploaded above. Please add bullets to separate talks. See further information on how to upload a file and link to it.

Other Workshops