Timed:DB Tutorial for TIMED

From CedarWiki
Jump to: navigation, search

Contents

CEDAR DB Tutorial for TIMED

Examples used: Sonde ISR Dec 2001, Poker Flat 4 Channel Photometer Dec 2001, CSU Lidar Jan 2002, Inuvik green line (~97 km) FPS Feb 2002 As of 2/19/02, all example data have summary plots but Sonde, and all but Inuvik data are accessible. However, the CSU lidar data organization will be altered soon. The Inuvik FPS data are awaiting recent cloud cover data, an important quality parameter, although these data are also available at another web site.

The 2002 Geophysical Calendar with global ISR committed dates is found at:
http://www.ises-spaceweather.org (Click on Geophysical Calendar)
The schedules for the ISR Coordinated World Days for 1998-2002 is at
http://www.eiscat.uit.no/URSI_ISWG
The 2002 Coordinated ISR dates start with the SELT campaign in March.

I) Starting Out

  1. Sign in on the [/index.html CEDAR main page]
  2. Along the left hand side lick on 'Data Services' From here you can either select 'Get/Plot Data' or 'Browse Plots'.

Recommend looking at summary plots first in order to see the documentation, and summary plots, which are a quick way to find out what data are in the Database (DB). You do not need to login to browse the catalog, look at the documentation web page, or the plots. Web pages of the instrumentor are linked in the summary plot web pages also, which can provide additional information. Also check to see what is "New data", find latest data available for the GBI's of TIMED.

II) See what is available

  1. Without login:
    1. Open the 'Data Services' folder and then on 'New data' or
      Click on 'Get/Plot Data', or 'Browse Plots', 'Browse Inventory'
    2. As of 2/15/02, there are 7 TIMED/GBI examples, 4 during TIMED, excluding the HF radars that self-serve, and also the MF radars, which are served by Scott Palo, and will be in the DB with some delay.
    3. Can click on Sonde ISR, Poker Flat 4 Channel Photometer, Inuvik FPS, or CSU lidar here and continue on as in IIIC3 or IIID3.
  2. Login:
    1. If this is the first time you have invoked your internet browser, you are required to enter your User ID (webname - often your last name in lower case), and your Password (bd$radec is the initial password). Then Click 'OK'. If this is not the first time, you will not be asked.
    2. Open the 'Data Services' folder and click on 'Get/Plot data'.
    3. Click on 'Select a date'. This brings up a selection for the year.
    4. As of 2/19/02, there are CSU lidar 2002 data officially in the DB, and there are plots of Feb 2002 Inuvik FPS data. Click on 2001 or 2002. This brings up a montly calendar. Click on December 2001 or January 2002. Jan 2002 shows Jan 11, 12 and 27, which are CSU lidar nights.
    5. As of 2/19/02, the two names are '80-SON-Sondrestrom I.S. Radar' and '4470-P4P-Poker Flat 4 Channel Photometer' for Dec 2001, and '6320-CSL-Colorado State Sodium LIDAR' for Jan 2002.
    6. Select a year and click 'Proceed'
    7. Select a month and click 'Proceed'
    8. Select a starting day and enter a number for the amount of days you want to cover.
    9. A list of instruments with available data will be displayed. Select an instrument and click 'Proceed' (e.g. 6320 - CSL Colorado State Sodium Lidar)
    10. The next screen displays the links for the data in a variety of return formats.
    11. Click on 'TAB' to return a tab-delimited ascii version of the data to the browser window.
    12. Optionally, you may click on 'Record Type' to select specific parameters, i.e. subset the data, and retrieve them. This is the best way to see exactly what is in the DB since 'What's New?' could be out of date, although the calendar lags behing the summary plots sometimes since the plots are usually done earlier to find any data problems.

III) See documentation and summary plots for an overview, also all data if wish:

  1. Open the 'Data Services' folder
  2. Click on Browse Plots
  3. Click on name of location of interest on the map, e.g. Poker Flat or Sondrestrom or Inuvik.

  1. Poker Flat 4 Channel Photometer (p4p 4470) example:
    1. Poker Flat has an MST Radar (defunct), a 4 Channel Photometer (Jim Hecht's GBI), and (soon) an imaging FPI (Mark Conde's GBI, now red line ~250 km)
    2. Click on 4 Channel Photometer
    3. This leads to the summary plot web page for Jim Hecht's 4 Channel Photometer at Poker Flat. He also has a very nice web page of nightly data plots for all nights. The technique requires low background light (night, low or no moonlight, clear or few clouds) and the presence of aurora. If there are no aurora, bad clouds, or high levels of background light, then the data are not sent to the CEDAR DB. If the conditions are OK, but sort of marginal, a quality flag is set to 0 (OK), while if the conditions for the night are generally good, this quality flag is set to 1 (good). ALL nights are plotted automatically in the Poker Flat web site, so check the CEDAR DB to confirm which nights are actually good data. There is a lag getting the data into the CEDAR DB, but Oct-Dec 2001 data are plotted and labelled 'good' or 'OK'. The 4 Channel Photometer at Fort Yukon also has its own web page of plots that appear the day after the data is taken, but the data are still in the process of calibration and so have not been submitted to the CEDAR DB.
    4. Much of the header documentation is listed on the DB web page, except for parameter code definitions, which must be obtained from the written header documentation (see IVD2b). There are also nightly summary plots, besides each is an opportunity to get ALL the data for that night. This is about 1 MB in ascii TAB form, so it is not bad to get it this way. If you get it this way, the first time you ask, you need to give your webname and password (see IIB2).
    5. There is a web site of the plots for the Poker Flat Photometer at http://gedds.pfrr.alaska.edu/aerospace/pokerflatdata where data from the night before are shown. However, much of the data is bad (no aurora, high background, bad clouds), and the background for each filter has been changed from the default on this web page to the data within the CEDAR DB.

  1. Sonde ISR (son 80) example:
    1. Sondrestrom has a digisonde, FPI, and ISR, and hopefully a lidar later. The lidar and other instruments are there, but not in the DB. The ISR is a TIMED/GBI (Jeff Thayer).
    2. Click on Incoherent Scatter Radar
    3. This leads to the summary plot web page for the Sonde ISR. There are several web links. A valuable link is the link to the schedule:
      http://isr.sri.com/radar.html
      Click there, and get to the ISR page and various yearly pdf documents of data taken, and projected schedules for the this and the next month. Check them out! The 2001 list shows data taken in Dec to be on Dec 6, 6-7, 9, 9-10, 10-14, 14-15, 16-17, but the only data in the CEDAR DB as of 2/08/02 is Dec 10-14, the World Days. Therefor, you must go to SRI for any of the data from the other days, which may or may not be in a suitable mode for comparisons. Approximately 30% of the Sonde ISR data is in the CEDAR DB. The rest is at SRI.
    4. Much of the header documentation is listed on the DB web page, except for parameter code definitions, which must be obtained from the written header documentation (see IVD2b). There are also 2-day summary plots, besides each is an opportunity to get ALL the data for those 2 days. If you choose to get the data this way, the first time you ask, you need to give your webname and password (see IIB2).
    5. NOTE: As of 2/13/02, the most recent Sonde ISR campaigns are not in the Sonde summary plot web page.
    6. In addition to these summary plots, ISR radars sometimes do MLTCS (Mesosphere-Lower Thermosphere Coupling Studies). The first ISR campaign for this is actually called SELT (Storm Effects in the Lower Thermosphere) and is a window between March 20-29, 2002. See the Millstone Hill web page on this and click on CEDAR-TIMED Campaigns, with past data at 'LTCS Campaigns'. The LTCS information can also be found on CedarWEB by clicking on 'Data Services' along the left hand side, then 'Browse plots' and scrolling to the bottom under 'Campaigns'.

  1. Inuvik FPS (ikf 5510) example:
    1. Only the FPS is at Inuvik, so this leads to the summary plot page. As of 02/19/02, the plots are up in the CEDAR DB, but not the data. There is a web link to: http://gedds.pfrr.alaska.edu/inuvik_FPS/default.htm This leads to plots and data files for days since Jan 30, 2002, up to 2 nights before the present. Most of these days are also plotted in the CEDAR DB web page for Inuvik. If you want data immediately, get it from their web page. The order is: YYMMDD, UThrfrac*100, azimuth (0=N, 90=E), elevation (0=horiz,90=vertical), Wn(m/s), Error in Wn, calculated emission (not used), error in calculated emission (not used), observed relative emission, error in observed relative emission, Tn (K), error in Tn.
    2. Much of the header documentation is listed on the DB web page, except for parameter code definitions, which must be obtained from the written header documentation (see IVD2b). There are also nightly summary plots, besides each is an opportunity to get ALL the data for that night. If you get it this way, the first time you ask, you need to give your webname and password (see IIB2).

  1. CSU Lidar (csl 6320) example:
    1. Ft Collins hosts the CSU lidar, and has hosted the MTM imager. Click on the lidar to get to the summary plot page.
    2. As of 02/19/02, the plots and data are available for Jan 2002, but the data are being re-organized so that al the winds and temperatures are in 1 file (kindat=18011).
    3. Much of the header documentation is listed on the DB web page, except for parameter code definitions, which must be obtained from the written header documentation (see IVD2b). There are also nightly summary plots, besides each is an opportunity to get ALL the data for that night. If you get it this way, the first time you ask, you need to give your webname and password (see IIB2).

IV) Get data and documentation:

  1. Open the 'Data Services' folder.
  2. Either click on 'Select a date' to see what is available and get to IIB7, and procede as in IVC1.
  3. Alternatively, click on 'Select an instrument', then select Instrument of interest such as '80-SON-Sondrestrom I.S. Radar', '4470 - P4P - Poker Flat 4 Channel Photometer' or '6630 - CSL - Colorado State University Lidar'. This list is in instrument order (ISR, indices, model outputs, HF radars, MST/MF/LF/meteor radars, digisonde ion drifts, spectrometers/spectrographs/spectrophotometers, photometers, FPIs, MIs, lidars, and imagers)
    1. Select year, month, start day and duration (clicking 'Proceed' between each stage. Choices are based on what is available in the catalog. See IIB3.

  1. Poker Flat 4 Channel Photometer (P4P 4470) example:
    1. As of 2/19/02, P4P shows data in Jan, Apr, Oct-Dec 2001.
    2. The Jan data is the standard MSIS profile. Get this in order to judge properly the calculated fo ratio between an [O] obs/model and this MSIS [O] profile.
      1. Click on 'Jan', shows Jan 31 is model day number for MSIS. NOW, fill out starting date and ending date. Only need month, day, and year. If you click on Jan 31 on calendar, this sets the start day. If you put '1' in the number of days and click calculate, this sets the end day to Feb 1. If you put in '0', you get start/end Jan 31.
      2. Click on 'Go' This shows a page where you can 'Get the data ..' with default 'TAB' option. This is an ascii table similar to CEDAR DB ascii format.
      3. Click on the button for TAB to see what other formats you can get. The most useful of these is INFO, which gets you the documentation about this data set. Another good option is OPeNDAP, which if your browser is correctly configured (email cedar_db@hao.ucar.edu for details or see the Tools folder), you can get OPeNDAP binary, which can be converted into many different formats by your configured browser (e.g. netCDF, HDF, CDF, etc). For now, choose INFO, and then 'Get the data' to get the documentation.
      4. This leads to a URL address. Click on it and get the header information about the MSIS profile. Now capture this using the top 'File' button and 'Save as'. You will also get the documentation for the data records if you have not restricted the date_time, but these are small records and you need them also to understand the data you will get.
      5. Now use the 'Back' button, and the 'Reload' button and 'Repost' to get to the 'Get the data' page. You should find the default TAB again, so click on 'Get the data'.
      6. This leads to a URL address. Click on it and get the data itself. (NOTE: Could have got here by changing the URL. Alter 'INFO' to 'TAB' Now capture this using the top 'File' button and 'Save as'. This is ONLY the MSIS profile IF you have chosen the start/end date correctly in step (a), but will be all the data in 2001 if you fail to define the start/end date properly!!! Since each night is about 1 MB of data, and there are or will be 53 nights of good P4P data in 2001 in the DB, it is wise to limit the nights you wish to get.
    3. Now use 'Back', and click 'Modify Query'. You most likely want to keep the instrument, but change the Date/Time. Alter the start/end dates by typing in new months/days, and duration. Then repeat the steps as in 3) to get the MSIS profile, but get the data values as a function of time instead.

  1. Sonde ISR (SON 80) example:
    1. As of 2/13/02, SON shows data in Feb, Apr, Jul, Sep, Oct, Nov and Dec 2001. As explained in IIID3, this is only about 30% of the data at SRI. Clicking on Dec shows data Dec 10-14, the World Day after TIMED launched.
    2. Set up starting/ending dates as Dec 10, 2001 and Dec 14, 2001.
    3. Click now on 'Record_Type' to see what types of data runs are available The result is 4 data types, where perhaps the most interesting are the 2 types with Te and Ti electron and ion temperatures.
    4. Click on 1 of the Record_Types with Te and Ti data before doing this. (The probable difference between the 2 Record types could be pulse length.) It is smart to choose a record type before you get the data, since you will otherwise get data you do not want such as altitudes for the power profiles (record type 9001) Therefore, if you have not chosen a record type, click on 'modify query' to do so. You can only choose 1 record type, not 2, so choose one of the Te,Ti record types (5501 and 5521). You could find out the difference by clicking on one, clicking on 'Go', and changeing to 'INFO'. To get the other one, use 'Back' etc to modify the query. Capture or print out these descriptions. They happen to be identical except for the kindat (record_type) and the volume name (son011210a or b). Therefore, you must get the data from each to confirm that they differ by the length of the pulse. And indeed, the pulse length is 20 for 5521, and is 320 for 5501.
    5. Click on 'Parameters' and select several parameters like: -560 (Te error bar), -550 (Ti error bar), 110 (altitude), 111 (increment to altitude), 132 and 133 (Beginning and ending azimuth angles), 142-143 (beginning and ending elevation angles), 160 and 170 (glat and glon of obs), 402 (pulse length - an important ISR parameter since short pulses are better for lower altitudes and long pulses are better for upper altitudes), 550 and 560 (Ti and Te temperatures). This selection gives you only a portion of the data.
    6. Click on 'Get the data ...' with TAB. We find that the pulse length for 5501 on volume a is 320 (320*1.e-6 sec -- known from INFO), with 21 heights from 97-1050 , while the pulse length for 5521 on volume b is 20 (20*1.e-6 sec from INFO), with 45 altitudes from 69-555 km. Obviously, the shorter pulse length is more appropriate for the lower altitudes and vice versa. The data can be saved with 'File' and 'Save As'. Note that missing values are -32767 integers. All the output is integer and must be understood in terms of the parameter code definitions found in INFO.

  1. Inuvik FPS (IKF 5510) example:
    1. As of 02/19/02, only the plots are up on the CEDAR DB, but not the data. There is a web link to: http://gedds.pfrr.alaska.edu/inuvik_FPS/default.htm This leads to plots and data files for days since Jan 30, 2002, up to 2 nights before the present. All but the latest of these days are also plotted in the CEDAR DB web page for Inuvik. If you want data immediately, get it from their web page. The order is: YYMMDD, UThrfrac*100, azimuth (0=N, 90=E), elevation (0=horiz,90=vertical), Wn(m/s), Error in Wn, calculated emission (not used), error in calculated emission (not used), observed relative emission, error in observed relative emission, Tn (K), error in Tn.

  1. CSU Lidar (csl 6320) example:
    1. As of 02/19/02, the plots and the data are up for Jan 2002. Proceed as in previous examples. The organization of the data will be changed soon to have an hourly file with vector winds and temperatures (kindat 18011). To get only this, choose Record_Type, and click on 6320 - 18011 when it becomes available. Then enter the dates and proceed as in IVD or IVE.

--Revised 05 Oct 2004 by emery@ucar.edu