Project

General

Profile

History

Introduction to the nightsum ยป

Summary tools

Common instructions

The summary tools are part of the nightsum product, which is installed on the SISPI nodes at CTIO (observer2.ctio.noao.edu, for example) and the DES cluster at Fermilab (des20.fnal.gov, for example). To run any of these programs, you need to log into a computer on which nightsum is installed and prepare the environment using eups (at CTIO) or UPS (at Fermilab):

bash$ setup nightsum

Help strings can be found for any of the programs here using the -h or --help arguments. For example:

bash$ progress --help
usage: progress [-h] [-d PATH] [-c CONFIG] [-l LOG] [-r] [-v] [-D DATE]

Generate tables and reports on survey progress

optional arguments:
  -h, --help            show this help message and exit
  -d PATH, --path PATH  the directory in which to write
  -c CONFIG, --config CONFIG
                        the name of the configuration file
  -l LOG, --log LOG     the name of the logfile
  -r, --regenerate      remake the page based on previously collected files
  -v, --verbose         be verbose with logging
  -D DATE, --date DATE  the start date of the night, local time
bash$

All dates given to programs in nightsum should be provided this format: YYYY-MM-DDTHH:mm:SSZ.

Progress

Purpose

The progress tool generates progress plots and data. The data table, written to the file accum_time.txt, which has a format that looks like this:

mjd     accum_exptime_hours     accum_sched_hours
56535.9720486111        0.45    0.0
56536.4211458333        3.35    10.7783333333333
56536.9724074074        3.35    10.7783333333333
56537.4203587963        5.275   21.5291666666667

This file uses a tab as a field separator. The mjd column marks time samples at the start and end of observing for each scheduled night. The accum_exptime_hours column gives the total hours of exposure time for all wide survey exposures not currently declared bad. The accum_sched_hours column gives the accumulated hours of scheduled DES time prior the the MJD.

progress generates three plots, accum_time_plot, progress_by_lst_hist, and hex_completion_map all of which are in both png and pdf format.

accum_time_plot shows the accumulated exposure time (of wide survey exposures not declared bad) by date in black, with a baseline (based on elapsed scheduled time and a nominal rate of 0.32 exposure time hours per scheduled hour; see DES docdb 6563) in green:

progress_by_lst_hist is a histogram of numbers of exposures by the local sidereal time at which they set. The blue represents exposures complete on the night as "current" in generating the plot; the green, exposures completed in nights prior to the "current" night; and the red, exposure not complete on that night or prior. This exposure time total includes only DES wide-survey exposures not declared "bad". Here is an example:

Instructions

To generate the progress plots and data, setup nightsum following the instrunctions in Common instructions above, then:

bash$ progress

With no arguments, progress generates the output for the current date and writes the files to the current directory. Either or both can be changed using the --path and --date arguments:

bash$ progress --path /my/favorite/dir --date 2013-10-27T10:00:00Z

Time usage

Purpose

Execution output

The primary output of time_usage is table of how time was used in a given time period (for example over a month):

Total nights: 14
Total full nights: 14
Total half nights: 0
Total open shutter time: 78.660278
Total gap times: 46.044024
Total start offset from nautical twilight: 8.549536
Mean start offset from nautical twilight: 0.610681
Total end offset from midpoint: 0.000000
Total end offset from nautical twilight: -1.676889
Mean end offset from nautical twilight: -0.119778
Total edge time: -2.205980

In this table, times are expressed in units of hours.

Long gap table

time_usage creates a file, long_gaps.txt, which lists instances where there were longs gaps between exposures, ordered with the longest gaps first. The purpose of this file is to help identify instances where there were problems. The file starts like this:

ExpID   Start time              duration
246054  2013-10-25 01:13:13Z    20675
242508  2013-10-10 06:55:29Z    2504
247254  2013-10-27 05:34:09Z    2321
247258  2013-10-27 06:20:57Z    2002
244123  2013-10-14 23:34:58Z    1434
240527  2013-10-02 04:43:17Z    686
242364  2013-10-09 23:45:14Z    559
246092  2013-10-25 08:50:48Z    549

The duration is in seconds. Note the first entry in the above example: there were clouds between 1:00 UT and 6:50 UT, during which time the observers took decal images. time_usage identified the corresponding long gap between science exposures.

Plots

The time_usage program generates several plots, including:

  • gap_hist.pdf, a histogram of gaps between exposures, as found in the night summary but including all exposures in a specified range of dates.
  • open_shutter_time_hist.pdf, a histogram of the total open shutter time by night.
  • start_time_hist.pdf, a histogram of when the first science exposures started relative to twilight, by night.

Instructions

To generate the time_usage plots and data, setup nightsum following the instrunctions in Common instructions above, then:

bash$ time_usage <START DATE> <END DATE>

Where <START DATE> and <END DATE> are the first and last nights in the date range to analyze. By default, time_usage places the results in a new directory named based on the date. For example:

bash$ time_usage 2013-10-01 2013-10-27

will create a directory called time_usage_2013-10-01_to_2013-10-27 and put all of its output files there. The output directory may be overridden with the --dir option.