Summary tools¶
Common instructions¶
The summary tools are part of the nightsum
product, which is installed on the SISPI nodes at CTIO (observer2.ctio.noao.edu, for example) and the DES cluster at Fermilab (des20.fnal.gov, for example). To run any of these programs, you need to log into a computer on which nightsum
is installed and prepare the environment using eups (at CTIO) or UPS (at Fermilab):
bash$ setup nightsum
Help strings can be found for any of the programs here using the -h
or --help
arguments. For example:
bash$ progress --help usage: progress [-h] [-d PATH] [-c CONFIG] [-l LOG] [-r] [-v] [-D DATE] Generate tables and reports on survey progress optional arguments: -h, --help show this help message and exit -d PATH, --path PATH the directory in which to write -c CONFIG, --config CONFIG the name of the configuration file -l LOG, --log LOG the name of the logfile -r, --regenerate remake the page based on previously collected files -v, --verbose be verbose with logging -D DATE, --date DATE the start date of the night, local time bash$
All dates given to programs in nightsum
should be provided this format: YYYY-MM-DDTHH:mm:SSZ
.
Progress¶
Purpose¶
The progress
tool generates progress plots and data. The data table, written to the file accum_time.txt
, which has a format that looks like this:
mjd accum_exptime_hours accum_sched_hours 56535.9720486111 0.45 0.0 56536.4211458333 3.35 10.7783333333333 56536.9724074074 3.35 10.7783333333333 56537.4203587963 5.275 21.5291666666667
This file uses a tab as a field separator. The mjd
column marks time samples at the start and end of observing for each scheduled night. The accum_exptime_hours
column gives the total hours of exposure time for all wide survey exposures not currently declared bad. The accum_sched_hours
column gives the accumulated hours of scheduled DES time prior the the MJD.
progress
generates three plots, accum_time_plot
, progress_by_lst_hist
, and hex_completion_map
all of which are in both png
and pdf
format.
accum_time_plot
shows the accumulated exposure time (of wide survey exposures not declared bad) by date in black, with a baseline (based on elapsed scheduled time and a nominal rate of 0.32 exposure time hours per scheduled hour; see DES docdb 6563) in green:
progress_by_lst_hist
is a histogram of numbers of exposures by the local sidereal time at which they set. The blue represents exposures complete on the night as "current" in generating the plot; the green, exposures completed in nights prior to the "current" night; and the red, exposure not complete on that night or prior. This exposure time total includes only DES wide-survey exposures not declared "bad". Here is an example:
Instructions¶
To generate the progress
plots and data, setup nightsum
following the instrunctions in Common instructions above, then:
bash$ progress
With no arguments, progress
generates the output for the current date and writes the files to the current directory. Either or both can be changed using the --path
and --date
arguments:
bash$ progress --path /my/favorite/dir --date 2013-10-27T10:00:00Z
Time usage¶
Purpose¶
Execution output¶
The primary output of time_usage
is table of how time was used in a given time period (for example over a month):
Total nights: 14 Total full nights: 14 Total half nights: 0 Total open shutter time: 78.660278 Total gap times: 46.044024 Total start offset from nautical twilight: 8.549536 Mean start offset from nautical twilight: 0.610681 Total end offset from midpoint: 0.000000 Total end offset from nautical twilight: -1.676889 Mean end offset from nautical twilight: -0.119778 Total edge time: -2.205980
In this table, times are expressed in units of hours.
Long gap table¶
time_usage
creates a file, long_gaps.txt
, which lists instances where there were longs gaps between exposures, ordered with the longest gaps first. The purpose of this file is to help identify instances where there were problems. The file starts like this:
ExpID Start time duration 246054 2013-10-25 01:13:13Z 20675 242508 2013-10-10 06:55:29Z 2504 247254 2013-10-27 05:34:09Z 2321 247258 2013-10-27 06:20:57Z 2002 244123 2013-10-14 23:34:58Z 1434 240527 2013-10-02 04:43:17Z 686 242364 2013-10-09 23:45:14Z 559 246092 2013-10-25 08:50:48Z 549
The duration is in seconds. Note the first entry in the above example: there were clouds between 1:00 UT and 6:50 UT, during which time the observers took decal images. time_usage
identified the corresponding long gap between science exposures.
Plots¶
The time_usage
program generates several plots, including:
gap_hist.pdf
, a histogram of gaps between exposures, as found in the night summary but including all exposures in a specified range of dates.open_shutter_time_hist.pdf
, a histogram of the total open shutter time by night.start_time_hist.pdf
, a histogram of when the first science exposures started relative to twilight, by night.
Instructions¶
To generate the time_usage
plots and data, setup nightsum
following the instrunctions in Common instructions above, then:
bash$ time_usage <START DATE> <END DATE>
Where <START DATE>
and <END DATE>
are the first and last nights in the date range to analyze. By default, time_usage
places the results in a new directory named based on the date. For example:
bash$ time_usage 2013-10-01 2013-10-27
will create a directory called time_usage_2013-10-01_to_2013-10-27
and put all of its output files there. The output directory may be overridden with the --dir
option.