Daily check list » History » Version 84

« Previous - Version 84/96 (diff) - Next » - Current version
Kathy Romer, 10/18/2017 06:58 PM

Run Manager's daily check list

Before 2pm

(Not all of this happens in practice, but things in bold definitely need to done every day.)

[added by Kathy R 10/17 - IMHO, other than the email about 4pm meeting, most of this could be left till 3pm....]

  1. Send out an email to to say whether the 4pm meeting is happening or not (this may have already been decided at the previous 4pm meeting, but it is still worth sending out that email).
  2. Perform Observer 1 and 2 End Of Night Checklists if the observers didn't do it (see their instructions).
  3. Write the Synopsis_Night logbook entry if the observers didn't do it.
  4. Publish the nightsum in the morning if the observers didn't do it.
  5. Complete CTIO end of night report if the observers didn't do it (if you had two operators, make sure both names appear). Check the calendar to see the status of the night reports.
  6. Be aware of our observers arrival to and departure from the mountain.
  7. Prepare the nightly observing plan page, including populating Almanac and Weather info (go to the Shifters page (link for Y5) and scroll to - and then click on - today's date.) Top tip: populate the new page with the contents from the previous night, and then edit. That way you won't forget the info to include.
  8. Check the instrument status (read the logbook, talk to people)
  9. Check the DM status
  10. Check SISPI status
  11. Check what scripts are needed/present for calibration
  12. Take a first look at the DESDM data quality results table Anything unusual should be discussed at the 4pm meeting. Typically you won't accept the results until after the 4pm meeting.
  13. Check diskspace. Log into one of the observer machines as user "sispi" (password on white board) and execute the commands below. If any disks are filling up (>90% for observer2, >93% for the pipeline machines) send an email to Klaus and Ann ( and )

added by Kathy R 10/17: I didn't have any success logging in using su sispi from observer3 but ssh -X sispi@observer3 did work

setup SISPI

Late afternoon

  1. Before the 4pm meeting check with telops if SISPI instance and the GUIs were restarted. You could also do this by checking if they completed a "2 PM Checklist" in the eLog, or looking at the "Uptime" field in the Architect Console GUI.
  2. Perform the aTmCam afternoon check
  3. Attend the 4pm meeting (this is usually chaired by someone at Fermilab, but occasionally you might have to do it)
  4. Edit the observing plan (you should have already started it, see "before 2pm" list above). Differences in the plan do occur from night to night: these are usually discussed at the 4pm meeting.
  5. Check that telops have started the afternoon calibration flat fielding scripts (see below for an overview of what they have to do).
  6. Perform the Late Afternoon Checklist in the logbook. See below, for some of the information that is asked for in the logbook, and for instructions to get onto that checklist.
  7. check the SN processing status [for this you need a password that would be the first 3 letters of your Services username + docdbpw (the inaccurate info about how many CCDs) ]
  8. Implement the data quality (= "fill out the Survey Table") If you didn't have a 4pm meeting, then you need to check with DESDM before doing the implementation if you think any of their true/false flags are erroneous. (Note that it is OK to implement the data quality later in the evening/night if you didn't do it in the afternoon.)
  9. Make sure observers are present and roles are defined/understood
Examples of things you will enter into the Late Afternoon Checklist in the logbook
  • names of the observers
  • the "SISPI Instance": The format is DECam_YYYYMMDD
  • name of the script used to perform the afternoon calibrations (most likely this will be prenight_zeros_flats.json)
  • confirmation that today's afternoon calibration has been completed
How to get to Late Afternoon Checklist in the logbook
  • Click on this link" (N.B. this works at CTIO even from laptops, but not from elsewhere)
  • login as DECamObserver (password on white board)
  • select New Entry from left side bar, and a new window pops up.
  • select "Afternoon Check List" from the Form menu and "DECam Logbook" from the Category menu. Click on "Summary" in Tags window.
  • after you've filled it in, press post (button at the end)
Overview of how calibrations are done (these will be done by Telops, don't attempt them yourself!)
  • Verify Dome is Closed
  • Verify Dome Flat Setup (ask Telops) Dome Flat setup
  • Turn VSUB on
  • Submit Afternoon Calibration Script (/user/sispi/decam/ExposureScripts/DECam/prenight_zeros_flats.json):
Overview of how calibrations are checked (this should be done by Observer2, but you might have to do it yourself)
  • Check a few exposures on the Comfort Display. Is the focal plane complete?
  • When script is done, in kentools (on observer2), perform "flatCheck"


  1. Make sure the observers understand what they are doing
  2. Make sure the observers get off to a good start
  3. Periodically check wind speed (TelOPS will lower/raise screens if needed)
  4. Be available to the observers if they have questions or problems
  5. Try to get some sleep