Project

General

Profile

Daily check list » History » Version 55

« Previous - Version 55/96 (diff) - Next » - Current version
Kathy Romer, 10/17/2017 02:42 PM


Run Manager's daily check list

Before 2pm

  1. Perform Observer 1 and 2 End Of Night Checklists if the observers didn't do it (see their instructions).
  2. Write the Synopsis_Night logbook entry if the observers didn't do it.
  3. Publish the nightsum in the morning if the observers didn't do it.
  4. Complete CTIO end of night report if the observers didn't do it (if you had two operators, make sure both names appear). Check the calendar to see the status of the night reports.
  5. Be aware of our observers arrival to and departure from the mountain.
  6. Prepare the nightly observing plan page, including populating Almanac and Weather info (go to the Shifters page and scroll to - and then click on - today's date)
  7. Check the instrument status (read the logbook, talk to people)
  8. Check the DM status
  9. Check SISPI status
  10. Check what scripts are needed/present for calibration
  11. Take a first look at the DESDM data quality results, survey table
    Anything unusual should be discussed at the 4pm meeting.
  12. Send out the agenda for the 4pm meeting (if to occur) to
  13. Check diskspace. Log into one of the observer machines as user "sispi" (password on white board)

added by Kathy R 10/17: I didn't have any success logging in using su sispi from observer3 but ssh -X sispi@observer3 did work

setup SISPI
check_diskspace

If any disks are filling up (>90% for observer2, >93% for the pipeline
machines) send an email to Klaus and Ann ( and
)

Late afternoon

  1. Before the 4pm meeting check with telops if SISPI instance and the GUIs were restarted. You could also do this by checking if they completed a "2 PM Checklist" in the eLog, or looking at the "Uptime" field in the Architect Console GUI.
  2. Ask telops to point telescope to white spot and fire afternoon calibration scripts
  3. Perform the aTmCam afternoon check
  4. Chair the 4pm meeting and present observing plan
  5. Edit the observing plan if needed
  6. Perform the Late Afternoon Checklist in the logbook:
  • Observer(s) Present:
  • SISPI Instance: This is now autogenerated. The format is DECam_YYYYMMDD
  • SISPI configured with correct DES PropID (2012B-0001 as of 12/5/2014)
  • SISPI Interlocks OK [Y/N]:
  • SISPI Interlocks that are not OK [List]:
  • Check Telemetry GUIs:
    Explanation of SISPI Telemetry GUI Plots
  • Check if Morning Calibration Completed [Y/N]:
  • Check the eLog to see if Telops did the "2 PM Checklist"
  • Calibration [NOTE: calibrations are often done by Telops. Just run flatCheck once done.]
    • Verify Dome is Closed
    • Verify Dome Flat Setup (ask Telops) Dome Flat setup
    • Turn VSUB on
    • Submit Afternoon Calibration Script (/user/sispi/decam/ExposureScripts/DECam/prenight_zeros_flats.json):
    • Calib Script Name (if not the standard script):
    • Check a few exposures on the Comfort Display. Is the focal plane complete?
    • When script is done, in kentools (on observer2), run "flatCheck" - see if plot is OK.
  • Check GPS Monitor is Operating [Y/N]:
    See that GPSMon is working
  • Comments:
    1. check the SN processing status [for this you need a password that would be the first 3 letters of your Services username + docdbpw (the inaccurate info about how many CCDs) ]
    2. Implement the data quality (= "fill out the Survey Table") If you didn't have a 4pm meeting, then you need to check with DESDM before doing the implementation if you think any of their true/false flags are erroneous. [ confession from Kathy R - this line was added because I tried to be too clever - sorry Robert, Klaus, 400+ DES members... ] It's OK to do this in the middle of the night if you didn't do it sooner--it won't mess up the data from that night.
    3. Make sure observers are present and roles are defined/understood

Night

  1. Make sure the observers understand what they are doing
  2. Make sure the observers get off to a good start
  3. Periodically check wind speed, TelOPS will lower/raise screens if needed.
  4. Be available to the observers if they have questions or problems
  5. Try to get some sleep