Troubleshooting For Observers

Sometimes SISPI will have a problem that prevents you from taking images. Here's how to resolve the issue and get back to taking data. An expert may perform other steps in the process of debugging or fixing the instance; these directions are the quickest way for a non-expert to get back to observing. (Sometimes SISPI will have a non-critical problem that doesn't prevent data taking. If that is your problem, see instructions here)

  1. Figure out where the problem is.
    • Something alerted you to the fact that SISPI is not working properly. Note what that was so that you can record it in the logbook.
    • You may have a broken interlock. Look at the LEDs in the Observer Console - if the "Interlock" light is red, then you've got a broken interlock. Check the Interlock Viewer page to see which interlock is broken. Note this for the logbook.
    • If the problem raised an alarm, the alarm button in the menu bar will show a red circle with a number in it. You can click on that button to get a dropdown list of new alarms, and look on the Alarm History page. The alarms will state which component and device they came from.
    • On the Observer Console, you can look in the System View for any components that are in the ERROR state, and in the OCS messages to see if they mention an error.
  2. Examine the problematic role.
    • Once you know which role is causing the problems, go over to the Architect Console and highlight that role. You will see the one latest log message in the log message box. If there is a red X next to the role instead of a green check, skip the reconfiguring step and go straight to restarting the role.
    • If you'd like, you can click on the "Full Log" button to see all the logs, including the ones that lead up to the problem. If there are any recent error messages, make note of them (just the timestamp and role name is fine; experts can look up the full log message later) so that you can put that information in the log book.
  3. Fast fix (if all roles are running) - reconfigure the instance.
    • If all the roles are up and running (have green check marks in the Architect Console), you may be able get back to a working state by reconfiguring the instance. Click on the "Configure" button in the Observer Console, which will trigger the roles to configure and reconnect with each other.
    • Make a note in the logbook describing the error and that it was solved by reconfiguring.
  4. Next fix - restarting the role.
    • If the previous step failed, or there's a red X next to any role in the Architect Console, you will need to restart the role instead of just reconfiguring it. Select the role in the Architect Console then click on "Start" (if the role is already stopped) or "Restart" (if the broken process is still running).
    • Once the process restarts, you will see a green check next to its name and some messages in the log window. Then go to the Observer Console and press the "Configure" button in the System Control tab.
    • Make a note in the logbook about the error, that reconfiguring didn't fix it, but that restarting the role and reconfiguring worked.
  5. Things are really broken - restart the instance.
    • If restarting the broken role does not work, you should restart the entire SISPI instance. The restart is fairly quick, so even for experts a restart may be quicker (if less informative) than debugging. If you are not an expert, you will need an expert to help with this. The directions for restarting the instance are in the Expert Guide section of the wiki.
    • Make a note in the logbook about the error and how you tried to fix it that didn't work, and that you are now restarting the instance.
    • Make a phone call to an expert. To dial out from the 4m control room to the US, do the following: