Expert Panel work (low priority)
0. Test and implement Keithley monitoring into Slow controls. For the Analog current monitoring with 2700, the set Range field is set every time we read back, is this fixable so it doesn't do
that? For the Pick off monitoring with 2410, establish a Source/sense configuration that will nicely read-out pick off voltage and get rid of the errors currently seen on the front panel. -- DONE!
1. Implement an “Abort” button that will let the expert stop a ramp --DONE (to most part, pending testing by the HV expert)
Sarah's notes on this: While the program is quite good already, I am a little concerned about inadvertencies on the expert’s part (mine). Would it be possible to add an “abort” button to the ramp?This would be
something that just stops the ramp in voltage wherever it is. So if one is going from 0 V to 10 kV, and the button is hit at 5 kV, the program breaks out of its loop and the resulting steady voltage is 5 kV. This
would allow us to issue larger ramp commands than 1 kV without having to choose between hitting the PS power button or letting the ramp continue if an issue is seen in the HV. It would also allow us to break
out of the ramp if one enters 71 instead of 71000 when wanting to increase the voltage.
2. Implement automatic logging into timestamped .txt files whenever expert GUI is initiated.
3. Make the expert python GUI read current state of the supply and populate the set fields.
4. Also caput set information at every ramp step to the slow controls GUI. Probably implement a different button "Ramp V" for example or into the desired V field.
4.1. Implement an additional password layer to access the HV expert panel (requested by the experts)
5. Remove read back buttons for which the read back bits don't work from the expert python GUI panel
Drift HV panels work (low priority)
7. Implement exponential notation for Desired R in the GUI. -- DONE!
8. Fix the ranges etc. on the plot that is displayed on the Drift HV panel
9. Need to switch to plotting current instead of raw voltage.
10. Need to fix protocol for waveform grab to handle 0 characters in waveform so they don’t terminate the “string”.
11. Need to change PREC on uB_TPCDrift_HV01_keithley/I_expected_calc to 2 in db (currently only temporarily set using caput.
MPOD ASICS LV:
0. Update the save/restore template for ASICS LV with updated variable naming - DONE!
1. The group on/off panel for ASICS LV has been updated. Need to test this - HALF DONE!
MPOD BIAS HV:
0. Make a save/restore template with updated Wirebias variables - DONE!
1. Implement a protection message box (more like a confirm message)m when trying to edit the groupSwitchHV button that is in the expert panel - DONE!
2. Implement the updated MIB file from http://file.wiener-d.com/software/net-snmp/ (The latest WIENER-CRATE-MIB.txt is in WIENER-CRATE-MIB-2734.zip) -- low priority
3. Clean up unnecessary files, test files or unused old version of files from the Wiener EPICS directories. -- low priority
PMT HV (high priority):
1. The save-Restore feature doesn't work for PMT HV. When you take a snapshot and try to restore it, it just sits quiet and you can see that the values are not changed. I tested their claim briefly yesterday and I confirm that the save Restore sits quietly when trying to restore saved settings.
2. Test Save-Restore feature (coordinate with Taritree or Matt) once fixed
3. Request to implement a "ramp down button to zero so that the shifter can turn off a tube. This may be useful in the future, but right now I would prefer either Jarrett, Matt or myself to ever change voltages. So this isn’t super urgent."
4. is there anyway to set the default ramp value to 10 (instead of 100)?
0. Make an overview panel that tells which sub-systems are on or off, a single view of all sub-systems.
COMMENTS FROM BONNIE ON SLOW CONTROLS SHIFTER LEVEL DOCUMENTATION:
Shift Documentation (my suggestions):
General comments: Headings for each system should be consistent. Information in the Operations Wiki Guides should be for understanding how to operate and monitor the detector, while day-to-day information (such as alarming systems) should be in a Daily Systems Log. Checklists for regular checks for different systems should not be in the guide but rather in a checklist area or part of required checklists during the shift.
Headers in all sections should follow: Overview, Shifters Guide, Troubleshooting, Experts
General note on Slow Mon: The acknowledged alarm list can get very long (308 tonight – I understand after a power outage but…) which makes me worry. Not sure what to suggest but if something that should not be acknowledged gets in this list and it's a huge list we could be in a bad state and not know it from shift to shift…..
Other general notes:
This documentation should match the structure of the others meaning Overview, Guide, Troubleshooting, Experts Only.
The documentation should be in a steady state for running. Transient information like which systems to look at now and which not to should be in a Daily Systems Log. The new shifter or as a reminder to an experienced shifter should use this guide to understand how to monitor the detector, not to determine what to monitor—that should go in a Daily Systems Log. Anything that needs to be checked once per shift or more should go into the shift checklists (being created)
There are multiple places where the shifter is told how to pay attention to the different alarm colors and sometimes in different ways for different systems. There should be one consistent set of responses for all systems described at the top. Any deviations from this (like don’t pay attention to minor alarms in system X right now) should be in the Daily Systems Log.
Below are comments on what is in the documentation now, and notes on using the documentation to run the system (what I had trouble with and some suggestions):
Note that says ignore any other instructions you might see in the low monitoring tabs below – this is dangerous. If there are inconsistent instructions below then they need to be removed…
Remove “New! Easier!” Hard to tell when things are really new…. In general in the notes below there are a number of places where I suggest things be streamlined to general running – like remove messages like “at this stage”. These instructions should be updated for steady state mode.
Where is the slow control Icon? Is it only on the console below the iMAC? Should be indicated. Include a small picture of the icon
Can more than one slow controls monitoring panel be launched? I was able to but is this advised? I think there is something special about the one on the monitor below the iMAC bc it has that yellow screen behind it? If so this should be noted somewhere.
General comments: Monitoring cryo in even the most basic way is a mystery to me. The “Cryotable” tab is super confusing. How, for example, could I verify that the pumps are on?
Could we make the font on the Alarm Area Panel bigger? I find myself especially with some of the alarm colors having trouble reading the boxes in particular on the upper monitors. Or maybe make them bold, or both?
The overview panel is very useful – but looks like it could be closed in the list of things to view on the top bar above it. Could this be a permanent button somewhere to set it apart from the other possible displays?
At the end of Important Notes it says: “Shifters are advised to sort the Alarm Table every now and then to make sure no new alarms are missed”. Shouldn’t this be more stringent – like: When the detector is on, (or you could omit this) Shifters should not normally have any alarms that have not been acknowledged. Acknowledged means properly addressed by following instructions of or contacting a system expert. All alarms should be addressed right away….
Monitoring Instructions During The Shift:
Remove “If you are first time shifters” bc all shifters should take note of the alarm panel at the beginning of their shift…. Later in this paragraph – given my above comment, should there ever be any current alarms? Maybe it should say that any current alarms at the beginning of the shift should be discussed in shift changeover as to their status…
Remove “at this stage” later in that sentence.
In the Important note: fine to ignore glitches but please define rapidly – change back within a second or within a minute? Also I would say instead of ignore – shifters should try to note any glitches in the elog so that system experts can fix them.
For the IGNORE and Most important subsystems: Maybe this is not the place to put this somewhat transient information. Where else could it be included? Perhaps in the Run Plan which shifters should read at the beginning of each shift? I worry that more experienced shifters won’t read this documentation but need this information.
I really don’t like (b) for cryo ifix. Acknowledge and completely ignore? Maybe we need a glossary of what cryo variables we can acknowledge without contacting an expert and what we can. How to monitor cryo is confusing – see my notes on this above too.
OnDetPower_TPCPS: First paragraph again seems like transient information – what to ignore right now and what to call an expert on – maybe should be in Run Plan.
How to reach the WireBias HV and ASICS LV panle is important but gets lost in this paragraph. Maybe this should come first and then a note to look at the Run Plan or Daily Alarm Plan (new idea).
There are notes on what to do for a minor alarm for TPCPS – again here, these should go somewhere where EVERY shifter will note them every shift.
Note from shift 7/31/2015: Is this still relevant? Same comments as above. Does not belong here.
Image here is out of date. Specifically, the buttons on TPC P5 are different from what is on the slow controls now.
Same comments as above. We should not need to repeat what to do with Major and Minor alarms unless there is a specific issue which should be noted in a Daily Alarm Log. The note in this section should also not be needed if shifters know to note a glitch in the elog to be addressed by systems experts.
DAQ Status: the (for those who want to understand this behavior) I think does not belong here. The information here should be what is needed by shifters and the backstory should go somewhere else….
PMTHV: Same comments as above. We should have a standard way to address all alarms and any transient response should be in a Daily Alarm Plan or Log.
Could you describe here how to see the PMT HV status? Took me help to determine with clicking how to get to the screen that showed me their was voltage on them. In general I could not figure out just how to tell if they were on or off easily.
Great description! I really like how it gives the shifter information on where temps are measured. Don't like the statement that all the rack protection should be green as this should already be a given for all systems (except in a Daily log) (Sorry I must sound like a broken record).
Same comment on note on glitches…
Why does it sound like its OK for this alarm to be yellow?
Please include a description first of where to find this and what It is, then the sentence on who to contact.
Great description! For inline monitoring and for what to keep an eye on – should be in Daily Log?
Making Shift Plots
This is great! But I’ve only noticed it for the first time and have been wondering how to do it! This is too buried. Can it go up at the top or have its own tab under the Slow Controls main heading?