Project

General

Profile

List of Ganglia metrics

Overview of UBOONE-PROD

  • UBOONE-PROD Cluster Load:
  • UBOONE-PROD Cluster Memory:
  • UBOONE-PROD Cluster CPU:
  • UBOONE-PROD Cluster Network:
  • load_one (shown for all machines separately and as a stacked histogram in UBOONE-PROD aggregated load_one):

Metrics for individual machines

Each of these ganglia metrics are available for all DAQ machines:
  • evb: ubdaq-prod-evb-priv
  • near1: ubdaq-prod-near1-priv
  • near2: ubdaq-prod-near2-priv
  • seb01: ubdaq-prod-seb-priv01
  • seb02: ubdaq-prod-seb-priv02
  • seb03: ubdaq-prod-seb-priv03
  • seb04: ubdaq-prod-seb-priv04
  • seb05: ubdaq-prod-seb-priv05
  • seb06: ubdaq-prod-seb-priv06
  • seb07: ubdaq-prod-seb-priv07
  • seb08: ubdaq-prod-seb-priv08
  • seb09: ubdaq-prod-seb-priv09
  • seb10: ubdaq-prod-seb-priv10
  • smc: ubdaq-prod-smc-priv
  • smc2: ubdaq-prod-smc2-priv
  • ws01: ubdaq-prod-ws01-priv
  • ws02: ubdaq-prod-ws02-priv

To navigate to the metrics for a given machine, choose that option in the drop-down box labelled

uBooNE@LArTF Grid > UBOONE-PROD > --Choose a Node

Host Overview

  • Load:
  • Memory:
  • CPU:
  • Network:

no_group metrics

  • fans_fastest_rpm:
  • fans_ok:
  • fans_slowest_rpm:
  • size_data:
  • system_temperature:
  • temperatures_ok:
  • voltages_ok:

CPU metrics

  • CPU aidle:
  • CPU Idle:
  • CPU Nice:
  • CPU System:
  • CPU User:
  • CPU wio:

Disk metrics

  • Disk Space Available:
  • Total Disk Space:
  • Maximum Disk Space Used:

Load metrics

  • Fifteen Minute Load Average:
  • Five Minute Load Average:
  • One Minute Load Average:

Memory metrics

  • Memory Buffers:
  • Cached Memory:
  • Free Memory:
  • Shared Memory:
  • Free Swap Space:

Network metrics

Direct way to see data flowing! Metrics are exactly what they sound like.

  • Bytes Received
  • Bytes Sent
  • Packets Received
  • Packets Sent

Process metrics

  • Total Running Processes:
  • Total Processes:

assemblerAppevb metrics (evb only)

  • Builder-AssembledEvents_Assemblerappevb: how many events are being assembled.
  • Builder-DroppedEventsRate_Assemblerappevb:
  • Builder-DroppedEvents_Assemblerappevb:
  • Builder-EventBuildTime_Assemblerappevb:
  • Builder-EventBuildTime_Average_Assemblerappevb:
  • Builder-Event_Counter_Diff_Max_Assemblerappevb:
  • Builder-Frame_Counter_Diff_Max_Assemblerappevb:
  • Builder-SerializeRecordTime_Assemblerappevb:
  • Builder-SoftwarePrescaleTime_Aassemblerappevb:
  • Builder-SoftwareTriggerTime_Aassemblerappevb:
  • Builder-TimeDifference_GPS-PPS_Local_Aassemblerappevb:
  • Builder-TimeDifference_GPS_Local_Aassemblerappevb:
  • Builder-TrigRate_BNB_Aassemblerappevb: BNB trigger rate
  • Builder-TrigRate_EXT_Aassemblerappevb: EXT trigger rate (note that the expected EXT trigger rate should be given in the name of the config file for the current run)
  • Builder-TrigRate_Laser_Aassemblerappevb: Laser trigger rate
  • Builder-TrigRate_NuMI_Aassemblerappevb: NuMI trigger rate
  • Builder-TrigRate_Other_Aassemblerappevb:
  • Builder-TrigRate_PMT_Beam_Aassemblerappevb:
  • Builder-TrigRate_PMT_Cosmic_Aassemblerappevb:
  • Builder-TrigRate_Paddle_Aassemblerappevb:
  • Builder-WriteDataRate_Aassemblerappevb:
  • Builder-WriteEventRate_Aassemblerappevb:
  • Builder-WriterQueue_Aassemblerappevb:
  • FEM-card-dissection-errors_Aassemblerappevb:
  • FEM-cards-per-crate_Aassemblerappevb:
  • FEM-channel-dissection-errors_Aassemblerappevb:
  • FEM-chs-per-crate_Aassemblerappevb:
  • PMT-window-per-event_Aassemblerappevb:
  • Receiver-ReceivedDataRate_Aassemblerappevb:
  • Receiver-ReceivedFragmentRate_Aassemblerappevb:
  • Receiver-RemainingEventStoreCapacity_Aassemblerappevb:
  • Recycler-AllocationRate_Aassemblerappevb:
  • Recycler-PoolSize_Aassemblerappevb:
  • TPC-channel-per-event_Aassemblerappevb:
  • TPC-missing-channel-trailer-rate_Aassemblerappevb:
The following metrics are given for every software trigger defined in the config file. For example, for the software trigger BNB_FEMBeamTriggerAlgo:
  • SoftwareTrigger-BNB_FEMBeamTriggerAlgo_EventsSinceLastPass_Aassemblerappevb:
  • SoftwareTrigger-BNB_FEMBeamTriggerAlgo_Rate_Aassemblerappevb:
  • SoftwareTrigger-BNB_FEMBeamTriggerAlgo_Rate_vs_RawTrigger_Aassemblerappevb: Rate of raw triggers issued by the detector divided by the rate of events passing the software trigger. Somewhat equivalent to 1/passing fraction, or an averaged number of events since the last one passed the software trigger. An increase in this metric indicates that there are more raw triggers being issued (and could be indicative of eg. a hot PMT)
  • SoftwareTrigger-BNB_FEMBeamTriggerAlgo_Time_Aassemblerappevb:
  • SoftwareTrigger-BNB_FEMBeamTriggerAlgo_Value_Aassemblerappevb:

not-setevb metrics (evb only)

  • FEM-card-dissection-errors_Anot-setevb:
  • FEM-cards-per-crate_Anot-setevb:
  • FEM-channel-dissection-errors_Anot-setevb:
  • FEM-chs-per-crate_Anot-setevb:

sebAppsebXX metrics (sebs only)

Shown for example for seb01.

  • DMP-CircularBufferOccupancy_Asebappseb01:
  • FEM-card-dissection-errors_Asebappseb01:
  • FEM-cards-per-crate_Asebappseb01:
  • FEM-channel-dissection-errors_Asebappseb01:
  • FEM-chs-per-crate_Asebappseb01:
  • FEMMonitorWorker Thread rate_Asebappseb01:
  • FEMMonitorWorker fem_bad_stat_Asebappseb01:
  • FEMMonitorWorker nu_block_id_Asebappseb01:
  • FEMMonitorWorker nu_nwords_Asebappseb01:
  • FEMMonitorWorker nu_read_lag_Asebappseb01:
  • FEMMonitorWorker sn_block_id_Asebappseb01:
  • FEMMonitorWorker sn_nwords_Asebappseb01:
  • FEMMonitorWorker sn_read_lag_Asebappseb01:
  • FEMMonitorWorker xmit_bad_stat_Asebappseb01:
  • FEMMonitorWorker xmit_frame_ctr_Asebappseb01:
  • FEMMonitorWorker xmit_trigger_ctr_Asebappseb01:
  • NU-AssemblerWriteDataRate_Asebappseb01:
  • NU-AssemblerWriteFragmentRate_Asebappseb01:
  • NU-DMA-CircularBufferOccupancy_Asebappseb01:
  • NU-DMA-ReadFragmentTime_Asebappseb01:
  • SN-DiskWriteFragmentRate_Asebappseb01:
  • SN-DiskWriteRate_Asebappseb01:
  • NU/SN-CircularBufferFillRate_Asebappseb01:
  • NU/SN-ConsumeFragmentDataRate_Asebappseb01:
  • NU/SN-ConsumeFragmentRate_Asebappseb01:
  • NU/SN-DMAPollingRate_Asebappseb01:
  • NU/SN-DMAReadChunksRate_Asebappseb01:
  • NU/SN-DMAReadRate_Asebappseb01:
  • NU/SN-FragmentOutboundQueueDepth_Asebappseb01:
  • NU/SN-FragmentOutboundQueueSize_Asebappseb01:
  • NU/SN-FragmentSourceEventDiff_Asebappseb01:
  • NU/SN-FragmentSourceFrameDiff_Asebappseb01:
  • NU/SN-ReadFragmentDataRate_Asebappseb01:
  • NU/SN-ReadFragmentRate_Asebappseb01:
  • NU/SN-WriteFragmentDataRate_Asebappseb01:
  • NU/SN-WriteFragmentRate_Asebappseb01:
  • Recycler-PoolSize_Asebappseb01:
  • TPC-channel-per-event_Asebappseb01:
  • TPC-missing-channel-trailer-rate_Asebappseb01: