Project

General

Profile

Updated 2012-03-28

Temporary instructions for verifying DCM Synchronization when running with 2 Master TDUs

Overview

We are currently running a mix of old and new DCMs, which in turn require using old and new Master TDUs. This means we have two totally independent timing systems for NDOS.

Checking the synchronization between these two timing systems is very tough, since the timing "ping" (aka, timing fiducial), which cannot be tied to a hardware clock, will vary between the two Master TDUs. (We normally don't care about the absolute timing of these pings, but just that the DCMs see them at the same time.)

checkSynch.sh

First, see this link for a refresher on running the checkSynch.sh script.

Possible Outputs:

Missing pings from one of the mTDUs

If the script claims "All" 7 DCMs are in synch (in fact, there are 11), then the ping is missing from one of the mTDUs:

All 7 DCMs in synch within 0 ticks/ ( Limit = 10)

(Latest ping=2012-Mar-21 20:21:09.556337750000 UTC)

This is probably because, when started "by hand", the TDUControl does not automatically select DCMs as the target of the timing ping. To fix it, check the "DCMs" box in the "Timing Fiducial" panel in the "Timing" pull-down menu of TDUControl:

Marked-up ping setup graphic

One or more DCMs is really out of synch

As in this case, there are 3 or more groups of timings. This is unambiguously bad.
Follow the recovery procedure.

Some DCMs vary from others by 4422897776 ticks = 69107777750 ns!  (Limit = 10 ticks)
PLEASE TAKE ACTION!

----------------------------------------------------
Timestamp=0x000fee15f6ae9df0 (2012-Mar-21 21:23:00.484727750000 UTC) for DCMs:

dcm-3-01-03
dcm-3-02-03
dcm-3-03-01
dcm-3-03-03
dcm-3-04-01
dcm-3-04-02

----------------------------------------------------
Timestamp=0x000fee14f6ae9df0 (2012-Mar-21 21:21:53.375863750000 UTC) for DCMs:

dcm-3-03-02

----------------------------------------------------
Timestamp=0x000fee15fe4eae60 (2012-Mar-21 21:23:02.483641500000 UTC) for DCMs:

dcm-3-01-01
dcm-3-01-02
dcm-3-02-01
dcm-3-02-02

DCMs are in synch within each mTDU

Some DCMs vary from others by 364760 ticks = 5699375 ns!  (Limit = 10 ticks)
DCM times are grouped by mTDU: synch may be OK, but please check Event Display to be sure!

----------------------------------------------------
Timestamp=0x000fee1148dd32f8 (2012-Mar-21 21:17:46.483915875000 UTC) for DCMs:

dcm-3-01-03
dcm-3-02-03
dcm-3-03-02
dcm-3-03-03
dcm-3-04-01
dcm-3-04-02

----------------------------------------------------
Timestamp=0x000fee1148e2c3d0 (2012-Mar-21 21:17:46.489615250000 UTC) for DCMs:

dcm-3-01-01
dcm-3-01-02
dcm-3-03-01
dcm-3-02-01
dcm-3-02-02

...but... the skew between the two sets of DCMs is arbitrary. See below.

Using the Event Display

If the checkSynch.sh script reports that the DCMs are synched within each Master TDU, use the event display to determine whether they are synched between the mTDUs. The following cases are indicative of a synch problem between the mTDUs:

  • Track in one view not having a match in the other (assuming instrumentation is present)
  • Track disappears at the boundary between mTDUs, despite not having left the detector as indicated in the other view.
  • Update: a new DCM was put in position 3-01-03 on 3/26/2012. That means that all of Diblock 1 is on the new timing chain (and thus the picture below is somewhat incorrect). The best way to see evidence of poor synchronization is to look for tracks disappearing at the boundary of the lowest part of the Y view in Diblock 2, corresponding to y<-80cm for 400cm<z<800cm.

~10 or so events should allow you to determine whether the two mTDUs are relatively synched. If not, follow the recovery procedure.

Event Display with bad sync between the mTDUs

How to Recover

  • The best way to recover is to end the run, then restart. Don't bother re-configuring or re-cooling the APDs.
  • However, if this does not work after 3 or 4 tries, try going back to "Reload hardware config", and proceeding as usual from there.
  • If that does not work, try a restart of the entire system.
  • If that doesn't work, contact an expert.