Project

General

Profile

Soudan to Ash River Time Transfer

Soudan Time Phase Measurement

The time phase offset of the Rb clock was measure against the master clock at Soudan. The measurement that was made was done using the Rb clock as the start for the time interval counter and the Soudan (Minos) master clock as the stop. The time offsets were recorded as follows:

Calib Corr. Measurement Time Phase (ms)
SF1608 07:24 240.432602262
SF1607 08:30 240.432578141
SF1607 09:00 240.432558283
SF1607 09:30 240.432541203

Delta T between Minos (t0) and Cs clock: 4884 ns (13:09)
Minos Round Trip time: 13436 ns (6718 ns one direction)

States of Ash River TDUs

The Ash River TDUs were found in the following configuration states (showing no errors)

Firmware versions (all TDUs): 2.17 and 1.7 for arm software.

TDU-Master-01 (Primary Operational TDU)

[root@tdu-master-ppc-01:testRelForSpillServer_Timing]$ tduRegDump 
------------------------------------------------------
Register        Value    Description
------------------------------------------------------
0x0000        0x0400    Control
              Bit-10: Enable Accelerator Events
0x0001        0x0008    Status
              Bit-03: Okay to SYNC
0x0002        0x0000    TDU Delay Value
0x0003        0x0000    Side Output Delay
0x0004        0x0000    Top Output Delay
0x0005        0xa3bf    GPS Time of Week Lo Word
0x0006        0x0003    GPS Time of Week Hi Word
0x0007        0x06f0    GPS Week
0x0008        0x01ff    Delay Offset
0x0009        0x0000    Control 2
0x000a        0x0028    1 PPS Verification Lo Word
0x000b        0x0000    1 PPS Verification Hi Word
0x000c        0x0000    Interrupt Mask
0x000d        0x0000    Early Sync
0x000e        0x0000    Error Register (Secondary)
0x000f        0x0000    Error Disable Mask
0x0010        0x9000    Preset Time Bytes 1,0
0x0011        0xf1cb    Preset Time Bytes 3,2
0x0012        0x1bfb    Preset Time Bytes 5,4
0x0013        0x001d    Preset Time Bytes 7,6
0x0014        0x0000    Error Register (Primary)
0x0015        0x0211    Firmware ID
0x0016        0x0003    TDU Type (Master/Slave)
0x0017        0x0000    Event Fifo Overflow count
0x0018        0x0000    --- Depricated --
0x0019        0x0000    Error Disable Mask (Primary)
0x001a        0x0000    Pulse Per Second Width
0x001b        0x0000    Event Timestamp Byte 1,0
0x001c        0x0000    Event Timestamp Byte 3,2
0x001d        0x0000    Event Timestamp Byte 5,4
0x001e        0x0000    Event Timestamp Byte 7,6
0x001f        0x0000    Event Number
0x0020        0x0000    Command Data
0x0021        0x0000    Command Address
0x0022        0x0000    Command Header
0x0023        0x0000    Event Decoding Mask
0x0024        0x0000    Interrupt Mask
0x0025        0x0000    GPS Lock Loss Counter
0x0026        0x0002    GPS Holdover Counter
0x0027        0x0000    GPS Antenna Fault Counter
0x0028        0x0000    GPS Lock Loss Timer
0x0030        0xa99b    Command History Time, Byte 1,0
0x0031        0xe604    Command History Time, Byte 3,2
0x0032        0x1b94    Command History Time, Byte 5,4
0x0033        0x001d    Command History Time, Byte 7,6
0x0034        0xabcd    Command History Data
0x0035        0x04fd    Command History Address
0x0036        0x0040    Command History Header
0x0037        0x007e    Command History Word Count
0x0040        0x9000    Decoded Time, Byte 1,0
0x0041        0xf1cb    Decoded Time, Byte 3,2
0x0042        0x1bfb    Decoded Time, Byte 5,4
0x0043        0x001d    Decoded Time, Byte 7,6
0x0050        0x001d    Time Snapshot, Bytes 7,6
0x0051        0x1c01    Time Snapshot, Bytes 5,4
0x0052        0x0b0a    Time Snapshot, Bytes 3,2
0x0053        0x4e7c    Time Snapshot, Bytes 1,0
0x00fe        ------    Internal Use

Time of last Sync:

<RelForSpillServer_Timing]$ NovaTimeConvert 0x001d1bfbf1cb9000                  

A NOvA base time of 8193543232000000 corresponds to...
  a UNIX time of 1390328112 sec, 0 nsec
  a calendar date of 2014-Jan-21 18:15:12.000000000000 UTC

TDU-Master-02 (Secondary)

[root@tdu-master-ppc-02:testRelForSpillServer_Timing]$ tduRegDump 
------------------------------------------------------
Register        Value    Description
------------------------------------------------------
0x0000        0x0400    Control
              Bit-10: Enable Accelerator Events
0x0001        0x0008    Status
              Bit-03: Okay to SYNC
0x0002        0x0000    TDU Delay Value
0x0003        0x0000    Side Output Delay
0x0004        0x0000    Top Output Delay
0x0005        0xb6b2    GPS Time of Week Lo Word
0x0006        0x0002    GPS Time of Week Hi Word
0x0007        0x06f0    GPS Week
0x0008        0x01ff    Delay Offset
0x0009        0x0000    Control 2
0x000a        0x1000    1 PPS Verification Lo Word
0x000b        0x0000    1 PPS Verification Hi Word
0x000c        0x0000    Interrupt Mask
0x000d        0x0000    Early Sync
0x000e        0x0000    Error Register (Secondary)
0x000f        0x0000    Error Disable Mask
0x0010        0x4000    Preset Time Bytes 1,0
0x0011        0xaae4    Preset Time Bytes 3,2
0x0012        0x1873    Preset Time Bytes 5,4
0x0013        0x001d    Preset Time Bytes 7,6
0x0014        0x0000    Error Register (Primary)
0x0015        0x0211    Firmware ID
0x0016        0x0003    TDU Type (Master/Slave)
0x0017        0x0000    Event Fifo Overflow count
0x0018        0x0000    --- Depricated --
0x0019        0x0000    Error Disable Mask (Primary)
0x001a        0x0000    Pulse Per Second Width
0x001b        0x0000    Event Timestamp Byte 1,0
0x001c        0x0000    Event Timestamp Byte 3,2
0x001d        0x0000    Event Timestamp Byte 5,4
0x001e        0x0000    Event Timestamp Byte 7,6
0x001f        0x0000    Event Number
0x0020        0x0000    Command Data
0x0021        0x0000    Command Address
0x0022        0x0000    Command Header
0x0023        0x1130    Event Decoding Mask
0x0024        0x0016    Interrupt Mask
0x0025        0x0000    GPS Lock Loss Counter
0x0026        0x0002    GPS Holdover Counter
0x0027        0x0000    GPS Antenna Fault Counter
0x0028        0x0000    GPS Lock Loss Timer
0x0030        0xdb46    Command History Time, Byte 1,0
0x0031        0x8e32    Command History Time, Byte 3,2
0x0032        0x1b91    Command History Time, Byte 5,4
0x0033        0x001d    Command History Time, Byte 7,6
0x0034        0xabcd    Command History Data
0x0035        0x04fd    Command History Address
0x0036        0x0040    Command History Header
0x0037        0x007d    Command History Word Count
0x0040        0x4000    Decoded Time, Byte 1,0
0x0041        0xaae4    Decoded Time, Byte 3,2
0x0042        0x1873    Decoded Time, Byte 5,4
0x0043        0x001d    Decoded Time, Byte 7,6
0x0050        0x001d    Time Snapshot, Bytes 7,6
0x0051        0x1c04    Time Snapshot, Bytes 5,4
0x0052        0xa844    Time Snapshot, Bytes 3,2
0x0053        0xa714    Time Snapshot, Bytes 1,0

Time of last Sync:

<RelForSpillServer_Timing]$ NovaTimeConvert 0x001d1873aae44000                

A NOvA base time of 8189659392000000 corresponds to...
  a UNIX time of 1390267427 sec, 0 nsec
  a calendar date of 2014-Jan-21 01:23:47.000000000000 UTC

TDU-Master-04 was found in a state where its powerpc board was not booted (stuck at the redboot prompt)
we did a soft reboot of the machine (reset from the redboot prompt). The state of the machine was then:

------------------------------------------------------
Register        Value    Description
------------------------------------------------------
0x0000        0x0400    Control
              Bit-10: Enable Accelerator Events
0x0001        0x0008    Status
              Bit-03: Okay to SYNC
0x0002        0x0000    TDU Delay Value
0x0003        0x0000    Side Output Delay
0x0004        0x0000    Top Output Delay
0x0005        0x0000    GPS Time of Week Lo Word
0x0006        0x0000    GPS Time of Week Hi Word
0x0007        0x0000    GPS Week
0x0008        0x01ff    Delay Offset
0x0009        0x0000    Control 2
0x000a        0x0031    1 PPS Verification Lo Word
0x000b        0x8000    1 PPS Verification Hi Word
0x000c        0x0000    Interrupt Mask
0x000d        0x0000    Early Sync
0x000e        0x0000    Error Register (Secondary)
0x000f        0x0000    Error Disable Mask
0x0010        0x0000    Preset Time Bytes 1,0
0x0011        0x0000    Preset Time Bytes 3,2
0x0012        0x0000    Preset Time Bytes 5,4
0x0013        0x0000    Preset Time Bytes 7,6
0x0014        0x0000    Error Register (Primary)
0x0015        0x0211    Firmware ID
0x0016        0x0002    TDU Type (Master/Slave)
0x0017        0x0000    Event Fifo Overflow count
0x0018        0x8000    --- Depricated --
0x0019        0x0000    Error Disable Mask (Primary)
0x001a        0x0000    Pulse Per Second Width
0x001b        0x0000    Event Timestamp Byte 1,0
0x001c        0x0000    Event Timestamp Byte 3,2
0x001d        0x0000    Event Timestamp Byte 5,4
0x001e        0x0000    Event Timestamp Byte 7,6
0x001f        0x0000    Event Number
0x0020        0x0000    Command Data
0x0021        0x0000    Command Address
0x0022        0x0000    Command Header
0x0023        0x0000    Event Decoding Mask
0x0024        0x0000    Interrupt Mask
0x0025        0x0002    GPS Lock Loss Counter
0x0026        0x0001    GPS Holdover Counter
0x0027        0x0000    GPS Antenna Fault Counter
0x0028        0x0000    GPS Lock Loss Timer
0x0030        0x0000    Command History Time, Byte 1,0
0x0031        0x0000    Command History Time, Byte 3,2
0x0032        0x0000    Command History Time, Byte 5,4
0x0033        0x0000    Command History Time, Byte 7,6
0x0034        0x0000    Command History Data
0x0035        0x0000    Command History Address
0x0036        0x0000    Command History Header
0x0037        0x0000    Command History Word Count
0x0040        0x0000    Decoded Time, Byte 1,0
0x0041        0x0000    Decoded Time, Byte 3,2
0x0042        0x0000    Decoded Time, Byte 5,4
0x0043        0x0000    Decoded Time, Byte 7,6
0x0050        0x0000    Time Snapshot, Bytes 7,6
0x0051        0x0000    Time Snapshot, Bytes 5,4
0x0052        0x0000    Time Snapshot, Bytes 3,2
0x0053        0x0000    Time Snapshot, Bytes 1,0

Time of last sync is not applicable (not used for det sync).

Time Stamping of Rb Clock with existing TDUs

The Rb clock's 1pps output was connected to the mdat input on the TDUs currently installed at ash river. The 1pps was then time stamped by each TDU independently.

The following commands were issues:

  • If an existing shared memory segment was in place (or needed to be cleared) it was removed with "ipcrm shm <shmid>"
  • Spill server stand alone app was started with: startSpillServerApp_Standalone.sh -t <tdu_name>. This script was in the testRel for these tests.
  • No other configurations were required to start reading out the 1Hz events.
  • 1Hz (input) from the RB clock are registered on the mdat line which in software is denoted as the BNB line.

Test Results

Test results were stored in TDU_Spill_Logs and named with the convention "SpillServerApp_Standalone_Log_<tdu_networkname>_<date stamp>_<timestamp>.txt

TDU-Master-01

Preliminary phase offset for existing primary master was:

759.574234 ms (240.425766 ms)

Standard Time Syncs were then issued.

The time stamps of the 1Hz were then recorded. This was performed 5 times interactively and the results were observed to have no significant deviation from the initial phase measurement. The sync series was then performed in continuous loops with a 12 second spacing between syncs and the results were logged to look for discontinuities in the timestamp logs.

TDU-Master-02

Preliminary phase offset for existing secondary master was:

759.574250 ms (240.42575 ms)

Standard Time Syncs were then issued.

The time stamps of the 1Hz were then recorded. This was performed 5 times interactively and the results were observed to have no significant deviation from the initial phase measurement. The sync series was then performed in continuous loops with a 12 second spacing between syncs and the results were logged to look for discontinuities in the timestamp logs.

TDU-Master-04

TDU-Master-04 was not active. (no clock started). We issued a sync series and had an unexpected result. The TDU issued a sync for:

[root@tdu-master-ppc-04:testRelForSpillServer_Timing]$ NovaTimeConvert 0x0028d3ced4002000

A NOvA base time of 11491884341927936 corresponds to...
  a UNIX time of 1441864691 sec, -29791232 nsec
  a calendar date of 2015-Sep-10 05:58:11.842624000000 UTC

We proceeded to scrub the system for this TDU.

A sync was then issued to start the clock. The result was a time phase offsite:

759.574218 ms (240.425782 ms)

Preliminary phase offset for existing secondary master was:
ms ( ms)

Standard Time Syncs were then issued.

The time stamps of the 1Hz were then recorded. This was performed 5 times interactively and the results were observed to have no significant deviation from the initial phase measurement. The sync series was then performed in continuous loops with a 12 second spacing between syncs and the results were logged to look for discontinuities in the timestamp logs.

TDU-Master-05 (New TDU)

The new TDU was connected to the 4th GPS antenna and allowed to settle for 20 minutes before tests were performed.

The first measurement of the phase offset yielded:

759.638156 ms (240.361844 ms)

This is an offset of 64us from the other clocks. We have seen failure modes like this in the past. We initiated a scrub of the new system (wrote 0x0004 to register 0x09)

The system came back with the same phase of:

759.638156 ms

The unit was scrubbed again. This time the system gave a phase offset of:

759.574218 ms (240.425782 ms)

which is in line with previous measurements from the other 3 TDUs.

The 1 pulse per second output was then activated (wrote a value of 0x0001 to register 0x001a which produces a pulse of width 7.6 micro seconds).

The time interval counter was fed this output pulse (into input 2) and the Rb clock 1 pps into input 1. The phase as measured by the interval counter was:

+240.425792 ms

This replaced and is now known as TDU-Master-01. The log file for the time interval counter measurement is below

TDU-Master-06 (New TDU)

This unit was racked in and allowed to settle after power on. The phase was against the internal tdu timestamp as:

759.574218 (240.425782 ms)

And then against the time interval counter:

+240.425815 ms

This unit replaced and is now known as TDU-Master-03. The log file for the time interval measurement is below.

TDU-Master-13 (New TDU)

This unit was racked in and allowed to settle after power on. The phase was measured using the time interval counter as:
+240.425845 ms

This unit replace and is now known as TDU-Master-02. The log file for the time interval measurement is below.