Project

General

Profile

The adinstbpm Primer - Quick explanation of adinstbpm operation basics

Logging In

To login you will need to first ssh to outland and then ssh to the BPM node:

outland$ ssh iotabp
X11 forwarding request failed on channel 0
                              NOTICE TO USERS

       This  is a Federal computer (and/or it is directly connected to a
       Fermilab local network system) that is the property of the United
       States Government.  It is for authorized use only.  Users (autho-
       rized or unauthorized) have no explicit or  implicit  expectation
       of privacy.

       Any  or  all uses of this system and all files on this system may
       be intercepted, monitored, recorded,  copied, audited, inspected,
       and  disclosed  to authorized site, Department of Energy  and law
       enforcement personnel, as  well as authorized officials of  other
       agencies,  both  domestic and foreign.  By using this system, the
       user consents to such interception, monitoring, recording,  copy-
       ing,  auditing,  inspection,  and disclosure at the discretion of
       authorized site or Department of Energy personnel.

       Unauthorized or improper use of this system may result in  admin-
       istrative  disciplinary  action and civil and criminal penalties.
       By continuing to use this system you indicate your  awareness  of
       and  consent to these terms and conditions of use.  LOG OFF IMME-
       DIATELY if you do not agree to  the  conditions  stated  in  this
       warning.

       Fermilab  policy  and  rules for computing, including appropriate
       use, may be found at http://www.fnal.gov/cd/main/cpolicy.html

[jdiamond@iotabp ~]$

Logging in requires that your username be in the node's /etc/passwd file. You may have to ask an expert to help add you to this file.

Become root

Most bpmcli operations require you to be root. Use ksu to become root:

[jdiamond@iotabp ~]$ ksu
[root@iotabp ~]#

Start bpmd

To start the BPM application (bpmd), change directory to the node's NFS mount and run the startup script:

[root@iotabp ~]# cd /fecode-bd/iotabp
[root@iotabp ~]# ./test-start.sh

bpmd should be running now. To verify, use the 'bpmcli show' command:

[root@iotabp iotabp]# ./bpmcli show

This should display a lot of useful information. Pay attention to the DAQ System section towards the end, where you will see the number of DAQ modules detected:

===== DAQ System =====

Number of DAQ modules present: 2

Taking a Measurement Manually

The bpmcli program can be used to manually arm the DAQ modules for a measurement:

[root@bbpmt1 bbpmts]# ./bpmcli arm -i0
ARM command sent to bpmd.

This will arm the DAQ modules to take a measurement. When a trigger is detected, the DAQ modules will interrupt the CPU to inform us that a measurement has been completed. You can use 'bpmcli show' to verify that the measurement was successful (look in the Measurements section) -

===== Measurements =====

  Idx  Status                    Code                 Armed                       Triggered                   Machine Context

  0    Triggered / Data Ready    OK                   2018-08-08 16:06:20.921574  2018-08-08 16:06:29.912742  Booster              $fe (NO TIME)

If the measurement status says "Armed" then the DAQ modules were not triggered.

Once triggered, the measurement data is stored in the DAQ module's memory. If you arm the DAQ modules again this data will be overwritten by the new measurement data. To buffer the data for transporting to ACNET or storing in a file you must transfer the data from the module's into a 'Reading'. There are a couple of different types of readings that you can make, but the one we're most interested in is a full Turn-by-Turn reading. To transfer the measurement data into a Turn-by-Turn Reading, use 'bpmcli readout -rtbt' (the -r option control what kind of reading is made).

[root@bbpmt1 bbpmts]# ./bpmcli readout -rtbt
READOUT command sent to bpmd.

Now you should see a new reading entry in the TBT Readings section from 'bpmcli show':

===== Measurement Readings =====

  Index   Mutex     Triggered                     Samples   1st Turn

  Closed-orbit (0 / 10) -

  Flash-orbit (0 / 100) -

  Flash-turn (0 / 100) -

  Raw (0 / 10) -

  Smooth-Orbit (0 / 10) -

  Turn-by-Turn (1 / 3) -
       0  UNLOCKED    2018-08-08 16:06:29.912742       256         0

The 'UNLOCKED' indicator means that nobody is reading or writing this Reading. If you see 'LOCKED' this means that someone else is either filling the buffer (e.g. the DAQ modules) or reading it (e.g. ACNET).

To view the contents of the Turn-by-Turn reading use 'bpmcli dump -rtbt -hmt'

This produces a lot of output. The first row of data contains the row headers and tells you what you're looking at. Unfortunately, CSV data doesn't line up at all. To limit the number of turns shown us the -s option and provide to it the number of turns to dump, i.e.:

./bpmcli dump -rtbt -s20 -hmt
The other options are:
  • -rtbt - This selects the turn-by-turn reading
  • -hmt - Actually this is three arguments:
    • h - Display headers
    • m - Display magnitudes (if only positions are desired then omitting this will cut-down on the amount of output)
    • t - Display the turn and bunch numbers (first two columns of output)

Diagnostics Messages

Diagnostics messages are logged to the system log file: /var/log/messages. You can use the tail command to examine the most recent messages (the -n option controls the number of lines dumped from the end of the file):

[root@bbpmt1 bbpmts]# tail /var/log/messages -n10
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BHP23S
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BHP22L
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BHP21L
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BVP23S
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BHP21S
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BVP22S
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BVP22L
Aug  8 15:46:42 bbpmt1 user.info syslog: boosterbpm_acnet started for Z:BHP23L
Aug  8 15:46:49 bbpmt1 user.warn bpm: trigger notification received outside of the Armed state
Aug  8 15:46:49 bbpmt1 user.err bpm: BPMError caught by _handler: DAQ system trigger failed

Diagnosing DAQ Module Initialization

One of the first things the test-start.sh script does is install the drivers (kernel modules) for the DAQ modules. Lots of diagnostics messages are written to the log file.

Here is an example of the messages from iotabp this morning:

Aug  8 10:59:28 iotabp kern.alert kernel: [   68.554186] iotabpm250x8mod: Inserting kernel module.
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.559385] iotabpm250x8mod: Registering as VME driver...
Aug  8 10:59:28 iotabp kern.err kernel: [   68.565222] No VME resource Attribute tests done
Aug  8 10:59:28 iotabp kern.err kernel: [   68.572134] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0x8138, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.580709] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.584026] magicNum = 00000000
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.588257] iotabpm250x8mod: Probe succeeded for device 0 in slot 1 (0x8800)
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.595608] magicNum = 00000000
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.599656] iotabpm250x8mod: Probe succeeded for device 1 in slot 2 (0x9000)
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.607256] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.610576] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0x9938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.619176] magicNum = 00000000
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.623330] iotabpm250x8mod: Probe succeeded for device 2 in slot 4 (0xa000)
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.630834] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.634161] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xa938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.err kernel: [   68.644843] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xb138, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.653462] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.658851] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xb938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.667423] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.672809] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xc138, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.681381] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.686764] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xc938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.695335] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.700719] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xd138, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.709324] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.714711] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xd938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.723295] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.728696] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xe138, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.737269] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.742651] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xe938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.751222] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.756606] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xf138, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.765177] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.err kernel: [   68.770562] vme_tsi148 0000:06:00.0: VME Bus Error at address: 0xf938, attributes: 800ce900
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.779148] magicNum = ffffffff
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.784513] iotabpm250x8mod: OK.
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.787964] iotabpm250x8mod: /dev/iotabpm250x8mod/dsp devices created.
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.794740] iotabpm250x8mod: /dev/iotabpm250x8mod/raw devices created.
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.801740] iotabpm250x8mod: /dev/iotabpm250x8mod/cfg devices created.
Aug  8 10:59:28 iotabp kern.alert kernel: [   68.808591] iotabpm250x8mod: Initialization complete, 3 of 16 slots used.

The 'vme_tsi148' messages come from the VME bridge chip and indicate a problem detected on the VME bus. In this output you can see several 'VME Bus Error' messages thrown when we probe for a device at an address that no module is responding to. All of the addresses have an offset of +0x138 because this is the FPGA version register. We expect to read 0x00000000 from this register if the device is a 250MHz 8-channel IOTA DAQ module.

A couple of messages report finding a DAQ module and display the base address it was detected at. The final message says initialization complete and reports that it found 3 devices.

Dump BPM Data to a File

Arm and readout a new measurement first:

[root@iotabp iotabp]# bpmcli arm -i0
ARM command sent to bpmd.
[root@iotabp iotabp]# bpmcli readout -rtbt
READOUT command sent to bpmd.

Use the 'dump' command to dump positions, magnitudes and intensity while redirecting output to a text file:

[root@iotabp iotabp]# bpmcli dump -rtbt -hmnt > /fecode-bd/iotabp/data.csv

OR if you're interested in raw data:

[root@iotabp iotabp]# bpmcli readout -rraw
READOUT command sent to bpmd.
[root@iotabp iotabp]# bpmcli dump -rraw -hmnt > /fecode-bd/iotabp/data.csv

Verify that the file has data in it:

[root@iotabp iotabp]# head /fecode-bd/iotabp/data.csv
Turn-by-Turn Reading (@ shared memory offset 0x292ca90)
Measurement Index: 6
  DAQ module profile:          raw
  State:                       Triggered / Data Ready
  Machine Context:             IOTA $fe (NO TIME)
  Armed timestamp:             2018-08-15 09:05:04.737404
  Triggered timestamp:         2018-08-15 09:05:04.884662

  Readout began timestamp:     2018-08-15 09:05:16.257368
  Readout completed timestamp: 2018-08-15 09:05:21.789740

Make sure that your nova user will be able to read the file:

[root@iotabp iotabp]# chmod 755 /fecode-bd/iotabp/data.csv

Now login to nova and copy the file you made from the target NFS area into your home directory:

[jdiamond@nova ~]$ cp /fecode-bd/vxworks_write/fe/ees/iotabp/data.csv .

Connect to nova with WinSCP and download.