Project

General

Profile

FC7

Manual

The user manual can be found here: fc7_user_manual.pdf

Channel numbering

The fiber numbering on the TCDS board when attached to the FC7 is as follows:

Top

3 7
2 6
1 5
0 4

Bottom

Firmware

Bin files and a changelog are here:
http://www.hep.ucl.ac.uk/~warren/g-2/firmware/?C=M;O=D
also at UCL /home/warren/g-2/gm2trackerdaq/hardware/FC7_firmware/fc7_4.0.7/fw/prj/fpga_fc7_ddr3_ise/*.bin

Program SD card

Instructions on programming the SD card to use for the FC7 firmware can be found in fc7_user_manual.pdf in the section "Indirect programming of FPGA via microSD card". Assuming you already have the .bin file, start from bullet 5 if you need to get the code for the first time, or bullet 7 is you have already gotten and built the code.

Program using card reader

Here is an example of the commands we send (assuming have .bin file and code is built). Note that root/sudo access is required.

WARNING: Make sure you use the correct device (as will format the disk). /dev/sdb is a common choice for device name (as the OS is installed on /dev/sda - do not erase this!).

This uses a bin (not bit) file.

cd ic_mmc_v1_6_0/imperial_mmc/tools/imgtool/bin
sudo /sbin/fdisk -l
sudo ./imgtool <device> format GoldenImage.bin
sudo ./imgtool <device> add GoldenImage.bin <desired FW>.bin

The device is probably something like /dev/sdb (unlike when using IPbus).

Once programmed, insert the SC card into the slot on the FC7 front (metal contacts facing away from board) and cycle the FC7 (by pulling out then pushing in the lever). A purple flashing LED near the bottom of the board front should appear eventually (after ~10s) if this has worked. Note that a flashing green LED near the FC7 top (and no other LEDs) means no firmware.

Occasionally this fails, meaning the firmware is never loaded by the FC7 when you plug the SC card in, e.g. the green LED stays flashing. One thing to look for is the mesage "Bad magic number in superblock" during the card loading, or generally the card loading being too fast (e.g. instantaneous). In this case you need to load the firmware onto the memory card again and it typically works.

Program via IPbus

WARNING: This doesn't seem to work (hangs whilst writing via IPbus) and needs investigating.

The following assumes you have the fc7 repository installed locally on your system, and it is pointed to by the environment variable FC7_ROOT.

This uses a bit (not bin) file (unlike when programming SD card using imgtool).

cd gm2trackerdaq/software
source setup.sh
make fc7                                                             #Build the FC7 common tools
cd $FC7_ROOT/fc7/tests                                               #Must run from this directory as tools have hard-coded paths
fc7-userimage-reprogram.exe -i <FC7 IP address> -f <bit file>
fc7-boot-userimage.exe -i <FC7 IP address>

Convert firmware .bit file to .bin file

If you have a .bit file but want a .bin file, you can convert as follows (needs to be on a PC with Xilinx tools like impact installed):

First set up Xilinx environment (XILINX_SETTINGS should be defined in your .env file to point to your Xilinx installation settings file, e.g. /opt/Xilinx/14.6/LabTools/settings64.sh):

source $XILINX_SETTINGS
promgen -w -p bin -c FF -o <output file stem> -s 32768 -u 0 <input bit file>

where <output file stem> is the filename you want the .bin file to have, but with out ".bin". E.g.

promgen -w -p bin -c FF -o $HOME/fc7_top -s 32768 -u 0 $HOME/fc7_top.bit

Set IP address

The following assumes you have the fc7 repository installed locally on your system, and it is pointed to by the environment variable FC7_ROOT.

python $FC7_ROOT/sw/fc7/tests/scripts/ipmi_helper.py -m <MCH IP> -s <slot num> -I <FC7 IP> -N -p

The "-p" is required to make the change persistent (write to FC7 flash).

Set MAC address

python $FC7_ROOT/sw/fc7/tests/scripts/ipmi_helper.py -m <MCH IP> -s <slot num> -M <MAC address> -N -p

<MAC address> are formatted as "fc:07:00:03:00:52", where "03" here is the sticker number we have put on the board and the "52" is the original CERN/Imperial serial number of the form: 1549000 52. The MAC address is enclosed in double quotes and has to be lowercase.

The "-p" is required to make the change persistent (write to FC7 flash).

Check the MAC address using:

arp -a <IP address>

MAC address conventions

All FC7s have the same MAC address by default which causes problems when they are on the same subnet. Therefore we need to change them to unique values.

The convention we are using is: FC:07:<sticker num>, where <sticker num> is the number of the white stickers on ech FC7 board, passed with zeros from the left (it must be 8 hex characters long). E.g. for FC7 with sticker number 001, the MAC would be FC:07:00:00:01.

Power cycle FC7 remotely

python $FC7_ROOT/sw/fc7/tests/scripts/ipmi_helper.py -m <MCH IP> -s <slot num> --hard_reset

Assembly

  • Heat sync
    • Put grey plastic square over the KINTEX-7 FPGA (bevel side down) so that the lips click under it
    • Put heat sink on so air flows through ridges vertically
    • Put metal clip vertically over it to hold in place
  • Bottom edge clip
    • This is the one with the level
    • Put it on bottom front edge of point with level point fowards
    • The screw hole should line up with the board hole, and the little clip should enter the slot on the FC7 board (the bottom of the two)
    • Place the plastic guide (the transparent one without a black bit inside) over the screw hole
    • Screw in
    • Clip the two plastic parts of the level handle over the metal level (align the plastic pins with the metal holes)
  • Top edge clip
    • Place the other transparent plastic guide (the one with the black bit) in the other edge clip (metal, no level) so that the screw holes align
    • Place the edge clip on the top front corner so that the screw holes align
    • Screw into place
  • Metal front plate and conductive strips
    • Don't need these
  • Make sure the following sets of switches are in the position photographed:


Trouble-shooting

Can ping FC7 but cannot connect via IPbus

This probably means that the AMC slot in the uTCA that the FC7 is plugged into is not enabled by the AMC13, meaning the FC7 is not receiving a clock. Enabled it (either using the DAQ, or AMC13Tool2.exe).

Board does not power up

If the FC7 does not have any LEDs on when plugged into the crates (and the crate is on), it might be because it is switched to benchtop mode. There is a little switch on the lower back of the board, switch this to "crate".

Gettings IPbus read/write timeouts often but not always

This happens when there are multiple FC7s on the same network with the same MAC address. When only one is powered on, it works every time. But with multiple then things get confused in the switch. Make sure all FC7s have unique MAC addresses (see above for how to set this).

imgtool for Mac users

Install gcc-4.9 on your MAC using homebrew:

Install homebrew if you don't have it:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" 
sudo brew prune
brew doctor

Then
brew install homebrew/versions/gcc49

 
cd /gm2trackerdaq/software/imgtool/imgtool 
make clean
make Board=FC7_0 
cd bin
diskutil list

Determine the correct X i.e /dev/diskX for the SD card: be careful here, get the wrong one and you could reformat the hard-drive of your MAC !

 
diskutil unmountDisk /dev/diskX
sudo ./imgtool /dev/diskX format GoldenImage.bin 

Get the appropriate firmware bin file and rename it be "GoldenImage.bin" and then write this bin file to the SD-card

 sudo ./imgtool /dev/diskX add GoldenImage.bin GoldenImage.bin