Project

General

Profile

FTS for the Laser Scanner » History » Version 5

Andrew Norman, 10/30/2012 04:52 PM

1 1 Andrew Norman
h1. FTS for the Laser Scanner
2 1 Andrew Norman
3 1 Andrew Norman
The file transfer service has been setup as a transport layer for data taken with the laser scanner at Ash River.  Setting up the system required some additional steps to make it possible to bridge the gap between the machine that runs the laser scanner software (which is a windows 7 laptop) and the linux based system that runs the file transfer service.
4 1 Andrew Norman
5 2 Andrew Norman
h1. General Topology and Dataflow
6 1 Andrew Norman
7 1 Andrew Norman
We have setup the FTS system in a slightly different manner for use with the laser scanner than we have with the rest of the DAQ.  The data is generated on a windows based pc that is connected to a laser scanner.  The data is then processed locally on the windows machine to generate a set of files for further analysis.  These files are bundled into a "tar" format archive file and placed in the c:\Export directory of the windows machine.
8 1 Andrew Norman
9 1 Andrew Norman
The machine running the FTS instance is then setup to mount the c:\Export directory from the windows machine, and performs a synchronization of the files that it finds in that area with another directory that resides on the FTS machine's local disk array.  The area that is sync'd to is currently setup as /assembly which is a symbolic link to /data-b/assembly (a directory on the secondary raid array of the disk).
10 1 Andrew Norman
11 1 Andrew Norman
This area (/assembly) is set as the dropbox location that the FTS system will look in for new files.
12 1 Andrew Norman
13 2 Andrew Norman
When a new file is found it is transfered via an FTS relay (being hosted from novasamgpvm01.fnal.gov) to an intermediate area on the Fermilab bluearc (/nova/ana/assembly_fts).  This area is set as the drop box for the relay FTS agent which then transfers the files to both the enstore tape system and to a well organized area on bluearc designed to be used for analysis (/nova/ana/assembly_ana).  In these areas the files are organized by directories corresponding to the numbers of the blocks and then again in directories corresponding to the layers numbers.
14 2 Andrew Norman
15 2 Andrew Norman
h2. External Mount
16 2 Andrew Norman
17 2 Andrew Norman
In order to mount the disk from the machine that is taking the laser scanner data the following steps were taken:
18 2 Andrew Norman
19 2 Andrew Norman
* The cifs file system extensions had to be installed (part of base Linux install on this machine)
20 2 Andrew Norman
* Optionally the cifs-utils package can be installed to aid with debugging (this was done on novadcs-far-master)
21 2 Andrew Norman
22 2 Andrew Norman
The file system is mounted using the command:
23 2 Andrew Norman
<pre>
24 2 Andrew Norman
mount -t cifs //198.124.69.26/Export -o ro,username=LAZER\ SCANNER,password=THEPASSWORD,forceuid,forcegid,uid=novaraw,gid=nova /mnt/laser_scan
25 2 Andrew Norman
</pre>
26 3 Andrew Norman
27 3 Andrew Norman
Note:  You must use the ip address (not the network name) in the mount command in order for the windows system to correctly recognize this.  Also, in order for the files in the volume to have the proper uid/gid combinations (so we can do things with them) we force a mapping to novaraw.nova so that the scripts that run the sync can be run as novaraw and do the right thing under rsync.
28 3 Andrew Norman
29 3 Andrew Norman
Now because we are working with a machine that lives on a different network (and can go up and down due to users cycling it) we need a way to make sure that if the mount is lost, we can restore it.  To do this we have a small script which is run from the crontab which checks if the volume is mounted and if it isn't, mounts it.
30 3 Andrew Norman
31 3 Andrew Norman
The script is called "mount-laser-scanner.sh" and must be run from root's crontab in order to mount the disk.
32 3 Andrew Norman
33 3 Andrew Norman
The script currently reads:
34 3 Andrew Norman
35 3 Andrew Norman
<pre>
36 3 Andrew Norman
#!/bin/bash
37 3 Andrew Norman
# This script ensures that the volume containing the 
38 3 Andrew Norman
# laser scanner data is mounted by the machine that 
39 3 Andrew Norman
# is running the FTS system
40 3 Andrew Norman
#
41 3 Andrew Norman
MOUNTPOINT=/mnt/laser_scan
42 3 Andrew Norman
SOURCEIP=198.124.69.26
43 3 Andrew Norman
SOURCEVOL=Export
44 3 Andrew Norman
USERNAME="LAZER SCANNER"
45 3 Andrew Norman
PASS=CENSORED
46 3 Andrew Norman
DATE=date
47 3 Andrew Norman
HOSTNAME=hostname
48 3 Andrew Norman
MOUNT_UID=novaraw
49 3 Andrew Norman
MOUNT_GID=nova
50 3 Andrew Norman
/bin/mountpoint -q $MOUNTPOINT
51 3 Andrew Norman
if [ $? -ne 0 ]; then 
52 3 Andrew Norman
  mount -t cifs //${SOURCEIP}/${SOURCEVOL} -o ro,username=LAZER\ SCANNER,passwor
53 3 Andrew Norman
d=${PASS},forceuid,forcegid,uid=${MOUNT_UID},gid=${MOUNT_GID} ${MOUNTPOINT}
54 3 Andrew Norman
  if [ $? -ne 0 ]; then
55 3 Andrew Norman
    echo Failed to mount $MOUNTPOINT on $HOSTNAME at $DATE
56 3 Andrew Norman
  else
57 3 Andrew Norman
    echo Remounted $MOUNTPOINT on $HOSTNAME at $DATE 
58 3 Andrew Norman
  fi
59 3 Andrew Norman
#else
60 3 Andrew Norman
# echo Already mounted /mnt/laser_scan 
61 3 Andrew Norman
fi
62 3 Andrew Norman
</pre>
63 3 Andrew Norman
64 3 Andrew Norman
Root's crontab is then setup as:
65 3 Andrew Norman
<pre>
66 3 Andrew Norman
#
67 3 Andrew Norman
MAILTO=anorman@fnal.gov
68 3 Andrew Norman
#
69 3 Andrew Norman
# Run every 10 Minutes to restore mount points
70 3 Andrew Norman
*/10 * * * * /root/bin/mount-nova-disks.sh
71 3 Andrew Norman
*/10 * * * * /root/bin/mount-laser-scanner.sh
72 3 Andrew Norman
</pre>
73 3 Andrew Norman
74 3 Andrew Norman
So that it checks every ten minutes to make sure that the mount is up.
75 4 Andrew Norman
76 4 Andrew Norman
h2. Sync from Laser to Local
77 4 Andrew Norman
78 4 Andrew Norman
The sync from the (remote) Laser scanner machine to the local disk on the FTS machine is accomplished through a simple rsync.  The script is located in the bin directory of the novaraw user on novadaq-far-datadisk-03.  The script uses a VERY simple locking mechanism to ensure that rsyncs don't pile up.  The script reads as:
79 4 Andrew Norman
<pre>
80 4 Andrew Norman
#!/bin/bash
81 4 Andrew Norman
#
82 4 Andrew Norman
# This is a script that synchronizes the data 
83 4 Andrew Norman
# that is stored on the laser scanner machine (windows 7)
84 4 Andrew Norman
# to an area on the data disk that is watched by the FTS system
85 4 Andrew Norman
#
86 4 Andrew Norman
# This script is designed to be run via a crontab
87 4 Andrew Norman
SRC_TREE=/mnt/laser_scan/
88 4 Andrew Norman
DEST_TREE=/assembly/
89 4 Andrew Norman
90 4 Andrew Norman
LOCKFILE=/tmp/lazer-scanner-copy-lock
91 4 Andrew Norman
# Take out a lock so that we don't get multiple rsyncs running at
92 4 Andrew Norman
# the same time
93 4 Andrew Norman
if [ -f $LOCKFILE ]; then
94 4 Andrew Norman
  # A lock exists so we check the age and clear it
95 4 Andrew Norman
  # if it is stale
96 4 Andrew Norman
  echo Lock Exists
97 4 Andrew Norman
else
98 4 Andrew Norman
  # No lock exists so we take one out
99 4 Andrew Norman
  touch $LOCKFILE
100 4 Andrew Norman
  # Now we perform our sync
101 4 Andrew Norman
  rsync -av ${SRC_TREE} ${DEST_TREE}
102 4 Andrew Norman
  # Clean up our lock file
103 4 Andrew Norman
  rm $LOCKFILE
104 4 Andrew Norman
fi
105 4 Andrew Norman
</pre>
106 4 Andrew Norman
107 4 Andrew Norman
The novaraw account then has a crontab that reads:
108 4 Andrew Norman
<pre>
109 4 Andrew Norman
#
110 4 Andrew Norman
#MAILTO=anorman@fnal.gov
111 4 Andrew Norman
#
112 4 Andrew Norman
# Run every 4 hours to sync data from the laser scanner machine
113 4 Andrew Norman
# to this machine for the FTS
114 4 Andrew Norman
30 */4 * * * /home/novaraw/bin/sync_laser_scanner_data.sh
115 4 Andrew Norman
</pre>
116 4 Andrew Norman
117 4 Andrew Norman
Which triggers an rsync every 4 hours between the areas.
118 4 Andrew Norman
119 4 Andrew Norman
h2. FTS instance
120 4 Andrew Norman
121 4 Andrew Norman
After the files are copied into the FTS dropbox, they are handed over to the FTS system for transfer, validation, deletion.  The status of files which are under FTS control can be found at:
122 4 Andrew Norman
123 5 Andrew Norman
http://novadaq-far-datadisk-03.fnal.gov:8888/fts/status
124 4 Andrew Norman
125 4 Andrew Norman
The FTS instance is installed under the novaraw user and is setup in the standard way.  The instructions for starting/stopping the FTS are found on the main FTS page.