Project

General

Profile

Development DCS intro

The development DCS is running on mu2edaq04 (as of August 1, 2018). This intro covers what is running and how to access it.

What is running on the development system (August 1, 2018)

The following are running, all under the mu2edcs user. All start automatically. (Exception: the postgres server does not start automatically as of 2018-08-01.)

  • Two EPICS IO Controllers; both are "soft" IOCs.
  • A test postgres server. (Start with pg_ctl start -o "-k /home/gahs/tmp/run" -D /home/gahs/testdb -l /home/gahs/testdb/logfile)
  • An ApacheMQ java message server (needed for the Control System Studio alarm system).
  • The CSS JMS2RDB application, which records alarm state changes and certain log messages in the database dcs_log.
  • The CSS ArchiveEngine, which records values from EPICS in the database dcs_archive.
  • The CSS AlarmServer, which monitors EPICS channels, sends alarm state change info to ApacheMQ, and records state in the database dcs_alarm.

Many of these processes are started inside "screen" to capture console output. (The IOCs also allow console input.)

The processes shown by ps and screen -ls are shown in a later section below.

In addition, a cron job periodically runs programs to import data from IFBeamDB and from Ganglia into EPICS.

How to access the development system

Since mu2edaq01.fnal.gov is the only host which accepts ssh connections from outside the pilot system, you need to log into mu2edaq01.fnal.gov first.
From there, you can do the following:

  • Look at the files in /home/mu2edcs/mu2e-dcs/. (The directory names and README files are hopefully self-explanatory.)
  • Set up epics and execute various command line tools, such as "caget", "cainfo", etc.
  • Start a Control System Studio gui.
  • Log into another node, such as mu2edaq04, and do the same things there.
  • Log into mu2edaq04 and look at the processes running there under the mu2edcs account using ps.
  • Log into mu2edaq04 as the mu2edcs user (if your kerberos principal is in the .k5login) and start and stop mu2edcs processes.

To set up EPICS and Control System Studio after logging in

Issue the following command:

source /home/mu2edcs/bin/setup-EPICS.sh

Useful EPICS commands

Examples are caget, cainfo, and camonitor. Issuing these commands with "-h" as the only option will print help. Documentation
can be found at https://epics.anl.gov/base/R3-15/5-docs/CAref.html#CommandTools.

These commands require the name of an EPICS "process variable" (PV). An example is Mu2e_Test_Test_1/ai. E.g., you could issue the command caget Mu2e_Test_Test_1/ai, which should print out the current value of the PV. More examples of PV variables can be found in the various gui panels (see below).

Start a control system studio gui.

(The very first time you run CSS on a host, there is an extra setup step. See below for that.)

Issue the following command: run_css. This will bring up the CSS gui.

First time setup

Only if you have never run a CSS gui under your own account before**, then do the following:

       mkdir -p $HOME/ControlSystemStudio/krb5--as-$(whoami)-on-ubdaq-prod-ws01/
       cp -a /home/gahs/ControlSystemStudio/krb5--as-ubooneshift-on-ubdaq-prod-ws01/CSS \
             $HOME/ControlSystemStudio/krb5--as-$(whoami)-on-ubdaq-prod-ws01/
       mkdir -p $HOME/.eclipse/org.eclipse.equinox.security/
       cp -a /home/gahs/.eclipse/org.eclipse.equinox.security/secure_storage  \
             $HOME/.eclipse/org.eclipse.equinox.security/

Interesting things to do in CSS

Various windows and panels can be opened and closed in CSS. Explore the Window > Open Perspective and Window > Open View menu items. See also "Help".

A particularly useful "View" is the Navigator. With this, you can open various operator panel interface (OPI) files and data browser plot (PLT) files.

Processes running on mu2edcs@mu2edaq04 as shown by ps and screen -ls

ps

As any user on mu2edaq04, you can see this:

[gahs@mu2edaq04 ~]$ ps aux|grep ^mu2edcs
mu2edcs    590  0.0  0.0 300612 11300 pts/9    S+   12:41   0:00 /home/mu2edcs/opt/jms2rdb-4.5.0/JMS2RDB -data workspace/ -pluginCustomization plugin_customization.ini
mu2edcs    633  0.1  0.4 14716496 156860 pts/9 Sl+  12:41   0:16 /usr/bin/java -Djava.awt.headless=true -jar /home/mu2edcs/opt/jms2rdb-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar -os linux -ws gtk -arch x86_64 -showsplash -launcher /home/mu2edcs/opt/jms2rdb-4.5.0/JMS2RDB -name JMS2RDB --launcher.library /home/mu2edcs/opt/jms2rdb-4.5.0//plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.551.v20171108-1834/eclipse_1630.so -startup /home/mu2edcs/opt/jms2rdb-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar --launcher.overrideVmargs -exitdata d508016 -data workspace/ -pluginCustomization plugin_customization.ini -vm /usr/bin/java -vmargs -Djava.awt.headless=true -jar /home/mu2edcs/opt/jms2rdb-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar
mu2edcs    698  0.0  0.0 236380  7228 ?        Ss   12:41   0:00 postgres: dcs_writer dcs_log 127.0.0.1(48158) idle
mu2edcs   5092  0.0  0.0 237092  6732 ?        Ss   Jul23   0:00 postgres: dcs_writer dcs_alarm 127.0.0.1(40796) idle
mu2edcs  10203  0.0  0.0  16316   868 pts/6    S    Jul23   0:00 dbus-launch --autolaunch d6b6ac88f8e24142bb3b7751c35b707e --binary-syntax --close-stderr
mu2edcs  10204  0.0  0.0  22728   832 ?        Ss   Jul23   0:00 /bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
mu2edcs  10206  0.0  0.0 152536  2724 ?        S    Jul23   0:01 /usr/libexec/gconfd-2
mu2edcs  15286  0.0  0.0 234944  9460 pts/6    S    Jul23   0:36 /usr/bin/postgres -D /home/mu2edcs/pgsql/data -k /home/mu2edcs/pgsql/run
mu2edcs  15287  0.0  0.0 194696  1604 ?        Ss   Jul23   0:00 postgres: logger process   
mu2edcs  15289  0.0  0.1 235044 35596 ?        Ss   Jul23   0:13 postgres: checkpointer process   
mu2edcs  15290  0.0  0.0 234944  2000 ?        Ss   Jul23   0:04 postgres: writer process   
mu2edcs  15291  0.0  0.0 234944  2188 ?        Ss   Jul23   0:03 postgres: wal writer process   
mu2edcs  15292  0.0  0.0 235992  3308 ?        Ss   Jul23   1:35 postgres: autovacuum launcher process   
mu2edcs  15293  0.0  0.0 194980  1952 ?        Ss   Jul23   1:12 postgres: stats collector process   
mu2edcs  17537  0.0  0.0 130236  1308 ?        Ss   Jul23   0:03 SCREEN -L -d -m -S BeamDataIoc env LD_LIBRARY_PATH=/mu2e/ups/epics/v3_15_4/Linux64bit+3.10-2.17-e10/lib/linux-x86_64:/mu2e/ups/gcc/v4_9_3a/Linux64bit+3.10-2.17/lib64:/mu2e/ups/gcc/v4_9_3a/Linux64bit+3.10-2.17/lib:/mu2e/ups/git/v2_15_1/Linux64bit+3.10-2.17/lib64 softIoc -D /mu2e/ups/epics/v3_15_4/Linux64bit+3.10-2.17-e10/dbd/softIoc.dbd BeamData.iocsh
mu2edcs  17541  0.0  0.0 1556916 4696 pts/2    Ssl+ Jul23  11:49 softIoc -D /mu2e/ups/epics/v3_15_4/Linux64bit+3.10-2.17-e10/dbd/softIoc.dbd BeamData.iocsh
mu2edcs  17545  0.0  0.0 130236  1316 ?        Ss   Jul23   0:03 SCREEN -L -d -m -S GangliaIoc env LD_LIBRARY_PATH=/mu2e/ups/epics/v3_15_4/Linux64bit+3.10-2.17-e10/lib/linux-x86_64:/mu2e/ups/gcc/v4_9_3a/Linux64bit+3.10-2.17/lib64:/mu2e/ups/gcc/v4_9_3a/Linux64bit+3.10-2.17/lib:/mu2e/ups/git/v2_15_1/Linux64bit+3.10-2.17/lib64 softIoc -D /mu2e/ups/epics/v3_15_4/Linux64bit+3.10-2.17-e10/dbd/softIoc.dbd Ganglia.iocsh
mu2edcs  17550  0.1  0.0 1691220 5272 pts/3    Ssl+ Jul23  16:19 softIoc -D /mu2e/ups/epics/v3_15_4/Linux64bit+3.10-2.17-e10/dbd/softIoc.dbd Ganglia.iocsh
mu2edcs  24162  0.1  1.0 7835272 360012 pts/6  Sl   Jul23  13:47 /usr/bin/java -Xms64M -Xmx1G -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/home/mu2edcs/opt/apache-activemq-5.15.4//conf/login.config -Dcom.sun.management.jmxremote -Djava.awt.headless=true -Djava.io.tmpdir=/home/mu2edcs/opt/apache-activemq-5.15.4//tmp -Dactivemq.classpath=/home/mu2edcs/opt/apache-activemq-5.15.4//conf:/home/mu2edcs/opt/apache-activemq-5.15.4//../lib/: -Dactivemq.home=/home/mu2edcs/opt/apache-activemq-5.15.4/ -Dactivemq.base=/home/mu2edcs/opt/apache-activemq-5.15.4/ -Dactivemq.conf=/home/mu2edcs/opt/apache-activemq-5.15.4//conf -Dactivemq.data=/home/mu2edcs/opt/apache-activemq-5.15.4//data -jar /home/mu2edcs/opt/apache-activemq-5.15.4//bin/activemq.jar start
mu2edcs  26020  0.0  0.0 130236  1312 ?        Ss   Jul23   0:05 SCREEN -L -d -m -S ArchiveEngine ./run_archiver
mu2edcs  26024  0.0  0.0 113136  1616 pts/7    Ss+  Jul23   0:00 /bin/bash ./run_archiver
mu2edcs  26050  0.0  0.0 130236  1316 ?        Ss   Jul23   0:03 SCREEN -L -d -m -S AlarmServer ./run_alarmserver
mu2edcs  26053  0.0  0.0 113136  1628 pts/8    Ss+  Jul23   0:00 /bin/bash ./run_alarmserver
mu2edcs  26823  0.0  0.0 114256   952 pts/7    S+   Jul23   0:00 /home/mu2edcs/opt/archive-engine-4.5.0/ArchiveEngine -engine Mu2e -port 4812 -data /home/mu2edcs/mu2e-dcs/apps/archiver_setup/archiver-workspace -pluginCustomization /home/mu2edcs/mu2e-dcs/apps/archiver_setup/archiver_settings-dev.ini
mu2edcs  26949  0.1  0.7 7145696 241540 pts/7  Sl+  Jul23  13:19 /usr/bin/java -Djava.awt.headless=true -Xms64m -Xmx256m -Declipse.exitdata= -jar /home/mu2edcs/opt/archive-engine-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar -os linux -ws gtk -arch x86_64 -showsplash -launcher /home/mu2edcs/opt/archive-engine-4.5.0/ArchiveEngine -name ArchiveEngine --launcher.library /home/mu2edcs/opt/archive-engine-4.5.0//plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.551.v20171108-1834/eclipse_1630.so -startup /home/mu2edcs/opt/archive-engine-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar --launcher.overrideVmargs -exitdata 3e0013 -engine Mu2e -port 4812 -data /home/mu2edcs/mu2e-dcs/apps/archiver_setup/archiver-workspace -pluginCustomization /home/mu2edcs/mu2e-dcs/apps/archiver_setup/archiver_settings-dev.ini -vm /usr/bin/java -vmargs -Djava.awt.headless=true -Xms64m -Xmx256m -Declipse.exitdata= -jar /home/mu2edcs/opt/archive-engine-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar
mu2edcs  26976  0.0  0.0 114256   956 pts/8    S+   Jul23   0:00 /home/mu2edcs/opt/alarm-server-4.5.0/AlarmServer -root Mu2e_alarm -data /home/mu2edcs/mu2e-dcs/apps/alarmserver_setup/alarm-workspace -pluginCustomization /home/mu2edcs/mu2e-dcs/apps/alarmserver_setup/alarmserver_settings-dev.ini
mu2edcs  27033  0.1  0.4 6143000 152080 pts/8  Sl+  Jul23  14:54 /usr/bin/java -Declipse.exitdata= -Djava.awt.headless=true -Xms64m -Xmx256m -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.defaultlog=fatal -jar /home/mu2edcs/opt/alarm-server-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar -os linux -ws gtk -arch x86_64 -showsplash -launcher /home/mu2edcs/opt/alarm-server-4.5.0/AlarmServer -name AlarmServer --launcher.library /home/mu2edcs/opt/alarm-server-4.5.0//plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.551.v20171108-1834/eclipse_1630.so -startup /home/mu2edcs/opt/alarm-server-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar --launcher.overrideVmargs -exitdata 3e8014 -console -root Mu2e_alarm -data /home/mu2edcs/mu2e-dcs/apps/alarmserver_setup/alarm-workspace -pluginCustomization /home/mu2edcs/mu2e-dcs/apps/alarmserver_setup/alarmserver_settings-dev.ini -vm /usr/bin/java -vmargs -Declipse.exitdata= -Djava.awt.headless=true -Xms64m -Xmx256m -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.defaultlog=fatal -jar /home/mu2edcs/opt/alarm-server-4.5.0//plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar
mu2edcs  27111  0.0  0.0 237092  7676 ?        Ss   Jul23   0:00 postgres: dcs_writer dcs_alarm 127.0.0.1(39908) idle
mu2edcs  27125  0.0  0.1 237104 39456 ?        Ss   Jul23   0:49 postgres: dcs_writer dcs_archive 127.0.0.1(39912) idle
mu2edcs  32742  0.0  0.0 130236  1216 ?        Ss   12:41   0:00 SCREEN -L -d -m -S JMS2RDB ./run_JMS2RDB
mu2edcs  32743  0.0  0.0 113136  1552 pts/9    Ss+  12:41   0:00 /bin/bash ./run_JMS2RDB

screen

Many of the slow control servers run inside GNU screen. As the mu2edcs user on mu2edaq04, you can see them like this:

[mu2edcs@mu2edaq04 ~]$ screen -ls
There are screens on:
    32742.JMS2RDB    (Detached)
    26050.AlarmServer    (Detached)
    26020.ArchiveEngine    (Detached)
    17545.GangliaIoc    (Detached)
    17537.BeamDataIoc    (Detached)
5 Sockets in /var/run/screen/S-mu2edcs.