Migrate DBU DataQuality table filling process to near-online
Andy Blake's jobs (previously run on Near/Far minos-om machines) need to be migrated to central site.
In particular at least the tables: DBUFARRUNQUALITY[VLD] and DBUHVFROMSINGLES[VLD] need to be filled for CALDRIFT calibration processing.
#1 Updated by Andrew Blake almost 8 years ago
I've reviewed the Data Quality software at the near and far detector.
The four modules appear to be up to date in CVS:
CandMorgue/FillNearRunQuality (Near Detector)
CandMorgue/FillFarRunQuality (Far Detector)
CandMorgue/FillHvFromSingles (Far Detector)
SpillTiming/FillSpillServerMon (Far Detector)
(A cvs diff reveals that Robert has fixed some of my bad programming...
sorry and thanks...)
I've committed the ND and FD job control macros to the CandMorgue package:
runDataQualityND.C (Near Detector)
runDataQualityFD.C (Far Detector)
The Near Detector code ought to work! The Far Detector code has never been
run over non-physics runs, so this will need testing.
The Near Detector hot/warm channel finder will also need a home.
#2 Updated by Andrew Blake almost 8 years ago
All Database-related Data Quality modules are now up to date, and committed to the CandMorgue package.
This includes the refurbished hot channel finder, which is committed to the CandMorgue package.
The modules should work on Near/Far and Physics/NonPhysics data (I've tested this on some sample data).
For Non-Physics data, the default mode is to just to skip over the data.
The CandMorgue package also contains job control macros: 'runDataQualityND.C' for the Near Detector;
and 'runDataQualityFD.C' for the Far Detector. These macros run all the necessary modules, including
the hot-channel finding.
The next step should be to set up something on the minos-nearline machine (for initial testing,
the DB writing should be disabled).
#3 Updated by Robert Hatcher almost 8 years ago
Andy Blake wrote on May 29, 2013 12:12,
I just tried logging on to the minos-nearline machine and running some software.
Things didn't work first time for me. If I source the development version of minossoft, and try running root or loon, I see errors relating to various missing libraries (libcrypto, libssl, libXpm...).
I get similar results at Cambridge, when I run my SLC5 minossoft on one of our SLC6 machines.
Any suggestions ?
So what's probably happened is that the code is built against these system-level libraries; at executable link time the exact version used is scribbled into the executable and the code won't run with newer libraries (i.e.
libXYZ.so.CD). The way we've dealt with this sort of thing in the past is by putting a copy of the older libraries in one of the directories in the
$LD_LIBRARY_PATH (perhaps the
MINOS_EXTERN area, or some special SLF5 area that would get added to the path by the setup script).
What would need to happen it to determine exactly which libraries (and versions) are missing. One tool for find them is:
ldd `which loon`
and looking for which libraries didn't resolve.
#6 Updated by Arthur Kreymer almost 8 years ago
Sorry, I thought that I have included everyone on the watch list
of the request to reinstall minos-nearline with SLF 5 matching minos50.
Apparently that registration got lost.
minos-nearline should now look just like minos50.
I have run a simple loon process successfully.
#9 Updated by Andrew Blake almost 8 years ago
Far Detector DQ processing is now running on the minos-nearline machine.
Last subrun of old processing was 50105/12 (2012-10-01)
First subrun of new processing is 51598/13 (2013-06-03)
(Everything in between will need to be recovered by running the FD DQ
'runDataQualityFD.C' macro over the raw data).
The processing is running from cron, under the mindata account. Things
look to be stable, and the DB entries look okay.
#10 Updated by Andrew Blake almost 8 years ago
I've just activated the ND DQ processing, on the nearline machine.
Currently, the DB write permissions are switched off. The processing will
instead write out Andy P's ROOT ntuples, which I'm storing here:
There look to be several bad channels in the Near Detector today - a good
testing ground for the bad channel finder!
#11 Updated by Andrew Blake almost 8 years ago
My DQ webpages are also active in their new locations:
But they're still under construction!