Project

General

Profile

Feature #23131

Add reporting of fragment IDs for missing fragments to the FragmentWatcher module

Added by Kurt Biery about 1 month ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
08/15/2019
Due date:
% Done:

0%

Estimated time:
Experiment:
-
Co-Assignees:
Duration:

Description

The FragmentWatcher art module (included in artdaq/ArtModules) already has reporting of the fragment IDs of individual Empty fragments. It would be helpful to also have this level of detail in the tracking of Missing fragments.

History

#1 Updated by Kurt Biery about 1 month ago

The candidate code changes are on the feature/23131_FragWatcher_MissingFragDetail branch in the artdaq repo. This branch was created based on the v3_06_00 tag.

In addition to adding detailed tracking and reporting of the fragment IDs for missing fragments, I removed blocks of code in the constructor and destructor that dealt with the metrics_pset. This code must have been a hold-over from whatever source code file(s) that I used as a starting point for the FragmentWatcher, but my current sense is that it was very much out-of-place there. The configuration snippet for the FragmentWatcher will likely never include information about metrics.

I also tried to organize the TRACE messages and explain their intention with comments in the code.

#2 Updated by Kurt Biery about 1 month ago

I tested these changes in an artdaq-demo system on sbnd-daq33 with the mediumsystem_with_routing_master sample configuration.

Here are sample file metrics entries:

[biery@sbnd-daq33 eventbuilder]$ pwd
/home/nfs/biery/gpmTest/daqlogs/eventbuilder
[biery@sbnd-daq33 eventbuilder]$ egrep 'EventsWith|EventsProcessed' evb_13508_metrics.log
...
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsProcessed: 80 events.
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsWith50PctMissingFragments: 0 events.
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsWith10PctMissingFragments: 16 events.
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsWithMissingFragments: 16 events.
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsWith10PctEmptyFragments: 76 events.
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsWithEmptyFragments: 76 events.
Thu Aug 15 08:26:33 2019: FileMetric: EventBuilder3_art1.EventsWith50PctEmptyFragments: 76 events.

Here are sample logfile messages:

[biery@sbnd-daq33 art]$ pwd
/home/nfs/biery/gpmTest/daqlogs/art
[biery@sbnd-daq33 art]$ grep -A 2 -i msg-w art-sbnd-daq33-2019081508254*.log | tail -15
art-sbnd-daq33-20190815082548-13508.log:%MSG-w EventBuilder3_art1_FragmentWatcher:  DAQ 15-Aug-2019 08:26:18 CDT Booted FragmentWatcher_module.cc:216
art-sbnd-daq33-20190815082548-13508.log-Event 224: total_fragments=9, fragmentIDs for empty_fragments: 1, 2, 3, 4, 5, 6, 8, 9
art-sbnd-daq33-20190815082548-13508.log-%MSG
art-sbnd-daq33-20190815082548-13508.log:%MSG-w EventBuilder3_art1_FragmentWatcher:  DAQ 15-Aug-2019 08:26:18 CDT Booted FragmentWatcher_module.cc:202
art-sbnd-daq33-20190815082548-13508.log-Event 225: total_fragments=9, fragmentIDs for missing_fragments: 7
art-sbnd-daq33-20190815082548-13508.log-%MSG
art-sbnd-daq33-20190815082548-13508.log:%MSG-w EventBuilder3_art1_FragmentWatcher:  DAQ 15-Aug-2019 08:26:18 CDT Booted FragmentWatcher_module.cc:216
art-sbnd-daq33-20190815082548-13508.log-Event 225: total_fragments=9, fragmentIDs for empty_fragments: 1, 2, 3, 4, 5, 6, 8, 9
art-sbnd-daq33-20190815082548-13508.log-%MSG
art-sbnd-daq33-20190815082548-13508.log:%MSG-w EventBuilder3_art1_FragmentWatcher:  DAQ 15-Aug-2019 08:26:18 CDT Booted FragmentWatcher_module.cc:202
art-sbnd-daq33-20190815082548-13508.log-Event 226: total_fragments=9, fragmentIDs for missing_fragments: 7
art-sbnd-daq33-20190815082548-13508.log-%MSG
art-sbnd-daq33-20190815082548-13508.log:%MSG-w EventBuilder3_art1_FragmentWatcher:  DAQ 15-Aug-2019 08:26:18 CDT Booted FragmentWatcher_module.cc:216
art-sbnd-daq33-20190815082548-13508.log-Event 226: total_fragments=9, fragmentIDs for empty_fragments: 1, 2, 3, 4, 5, 6, 8, 9
art-sbnd-daq33-20190815082548-13508.log-%MSG

Here are sample TRACE messages:

19622 08-15 08:26:16.909988  13507  13507   1      EventBuilder1_art1_FragmentWatcher wrn . Event 211: total_fragments=9, fragmentIDs for empty_fragments: 1, 2, 3, 4, 5, 6, 8, 9
19623 08-15 08:26:16.909812  13507  13507   1      EventBuilder1_art1_FragmentWatcher wrn . Event 211: total_fragments=9, fragmentIDs for missing_fragments: 7
19624 08-15 08:26:16.909797  13507  13507   1      EventBuilder1_art1_FragmentWatcher d04 . Event 211: this event: total_fragments=9, missing_fragments=1, empty_fragments=8 (66 events processed)
19724 08-15 08:26:16.795336  13507  13507   1      EventBuilder1_art1_FragmentWatcher d07 . Event 210: events_with_50pct_missing_fragments=0, events_with_50pct_empty_fragments=65
19725 08-15 08:26:16.795326  13507  13507   1      EventBuilder1_art1_FragmentWatcher d07 . Event 210: events_with_10pct_missing_fragments=11, events_with_10pct_empty_fragments=65
19726 08-15 08:26:16.795321  13507  13507   1      EventBuilder1_art1_FragmentWatcher d06 . Event 210: events_with_missing_fragments=11, events_with_empty_fragments=65
19727 08-15 08:26:16.795250  13507  13507   1      EventBuilder1_art1_FragmentWatcher wrn . Event 210: total_fragments=9, fragmentIDs for empty_fragments: 1, 2, 3, 4, 5, 6, 8, 9
19728 08-15 08:26:16.795146  13507  13507   1      EventBuilder1_art1_FragmentWatcher wrn . Event 210: total_fragments=9, fragmentIDs for missing_fragments: 7
19729 08-15 08:26:16.795136  13507  13507   1      EventBuilder1_art1_FragmentWatcher d04 . Event 210: this event: total_fragments=9, missing_fragments=1, empty_fragments=8 (65 events processed)
19739 08-15 08:26:16.794524  13507  13507   1      EventBuilder1_art1_FragmentWatcher d07 . Event 209: events_with_50pct_missing_fragments=0, events_with_50pct_empty_fragments=64

The local changes to the EventBuilder configuration files were the following:

--- a/simple_test_config/mediumsystem_with_routing_master/EventBuilder1.fcl
+++ b/simple_test_config/mediumsystem_with_routing_master/EventBuilder1.fcl
@@ -27,6 +27,7 @@ daq: {
        send_requests: true
        request_port: 3001
        request_address: "227.128.12.26" 
+       multicast_interface_ip: "192.168.230.0" 
        expected_art_event_processing_time_us: 2000000

     routing_token_config: {
@@ -46,8 +47,8 @@ daq: {
        evbFile: {
          metricPluginType: "file" 
          level: 4
-         fileName: "eventbuilder/evb_%UID%_metrics.log" 
-         absolute_file_path: false
+         fileName: "/home/nfs/biery/gpmTest/daqlogs/eventbuilder/evb_%UID%_metrics.log" 
+         absolute_file_path: true
          uniquify: true
        }
   }
@@ -61,12 +62,16 @@ outputs: {

 physics: {
   analyzers: {
-
+    fragmentwatcher: {
+      metrics_reporting_level: 1
+      mode_bitmask: 7
+      module_type: "FragmentWatcher" 
+    }
   }

-  producers: {
-  }
+  producers: { }

+  a1: [ fragmentwatcher ]
   my_output_modules: [ rootNetOutput ]
 }
 source: {



Also available in: Atom PDF