History

Table of contents
artdaq Release testing documentation and results

artdaq Release testing documentation and results¶

There is currently no particular order to these tests, the numbers are provided for easy reference to a particular test or stage of testing.
Note that the metrics reported here are for comparing artdaq versions only. They are not indicative of artdaq performance on any given machine, and should not be used as either guarantees or limitations.
The tests are predicated on a version of artdaq_demo being available from SciSoft.

artdaq¶

Make sure that artdaq builds without errors or warnings
Make sure that all artdaq packages pass their built-in test suites
Check that quick-mrb-start.sh functions properly -- run without parameters
Perform transfer_driver tests (See transfer_driver tests below):
1. Large fragments (100 MB) x 10000, record rate for Shmem, TCP, MPI
2. Small fragments (5 KB) x 1000000, record rate for Shmem, TCP, MPI (Originally 1K fragments)
Perform artdaqDriver tests:
1. test1: 10,000 1 MB events, record time
2. test2: 1,000,000 1 KB events, record time
3. test3: 10,000 1 MB events without disk-writing, record time
4. test4: 10,000 1 MB events with Binary disk-writing to /dev/null, record time (new for v2_03_00, run for v2_02_01)
Run quick-mrb-start.sh --run-demo
1. Make sure the demo runs as expected
2. Make sure that the output data file is created
  1. Run rawEventDump.fcl over the data file
  2. Run toyDump.fcl over the data file
3. Store data file in Redmine as version reference
Run the DAQInterface example configurations on a low-performance machine (e.g. ironwork)
1. Make sure each example runs as expected for 200s
  1. Make sure the output data file is created
  2. Run verification FCL jobs on data file
2. Make sure each example runs as expected for 3 runs of 60s each
  1. Make sure the output data file is created
  2. Run verification FCL jobs on data file
Run the DAQInterface example configurations on a high-performance machine (e.g. woof, mu2edaq) New as of v3_03_01
1. Make sure each example runs as expected for 200s
  1. Make sure the output data file is created
  2. Run verification FCL jobs on data file
2. Make sure each example runs as expected for 3 runs of 60s each
  1. Make sure the output data file is created
  2. Run verification FCL jobs on data file
Run the DAQInterface example configurations on multiple machines (e.g. mu2edaq10, mu2edaq11, mu2edaq12) New as of v3_03_01
1. Make sure each example runs as expected for 200s
  1. Make sure the output data file is created
  2. Run verification FCL jobs on data file
2. Make sure each example runs as expected for 3 runs of 60s each
  1. Make sure the output data file is created
  2. Run verification FCL jobs on data file
Test version reference data files from Redmine -- note if version incompatibility exists
Test previous version of artdaq with current reference data files -- note if data files are not backwards-compatible
1. Run quick-mrb-start.sh --tag [previous version tag] in new directory
2. See compatibility test notes

transfer_driver performance metrics¶

Version	Qualifiers	ironwork.fnal.gov
		Large fragments				Small fragments
		Shmem		TCP		Shmem		TCP
		Receive	Send	Receive	Send	Receive	Send	Receive	Send
v3_00_03	s64:e15	2.34 GB/s		1.01 GB/s		420.96 MB/s		444.53 MB/s
v3_01_00	s50:e14:prof	2.30 GB/s	1.99 GB/s	926.47 MB/s	872.39 MB/s	74.94 MB/s	351.69 MB/s	64.18 MB/s	212.79 MB/s
v3_02_00	s64:e15:prof	2.15 GB/s	1.88 GB/s	938.13 MB/s	875.16 MB/s	59.18 MB/s	146.00 MB/s	63.89 MB/s	194.76 MB/s
v3_02_00	s67:e15:prof	2.12 GB/s	1.84 GB/s	859.29 MB/s	801.72 MB/s	58.75 MB/s	143.37 MB/s	64.02 MB/s	194.70 MB/s
v3_02_00	s67:e17:prof	2.11 GB/s	1.79 GB/s	861.81 MB/s	796.98 MB/s	59.34 MB/s	145.45 MB/s	63.11 MB/s	185.16 MB/s
v3_02_00	s67:c2:prof	2.12 GB/s	1.82 GB/s	877.55 MB/s	815.30 MB/s	77.91 MB/s	416.93 MB/s	66.78 MB/s	223.55 MB/s
v3_02_01	s65:e15:prof	4.38 GB/s	4.34 GB/s	2.65 GB/s	2.63 GB/s	93.91 MB/s	3.09 GB/s	80.88 MB/s	541.85 MB/s
v3_03_00	s64:e15:prof	4.01 GB/s	3.97 GB/s	3.33 GB/s	3.30 GB/s	44.01 MB/s	100.39 MB/s	60.08 MB/s	186.07 MB/s
v3_03_01	s65:e15:prof	2.23 GB/s	1.91 GB/s	1.01 GB/s	963.08 MB/s	91.23 MB/s	1.36 GB/s	77.86 MB/s	448.45 MB/s
v3_03_02	s65:e15:prof	2.32 GB/s	2.06 GB/s	1.03 GB/s	979.98 MB/s	91.44 MB/s	2.21 GB/s	81.27 MB/s	605.96 MB/s
v3_04_00	s65:e15:prof	3.43 GB/s	3.38 GB/s	3.14 GB/s	3.10 GB/s	41.57 MB/s	372.27 MB/s	48.13 MB/s	241.53 MB/s
v3_05_00	s67:e15:prof	2.65 GB/s	2.26 GB/s	2.32 GB/s	2.22 GB/s	88.12 MB/s	1.32 GB/s	68.47 MB/s	305.27 MB/s
v3_05_00	s82:e19:prof	2.35 GB/s	2.01 GB/s	2.63 GB/s	2.59 GB/s	88.89 MB/s	1.70 GB/s	72.08 MB/s	320.52 MB/s
v3_06_00	s67:e15:prof	4.11 GB/s	4.08 GB/s	3.24 GB/s	3.21 GB/s	50.03 MB/s	152.74 MB/s	54.53 MB/s	161.81 MB/s
v3_06_01	s67:e15:prof	4.24 GB/s	4.21 GB/s	3.38 GB/s	3.35 GB/s	86.79 MB/s	1.05 GB/s	75.73 MB/s	407.36 MB/s
v3_07_01	s85:e17:prof	3.62 GB/s	3.46 GB/s	3.42 GB/s	3.38 GB/s	86.54 MB/s	1.06 GB/s	74.51 MB/s	384.18 MB/s
v3_07_02	s85:e17:prof	4.40 GB/s	4.36 GB/s	3.55 GB/s	3.51 GB/s	86.88 MB/s	1.09 GB/s	76.96 MB/s	446.08 MB/s
v3_08_00	s85:e17:prof	1.69 GB/s	2.00 GB/s	485.83 MB/s	463.72 MB/s	85.41 MB/s	925.61 MB/s	75.01 MB/s	396.88 MB/s
v3_08_00	s96:e19:prof	4.33 GB/s	4.28 GB/s	3.51 GB/s	3.47 GB/s	84.93 MB/s	925.83 MB/s	75.18 MB/s	400.25 MB/s
v3_08_00	s97:c7:prof	4.32 GB/s	4.28 GB/s	3.44 GB/s	3.40 GB/s	74.27 MB/s	397.21 MB/s	74.57 MB/s	382.90 MB/s
v3_09_00	s96:e19:prof	2.22 GB/s	1.98 GB/s	2.52 GB/s	2.11 GB/s	85.89 MB/s	1.05 GB/s	74.98 MB/s	405.97 MB/s
v3_09_01	s96:e19:prof	2.16 GB/s	1.91 GB/s	2.54 GB/s	2.16 GB/s	85.38 MB/s	946.16 MB/s	72.03 MB/s	338.29 MB/s
v3_09_01	s100:e20:prof	2.07 GB/s	1.75 GB/s	2.46 GB/s	2.03 GB/s	84.30 MB/s	888.52 MB/s	72.54 MB/s	346.40 MB/s
v3_09_02	s96:e19:prof	2.06 GB/s	1.76 GB/s	2.47 GB/s	2.05 GB/s	86.09 MB/s	1.05 GB/s	75.34 MB/s	405.48 MB/s

artdaqDriver performance metrics¶

Version	Qualifiers	ironwork.fnal.gov
Version	Qualifiers	test1 (Frags (MB)/s)	test2 (Frags (KB)/s)	test3 (Frags (MB)/s)	test4 (Frags (MB)/s)
v1_13_03	s35:e10:eth	138	6952	1941
v2_00_00	s41:e10:eth	161	6613	2771
v2_01_00	s41:e10:eth	181	6486	3189
v2_02_01	s41:e10:eth	135	6551	3147	661
v2_02_01	s46:e14:eth	212	7964	3201
v2_03_00	s46:e10:eth	166	7468	4655	522
v2_03_00	s48:e14:eth	137	7577	4682	551
v2_03_01	s46:e10:eth	203	7486	4589	521
v2_03_01	s50:e14:eth	204	7785	8842	552
v2_03_02	s46:e10:eth	151	7464	4647	522
v2_03_02	s50:e14:eth	141	7773	8826	551
v3_00_02	s50:e14	216	5519	1582	503
v3_00_03	s64:e15	178	4997	2912	488
v3_01_00	s50:e14	157	4726	1751	481
v3_02_00	s64:e15:prof	191	3703	1997	402
v3_02_00	s67:e15:prof	195	3519	1464	406
v3_02_00	s67:e17:prof	166	3757	1765	407
v3_02_00	s67:c2:prof	182	4445	1637	542
v3_02_01	s65:e15:prof	227	6889	1852	488
v3_03_00	s64:e15:prof	180	5613	2543	425
v3_03_01	s64:e15:prof	238	5389	2486	420
v3_03_02	s64:e15:prof	251	5798	1907	498
v3_04_00	s65:e15:prof	236	6324	1744	507
v3_05_00	s67:e15:prof	241	5644	1793	490
v3_05_00	s82:e19:prof	224	3685	3466	499
v3_06_00	s67:e15:prof	226	3573	1635	45
v3_06_01	s67:e15:prof	234	5837	1833	474
v3_07_01	s85:e17:prof	231	6077	2456	475
v3_07_02	s85:e17:prof	158	6383	3261	498
v3_08_00	s85:e17:prof	262	6025	3097	474
v3_08_00	s96:e19:prof	241	6084	3252	475
v3_08_00	s96:c7:prof	256	3163	2462	524
v3_09_00	s96:e19:prof	243	3223	2470	499
v3_09_01	s96:e19:prof	254	3147	2434	498
v3_09_01	s100:e20:prof	209	2027	2287	473
v3_09_02	s96:e19:prof	261	3131	2418	474

DAQInterface Example Configurations¶

Run via command line:
$ run_demo.sh --config <configuration> --comps "<components>" [--no_om]
or, more robustly:
$ ipcrm -a;. ./run_demo.sh --config <configuration> --comps "<components>" --no_om --runduration 20
For example:
$ ipcrm -a;. ./run_demo.sh --config request_based_dataflow_example --comps component{01..03} --no_om --runduration 20
Note: These examples will (should) run easily on single node; extending/modifying these examples to run on multiple nodes may
requiring node and/or network administration (to enable multicast, for example).

Number	Configuration	Components	Online Monitoring	Verification FCL	Extra run_demo Options
1	demo* (Default)	component01 component02 (Default)	Yes	rawEventDump.fcl, toyDump.fcl
2	ascii_simulator_example	component01	No	rawEventDump.fcl, asciiDump.fcl
3	combined_eb_and_dl	component01 component 02 ... component04	No	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/combined_eb_and_dl/boot.txt
4	config_includes	component01 component 02 ... component04	Yes	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/config_includes/boot.txt
5	demo_largesystem	component01 component02 ... component19	Yes	rawEventDump.fcl, toyDump.fcl
6	eventbuilder_diskwriting	component01 component02 (Default)	No	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/eventbuilder_diskwriting/boot.txt
7	file_closing_example	component01 component02 (Default)	Yes	rawEventDump.fcl, toyDump.fcl
8	mediumsystem_with_routing_master	component01 component02 ... component10	Yes	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/mediumsystem_with_routing_master/boot.txt
9	multiple_art_processes_example	component01 component02 (Default)	Yes	rawEventDump.fcl, toyDump.fcl
10	multiple_dataloggers	component01 component 02 ... component04	Yes	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/multiple_dataloggers/boot.txt
11	request_based_dataflow_example	component01 component02 component03	Yes	rawEventDump.fcl, toyDump.fcl, wfViewer.fcl
12	routing_master_example	component01 component02 (Default)	Yes	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/routing_master_example/boot.txt
13	subrun_example	component01 component 02 ... component04	Yes	rawEventDump.fcl, toyDump.fcl	--bootfile $ARTDAQ_DAQINTERFACE_DIR/simple_test_config/subrun_example/boot.txt
14	demo	component01 component02 (Default)	Yes	rawEventDump.fcl, toyDump.fcl	--om_fhicl TransferInputShmemWithDelay (Hung Online Monitor example)

*: Run during quick-mrb-start --run-demo test

Testing Results¶

v3_09_02¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_09_02 with no options.
Results posted above for s96:e19 for performance comparison
Results posted above for s96:e19 performance comparison
See below
1. Example runs, Bug #25111 caused problems
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e19:s96:prof: https://cdcvs.fnal.gov/redmine/attachments/61466/artdaqdemo_r000001_sr01_20201022T191604_1_dl1.root
See below
See below
See below
artdaq v3_09_02 correctly reads reference data files from v3_09_01
artdaq v3_09_01 can read files from v3_09_02

DAQInterface simple_test_configs for v3_09_02 s96:e19¶

s96-e19	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	2	21	22	23		1	20	21	22		1	20	21	22
circular_buffer_mode_example	3	24	25	26		2	23	24	25		2	23	24	25
circular_buffer_mode_withRM	4	27	28	29		3	26	27	28		3	75	76	77	Increased eventbuilder timeout to 60
complex_subsystems	5	30	31	32		4	29	30	31		4	27	28	29
complicated_subsystems	6	33	34	35		5	32	33	34		5	30	31	32
config_includes	7	36	37	38		6	35	36	37		6	33	34	35
demo	8	39	40	41		7	38	39	40		7	36	37	38
demo_largesystem	9	42	43	44		8	41	42	43		8	39	40	41
eventbuilder_diskwriting	10	45	46	47		9	44	45	46		9	42	43	44
file_closing_example	11	48	49	50		10	47	48	49		10	45	46	47
mediumsystem_with_routing_manager	12	51	52	53		11	50	51	52		11	48			Disk write rate limitation
mu2e_sample_system	13	54	55	56		12	53	54	55		12	59	50	51
multiple_art_processes_example	14	57	58	59		13	56	57	58		13	52	53	54
multiple_dataloggers	15	60	61	62		14	59	60	61		14	55	56	57
request_based_dataflow_example	16	63	64	65		15	62	63	64		15	58	59	60
routing_manager_example	17	66	67	68		16	65	66	67		16	61	62	63
simple_subsystems	18	69	70	71		17	68	69	70		17	64	65	66
subrun_example	19	72	73	74		18	71	72	73		18	67	68	69
demo (Hung online monitor)	20	75	76	77		19	74	75	76		19	70	71	72

v3_09_01¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_09_01 with no options.
Results posted above for s96:e19 for performance comparison
Results posted above for s96:e19 performance comparison
See below
1. Example runs s100 build necessitated using an older version of .setup_fhiclcpp, python version conflict?
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e19:s96:prof: https://cdcvs.fnal.gov/redmine/attachments/60356/artdaqdemo_r000007_sr01_20200815T180716_1_dl1.root
4. e20:s100:prof: https://cdcvs.fnal.gov/redmine/attachments/60357/artdaqdemo_r000007_sr01_20200817T012533_1_dl1.root
See below
See below
See below
artdaq v3_09_01 correctly reads reference data files from v3_09_00
artdaq v3_09_00 can read files from v3_09_01, including s96 reading s100 data files

DAQInterface simple_test_configs for v3_09_01 s96:e19¶

s96-e19	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	1	20	21	22		1	20	21	22		77	96	97	98
circular_buffer_mode_example	2	23	24	25		2	23	24	25		78	99	100	101
circular_buffer_mode_withRM	3	26	27	28		3	26	27	28		79	102	103	104
complex_subsystems	4	29	30	31		4	29	30	31		80	105	106	107
complicated_subsystems	5	32	33	34		5	32	33	34		81	108	109	110
config_includes	6	35	36	37		6	35	36	37		82	111	112	113
demo	7	38	39	40		7	38	39	40		83	114	115	116
demo_largesystem	8	41	42	43		8	41	42	43		84	117	118	119
eventbuilder_diskwriting	9	44	45	46		9	44	45	46		85	120	121	122
file_closing_example	10	47	48	49		10	47	48	49		86	123	124	125
mediumsystem_with_routing_manager	11	50	51	52		11	50	51	52		87	152			Issue with over-size Fragments
mu2e_sample_system	12	53	54	55		12	53	54	55		88	128	129	130
multiple_art_processes_example	13	56	57	58		13	56	57	58		89	131	132	133
multiple_dataloggers	14	59	60	61		14	59	60	61		90	134	135	136
request_based_dataflow_example	15	62	63	64		15	62	63	64		91	137	138	139
routing_manager_example	16	65	66	67		16	65	66	67		92	140	141	142
simple_subsystems	17	68	69	70		17	68	69	70		93	143	144	145
subrun_example	18	71	72	73		18	71	72	73		94	146	147	148
demo (Hung online monitor)	19	74	75	76		19	74	75	76		95	149	150	151

DAQInterface simple_test_configs for v3_09_01 s100:e20¶

s100-e20	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	1	20	21	22		1	20	21	22		77	96	97	98
circular_buffer_mode_example	2	23	24	25		2	23	24	25		78	99	100	101
circular_buffer_mode_withRM	3	76	77	78	EventBuilder stop timeout	3	26	27	28		79	151	152		Disk write rate limitation
complex_subsystems	4	27	28	29		4	29	30	31		80	103	104	105
complicated_subsystems	5	30	31	32		5	32	33	34		81	106	107	108
config_includes	6	33	34	35		6	35	36	37		82	109	110	111
demo	7	36	37	38		7	38	39	40		83	112	113	114
demo_largesystem	8	39	40	41		8	41	42	43		84	115	116	117
eventbuilder_diskwriting	9	42	43	44		9	44	45	46		85	118	119	120
file_closing_example	10	45	46	47		10	47	48	49		86	121	122	123
mediumsystem_with_routing_manager	11	48	49	50		11	50	51	52		87	153			Disk write rate limitation
mu2e_sample_system	12	51	52	53		12	53	54	55		88	125	126	127
multiple_art_processes_example	13	54	55	56		13	56	57	58		89	128	129	130
multiple_dataloggers	14	57	58	59		14	59	60	61		90	131	132	133
request_based_dataflow_example	15	60	61	62		15	62	63	64		91	134	135	136
routing_manager_example	16	63	64	65		16	65	66	67		92	137	138	139
simple_subsystems	17	66	67	68		17	68	69	70		93	140	141	142
subrun_example	18	69	70	71		18	71	72	73		94	143	144	145
demo (Hung online monitor)	19	72	73	74		19	74	75	76		95	146	147	148

v3_09_00¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_09_00 with no options.
Results posted above for s96:e19 for performance comparison
Results posted above for s96:e19 performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e19:s96:prof: https://cdcvs.fnal.gov/redmine/attachments/60201/artdaqdemo_r000001_sr01_20200731T010219_1_dl1.root
See below
See below
See below
artdaq v3_09_00 correctly reads reference data files from v3_08_00 after Hotfix 4
artdaq v3_08_00 can not read files from v3_09_00 due to the addition of RawFragmentHeader to the data files

DAQInterface simple_test_configs tests for v3_09_00¶

s96-e19	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	1	20	21	22		2	21	22	23		74	93	94	95
circular_buffer_mode_example	2	23	24	25		3	24	25	26		75	96	97	98
circular_buffer_mode_withRM	3	26	27	28		4	161	162	163	EVB Stop Timeout/Hotfix 3	76	99	100	101
complex_subsystems	4	29	30	31		5	28	29	30		77	102	103	104
complicated_subsystems	5	32			EVB Stop Timeout	6	149	150	151	EVB Stop Timeout/Hotfix 3	78	152	153	154	EVB Stop Timeout/Hotfix 3
config_includes	6	33	34	35		7	32	33	34		79	106	107	108
demo	7	36	37	38		8	35	36	37		80	109	110	111
demo_largesystem	8	39	40	41		9	38	39	40		81	112	113	114
eventbuilder_diskwriting	9	42	43	44		10	41	42	43		82	115	116	117
file_closing_example	10	45	46	47		11	44	45	46		83	118	119	120
mediumsystem_with_routing_manager	11	48	49	50		12	47	48	49		84	158	159	160	EVB Stop Timeout/Hotfix 3,Local Disk DataLogging
mu2e_sample_system NEW CONFIG	12	51	52	53		13	50	51	52		85	122	123	124
multiple_art_processes_example	13	54	55	56		14	53	54	55		86	125	126	127
multiple_dataloggers	14	57	58	59		15	56	57	58		87	128	129	130
request_based_dataflow_example	15	60	61	62		16	59	60	61		88	131	132	133
routing_master_example	16	63	64	65		17	62	63	64		89	134	135	136
simple_subsystems	17	66	67	68		18	65	66	67		90	137	138	139
subrun_example	18	69	70	71		19	68	69	70		91	140	141	142
demo (Hung online monitor)	19	72	73	74		20	71	72	73		92	143	144	145

v3_08_00¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_08_00 with no options.
Results posted above for s85:e17, s96:c7 and s96:e19 for performance comparison
Results posted above for s85:e17, s96:c7 and s96:e19 performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e17:s85:prof: https://cdcvs.fnal.gov/redmine/attachments/58816/artdaqdemo_r000007_sr01_20200325T132834_1_dl1.root
4. e19:s96:prof: https://cdcvs.fnal.gov/redmine/attachments/58817/artdaqdemo_r000007_sr01_20200330T134038_1_dl1.root
See below
See below
See below
artdaq v3_08_00 (both qualifier sets) correctly reads reference data files from v3_07_02
artdaq v3_07_02 correctly reads reference data files from v3_08_00 (both qualifier sets)

DAQInterface simple_test_configs tests for v3_08_00¶

s85-e17	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	1	19	20	21		1	19	20	21		1	8	9	10
circular_buffer_mode_example	2	22	23	24		2	22	23	24		2	22	23	24
circular_buffer_mode_withRM	3	25	26	27		3	25	26	27		3	25	26	27
complex_subsystems	4	28	29	30		4	28	29	30		15	20	21		DataLogger times out (#24262)
complicated_subsystems	5	31	32	33		5	31	32	33		16	17	18	19
config_includes	6	34	35	36		6	34	35	36		6	34	35	36
demo	7	37	38	39		7	37	38	39		7	37	38	39
demo_largesystem	8	40	41	42		8	40	41	42		8	40	41	42
eventbuilder_diskwriting	9	43	44	45		9	43	44	45		9	43	44	45
file_closing_example	10	46	47	48		10	46	47	48		10	46	47	48
mediumsystem_with_routing_master	11	49	50	51		11	49	50	51		11	49	50	51
multiple_art_processes_example	12	52	53	54		12	52	53	54		12	52	53	54
multiple_dataloggers	13	55	56	57		13	55	56	57		13	55	56	57
request_based_dataflow_example	14	58	59	60		14	58	59	60		14	2	3	4	Re-run
routing_master_example	15	61	62	63		15	61	62	63		15	60	61	62
simple_subsystems	16	64	65	66		16	64	65	66		16	63	64	65
subrun_example	17	72	73	74	See #24242	17	67			See #24242	17	66			See #24242
demo (Hung online monitor)	18	68	69	70		18	68	69	70		18	67	68	69

s96-e19	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	1	19	20	21		1	19	20	21		90	1	2	3
circular_buffer_mode_example	2	22	23	24		2	22	23	24		91	4	5	6
circular_buffer_mode_withRM	3	25	26	27		3	25	26	27		92	7	8	9
complex_subsystems	4	28	29	30		4	28	29	30		93	10			DataLogger times out (#24262)
complicated_subsystems	5	31	32	33		5	31	32	33		94	11			DataLogger times out (#24262)
config_includes	6	34	35	36		6	34	35	36		95	12	13	14
demo	7	37	38	39		7	37	38	39		96	15	16	17
demo_largesystem	8	40	41	42		8	40	41	42		97	18	19	20
eventbuilder_diskwriting	9	43	44	45		9	43	44	45		98	21	22	23
file_closing_example	10	46	47	48		10	46	47	48		99	24	25	26
mediumsystem_with_routing_master	11	49	50	51		11	49	50	51		100	27	28		DataLogger times out (#24262)
multiple_art_processes_example	12	52	53	54		12	52	53	54		101	30	31	32
multiple_dataloggers	13	55	56	57		13	55	56	57		102	33	34	35
request_based_dataflow_example	14	58	59	60		14	58	59	60		103	36	37	38
routing_master_example	15	61	62	63		15	61	62	63		104	39	40	41
simple_subsystems	16	64	65	66		16	64	65	66		105	42	43	44
subrun_example	17	67			See #24242	17	67			See #24242	106	45	46	47
demo (Hung online monitor)	18	68	69	70		18	68	69	70		107	48	49	50

v3_07_02¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_07_02 with no options.
Results posted above for s85:e17 for performance comparison
Results posted above for s85:e17 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e17:s85:prof: https://cdcvs.fnal.gov/redmine/attachments/57930/artdaqdemo_r000001_sr01_20200207T234640_1_dl1.root
See below
See below
See below
artdaq v3_07_02 correctly reads reference data files from v3_07_01
artdaq v3_07_01 correctly reads reference data files from v3_07_02

DAQInterface simple_test_configs tests for v3_07_02¶

s85-e17	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	2	18	19	20		1	17	18	19		65	128	129	130	Issue running on multiple nodes, TCP related. Bug #24072
circular_buffer_mode_example	3	21	22	23		2	20	21	22		66	83	84	85
circular_buffer_mode_withRM	4	24	25	26		3	23	24	25		67	86	87	88
config_includes	5	27	28	29		4	26	27	28		68	89	90	91
demo	6	30	31	32		5	29	30	31		69	92	93	94
demo_largesystem	7	33	34	35		6	32	33	34		70	95	96	97
eventbuilder_diskwriting	8	36	37	38		7	35	36	37		71	98	99	100
file_closing_example	9	39	40	41		8	38	39	40		72	101	102	103
mediumsystem_with_routing_master	10	42	43	44		9	41	42	43		73	104	105	106
multiple_art_processes_example	11	45	46	47		10	44	45	46		74	107	108	109
multiple_dataloggers	12	48	49	50		11	47	48	49		75	110	111	112
request_based_dataflow_example	13	51	52	53		12	50	51	52		76	113	114	115
routing_master_example	14	54	55	56		13	53	54	55		77	116	117	118
simple_subsystems	15	57	58	59		14	56	57	58		78	119	120	121
subrun_example	16	60	61	62		15	59	60	61		79	122	123	124
demo (Hung online monitor)	17	63	64	65		16	62	63	64		80	125	126	127

v3_07_01¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_07_01 with no options.
Results posted above for s85:e17 for performance comparison
Results posted above for s85:e17 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e17:s85:prof: https://cdcvs.fnal.gov/redmine/attachments/57394/artdaqdemo_r000001_sr01_20191209T153844_1_dl1.root
See below
See below
See below
artdaq v3_07_01 correctly reads reference data files from v3_06_01
artdaq v3_06_01 correctly reads reference data files from v3_07_01

DAQInterface simple_test_configs tests for v3_07_01¶

s85-e17	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	2	18	19	20		1	17	18	19		65	127	128	129	Issues running on multiple mu2edaq nodes, DataLogger crashed on subsequent runs, possibly related to TCP reconnections?
circular_buffer_mode_example	3	21	22	23		2	20	21	22		66	83	84	85
circular_buffer_mode_withRM	4	65	66	67	Had to extend EventBuilder transition timeout	3	23	24	25		67	130	131	132	Had to extend EventBuilder transition timeout
config_includes	5	26	27	28		4	26	27	28		68	88	89	90
demo	6	29	30	31		5	29	30	31		69	91	92	93
demo_largesystem	7	32	33	34		6	32	33	34		70	94	95	96
eventbuilder_diskwriting	8	35	36	37		7	35	36	37		71	97	98	99
file_closing_example	9	38	39	40		8	38	39	40		72	100	101	102
mediumsystem_with_routing_master	10	41	42	43		9	41	42	43		73	103	104	105
multiple_art_processes_example	11	44	45	46		10	44	45	46		74	106	107	108
multiple_dataloggers	12	47	48	49		11	47	48	49		75	109	110	111
request_based_dataflow_example	13	50	51	52		12	50	51	52		76	112	113	114
routing_master_example	14	53	54	55		13	53	54	55		77	115	116	117
simple_subsystems	15	56	57	58		14	56	57	58		78	118	119	120
subrun_example	16	59	60	61		15	59	60	61		79	121	122	123
demo (Hung online monitor)	17	62	63	64		16	62	63	64		80	124	125	126

v3_06_01¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_06_01 with no options.
Results posted above for s67:e15 for performance comparison
Results posted above for s67:e15 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s67:prof: https://cdcvs.fnal.gov/redmine/attachments/55289/artdaqdemo_r000001_sr01_20190925T180547_1_dl1.root
See below
See below
See below
artdaq v3_06_01 correctly reads reference data files from v3_06_00
artdaq v3_06_00 correctly reads reference data files from v3_06_01

DAQInterface simple_test_configs tests for v3_06_01¶

Each cell contains the run number from the test run. Each test run has had toyDump run on it, with both the text and binary output verified.

s67-e15	Ironwork					Mu2edaq12					Mu2edaq multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	2	18	19	20		1	17	18	19		1	67	68	69	Issues running on multiple mu2edaq nodes, DataLogger crashed on subsequent runs, possibly related to TCP reconnections?
circular_buffer_mode_example	3	21	22	23		2	20	21	22		2	19	20	21
circular_buffer_mode_withRM	4	65	66	67	1st run timeout stopping eventbuilder after run 2	3	63	64	65	1st run timeout stopping eventbuilder after run 2	3	22	23	24
config_includes	5	26	27	28		4	24	25	26		4	25	26	27
demo	6	29	30	31		5	27	28	29		5	28	29	30
demo_largesystem	7	32	33	34		6	30	31	32		6	31	32	33
eventbuilder_diskwriting	8	35	36	37		7	33	34	35		7	34	35	36
file_closing_example	9	38	39	40		8	36	37	38		8	37	38	39
mediumsystem_with_routing_master	10	41	42	43		9	39	40	41		9	40	41	42
multiple_art_processes_example	11	44	45	46		10	42	43	44		10	43	44	45
multiple_dataloggers	12	47	48	49		11	45	46	47		11	46	47	48
request_based_dataflow_example	13	50	51	52		12	48	49	50		12	49	50	51
routing_master_example	14	53	54	55		13	51	52	53		13	52	53	54
simple_subsystems	15	56	57	58		14	54	55	56		14	55	56	57
subrun_example	16	59	60	61		15	57	58	59		15	58	59	60
demo (Hung online monitor)	17	62	63	64		16	60	61	62		16	61	62	63

v3_06_00¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_06_00 with no options.
Results posted above for s67:e15 for performance comparison
Results posted above for s67:e15 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s67:prof: https://cdcvs.fnal.gov/redmine/attachments/54808/artdaqdemo_r000001_sr01_20190812T164838_1_dl1.root
See below
See below
See below
artdaq v3_06_00 correctly reads reference data files from v3_05_00 (s67:e15 read both s67:e15 and s82:e19)
artdaq v3_05_00 can not read data file from v3_06_00 due to Fragment header version change!

DAQInterface simple_test_configs tests for v3_06_00¶

Each cell contains the run number from the test run. Each test run has had toyDump run on it, with both the text and binary output verified.

s67-e15	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
ascii_simulator_example	1	17	18	19		1	17	18	19		65	131	132	133	Issues running on multiple mu2edaq nodes, DataLogger crashed on subsequent runs, possibly related to TCP reconnections?
circular_buffer_mode_example	2	20	21	22		2	20	21	22		66	83	84	85
circular_buffer_mode_withRM	3	23	24	25		3	23	24	25		67	86	87	88
config_includes	4	26	27	28		4	26	27	28		68	89	90	91
demo	5	29	30	31		5	29	30	31		69	92	93	94
demo_largesystem	6	32	33	34		6	32	33	34		70	95	96	97
eventbuilder_diskwriting	7	35	36	37		7	35	36	37		71	98	99	100
file_closing_example	8	38	39	40		8	38	39	40		72	101	102	103
mediumsystem_with_routing_master	9	41	42	43		9	41	42	43		73	104	105	106
multiple_art_processes_example	10	44	45	46		10	44	45	46		74	107	108	109
multiple_dataloggers	11	47	48	49		11	47	48	49		75	110	111	112
request_based_dataflow_example	12	50	51	52		12	50	51	52		76	113	114	115
routing_master_example	13	53	54	55		13	53	54	55		77	116	117	118
simple_subsystems	14	56	57	58		14	56	57	58		78	119	120	121
subrun_example	15	59	60	61		15	59	60	61		79	122	123	124
demo (Hung online monitor)	16	62	63	64		16	62	63	64		80	125	126	127

v3_05_00¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_05_00 with no options.
Results posted above for s67:e15 and s82:e19 for performance comparison
Results posted above for s67:e15 and s82:e19 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e19:s82:prof: https://cdcvs.fnal.gov/redmine/attachments/53523/artdaqdemo_r000001_sr01_20190503T152053_1_dl1.root
  e15:s67:prof: https://cdcvs.fnal.gov/redmine/attachments/53748/artdaqdemo_r000001_sr01_20190508T193635_1_dl1.root
See below
See below
See below
artdaq v3_05_00 successfully reads data file from v3_04_00 in both e15:s67 and e19:s82 configurations
artdaq v3_04_00 successfully reads data files from v3_05_00 from both e15:s67 and e19:s82 configurations

DAQInterface simple_test_configs tests for v3_05_00¶

Each cell contains the run number from the test run. Each test run has had toyDump run on it, with both the text and binary output verified.

s82-e19	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
demo* (Default)	1	14	15	16		1	15	16	17		2	16	17	18
ascii_simulator_example	2	17	18	19		2	18	19	20		3	19	20	21
combined_eb_and_dl	NR	NR	NR	NR	Deprecated	X	X	X	X	Deprecated	X	X	X	X	Deprecated
config_includes	3	20	21	22		4	24	25	26		5	56	57	58	Extended DAQInterface timeouts
demo_largesystem	4	23	24	25		5	27	28	29		6	28	29	30
eventbuilder_diskwriting	5	26	27	28		6	30	31	32		7	31	32	33
file_closing_example	6	29	30	31		7	33	34	35		8	34	35	36
mediumsystem_with_routing_master	7	32	33	34		8	55	56	57	Had to re-run with longer EventBuilder timeout	9	63	64	65	Extended DAQInterface timeouts
multiple_art_processes_example	8	35	36	37		9	37	38	39		10	39	40	41
multiple_dataloggers	9	38	39	40		10	40	41	42		11	42	43	44
request_based_dataflow_example	10	41	42	43		11	43	44	45		12	45	46	47
routing_master_example	11	44	45	46		12	46	47	48		13	137	138	139	Extended DAQInterface timeouts. Bug #22576 noticed and patched
subrun_example	12	47	48	49		13	49	50	51		14	50	51	52
demo	13	50	51	52		14	52	53	54		15	53	54	55

s67-e15	Ironwork					Mu2edaq12					Mu2edaq Multiple
Configuration	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes	Long Run	Short Run 1	Short Run 2	Short Run 3	Notes
demo* (Default)	1	15	16	17		1	15	16	17		1	15	16	17
ascii_simulator_example	2	18	19	20		2	18	19	20		2	18	19	20
combined_eb_and_dl	X	X	X	X	Deprecated	X	X	X	X	Deprecated	X	X	X	X	Deprecated
config_includes	4	24	25	26		4	24	25	26		4	24	25	26
demo_largesystem	5	27	28	29		5	27	28	29		5	27	28	29
eventbuilder_diskwriting	6	30	31	32		6	30	31	32		6	30	31	32
file_closing_example	7	33	34	35		7	33	34	35		7	33	34	35
mediumsystem_with_routing_master	8	36	37	38		8	36	37	38		8	56	57	58
multiple_art_processes_example	9	39	40	41		9	39	40	41		9	38	39	40
multiple_dataloggers	10	42	43	44		10	42	43	44		10	41	42	43
request_based_dataflow_example	11	45	46	47		11	45	46	47		11	44	45	46
routing_master_example	12	48	49	50		12	48	49	50		12	47	48	49
subrun_example	13	51	52	53		13	51	52	53		13	50	51	52
demo	4	54	55	56		14	54	55	56		14	53	54	55

v3_04_00¶

Builds using Jenkins
Tests run on Jenkins
quick-mrb-start.sh downloads and installs artdaq_demo v3_04_00 with no options.
Results posted above for s65:e15 for performance comparison
Results posted above for s65:e15 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s65:prof: https://cdcvs.fnal.gov/redmine/attachments/52556/artdaqdemo_r000001_sr01_20190308T171927_1_dl1.root
See Below
1. demo
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. toyDump reports expected Fragments
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. toyDump reports expected Fragments
2. ascii_simulator_example:
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
3. combined_eb_and_dl
  1. ~~Destinations block for components not generated correctly by DAQInterface~~ Bug #21856
    1. Output file is created and closed successfully
    2. Output file contains no events!
  2. Only first run succeeds
    1. Output file is created and closed successfully
    2. Output file contains no events!
4. config_includes
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows 2 TOY1 and 2 TOY2 Fragments per event
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
5. demo_largesystem
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows 10 TOY1 and 9 TOY2 Fragments per event
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
6. eventbuilder_diskwriting
  1. ~~Example has issues due to empty destinations block in EventBuilder~~ Bug #21856
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
  2. Example runs for 2 runs
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
7. file_closing_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. Each file has exactly 100 events, except the last
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Each file has exactly 100 events, except the last
8. mediumsystem_with_routing_master
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
9. multiple_art_processes_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
10. multiple_dataloggers
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
11. request_based_dataflow_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
12. routing_master_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
13. subrun_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
14. demo (Hung online monitor)
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
See Below
1. demo
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. toyDump reports expected Fragments
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. toyDump reports expected Fragments
2. ascii_simulator_example:
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
3. combined_eb_and_dl
  1. ~~Destinations block for components not generated correctly by DAQInterface~~ Bug #21856
    1. Output file is created and closed successfully
    2. Output file contains no events!
  2. Only first run succeeds
    1. Output file is created and closed successfully
    2. Output file contains no events!
4. config_includes
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows 2 TOY1 and 2 TOY2 Fragments per event
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
5. demo_largesystem
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows 10 TOY1 and 9 TOY2 Fragments per event
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
6. eventbuilder_diskwriting
  1. ~~Example has issues due to empty destinations block in EventBuilder~~ Bug #21856
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
  2. Example runs for 2 runs
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
7. file_closing_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. Each file has exactly 100 events, except the last
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Each file has exactly 100 events, except the last
8. mediumsystem_with_routing_master
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
9. multiple_art_processes_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
10. multiple_dataloggers
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
11. request_based_dataflow_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
12. routing_master_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
13. subrun_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
14. demo (Hung online monitor)
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
v3_04_00 correctly reads reference file from v3_03_02.
v3_03_02 s64 correctly reads s65 reference file from v3_04_00.

v3_03_02¶

Builds using Jenkins
Tests run on Jenkins, some have to be retried (all succeed by second attempt)
quick-mrb-start.sh downloads and installs artdaq_demo v3_03_02 with no options.
Results posted above for s64:e15 for performance comparison
Results posted above for s64:e15 for performance comparison
See below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s64:prof: https://cdcvs.fnal.gov/redmine/attachments/52068/artdaqdemo_r000001_sr01_20190206T161050_1_dl1.root
See Below
1. demo
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. toyDump reports expected Fragments
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. toyDump reports expected Fragments
2. ascii_simulator_example:
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
3. combined_eb_and_dl
  1. ~~Destinations block for components not generated correctly by DAQInterface~~ Bug #21856
    1. Output file is created and closed successfully
    2. Output file contains no events!
  2. Only first run succeeds
    1. Output file is created and closed successfully
    2. Output file contains no events!
4. config_includes
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows 2 TOY1 and 2 TOY2 Fragments per event
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
5. demo_largesystem
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows 10 TOY1 and 9 TOY2 Fragments per event
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
6. eventbuilder_diskwriting
  1. ~~Example has issues due to empty destinations block in EventBuilder~~ Bug #21856
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
  2. Example runs for 2 runs
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
7. file_closing_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. Each file has exactly 100 events, except the last
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Each file has exactly 100 events, except the last
8. mediumsystem_with_routing_master
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
9. multiple_art_processes_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
10. multiple_dataloggers
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
11. request_based_dataflow_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
12. routing_master_example
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
13. subrun_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
14. demo (Hung online monitor)
  1. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. ToyDump shows expected output
See Below
1. demo
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
2. ascii_simulator_example:
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. Only expected strings are present in output
  2. Example runs without errors
    1. Output file is created and closed successfully
    2. Only expected strings are present in output
3. combined_eb_and_dl
  1. ~~Destinations block for components not generated correctly by DAQInterface~~ Bug #21856
    1. Output file is created and closed successfully
    2. Output file contains no events!
  2. ~~Destinations block for components not generated correctly by DAQInterface~~
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
4. config_includes
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
5. demo_largesystem
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
6. eventbuilder_diskwriting
  1. ~~Example has issues due to empty destinations block in EventBuilder~~ Bug #21856
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
  2. ~~Example has issues due to empty destinations block in EventBuilder~~
    1. Output file is created but NOT closed successfully
    2. Output files are not closed, cannot run art
7. file_closing_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
8. mediumsystem_with_routing_master
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. ~~Example only runs correctly for first run. Problems appear to be due to DataLogger art process.~~ Bug #21863
9. multiple_art_processes_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
10. multiple_dataloggers
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
11. request_based_dataflow_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
12. routing_master_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
13. subrun_example
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
14. demo (Hung online monitor)
  1. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output
  2. Example runs without errors
    1. Output files are created and closed successfully
    2. ToyDump shows expected output

v3_03_01¶

Builds using Jenkins
Tests run on Jenkins, some have to be retried (all succeed by second attempt)
quick-mrb-start.sh downloads and installs artdaq_demo v3_03_01 with no options.
Results posted above for s64:e15 for performance comparison
Results posted above for s64:e15 for performance comparison
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s64:prof: https://cdcvs.fnal.gov/redmine/attachments/51030/artdaqdemo_r000001_sr01_20181024T181508_1_dl1.root
See Below
1. demo
  1. Demo run as part of quick-mrb-start tests, above
  2. Example runs without errors
  3. Output file is created and closed successfully
  4. ToyDump reports expected Fragments
2. ascii_simulator_example:
  1. Example runs correctly
  2. Example runs without errors
  3. Output file is created and closed successfully
  4. asciiDump reports only expected strings in output
3. combined_eb_and_dl
  1. Example is not supported by DAQInterface
  2. Example is not supported by DAQInterface
  3. Output file is created and closed successfully for DL1
  4. Output file has no events!
4. config_includes
  1. Example runs without errors
  2. Example runs without errors
  3. Output file is created and closed successfully
  4. ToyDump shows 2 TOY1 and 2 TOY2 Fragments per event
5. demo_largesystem
  1. Example runs without errors
  2. Example runs without errors
  3. Output file is created and closed successfully
  4. ToyDump shows 10 TOY1 and 9 TOY2 Fragments per event
6. eventbuilder_diskwriting
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected Fragments in each file, evenly distributed
7. file_closing_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. Each file has exactly 100 events, except the last
8. mediumsystem_with_routing_master
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
9. multiple_art_processes_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
10. multiple_dataloggers
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
11. request_based_dataflow_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
12. routing_master_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
13. subrun_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
14. demo (Hung online monitor)
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected ouput
See Below
1. demo
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
2. ascii_simulator_example:
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events
3. combined_eb_and_dl
  1. Example is not supported by DAQInterface
  2. Example is not supported by DAQInterface
  3. Output files are created and closed successfully
  4. Output file has no events!
4. config_includes
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
5. demo_largesystem
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
6. eventbuilder_diskwriting
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
7. file_closing_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
8. mediumsystem_with_routing_master
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
9. multiple_art_processes_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
10. multiple_dataloggers
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
11. request_based_dataflow_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
12. routing_master_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
13. subrun_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
14. demo (Hung online monitor)
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
See Below
1. demo
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
2. ascii_simulator_example:
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
3. combined_eb_and_dl
  1. Example is not supported by DAQInterface
  2. Example is not supported by DAQInterface
  3. Output files are created and closed successfully
  4. Output file has no events!
4. config_includes
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
5. demo_largesystem
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
6. eventbuilder_diskwriting
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
7. file_closing_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
8. mediumsystem_with_routing_master
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
9. multiple_art_processes_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
10. multiple_dataloggers
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
11. request_based_dataflow_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
12. routing_master_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
13. subrun_example
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
14. demo (Hung online monitor)
  1. Example runs without errors
  2. Example runs without errors
  3. Output files are created and closed successfully
  4. ToyDump shows expected numbers of events and TOY Fragments
v3_03_01 correctly reads reference file from v3_03_00.
v3_03_00 s65 correctly reads s64 reference file from v3_03_01.

v3_03_00¶

Builds using Jenkins
Tests run on Jenkins, some have to be retried (all succeed by second attempt)
quick-mrb-start.sh downloads and installs artdaq_demo v3_03_00 with no options.
Results posted above for s64:e15 for performance comparison
Results posted above for s64:e15 for performance comparison
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s64:prof: https://cdcvs.fnal.gov/redmine/attachments/50192/artdaqdemo_r000001_sr01_20180926T154552_1_dl1.root
See Below
1. Demo run as part of quick-mrb-start tests, above
2. ascii_simulator_example:
  1. Example runs correctly
  2. Output file is created and closed successfully
  3. asciiDump reports only expected strings in output
3. demo_largesystem
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 10 TOY1 and 9 TOY2 fragments in each event
4. request_based_dataflow_example
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 3 Fragments/event. wfViewer shows expected distributions
5. routing_master_example
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 2 Fragments/event.
6. demo (Hung Online Monitor)
  1. Example runs and stops correctly, once OM processes complete, shared memory segments are removed
  2. Output file is created and closed successfully
  3. toyDump shows expected output (all events present)
7. demo_largesystem (Multi-node)
  1. Example run with 10 BRs, DL, DI on mu2edaq04, 9 BRs, EB on mu2edaq01
  2. Output data file was created and closed successfully
  3. toyDump shows expected output
8. demo (Multiple runs)
  1. Example runs and stops correctly
  2. Output files are created and closed successfully
  3. toyDump shows expected output
v3_03_00 correctly reads reference file from v3_02_01.
v3_02_01 s65 correctly reads s64 reference file from v3_03_00. NOTE: Reference data file does not contain Container Fragments; v3_03_00 files containing new Container fragments can not be read by previous versions!

v3_02_01¶

Builds using Jenkins
Tests run on Jenkins, some have to be retried (all succeed by second attempt)
quick-mrb-start.sh downloads and installs artdaq_demo v3_02_01 with no options.
Results posted above for s65:e15 for performance comparison. Note that v3_02_01 contains changes informed by profile-guided optimization, which is why the numbers are so different.
Results posted above for s65:e15 for performance comparison
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s65:prof: https://cdcvs.fnal.gov/redmine/attachments/48131/artdaqdemo_r000001_sr01_20180608T133613_1_dl3.root
See Below
1. Demo run as part of quick-mrb-start tests, above
2. ascii_simulator_example:
  1. Example runs correctly
  2. Output file is created and closed successfully
  3. asciiDump reports only expected strings in output
3. demo_largesystem
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 10 TOY1 and 9 TOY2 fragments in each event
4. request_based_dataflow_example
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 3 Fragments/event. wfViewer shows expected distributions
5. routing_master_example not fully set up, skipped
6. demo (Hung Online Monitor)
  1. Example runs and stops correctly, once OM processes complete, shared memory segments are removed
  2. Output file is created and closed successfully
  3. toyDump shows expected output (all events present)
7. demo_largesystem (Multi-node)
  1. Example run with 3 BRs, DL, DI on mu2edaq01, 5 BRs on mu2edaq04, 6 BRs, EB on mu2edaq05, 5 BRs, EB on mu2edaq06
  2. Output data file was created and closed successfully
  3. toyDump shows expected output
8. demo (Multiple runs)
  1. Example runs and stops correctly
  2. Output files are created and closed successfully
  3. toyDump shows expected output
v3_02_01 correctly reads reference file from v3_02_00.
v3_02_00 s64 correctly reads s65 reference file from v3_02_01.

v3_02_00¶

Builds using Jenkins
Tests run on Jenkins, some have to be retried (all succeed by second attempt)
quick-mrb-start.sh downloads and installs artdaq_demo v3_02_00 with no options. Added code to setup appropriate version of artdaq_mpich_plugin
Results posted above for s64:e15 (default qual) and 3 compiler-variants for s67 (e15, e17, and c2) for performance comparison
Results posted above for s64:e15 (default qual) and 3 compiler-variants for s67 (e15, e17, and c2) for performance comparison
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s64: https://cdcvs.fnal.gov/redmine/attachments/47419/artdaqdemo_r000003_sr01_20180507T221153_1_dl3.root
See Below
1. Demo run as part of quick-mrb-start tests, above
2. ascii_simulator_example:
  1. Example runs with changes to configuration file names
  2. Output file is created and closed successfully
  3. asciiDump reports only expected strings in output
3. demo_largesystem
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 10 TOY1 and 9 TOY2 fragments in each event
4. request_based_dataflow_example
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 3 Fragments/event. wfViewer shows expected distributions
5. routing_master_example not fully set up, skipped
6. demo (Hung Online Monitor)
  1. Example runs and stops correctly
  2. Output file is created and closed successfully
  3. toyDump shows expected output (all events present)
7. demo_largesystem (Multi-node)
  1. Had issues getting daqinterface running, eventually was able to run example between mu2edaq01 and mu2edaq10, and mu2edaq11 and mu2edaq10
  2. Output data file was created and closed successfully
  3. toyDump shows expected output
8. demo (Multiple runs)
  1. Example runs and stops correctly (Dispatcher died with strange error in run 1)
  2. Output files are created and closed successfully
  3. toyDump shows expected output
v3_02_00 correctly reads reference file from v3_01_00.
s64 data format is not readable by s50 (v3_01_00).

v3_01_00¶

Builds using Jenkins.
Tests run correctly on ironwork and mu2edaq01. Issues testing on buildservice nodes persist.
quick-mrb-start.sh downloads and installs artdaq_demo v3_01_00 with no options
Results posted above for s50:e14
Results posted above for s50:e14
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e14:s50: https://cdcvs.fnal.gov/redmine/attachments/47143/artdaqdemo_r000001_sr01_20180416T185810_1_dl1.root
See below
1. Demo run as part of quick-mrb-start tests, above
2. ascii_simulator_example:
  1. Example runs
  2. Output file is created and closed successfully
  3. asciiDump reports only expected strings in output
3. demo_largesystem
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 10 TOY1 and 9 TOY2 fragments in each event
4. request_based_dataflow_example
  1. Example runs
  2. Output file is created and closed successfully
  3. toyDump shows 3 Fragments/event. wfViewer shows expected distributions
5. routing_master_example not fully set up, skipped
6. demo (Hung Online Monitor)
  1. Example runs and stops correctly
  2. Output file is created and closed successfully
  3. toyDump shows expected output
7. demo_largesystem (Multi-node)
  1. Demo runs as expected on mu2edaq01 and mu2edaq10
  2. Output data file is created and closed successfully
  3. toyDump shows expected output (10 TOY1 and 9 TOY2 Fragments per event)
v3_01_00 s50:e14 correctly reads s50:e14 reference file from v3_00_02. s64 data format (i.e. v3_00_03 reference file) is not readable by s50.
v3_00_03 s64 correctly reads v3_01_00 reference data file

v3_00_03 (from here down, uses old test procedure at end of page)¶

Builds using Jenkins.
Builds using Jenkins. artdaq tests disabled due to shared memory issues on build nodes. mrb t and buildtool -t tests pass.
quick-mrb-start.sh downloads and installs artdaq_demo v3_00_03 with no options
Results posted above for s64:e15
Results posted above for s64:e15
See Below
1. Example runs, after adding "export DAQINTERFACE_PORT=5570" to DAQInterface/user_sourcefile_example
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e15:s64: https://cdcvs.fnal.gov/redmine/attachments/download/46400/artdaqdemo_r000001_sr01_20180220T151305_1_dl1.root
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. Only expected strings are produced in output
3. e15:s64: https://cdcvs.fnal.gov/redmine/attachments/download/46401/artdaqdemo_r000101_sr01_20180220T002736_1.root
See Below
1. Runs as expected
2. Data file created correctly.
  1. Runs as expected
  2. Updated toyDump.fcl runs without issue
  3. Expected distributions appear. Note that on event display, WFViewer quickly spooled through each fragment in the container (flashing plot).
v3_00_03 correctly reads s50:e14 reference file from v3_00_02
art s50 (in artdaq v3_00_02) CANNOT read s64 reference data file from v3_00_03!!!
DEPRECATED Test not run
See Below
1. DEPRECATED Test not run
2. System runs without errors. Processed 374 + 226 events in 60 seconds (DL 1 + DL 2)

v3_00_02¶

Builds using Jenkins
Builds using Jenkins. artdaq tests disabled due to shared memory issues on build nodes. mrb t and buildtool -t tests pass.
quick-mrb-start.sh downloads and installs artdaq_demo v3_00_02 with no options
Results posted above for e14:s50
Results posted above for e14:s50
See below
1. Runs, art analysis processes show plots. CheckIntegrity is being used, so plots are of pattern distribution (not very interesting)
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e14:s50: https://cdcvs.fnal.gov/redmine/attachments/download/46224/artdaqdemo_r000001_sr01_20180207T142924_1_dl1.root
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. Only expected strings are produced in output
3. e14:s50: https://cdcvs.fnal.gov/redmine/attachments/download/46225/artdaqdemo_r000101_sr01_20180207T145904_1.root
See Below
1. Runs as expected
2. Data file created correctly.
  1. Runs as expected
  2. num_adcs_to_show had to be adjusted from 0 to 1 (error about requesting more adcs than are present in the fragment)
  3. Expected distributions appear. Note that on event display, WFViewer quickly spooled through each fragment in the container (flashing plot).
v3_00_02 properly reads v2_03_03 e10:s46 reference data files
artdaq v2_03_03 reads v3_00_02 reference data files correctly.
DEPRECATED Test not run
See Below
1. Test runs successfully (logs to TRACE)
2. System runs without errors. Processed 373 + 226 events in 60 seconds (DL 1 + DL 2)

v2_03_03¶

Builds using Jenkins
Builds using Jenkins
Results posted above for e14:s50
Results posted above for e14:s50
See below
1. Runs, art analysis threads show plots, normal distribution appears (had to run multiple times on e10:s46 to get run actually going)
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/43267/artdaqdemo_r000001_sr01_20170817T161315_1_0.root,
  e14:s50: https://cdcvs.fnal.gov/redmine/attachments/download/43266/artdaqdemo_r000001_sr01_20170817T165621_1_0.root
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. Only expected strings are produced in output
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/43270/artdaqdemo_r000101_sr01_20170817T175751_1.root,
  e14:s50: https://cdcvs.fnal.gov/redmine/attachments/download/43271/artdaqdemo_r000101_sr01_20170817T180008_1.root
See Below
1. Runs as expected
2. Data file created correctly.
  1. Runs as expected
  2. num_adcs_to_show had to be adjusted from 0 to 1 (error about requesting more adcs than are present in the fragment)
  3. Expected distributions appear
Both e14 and e10 versions of v2_03_02 read previous data files correctly.
v2_03_00 reads data files correctly.
TODO: Test Infiniband build of artdaq (Currently using ds50daq)
1. TODO: Build artdaq with ib qualifier
2. TODO: Run transfer_driver_mpi across the ib link
3. TODO: Run ds50daq with new version of artdaq
See Below
1. Test runs successfully (logs to TRACE)
2. System runs without errors. Processed 495 + 104 events in 60 seconds (DL 1 + DL 2)
  Note: this version of artdaq appears to be unusually verbose. We should double-check the Debug-level messages from trying to understand the s50 issues.

v2_03_02¶

Builds using Jenkins
Builds using Jenkins
Results posted above for both e10:s46 (defaultqual) and e14:s50
Results posted above for both e10:s46 (defaultqual) and e14:s50
See below
1. Runs, art analysis threads show plots, normal distribution appears (had to run multiple times on e10:s46 to get run actually going)
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/43267/artdaqdemo_r000001_sr01_20170817T161315_1_0.root,
  e14:s50: https://cdcvs.fnal.gov/redmine/attachments/download/43266/artdaqdemo_r000001_sr01_20170817T165621_1_0.root
See Below
1. Example runs
2. Output data file is created
  1. Event dump succeeds
  2. Only expected strings are produced in output
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/43270/artdaqdemo_r000101_sr01_20170817T175751_1.root,
  e14:s50: https://cdcvs.fnal.gov/redmine/attachments/download/43271/artdaqdemo_r000101_sr01_20170817T180008_1.root
See Below
1. Runs as expected
2. Data file created correctly.
  1. Runs as expected
  2. num_adcs_to_show had to be adjusted from 0 to 1 (error about requesting more adcs than are present in the fragment)
  3. Expected distributions appear
Both e14 and e10 versions of v2_03_02 read previous data files correctly.
v2_03_00 reads data files correctly.
TODO: Test Infiniband build of artdaq (Currently using ds50daq)
1. TODO: Build artdaq with ib qualifier
2. TODO: Run transfer_driver_mpi across the ib link
3. TODO: Run ds50daq with new version of artdaq
See Below
1. Test runs successfully (logs to TRACE)
2. System runs without errors. Processed 495 + 104 events in 60 seconds (DL 1 + DL 2)
  Note: this version of artdaq appears to be unusually verbose. We should double-check the Debug-level messages from trying to understand the s50 issues.

v2_03_01¶

Builds using Jenkins
Builds using Jenkins
Results posted above for both e10:s46 (defaultqual) and e14:s50
Results posted above for both e10:s46 (defaultqual) and e14:s50
See below
1. Runs, art analysis threads show plots, normal distribution appears
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/43062/artdaqdemo_r000001_sr01_20170808T202228_1_0.root
  e14:s50: TEST FAILED!

v2_03_00¶

Builds using Jenkins
Builds using Jenkins
Results posted above for both e10:s46 (defaultqual) and e14:s48
Results posted above for both e10:s46 (defaultqual) and e14:s48
See below
1. Runs, art analysis threads show plots, normal distribution appears
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/40825/artdaqdemo_r000001_sr01_20170602T183923_1_0.root
  e14:s48: https://cdcvs.fnal.gov/redmine/attachments/download/40826/artdaqdemo_r000001_sr01_20170602T200045_1_0.root
See Below
1. Example runs, be sure to delete .fcl files from previous test!
2. Output data file is created
  1. Event dump succeeds
  2. Only expected strings are produced in output
3. e10:s46: https://cdcvs.fnal.gov/redmine/attachments/download/40943/artdaqdemo_r000101_sr01_20170605T172426_1.root,
  e14:s48: https://cdcvs.fnal.gov/redmine/attachments/download/40944/artdaqdemo_r000101_sr01_20170605T172812_1.root
See Below
1. Runs as expected
2. Data file created correctly.
  1. Runs as expected
  2. num_adcs_to_show had to be adjusted from 0 to 1 (error about requesting more adcs than are present in the fragment)
  3. Expected distributions appear
Both e14 and e10 versions of v2_03_00 read previous data files correctly.
v2_02_01 reads data files correctly.
TODO: Test Infiniband build of artdaq (Currently using ds50daq)
1. TODO: Build artdaq with ib qualifier
2. TODO: Run transfer_driver_mpi across the ib link
3. TODO: Run ds50daq with new version of artdaq
See Below
1. Test runs successfully (logs to TRACE)
2. System runs without errors. Processed 225 + 373 events in 60 seconds (DL 1 + DL 2)

v2_02_01¶

Builds using Jenkins
Builds using Jenkins
Results posted above for both e10:s41 (defaultqual) and e14:s46
Results posted above for both e10:s41 (defaultqual) and e14:s46
See below
1. Runs, art analysis threads show plots, normal distribution appears
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. e10:s41: https://cdcvs.fnal.gov/redmine/attachments/download/39105/artdaqdemo_r000101_sr01_20170314T211522_1.root,
  e14:s46: https://cdcvs.fnal.gov/redmine/attachments/download/39111/artdaqdemo_r000101_sr01_20170315T132812_1.root
See Below
1. Example runs, be sure to delete .fcl files from previous test!
2. Output data file is created
  1. Event dump succeeds
  2. Only expected strings are produced in output
3. e10:s41: https://cdcvs.fnal.gov/redmine/attachments/download/39106/artdaqdemo_r000101_sr01_20170314T213249_1.root,
  e14:s46: https://cdcvs.fnal.gov/redmine/attachments/download/39112/artdaqdemo_r000101_sr01_20170315T133053_1.root
See Below
1. Runs on e14 and e10. Identified and quick-patched a race condition in CommandableFragmentGenerator
  There's an issue with the ToySimulator's pattern mode on e14...instead of a monotonically-increasing value, the maximum value is repeated. This will be patched for the next version of artdaq_demo.
2. Data file created correctly.
  1. Runs as expected
  2. num_adcs_to_show had to be adjusted from 0 to 1 (error about requesting more adcs than are present in the fragment)
Both e14 and e10 versions of v2_02_01 read previous data files correctly.
artdaq v2_01_00 reads both e10 and e14 reference data files without issue.
TODO: Test Infiniband build of artdaq (Currently using ds50daq)
1. TODO: Build artdaq with ib qualifier
2. TODO: Run transfer_driver_mpi across the ib link
3. TODO: Run ds50daq with new version of artdaq

v2_01_00¶

Builds using Jenkins
Builds using Jenkins (Jenkins builds run tests)
Results posted above
1. Created standard fhicl files transfer_driver_{mpi,tcp,shmem}_100M.fcl
2. Created standard fhicl files transfer_driver_{mpi,tcp,shmem}_1K.fcl
Results posted above
1. Created standard fhicl file driver_test1.fcl -- results posted above
2. Created standard fhicl file driver_test2.fcl -- results posted above
3. Created standard fhicl file driver_test3.fcl -- results posted above
See below
1. Runs, art analysis threads show plots, normal distribution appears
2. Output data file is created
  1. Event dump succeeds
  2. toyDump succeeds
3. https://cdcvs.fnal.gov/redmine/attachments/download/38513/artdaqdemo_r000101_sr01_20170214T155625.root
See below
1. Example runs -- found issue with runAsciiSimulatorExample.sh, fixed in develop
2. Data file created
  1. Data file has fragments
  2. Only expected strings are produced in output
3. https://cdcvs.fnal.gov/redmine/attachments/download/38528/artdaqdemo_r000101_sr01_20170215T150506.root
See below
1. Example runs -- had to open firewall port so that multicast trigger messages are sent
2. Data file created
  1. Data file has 600 events, 2 TOY1 and a Container fragment per event.
  2. Container fragments are unpacked correctly. There are 6 or 7 fragments in each event (DAQ + DCS + 4-5 CRV)
No issues. All files read correctly.
Had to modify toyDump.fcl and asciiDump.fcl to add raw_data_label and frag_type configuration parameters for artdaq v2_00_00. Otherwise, files read fine
TODO: Test Infiniband build of artdaq (Currently using ds50daq)
1. TODO: Build artdaq with ib qualifier
2. TODO: Run transfer_driver_mpi across the ib link
3. TODO: Run ds50daq with new version of artdaq

Testing notes¶

transfer_driver tests¶

. products/setup
. setupARTDAQDEMO
for test in 5K 100M;do
  for type in shmem tcp;do
    runTransferTest.sh srcs/artdaq/test/TransferPlugins/fcl/transfer_driver_${type}_${test}.fcl 2 >|transfer_driver_${type}_${test}.log 2>&1
  done
done
# Newline for copy/paste execution

Since v3_00_00, older tests are documented here

driver tests¶

. products/setup
. setupARTDAQDEMO
olddir=$PWD
cd /tmp
for test in 1 2 3 4;do
  time artdaqDriver -c $olddir/srcs/artdaq_demo/tools/fcl/driver_test${test}.fcl >driver_test${test}.log 2>&1
done
rm driver_test*.root
mv driver_tets*.log $olddir
cd $olddir
# Newline for copy/paste execution

requestBasedDataFlow example¶

Make sure that port 3001/udp is open so that the data requests get through
On SL7:

firewall-cmd --add-port 3001/udp

Examining results from run_integration_tests.sh (New as of v3_06_00)¶

source setupARTDAQDEMO
cd daqdata
for file in *.root;do echo $file;rm out.bin;art -c toyDump.fcl $file >$file.toydump 2>&1 && hexdump -C out.bin >$file.hexdump;done

Examine output from toydump to determine that events contain expected Fragments.

Testing file compatibility¶

Use asciiDump.fcl and toyDump.fcl over reference source file:
(in a directory with the version being tested set up, i.e. use ./quick-mrb-start.sh --tag <VERSION>)

art -c $ARTDAQDEMO_REPO/artdaq-demo/ArtModules/fcl/toyDump.fcl /path/to/reference/file.root
art -c $ARTDAQDEMO_REPO/artdaq-demo/ArtModules/fcl/asciiDump.fcl /path/to/ASCII/reference/file.root

Routing Tests¶

Make sure that ports 35555/tcp, 35556/udp, 35557/udp, 36555/tcp, 36556/udp, and 36557/udp are open so that routing tokens get through

Old (Pre-artdaq v3_01_00) transfer_driver test results¶

Version	Qualifiers	ironwork.fnal.gov
		Large fragments			Small fragments
		Shmem	TCP	MPI	Shmem	TCP	MPI
v2_00_00*	s41:e10:eth	3.63 GB/s	151.48 MB/s	6.30 GB/s	7.43 MB/s	3.38 MB/s	52.24 MB/s
v2_01_00*	s41:e10:eth	4.36 GB/s	145.31 MB/s	2.99 GB/s	54.27 KB/s	30.49 MB/s	59.72 MB/s
v2_01_00	s41:e10:eth	4.28 GB/s	148.91 MB/s	7.05 GB/s	3.11 MB/s	109.54 MB/s	13.09 MB/s
v2_02_01	s41:e10:eth	4.34 GB/s	3.64 GB/s	6.20 GB/s	937.37 MB/s	680.02 MB/s	145.33 MB/s
v2_02_01	s46:e14:eth	4.44 GB/s	4.03 GB/s	6.88 GB/s	763.93 MB/s	579.11 MB/s	109.29 MB/s
v2_03_00	s46:e10:eth	4.37 GB/s	2.35 GB/s	6.66 GB/s	363.57 MB/s	429.90 MB/s	26.14 MB/s
v2_03_00	s48:e14:eth	1.55 GB/s	1.47 GB/s	6.68 GB/s	455.74 MB/s	510.53 MB/s	24.90 MB/s
v2_03_01	s46:e10:eth	4.44 GB/s	2.34 GB/s	6.88 GB/s	1.60 GB/s	1.12 GB/s	27.41 MB/s
v2_03_01	s50:e14:eth	4.37 GB/s	3.09 GB/s	6.98 GB/s	1.81 GB/s	1.05 GB/s	16.42 MB/s
v2_03_02	s46:e10:eth	4.35 GB/s	3.58 GB/s	6.61 GB/s	2.35 GB/s	999.97 MB/s	26.64 MB/s
v2_03_02	s50:e14:eth	2.81 GB/s	1.12 GB/s	3.95 GB/s	1.71 GB/s	861.82 MB/s	16.49 MB/s
v2_03_03+	s50:e14:eth	2.27 GB/s	1.47 GB/s	3.08 GB/s	479.39 MB/s	383.17 MB/s	15.75 MB/s
v3_00_02=	s50:e14	4.28 GB/s	4.00 GB/s		629.30 MB/s	876.74 MB/s
v3_00_03=	s64:e15	2.34 GB/s	1.01 GB/s		420.96 MB/s	444.53 MB/s

* "Small Fragments" test done with 1 KB fragments, instead of 5 KB.
+ Metrics disabled in .fcl files as an issue with them was discovered and previous versions did not have metrics enabled.
= MPI tests not run, MPI transfer is currently depreciated

Old (Pre-artdaq v3_01_00) Testing Procedure¶

Make sure that artdaq builds without errors or warnings
Make sure that all artdaq packages pass their built-in test suites
Check that quick-mrb-start.sh functions properly -- run without parameters
Perform transfer_driver tests (See transfer_driver tests below):
1. Large fragments (100 MB) x 10000, record rate for Shmem, TCP, MPI
2. Small fragments (5 KB) x 1000000, record rate for Shmem, TCP, MPI (Originally 1K fragments)
Perform artdaqDriver tests:
1. test1: 10,000 1 MB events, record time
2. test2: 1,000,000 1 KB events, record time
3. test3: 10,000 1 MB events without disk-writing, record time
4. test4: 10,000 1 MB events with Binary disk-writing to /dev/null, record time (new for v2_03_00, run for v2_02_01)
Run quick-mrb-start.sh --run-demo
1. Make sure the demo runs as expected
2. Make sure that the output data file is created
  1. Run rawEventDump.fcl over the data file
  2. Run toyDump.fcl over the data file
3. Store data file in Redmine as version reference
Run the asciiSimulator example (runAsciiSimulatorExample.sh)
1. Make sure that the example runs as expected
2. Make sure that the output data file is created
  1. Run rawEventDump.fcl over the data file
  2. Run asciiDump.fcl over the data file -- make sure that the proper strings are displayed
3. Store data file in Redmine as version reference
Run the requestBasedDataFlow example (runRequestBasedDataFlowExample.sh)
1. Make sure that the example runs as expected
2. Make sure that the output data file is created
  1. Run rawEventDump.fcl over the data file
  2. Run toyDump.fcl over the data file -- make sure that Container fragments are unpacked correctly
  3. (New for v2_03_00) Run wfViewer.fcl over the data file -- make sure that three plots appear on each window. Frag 0: Gaussian, Frag 1: Pattern (flat distribution with sharp cut-off), Frag 2: Uniform
Test version reference data files from Redmine -- note if version incompatibility exists
Test previous version of artdaq with current reference data files -- note if data files are not backwards-compatible
1. Run quick-mrb-start.sh --tag [previous version tag] in new directory
2. See compatibility test notes
DEPRECATED Test Infiniband build of artdaq (Currently using ds50daq)
1. Build artdaq with ib qualifier
2. Run transfer_driver_mpi across the ib link
3. Run ds50daq with new version of artdaq
Test Routing Master Functionality (new as of artdaq v2_03_00)
1. DEPRECATED (Part of artdaq test suite) Run mpirun -np 3 routing_master -c artdaq/test/Application/Routing/routing_master_t.fcl
2. Run 2x2x2x2 artdaq_demo system

Files (1)

Project

General

Profile

artdaq

Wiki

artdaq Release testing documentation and results¶

artdaq¶

transfer_driver performance metrics¶

artdaqDriver performance metrics¶

DAQInterface Example Configurations¶

Testing Results¶

v3_09_02¶

DAQInterface simple_test_configs for v3_09_02 s96:e19¶

v3_09_01¶

DAQInterface simple_test_configs for v3_09_01 s96:e19¶

DAQInterface simple_test_configs for v3_09_01 s100:e20¶

v3_09_00¶

DAQInterface simple_test_configs tests for v3_09_00¶

v3_08_00¶

DAQInterface simple_test_configs tests for v3_08_00¶

v3_07_02¶

DAQInterface simple_test_configs tests for v3_07_02¶

v3_07_01¶

DAQInterface simple_test_configs tests for v3_07_01¶

v3_06_01¶

DAQInterface simple_test_configs tests for v3_06_01¶

v3_06_00¶

DAQInterface simple_test_configs tests for v3_06_00¶

v3_05_00¶

DAQInterface simple_test_configs tests for v3_05_00¶

v3_04_00¶

v3_03_02¶

v3_03_01¶

v3_03_00¶

v3_02_01¶

v3_02_00¶

v3_01_00¶

v3_00_03 (from here down, uses old test procedure at end of page)¶

v3_00_02¶

v2_03_03¶

v2_03_02¶

v2_03_01¶

v2_03_00¶

v2_02_01¶

v2_01_00¶

Testing notes¶

transfer_driver tests¶

driver tests¶

requestBasedDataFlow example¶

Examining results from run_integration_tests.sh (New as of v3_06_00)¶

Testing file compatibility¶

Routing Tests¶

Old (Pre-artdaq v3_01_00) transfer_driver test results¶

Old (Pre-artdaq v3_01_00) Testing Procedure¶