Project

General

Profile

Feature #13877

Provide dataprep module that handles one APA at a time

Added by David Adams about 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Start date:
09/19/2016
Due date:
% Done:

0%

Estimated time:
Duration:

Description

The new dataprep module DataPrepModule and accompanying service StandardRawDigitPrepService (see #12701) build recob::Wire for the entire detector at once. This is likely unnecessarily using a large amount of memory to store the transient data (AdcChannelData). See #13749. For detectors with many APAs, we should be able to reduce this significantly by processing each APA independently and discarding the transient data for each immediately after processing. The request here is to provide such a module.


Related issues

Blocked by LArSoft - Feature #9264: Support reconstruction of objects from channel/time spaceClosed06/22/2015

History

#1 Updated by David Adams about 4 years ago

We would like the new module and service(s) to work for any detector or experiment. This implies we must be able to discover the channel-to-APA mappings from a shared larsoft interface, most likely the Geometry service. I believe this is covered by #9264.

#2 Updated by David Adams about 4 years ago

  • Subject changed from Provide dataprep service that handles one APA at a a time to Provide dataprep module that handles one APA at a a time

#3 Updated by David Adams about 4 years ago

  • Subject changed from Provide dataprep module that handles one APA at a a time to Provide dataprep module that handles one APA at a time

#4 Updated by David Adams about 4 years ago

  • Blocked by Feature #9264: Support reconstruction of objects from channel/time space added

#5 Updated by David Adams about 4 years ago

Feature #9264 provides an iterator over ROPs (readout planes) rather than APAs but as long as we don't want to handle more that one ROP at a time in data prep, looping over ROPs should reduce memory consumption even more than a loop over APAs.

#6 Updated by David Adams almost 4 years ago

There are now services that group channels by ROP and APA in dunetpc/dune/DuneCommon/Service. These provide the interface ChannelGroupService. I plan to modify the dataprep service to make use of that interface.

#7 Updated by David Adams almost 4 years ago

The service interface RawDigitPrepService and implementation StandardRawDigitPrepService are modified to take the input digits from the transient ADC channel data map instead of from a vector of digits. The module DataPrepModule is modified to fill the map and no longer pass the digit vector.

The changes will facilitate add a module which process channels by groups instead the whole detector at once.

#8 Updated by David Adams almost 4 years ago

I have verified that the component test for StandardRawDigitPrepService passes and that (by eye) the results are the same for a small MC sample.

#9 Updated by David Adams almost 4 years ago

I have modified DataPrepModule to add the option to process one channel group at a time. It is controlled by the fcl param DoGroups. I processed a small 35t no-ZS sim sample without groups and with APA and ROP groups. By eye, the prepared data looks the same for all three. The memory use (from the larsoft report) is 1.65 GB without groups, 1.48 with APA groups and 1.44 with ROP groups.

I have modified the default data prep configuration (producer_adcprep) to use groups and have added the APA channel group service to the configurations for 35t, 10kt and protoDUNE in services_dune.fcl. To disable use of groups, use this line:

physics.producers.caldata.DoGroups: false

I am about to commit these changes to dunetpc develop.

#10 Updated by David Adams almost 4 years ago

  • Status changed from Assigned to Closed

I also did a check with a small FD workspace sample. Again results are the same with and without groups. The memory without is 1.24 and with is 1.15 GB.

Closing this report.



Also available in: Atom PDF