Project

General

Profile

Feature #11777

Move 35t and 10kt production to the SimWireDUNE module

Added by David Adams over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Start date:
02/19/2016
Due date:
% Done:

0%

Estimated time:
Duration:

Description

We would like to move 35t production to the SimWireDUNE module and decommission SimWireDUNE35t.

Note that the new module includes the noise scale fix in
https://cdcvs.fnal.gov/redmine/issues/11470

However there is still an issue with random number seeds that must be resolved for production:
https://cdcvs.fnal.gov/redmine/issues/11695

We will also need to update the FCL files used for production. We can go ahead and do that now as there are seeds that are adequate for testing small samples (one job).

I assign this to Tingjun who maintains the production FCL. Please assign it back to me if you want me to work on that piece.

fdsignal.png (12.9 KB) fdsignal.png David Adams, 04/18/2016 12:23 PM
rawtracks_nonoise.png (23.2 KB) rawtracks_nonoise.png David Adams, 04/18/2016 12:58 PM
rawtracks_coll_nonoise.png (12 KB) rawtracks_coll_nonoise.png David Adams, 04/18/2016 12:58 PM
rawtracks_ind_nonoise.png (11.7 KB) rawtracks_ind_nonoise.png David Adams, 04/18/2016 12:58 PM
shaping_diff.txt (8.26 KB) shaping_diff.txt David Adams, 04/18/2016 01:38 PM
Screen Shot 2016-04-18 at 2.43.50 PM.png (99.7 KB) Screen Shot 2016-04-18 at 2.43.50 PM.png Tingjun Yang, 04/18/2016 02:46 PM
raw_evt1_mum_v050800.png (24.3 KB) raw_evt1_mum_v050800.png David Adams, 04/19/2016 09:12 AM
1689271_0_noDAQ_reco_OldDetsim.txt (15.5 KB) 1689271_0_noDAQ_reco_OldDetsim.txt Output from running the OldDetsim Thomas Warburton, 04/26/2016 12:03 PM
NewDetsim.fcl (2.61 KB) NewDetsim.fcl fcl file to run the NewDetsim Thomas Warburton, 04/26/2016 12:03 PM
OldDetsim.fcl (1.94 KB) OldDetsim.fcl fcl file to run the OldDetsim Thomas Warburton, 04/26/2016 12:03 PM
1689271_0_noDAQ_reco_NewDetsim.txt (7.1 MB) 1689271_0_noDAQ_reco_NewDetsim.txt Output from running the NewDetsim Thomas Warburton, 04/26/2016 12:03 PM

History

#1 Updated by Tingjun Yang over 3 years ago

  • Assignee changed from Tingjun Yang to David Adams

Hi David,

I am reassigning this back to you since you know the details of the new detsim. We would like to modify the following two fcl files in dunetpc:
fcl/dune35t/detsim/standard_detsim_dune35t.fcl
fcl/dunefd/detsim/standard_detsim_dune10kt.fcl

Thanks,
Tingjun

#2 Updated by Thomas Warburton over 3 years ago

Hi David,

What is the status of this?

I would like to start trying to split Monte Carlo.

#3 Updated by David Adams over 3 years ago

We are waiting for Gianluca to complete this: https://cdcvs.fnal.gov/redmine/issues/11125. I have just pinged him.

And then I have update our code and do some tests. I will try to start on that next week

#4 Updated by David Adams over 3 years ago

No response from Gianluca. I have pinged him again.

I am working on top-level FCL now. My working file is dunetpc/fcl/dune35t/detsim/new_standard_detsim_dune35t.fcl. I will later drop the prefix "new_".

Note that most of the service references in that file are defined in dunetpc/dune/DetSim/detsimmodules_dune.fcl.

We need to make sure the definitions are correct in the latter and that we have included the desired options in the former.

Right now I am having problems with the pedestal service. I have the definition:

pedlegacy: {
  service_provider: SIOVDetPedestalService
  DetPedestalRetrievalAlg: {
    AlgName:            "DetPedestalRetrievalAlg" 
    DatabaseRetrievalAlg: {
      AlgName: "DatabaseRetrievalAlg" 
      DBFolderName:  "" 
      DBUrl: "" 
      DBTag: "" 
    }
    UseDB: false
    UseFile: false
    DefaultCollMean:  500.0
    DefaultCollRms:     0.01
    DefaultIndMean:  1800.0
    DefaultIndRms:      0.01
    DefaultMeanErr:     0.0
    DefaultRmsErr:      0.0
  }
}

but get this error:

%MSG
bool dune::DetPedestalDUNE::Update(uint64_t) Called with run 1
bool dune::DetPedestalDUNE::Update(uint64_t) Method of determining pedestals is undefined! Either set UseDB or CSVFileName in fhicl. Begin processing the 1st record. run: 1 subRun: 0 event: 1 at 15-Mar-2016 08:55:51 EDT
DetPedestalDUNE: Validity time is not set!  Aborting.

Tingjjun and Jonathon, can you confirm we want to use fixed pedestals and tell how to correct the above FCL to allow that? It was working a month or two ago.

#5 Updated by Jonathan Paley over 3 years ago

  • Status changed from New to Feedback

Unfortunately I am not expert enough in art/fhicl to quickly help out here, but I don't understand why we would be using this fhicl for the pedestals, or DetPedestalRetrievalAlg. We should be using the DUNE pedestal service, DetPedestalServiceDUNE. That said, I think this would fail even if we made the switch because the Utilities/pedestals_dune.fcl has not been updated. I'll take care of that now.

#6 Updated by Tingjun Yang over 3 years ago

Hi David,

As Jon pointed out, I think you need to use the DUNE pedestal service (DetPedestalService), which is defined in:
Utilities/services_dune.fcl
But I think Jon needs to update pedestals_dune.fcl to include the new fcl parameters.

For MC simulation, we want to set UseDefaults to true and set the default pedestals for U/V/Y planes so the service won't try to connect to database.

Tingjun

#7 Updated by David Adams over 3 years ago

  • Status changed from Feedback to Assigned

I will change the code to use DetPedestalService/DetPedestalProvider interfaces. What behavior do we want for simulation? Previously we had one fixed value for the collection channels and a different value for the induction. Do we have an implementation?

#8 Updated by David Adams over 3 years ago

We are already using DetPedestalService/DetPedestalProvider. The FCL block in comment 4 is irrelevant and I have removed it from the code. Sorry for confusing matters.

But I still get the error in comment 4.

I would like to add a line something like

services.DetPedestalService: @local::ped35tfixed

where ped35tfixed is defined in prolog (in pedestals_dune.fcl?) with the desired behavior.

Again the old behavior was collection pedestal = 500, induction pedestal = 1800.

#9 Updated by Tingjun Yang over 3 years ago

Hi David,

Jon has updated pedestals_dune.fcl. I think we need to set the following parameters:
UseDB: false
UseDefaults: true
DefaultMeanU: 750.
DefaultMeanErrU: 0.
DefaultRmsU: 5.
DefaultRmsErrU: 0.
DefaultMeanV: 750.
DefaultMeanErrV: 0.
DefaultRmsV: 5.
DefaultRmsErrV: 0.
DefaultMeanZ: 750.
DefaultMeanErrZ: 0.
DefaultRmsZ: 5.
DefaultRmsErrZ: 0.

We need to replace 750 with the actual values. Actually right now we are using the same pedestal for induction and collection wires. I am not sure what we should use in the MC.

Tingjun

#10 Updated by David Adams over 3 years ago

Tingjun:

Are we presently using this FCL for production: standard_detsim_dune35t.fcl?

When I use it, I get pedestals of 500 and 1800, consistent with the daq parameters I find in my FCL dump:

      CollectionCalibPed: 500
      CollectionPed: 500
      InductionCalibPed: 1800
      InductionPed: 1800

The new producer will obtain the pedestals from the pedestal service. The old one had them as parameters.

If there are no objections, I will start with these values.

#11 Updated by Tingjun Yang over 3 years ago

Hi David,

Most of the 35t data were taken using the same pedestal 500 for both induction and collection wires. Maybe it would be good to use that for the simulation. Also we use noise RMS 2.5 ADC in the past, but the real noise is at least 10 ADC. I don't know if we want to change that.

Tingjun
David Adams wrote:

Tingjun:

Are we presently using this FCL for production: standard_detsim_dune35t.fcl?

When I use it, I get pedestals of 500 and 1800, consistent with the daq parameters I find in my FCL dump:
[...]

The new producer will obtain the pedestals from the pedestal service. The old one had them as parameters.

If there are no objections, I will start with these values.

#12 Updated by David Adams over 3 years ago

Perhaps we should get feedback from the 35-ton analysis group on what configuration(s) should be used for future 35-ton simulation. I do think we want to improve the noise model--it is one of the reasons I started working on this. If we try to match the pedestals, we should also make sure we have consistent gains.

I would like to start by getting close to the old simulation results and then later tune the simulation to match the detector.

#13 Updated by David Adams over 3 years ago

I get this error from DetPedestalDUNE:

---- OtherArt BEGIN
  ServiceCreation
  ---- Type mismatch BEGIN

    Unsuccessful attempt to convert FHiCL parameter 'Run' to type 'bool'.

    [Specific error:]
    ---- Type mismatch BEGIN
      error in bool string:
      0
      at or before:
    ---- Type mismatch END
  ---- Type mismatch END
  cet::exception caught during construction of service type lariov::DetPedestalServiceDUNE:
---- OtherArt END

To resolve it, I am changing
fVldTime = p.get<bool>("Run",0);

to
fVldTime = p.get<int>("Run",0);

Please let me know if I have misunderstood.

#14 Updated by David Adams over 3 years ago

I am finally able to get DetPedestalDUNE to return fixed pedestals. However it appears that it always returns the value set for unknown orientation instead of using the geometry to properly look up and use the value assigned for U, V or Z. Should I fix this? --david

#15 Updated by David Adams over 3 years ago

There was no response to my last comment. I plan to add a new service likely called FixedDetPedestalService that can be used in place of DetPedestalServiceDUNE that will return pedestals with fixed, separate values for U, V and Z orientations. This way those who want to take pedestals from the DB won't have to deal with geometry and those who want geometry-based pedestals won't have to deal with the DB.

The configuration of DetPedestalServiceDUNE was in services_dune.fcl while the configuration of its provider was in pedestals_dune.fcl. I have moved both of these to the latter file so the configuration is all in one place. I also dropped the extraneous definition of ChannelMapService within the DetPedestalServiceDUNE block.

#16 Updated by Jonathan Paley over 3 years ago

David Adams wrote:

There was no response to my last comment. I plan to add a new service likely called FixedDetPedestalService that can be used in place of DetPedestalServiceDUNE that will return pedestals with fixed, separate values for U, V and Z orientations. This way those who want to take pedestals from the DB won't have to deal with geometry and those who want geometry-based pedestals won't have to deal with the DB.

Adding a new service seems unnecessary to me. Why not just fix the existing service to do what you want? I thought that was what you were proposing in your previous message.

#17 Updated by David Adams over 3 years ago

I suppose it is somewhat a matter of taste (if that can be distinguished from good design). Where there are two distinct implementations with different dependencies, I would prefer to have two distinct services. I will add a second block to pedestals_dune.fcl so that users can choose between the two options in FCL without needing to know about the choice of one or two services. Thanks for following the discussion. --da

#18 Updated by David Adams over 3 years ago

I have added FixedDetPedestalService in dunetpc/dune/DetSim/Services and have added a default configuration to pedestals_dune.fcl. Changes are committed to dunetpc.

#19 Updated by Tingjun Yang over 3 years ago

I also prefer to have one service for the pedestal. It seems Jon's service has all the features we need (he added the support to configure pedestal for different plane). Jon, I know you are busy with the Nova meeting, but would you have time to take a look why it did not work for David?

Thanks,
Tingjun

#20 Updated by David Adams over 3 years ago

Tingjun, please take a look at what is in pedestals_dune.fcl now. We have two configuration and can easily add more or users can start from these. The description for the fixed params is simple. I would prefer to remove the fixed params from the old service so that both descriptions would be as simple as possible.

There is a little extra complexity because of the service/provider model. As I have argued in other forums, we should should also avoid that but I followed it to keep the same interface as was used in the first service.

Service users access both options via the same interface and so won't need to look at, compile or link against the two implementations.

#21 Updated by David Adams over 3 years ago

I have modified configurations to use the new fixed pedestals and to drop the addition of noise in the pedestal addition service. The results for the new detector simulation are vary similar to the old when the zero suppression is disabled.

#22 Updated by David Adams over 3 years ago

The production FCL for single-particle 35t DetSim has been updated to use the new DetSim module.
This is dunetpc/fcl/dune35t/detsim/standard_detsim_dune35t.fcl.

The old version of that file, which used the old DetSim module, is renamed to
dunetpc/fcl/dune35t/detsim/old_standard_detsim_dune35t.fcl.

The following items are needed before going into production:
-- Update the FCL to use random number seeds from a seed service that supports services. This requires a seed service update promised by Gianluca for many months.
-- Update the FCL for multi-particle 35t production
-- Update the FCL for FD production. This may require new services.

#23 Updated by David Adams over 3 years ago

I have switched the 35t cosmic to the new DetSim module.

I have also turned of sticky bit suppression in zslegacy because we are not adding sticky bits.

I see now that the old simulation appears noisy because it has a higher noise level, not because of a problem with ZS. We should increase the noise level in the new simulation.

#24 Updated by David Adams over 3 years ago

The old detsim has noise sigma = 2.531 for singles and 7.16 for cosmics. The diff is the old noise depends on the # ticks.

The new detsim as 2.50 for both and has legacy noise suppression enabled. If noise level is increase, ZS will have to be removed or retuned.

#25 Updated by David Adams over 3 years ago

I am now trying to look at the FD simulation. Running with this FCL:

#include "standard_detsim_dune10kt_workspace.fcl" 

I get very noisy raw data (RMS ~150 ADC counts). Figure is attached.

Tingjun, is this what I should expect?

da

#26 Updated by Tingjun Yang over 3 years ago

Hi David,

This is not expected. Did you run the fcl file on a FD simulation file?

Tingjun
David Adams wrote:

I am now trying to look at the FD simulation. Running with this FCL:
[...]
I get very noisy raw data (RMS ~150 ADC counts). Figure is attached.

Tingjun, is this what I should expect?

da

#27 Updated by David Adams over 3 years ago

Sorry, I should have been more complete. I first generated a file with:

prod_muminus_0.1-5.0GeV_isotropic_dune10kt_workspace.fcl

and then ran geant over the output of that with
standard_g4_dune10kt_workspace.fcl

The output of that was used as input to the FD detsim above.

I am using dunetpc from last Friday, specifically tag 4fdfc72e7d3bcd9b4b708d8536652c8bfa9d582c.

da

#28 Updated by David Adams over 3 years ago

If i drop the noise in detsim with

physics.producers.daq.NoiseOn:   0

the I do see tracks. See the attached plots.

#29 Updated by David Adams over 3 years ago

I see we have two shaping services: SignalShapingServiceDUNE10kt and SignalShapingServiceDUNE35t but they appear to be the same except for class name. I would like to copy one of these to a new service SignalShapingServiceDUNE and use it in place of these for detsim. I would mark the others as obsolete.

Any objections?

da

#30 Updated by David Adams over 3 years ago

I attach a diff for the two services in the previous message.

#31 Updated by Tingjun Yang over 3 years ago

Hi David,

I run the fcl files you specified and the result looks fine. I am using my feature branch though tjyang_mcc6. But I don't think I change anything related to noise simulation.

Tingjun

David Adams wrote:

Sorry, I should have been more complete. I first generated a file with:
[...]
and then ran geant over the output of that with
[...]
The output of that was used as input to the FD detsim above.

I am using dunetpc from last Friday, specifically tag 4fdfc72e7d3bcd9b4b708d8536652c8bfa9d582c.

da

#32 Updated by David Adams over 3 years ago

I will try v05_08_00 and see if the problem persists.

#33 Updated by David Adams over 3 years ago

There is much less noise in v05_08_00. See figure. I will try v05_09_00.

#34 Updated by David Adams over 3 years ago

The noise is also low in v05_09_00 and is low when I go back to the (new) head for dunetpc. I.e. the problem seems to have gone away.

I go back to merging the shaping services. We then need to switch to the new seed service (that allows seeds outside of modules). And then maybe ready for production.

#35 Updated by David Adams over 3 years ago

In my sw9 area I am generating ref samples 0 with this dunetpc tag

commit 663d31fc078afa14a6d5924296617bd374beccc2
Author: David Adams <dladams@bnl.gov>
Date:   Wed Apr 20 11:44:56 2016 -0400

    Add new FD detsim FCL.

for prodfd/olddetsim_mum and prod35t/olddetsim.

#36 Updated by David Adams over 3 years ago

I have consolidated the signal shaping services into SignalShapingServiceDUNE and updated the DetSim classes and FCL to use the new service in place of SignalShapingServiceDUNE35t and SignalShapingServiceDUNE10kt. The new tag is

commit 938629fffb1b8ce91d6675b7d5876a53c676da9f
Author: David Adams <dladams@bnl.gov>
Date:   Thu Apr 21 12:07:06 2016 -0400

    Move old destim modules to the new signal shaping.

    See https://cdcvs.fnal.gov/redmine/issues/11777.

and I have verified the old detsim for 35t and 10kt give the same results as the reference in comment #35 above.

#37 Updated by David Adams over 3 years ago

The new DetSim seems to be working. The FCL file names are (for now) prefixed with "new_, e.g. new_standard_detsim_dune10kt_workspace.fcl.

However the noise levels and suppression seem inconsistent for the old simulation: The noise RMS is 3.5 ADC counts and the ZS seems to be that tuned to remove noise for and RMS of 2.5 ADC counts. I suspect it was intended to do FD simulation with the lower noise level but the 35-ton params were used and the larger number of ticks in FD leads to a higher noise level.

We should decide which noise level we want as the default for FD simulation and where we want to set the ZS.

#38 Updated by Tingjun Yang over 3 years ago

Hi David,

Thanks for implementing this. I think you are right. We intended to set noise RMS to 2.5 ADC but it was actually higher because of longer readout window. I think for MCC6, we should set noise RMS to 2.5 ADC again. It would be great if this can be finalized so we can start testing.

One related issue is pedestal. Before we always set pedestal to 0 for FD simulation. Is this still the case?

Thanks,
Tingjun
David Adams wrote:

The new DetSim seems to be working. The FCL file names are (for now) prefixed with "new_, e.g. new_standard_detsim_dune10kt_workspace.fcl.

However the noise levels and suppression seem inconsistent for the old simulation: The noise RMS is 3.5 ADC counts and the ZS seems to be that tuned to remove noise for and RMS of 2.5 ADC counts. I suspect it was intended to do FD simulation with the lower noise level but the 35-ton params were used and the larger number of ticks in FD leads to a higher noise level.

We should decide which noise level we want as the default for FD simulation and where we want to set the ZS.

#39 Updated by David Adams over 3 years ago

The old FD production files FFF.fcl are renamed to old_FFF.fcl so the new destim files can use the original names.

#40 Updated by Tingjun Yang over 3 years ago

Hi David,

Could you also update the fcl files with 1x2x6 in the name? Those are the fcl files we plan to use in MCC6 (replacing workspace geometry).

Thanks,
Tingjun
David Adams wrote:

The old FD production files FFF.fcl are renamed to old_FFF.fcl so the new destim files can use the original names.

#41 Updated by David Adams over 3 years ago

I saw those files came in as I was making my change. All files using the old Detsim are now named old_*. The files standard_* use the new Detsim.

I also updated all to use the generic signal shaping service.

Here is what we have now:

old_standard_detsim_dune10kt_1x2x6.fcl
old_standard_detsim_dune10kt_3mmpitch_1x2x6.fcl
old_standard_detsim_dune10kt_3mmpitch_workspace.fcl
old_standard_detsim_dune10kt_45deg_1x2x6.fcl
old_standard_detsim_dune10kt_45deg_workspace.fcl
old_standard_detsim_dune10kt.fcl
old_standard_detsim_dune10kt_workspace.fcl
standard_detsim_dune10kt_1x2x6.fcl
standard_detsim_dune10kt_3mmpitch_1x2x6.fcl
standard_detsim_dune10kt_3mmpitch_workspace.fcl
standard_detsim_dune10kt_45deg_1x2x6.fcl
standard_detsim_dune10kt_45deg_workspace.fcl
standard_detsim_dune10kt.fcl
standard_detsim_dune10kt_workspace.fcl

Should I remove standard*workspace*.fcl and validate standard*1x2x6.fcl?

#42 Updated by Tingjun Yang over 3 years ago

Hi David,

I think it is important to validate 1x2x6 geometry for now since that's the default geometry for MCC6. I am ok with removing workspace fcl files.

By the way, it seems protondune is affected by the recent changes. For example, I saw the following when running a reconstruction job:
lar c protoDUNE_reco.fcl gen_protoDune_muon_200MeV_mono_g4_detsim.root
cet::exception caught in art
---
EventProcessorFailure BEGIN
An exception occurred during current event processing
---- ScheduleExecutionFailure BEGIN
ProcessingStopped.

---- ServiceNotFound BEGIN
Service unable to find requested service with compiler type name 'util::SignalShapingServiceDUNE10kt'.
cet::exception going through module CalWireDUNE10kt/caldata run: 1 subRun: 0 event: 1
---- ServiceNotFound END
Exception going through path reco
---- ScheduleExecutionFailure END
cet::exception caught in EventProcessor and rethrown
------------------------------------------------------------

Could you please also validate the protodune simulation and reco fcl files?
srcs/dunetpc/fcl/protodune/

Thanks,
Tingjun

David Adams wrote:

I saw those files came in as I was making my change. All files using the old Detsim are now named old_*. The files standard_* use the new Detsim.

I also updated all to use the generic signal shaping service.

Here is what we have now:
[...]

Should I remove standard*workspace*.fcl and validate standard*1x2x6.fcl?

#43 Updated by Tingjun Yang over 3 years ago

Hi again,

I forgot to mention that I just merged feature/etw_protoduneMCC into develop so please update before looking at protodune fcl files.

Tingjun

Tingjun Yang wrote:

Hi David,

I think it is important to validate 1x2x6 geometry for now since that's the default geometry for MCC6. I am ok with removing workspace fcl files.

By the way, it seems protondune is affected by the recent changes. For example, I saw the following when running a reconstruction job:
lar c protoDUNE_reco.fcl gen_protoDune_muon_200MeV_mono_g4_detsim.root
cet::exception caught in art
---
EventProcessorFailure BEGIN
An exception occurred during current event processing
---- ScheduleExecutionFailure BEGIN
ProcessingStopped.

---- ServiceNotFound BEGIN
Service unable to find requested service with compiler type name 'util::SignalShapingServiceDUNE10kt'.
cet::exception going through module CalWireDUNE10kt/caldata run: 1 subRun: 0 event: 1
---- ServiceNotFound END
Exception going through path reco
---- ScheduleExecutionFailure END
cet::exception caught in EventProcessor and rethrown
------------------------------------------------------------

Could you please also validate the protodune simulation and reco fcl files?
srcs/dunetpc/fcl/protodune/

Thanks,
Tingjun

David Adams wrote:

I saw those files came in as I was making my change. All files using the old Detsim are now named old_*. The files standard_* use the new Detsim.

I also updated all to use the generic signal shaping service.

Here is what we have now:
[...]

Should I remove standard*workspace*.fcl and validate standard*1x2x6.fcl?

#44 Updated by David Adams over 3 years ago

  • Subject changed from Move 35t production to the SimWireDUNE module to Move 35t and 10kt production to the SimWireDUNE module

The title is modified to reflect that the scope of this ticket is expanded to include 10kt (FD) production.

#45 Updated by Tingjun Yang over 3 years ago

I still see lots of util::SignalShapingServiceDUNE10kt and util::SignalShapingServiceDUNE35t instances. Are they supposed to be replaced by util::SignalShapingServiceDUNE?

Tingjun
<dunegpvm05.fnal.gov> ack SignalShapingServiceDUNE10kt
AnaTree/CMakeLists.txt
12: SignalShapingServiceDUNE10kt_service

CalData/CMakeLists.txt
11: SignalShapingServiceDUNE10kt_service

CalData/CalGausHFDUNE10kt_module.cc
33:#include "dune/Utilities/SignalShapingServiceDUNE10kt.h"
205: art::ServiceHandle<util::SignalShapingServiceDUNE10kt> sss;

CalData/CalWireDUNE10kt_module.cc
42:#include "dune/Utilities/SignalShapingServiceDUNE10kt.h"
141: art::ServiceHandle<util::SignalShapingServiceDUNE10kt> sss;

CalData/DeconvGausHFDUNE10kt_module.cc
35:#include "dune/Utilities/SignalShapingServiceDUNE10kt.h"
213: art::ServiceHandle<util::SignalShapingServiceDUNE10kt> sss;

Utilities/CMakeLists.txt
30:simple_plugin( SignalShapingServiceDUNE10kt "service"

Utilities/SignalShapingServiceDUNE10kt.h
3:/// \file SignalShapingServiceDUNE10kt.h
42: class SignalShapingServiceDUNE10kt {
47: SignalShapingServiceDUNE10kt(const fhicl::ParameterSet& pset,
49: ~SignalShapingServiceDUNE10kt();
83: void init() const{const_cast<SignalShapingServiceDUNE10kt*>(this)->init();}
166:template <class T> inline void util::SignalShapingServiceDUNE10kt::Convolute(unsigned int channel, std::vector<T>& func) const
188:template <class T> inline void util::SignalShapingServiceDUNE10kt::Deconvolute(unsigned int channel, std::vector<T>& func) const
206:DECLARE_ART_SERVICE(util::SignalShapingServiceDUNE10kt, LEGACY)

Utilities/SignalShapingServiceDUNE10kt_service.cc
2:/// \file SignalShapingServiceDUNE10kt_service.cc
6:#include "dune/Utilities/SignalShapingServiceDUNE10kt.h"
20:util::SignalShapingServiceDUNE10kt::SignalShapingServiceDUNE10kt(const fhicl::ParameterSet& pset,
25: mf::LogInfo("SignalShapingServiceDUNE10kt") << "Deprecated: Consider using SignalShapingServiceDUNE";
32:util::SignalShapingServiceDUNE10kt::~SignalShapingServiceDUNE10kt()
38:void util::SignalShapingServiceDUNE10kt::reconfigure(const fhicl::ParameterSet& pset)
78: mf::LogInfo("SignalShapingServiceDUNE10kt") << "Getting Filter from .fcl file";
105: mf::LogInfo("SignalShapingServiceDUNE10kt") << " using filter from .root file ";
183:util::SignalShapingServiceDUNE10kt::SignalShaping(unsigned int channel) const
212:double util::SignalShapingServiceDUNE10kt::GetASICGain(unsigned int const channel) const
235:double util::SignalShapingServiceDUNE10kt::GetShapingTime(unsigned int const channel) const
257:double util::SignalShapingServiceDUNE10kt::GetRawNoise(unsigned int const channel) const
297:double util::SignalShapingServiceDUNE10kt::GetDeconNoise(unsigned int const channel) const
339:void util::SignalShapingServiceDUNE10kt::init()
399:void util::SignalShapingServiceDUNE10kt::SetFieldResponse()
488: mf::LogInfo("SignalShapingServiceDUNE10kt") << " using the old field shape ";
518:void util::SignalShapingServiceDUNE10kt::SetElectResponse(double shapingtime, double gain)
593:void util::SignalShapingServiceDUNE10kt::SetFilters()
661:void util::SignalShapingServiceDUNE10kt::SetResponseSampling()
748:int util::SignalShapingServiceDUNE10kt::FieldResponseTOffset(unsigned int const channel) const
776: DEFINE_ART_SERVICE(SignalShapingServiceDUNE10kt)

Utilities/SignalShapingServiceDUNE35t_service.cc
26: mf::LogInfo("SignalShapingServiceDUNE10kt") << "Deprecated: Consider using SignalShapingServiceDUNE";

Utilities/test/SignalShapingDUNE10ktTest_module.cc
16:#include "dune/Utilities/SignalShapingServiceDUNE10kt.h"
90: art::ServiceHandle<util::SignalShapingServiceDUNE10kt> sss;

Utilities/test/sss_dune10kt.fcl
18:services.user.SignalShapingServiceDUNE10kt: @local::dune10kt_signalshapingservice
<dunegpvm05.fnal.gov> ack SignalShapingServiceDUNE35t
AnaTree/CMakeLists.txt
13: SignalShapingServiceDUNE35t_service

CalData/CMakeLists.txt
12: SignalShapingServiceDUNE35t_service

CalData/CalGausHFDUNE35t_module.cc
33:#include "dune/Utilities/SignalShapingServiceDUNE35t.h"
232: art::ServiceHandle<util::SignalShapingServiceDUNE35t> sss;

CalData/CalWireDUNE35t_module.cc
43:#include "dune/Utilities/SignalShapingServiceDUNE35t.h"
151: art::ServiceHandle<util::SignalShapingServiceDUNE35t> sss;

CalData/DeconvGausHFDUNE35t_module.cc
35:#include "dune/Utilities/SignalShapingServiceDUNE35t.h"
220: art::ServiceHandle<util::SignalShapingServiceDUNE35t> sss;

DetSim/Service/test/test_ExponentialChannelNoiseService.cxx
111: ofstream ofile("test_SignalShapingServiceDUNE35t_services.fcl");
115: scfg = "test_SignalShapingServiceDUNE35t_services.fcl";

Utilities/CMakeLists.txt
52:simple_plugin( SignalShapingServiceDUNE35t "service"

Utilities/SignalShapingServiceDUNE10kt_service.cc
151: mf::LogInfo("SignalShapingServiceDUNE35t") << " using the field response provided from a .root file " ;
205: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
228: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
252: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
273: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
313: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
765: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"

Utilities/SignalShapingServiceDUNE34kt_service.cc
148: mf::LogInfo("SignalShapingServiceDUNE35t") << " using the field response provided from a .root file " ;
202: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
225: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
249: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
270: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
310: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
763: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"

Utilities/SignalShapingServiceDUNE35t.h
3:/// \file SignalShapingServiceDUNE35t.h
43: class SignalShapingServiceDUNE35t {
48: SignalShapingServiceDUNE35t(const fhicl::ParameterSet& pset,
50: ~SignalShapingServiceDUNE35t();
85: void init() const{const_cast<SignalShapingServiceDUNE35t*>(this)->init();}
170:template <class T> inline void util::SignalShapingServiceDUNE35t::Convolute(unsigned int channel, std::vector<T>& func) const
192:template <class T> inline void util::SignalShapingServiceDUNE35t::Deconvolute(unsigned int channel, std::vector<T>& func) const
210:DECLARE_ART_SERVICE(util::SignalShapingServiceDUNE35t, LEGACY)

Utilities/SignalShapingServiceDUNE35t_service.cc
2:/// \file SignalShapingServiceDUNE35t_service.cc
6:#include "dune/Utilities/SignalShapingServiceDUNE35t.h"
21:util::SignalShapingServiceDUNE35t::SignalShapingServiceDUNE35t(const fhicl::ParameterSet& pset,
33:util::SignalShapingServiceDUNE35t::~SignalShapingServiceDUNE35t()
39:void util::SignalShapingServiceDUNE35t::reconfigure(const fhicl::ParameterSet& pset)
81: mf::LogInfo("SignalShapingServiceDUNE35t") << "Getting Filter from .fcl file" ;
107: mf::LogInfo("SignalShapingServiceDUNE35t") << " using filter from .root file " ;
153: mf::LogInfo("SignalShapingServiceDUNE35t") << " using the field response provided from a .root file " ;
185:util::SignalShapingServiceDUNE35t::SignalShaping(unsigned int channel) const
207: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
214:double util::SignalShapingServiceDUNE35t::GetASICGain(unsigned int const channel) const
231: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
238:double util::SignalShapingServiceDUNE35t::GetShapingTime(unsigned int const channel) const
255: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
260:double util::SignalShapingServiceDUNE35t::GetRawNoise(unsigned int const channel) const
276: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
300:double util::SignalShapingServiceDUNE35t::GetDeconNoise(unsigned int const channel) const
316: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
341:void util::SignalShapingServiceDUNE35t::init()
401:void util::SignalShapingServiceDUNE35t::SetFieldResponse()
491: mf::LogInfo("SignalShapingServiceDUNE35t") << " using the old field shape " ;
522:void util::SignalShapingServiceDUNE35t::SetElectResponse(double shapingtime, double gain)
595:void util::SignalShapingServiceDUNE35t::SetFilters()
665:void util::SignalShapingServiceDUNE35t::SetResponseSampling()
774:int util::SignalShapingServiceDUNE35t::FieldResponseTOffset(unsigned int const channel) const
791: throw cet::exception("SignalShapingServiceDUNE35t")<< "can't determine"
802: DEFINE_ART_SERVICE(SignalShapingServiceDUNE35t)

Utilities/services_dune.fcl
254:dune35t_simulation_services.SignalShapingServiceDUNE35t: @local::dune35t_signalshapingservice

Utilities/test/SignalShapingDUNE35tTest_module.cc
16:#include "dune/Utilities/SignalShapingServiceDUNE35t.h"
90: art::ServiceHandle<util::SignalShapingServiceDUNE35t> sss;

Utilities/test/sss_dune35t.fcl
18:services.user.SignalShapingServiceDUNE35t: @local::dune35t_signalshapingservice

#46 Updated by David Adams over 3 years ago

Yes, replace util::SignalShapingServiceDUNE10kt and util::SignalShapingServiceDUNE35t with util::SignalShapingServiceDUNE. --da

#47 Updated by David Adams over 3 years ago

I see that the detsim geometry jobs pick up geometry from the input geant file and geant jobs pick up geometry from the input gen file. This geometry read from the file overrides the one specified in the job FCL. Because of this, I have always been validating the old ws geometry even where I intended to validate the new version. I have modified the dun_extensions validation scripts to remedy this, i.e. to specify all geometries for gen and geant and make consistent specification of geometry.

I have been using the single muminus samples for validation but I see there is no 3 mm or 45 degree gen file for these. I am adding these to dunetpc/fcl/prodfd/gen.

#48 Updated by David Adams over 3 years ago

The status of the new DetSim module was presented by me at the FD reco meeeting today:

https://indico.fnal.gov/conferenceDisplay.py?confId=11982

Xin suggested we should truncate rather than round when converting the floating ADC signal to the integer ADC count, I think, because of the nature of the ADC. But I don't think the device matters. What is relevant is how we define the pedestal. To me it is most natural to define it with rounding. E.g. an ADC count of 1020 with pedestal 1000 means the (uncalibrated) charge is in the range (19.5, 20.5) ADC counts.

Do others agree or disagree?

#49 Updated by Thomas Warburton over 3 years ago

Hi,

I am not sure how much testing has been done on reconstruction with the full far detector geometry but I just did a run and am very concerned with what I have just seen. I attach the fcl files I used to do the comparison along with the output I generated to the screen.
I got this output by adding the line: if (rois.size()) std::cout << "Xin: " << max << " "<< channel << " " << deconNoise << " " << rois.size() << std::endl; at line 249 of the module.
The directory where I ran everything from is here; /dune/data2/users/warburton/MUSUN/NewDetsim/.
I was running on the 8th event ( as I know that has a particle in TPC 154 ), not that that makes a difference though as I have seen it stall on all events.

As you will notice the new detsim gives many many more ROI's, do we desire this large an increase?
I left the caldata stage running for a number of hours earlier this week and it never exited the script. Supposedly because it was struggling to make all of the necessary SimWires.

I also noticed that the file size double when running the new detsim as compared to running the old detsim.

Cheers.

#50 Updated by David Adams over 3 years ago

Karl:

I am just seeing your last comment. Sorry to not be following more closely.

I have not attempted any reco with the full FD geometry. I will try running your OldDetsim and NewDetsim through DetSim and see if I see significant differences. Can you see the same problem in simplified geometries? This would be easier to study.

#51 Updated by David Adams over 3 years ago

  • Status changed from Assigned to Closed

A new seed service LArSeedService has been added to larsim that allows seeds to be assigned outside of modules. I have modified the relevant DetSim services to use this new service. In each case, one may override use of the service by setting the FCL parameter RandomSeed to a nonzero value which is then used as the seed.

This item will now be closed.

#52 Updated by David Adams over 3 years ago

  • Status changed from Closed to Assigned

The last comment should have gone (and is copied to) https://cdcvs.fnal.gov/redmine/issues/11695. This ticket is not yet complete and I reopen it.

#53 Updated by Thomas Warburton over 3 years ago

Hi David,

I have just ran the detsim stage on a sample of 10 events which I generated for the 1x2x6 geometry.
I see that the new and old detsim modules take the same amount of time to run.
However, caldata takes significantly longer to run after the new detsim module ( 275 vs 32 seconds for the 10 events).

I see from the other ticket that you configured the noise to be 2.5 ADC counts using the old module. Could the noise be larger with the new module?

I also ran the new module on the 10 kt geometry and caldata was able to complete in roughly the same time as it would when using the old module.

#54 Updated by David Adams over 3 years ago

Karl:

The noise should be 2.5 ADC counts for both old and new in v05_09_01. The detsim job produces histograms that show this noise level in both cases albeit with different names. I do think we need to understand why caldata would be different for the two. I will have a look. --david

#55 Updated by David Adams over 3 years ago

I get the same results as Karl. The new and old Detsim take the same amount of time (3.8 s/event) using either my or Karl's configuration. However, with (Karl's configuration), caldata takes 10X longer after the new Detsim (17 s/event vs 2.2 sec/event). Caldata and Detsim were run in the same job to generate these numbers. I will continue to investigate.

#56 Updated by David Adams over 3 years ago

My analysis program shows that the pedestal-subtracted raw data produced by the old and new detsim are very similar but there are significant differences in the raw data before unpacking. The pedestals are zero in the old data and nonzero in the new. An empty channels are absent in the old detsim sample. I suspect it is the latter that is causing the change observed in caldata. I will add the option to exclude empty channels to the new detsim and then check the effect.

#57 Updated by David Adams over 3 years ago

I have added and enabled by default an option to skip empty channels in the new detsim and that was the issue. Caldata following new default detsim now takes 2.3 s/event. If I instead keep empty channels in Detsim, caldata takes 16 s/event. This change will go in the next release--see https://cdcvs.fnal.gov/redmine/issues/12412.

#58 Updated by David Adams over 3 years ago

  • Status changed from Assigned to Closed

The new DetSim module can now be used for production. The DetSim services take seeds from the new seed service and the example FCL configurations for 35t and 10kt have been updated to use the new module. The changes are going into dunetpc v05_10_00.

I close this ticket.



Also available in: Atom PDF