Project

General

Profile

Geant4MT

Notes from meeting with Andrea and plans to move forward:

What do we need to do to move to the new 10.1 MT event loop in artg4?

————————
Wrap the Action “new” calls in a factory-like function.

In artg4Main_module.cc there are calls like this:

 runManager_ -> SetUserAction(new ArtG4SteppingAction);
  runManager_ -> SetUserAction(new ArtG4StackingAction);
  runManager_ -> SetUserAction(new ArtG4EventAction);
  runManager_ -> SetUserAction(new ArtG4TrackingAction);
  runManager_ -> SetUserAction(new ArtG4RunAction);

In the MT system, you must derive a new class from G4VUserActionInitialization,
adding a “Build” member function. The build member will look something like this:

class ArtG4UserActionInit
{
 …
 void Build {
  runManager_ -> SetUserAction(new ArtG4SteppingAction);
  runManager_ -> SetUserAction(new ArtG4StackingAction);
  runManager_ -> SetUserAction(new ArtG4EventAction);
  runManager_ -> SetUserAction(new ArtG4TrackingAction);
  runManager_ -> SetUserAction(new ArtG4RunAction);
  }
};

This forms the factory function for Geant4MT to use.

In artG4Main_module, the the call to SetUserInitialization(…)

void artg4Main::beginJob() {
 …
 // Declare the detector construction to Geant
  runManager_->SetUserInitialization(new ArtG4DetectorConstruction);
  runManager_->SetUserInitialization(new ArtG4UserActionInit);
  …
}

—————

There is a merge procedure that pulls together all stepping data info one G4Run.
Sounds like “BeamOnBeginRun” may need modifications.

The current code that cause G4 to process an event is:

void artg4Main::produce(art::Event& e) {
  … 
  // Begin event
  runManager_ -> BeamOnDoOneEvent(e.id().event());
  // Done with the event
  runManager_ -> BeamOnEndEvent();
  …
}

I’m not sure from my notes whether or not this needs to change, but BeamOnDoOneEvent looks like it does not exist in RunManager in 10.1. Many it needs to be replaced by “BeamOn” ?

—————

Geometry initialization in artg4 looks okay.
Need to look at sensitive detectors. On-per-thread is generated / used (this is the default) if not totally thread safe.
Must be careful to use new interface function of sensitive detector - separate steps now.

Generate event call -
each thread will call this, independent of passed event number to get the generate generator data for its G4Event.

Artg4 primary generator action:
• can there be one of these per event? (this is necessary)
• is ActionHolderService thread safe? (generates primaries)
• can be one action (or not) instances
• just using the particle gun
• there will be one G4 event created per thread

art::Events come in at “End Event” call, where everyone needs to dump G4 info the art::Event.

————————

An example of where a data product is pushed into the art::Event is

common/ring/Ring_service.hh
57:     virtual void doFillEventWithArtHits(G4HCofThisEvent * hc) override;

This code ends up doing something like this:

std::unique_ptr<RingArtRecordCollection> myArtHits(new RingArtRecordCollection);
…
art::ServiceHandle<artg4::DetectorHolderService> detectorHolder;
  art::Event & e = detectorHolder -> getCurrArtEvent();

  // Put the hits into the event
  e.put(std::move(myArtHits), category());

——————

Here is the place where fill is triggered for all the sensitive volumes:

void artg4::ArtG4EventAction::EndOfEventAction(const G4Event * currentEvent)
{
  // Convert geant hits to art for DETECTORS
  art::ServiceHandle<artg4::DetectorHolderService> dhs;
  dhs -> fillEventWithArtHits( currentEvent->GetHCofThisEvent() );

  // Run EndOfEventAction
  art::ServiceHandle<ActionHolderService> ahs;
  ahs -> endOfEventAction(currentEvent);

  // Every ACTION needs to write out their event data now, if they have any
  // (do this within ArtG4EventAction) since some still need to be within
  // Geant
  ahs -> fillEventWithArtStuff();
}

——————
So what’s the overall plan?
(it is unclear how to actually implement all these things at this point)

1) Configure Geant4MT to propagate between 1000-10000 particles, producing the same number of G4Events using many threads.

2) Convert these G4Events to art products (as is done now) in the sensitive volume code, but without putting the results into the art::Event.

3) Put the art data products into a list - perhaps one list per sensitive volume object, but perhaps within the service that holds them. The “doFillEventWithArtHits” would like do this adding the list instead of doing the “e.put” shown above.

4) after G4 has completed processing of all the G4Events, either before or after the call to “runManager_ -> BeamOnEndEvent();”, we must move all the collected art data products into the current art::Event. Things are back to sequential at this point.

Note that there will be one art product instance per inserted art data product of a particular type. For example, RingArtRecordCollection will have 1000-10000 instances inserted into the current event.

An alternative to this is to make a collection of RingArtRecordCollection objects. What we do will depend on the overhead needed (memory).

A way to handle (4) might be to add a new member function to the DetectorBase called “drainCollectedData” and imagine that “fillEventWithArtHits” is really called “collectArtProducts”. In this setting, “fillEventWithArtHits” transforms the data into the collection (art product) object and stores it into an internal list (the “push” onto this list needs to be thread-safe). The “drainCollectedData” is given an art::Event and it goes through the internal product list, moving the items into the art::Event. It then clears the internal list.

Dealing with the writing to the art::Event

I wonder if this will work. The current art::Event is tucked away into the artg4 serivces. What if we change the thing that is tucked away into something that looks like the code below. Note that I typed this in quickly and it probably has problems. The main idea is to buffer the objects in the the "HeldThings" list and put them really into the event at the right time (when the stashed event is cleared, which ought to be when no multi-threading is active in Geant4.

struct HeldThing
{ 
  virtual ~HeldThing() { } 
  virtual void put(art::Event& e) =0;
};

template <typename PROD>
struct HeldProduct : public HeldThing
{
  ~HeldProduct() { cout << "HeldProduct going away " << inst_ << "\n"; }
  HeldProduct(std::unique_ptr<PROD>&& p, std::string const& inst):prod_(std::move(p)), inst_(inst) { }
  void put(art::Event& e) { e.put(std::move(prod_),inst_); }

  std::unique_ptr<PROD> prod_;
  std::string inst_;
};

class FakeEvent
{
public:
  FakeEvent(art::Event& e): mutex_(), pen_(), e_(e) { }
  ~FakeEvent()
  {
    for( auto& p : pen_) { p->put(e_); }
  }

  template <typename PROD> void put(std::unique_ptr<PROD> && product,
                    std::string const& productInstanceName)
  {
    std::lock_guard<std::mutex> l(mutex_);
    pen_.push_back( std::unique_ptr<HeldProduct<PROD>>(
                           new HeldProduct<PROD>(std::move(product),
                                     productInstanceName)
                           ));
  }
private:
 std::mutex mutex_;
  std::list<std::unique_ptr<HeldThing>> pen_;
  art::Event& e_;
};

void func(art::Event& e)
{
  FakeEvent fe(e);
  std::unique_ptr<double> p1(new double(5.5));
  std::unique_ptr<int> p2(new int(5));
  fe.put(std::move(p1),"1");
  fe.put(std::move(p2),"2");
}

int main()
{
  art::Event e;
  func(e);
  return 0;
}

A better name for FakeEvent is certainly HoldingPen.

what about the product instance problem?

With the above changes, there is still the issue of identifying what detector product instance goes with what G4Event. There are many different product instances (with several different types) put in for each G4Event. G4Event has a GetEventID() that can be used. This will need to be passed to on the call to put() to the FakeEvent

In the simplest implementation of the above code, each put will need a generated instance number. Recall that art requires that every product instance name be pre-registered. This means that each product like this in the event will have as its identity the module_label and the instance_name, where the instance name is a number as a string. This is inconvenient. A better way to deal with this is perhaps to create a "collection of collections" data product for each thing that can be inserted in the event.

<lcgdict>
    <!-- For RingHits - note that using the typedef will not work here-->
    <class name="gm2ringsim::RingArtRecord"/>
    <class name="std::vector< gm2ringsim::RingArtRecord >"/>
    <class name="art::Wrapper< std::vector< gm2ringsim::RingArtRecord > >"/>
    <class name="std::vector<std::vector< gm2ringsim::RingArtRecord > >"/>
    <class name="art::Wrapper< std::vector<std::vector< gm2ringsim::RingArtRecord > > >"/>
   <!-- or include the identity also -->
    <class name="std::vector<std::pair<int, std::vector< gm2ringsim::RingArtRecord > > >"/>
    <class name="art::Wrapper< std::vector<std::pair<int, std::vector< gm2ringsim::RingArtRecord > > > >"/>
</lcgdict>

With a few changes, the above code can be used to hold the collections of data product with ID by type, and that held object could just be put directly into the event.

calling sequence example

  ... produce ...

  FakeEvent fe(e);
  actionHolder -> setCurrArtEvent(fe);
  detectorHolder -> setCurrArtEvent(fe);

  // Begin event
  runManager_ -> BeamOnDoOneEvent(e.id().event());
  // Done with the event
  runManager_ -> BeamOnEndEvent();

  // new calls
  actionHolder -> ClearCurrentEvent();
  detectorHolder->ClearCurrentEvent();

Or course the better solution to the setCurrArtEvent / ClearCurrentEvent is to use a little sentry object.

Another way

The attached file test3.cc has the code necessary for collecting up all the art data products that would have been "put" directly into the event. What it does at the end is to run through all the different types of product, and for each type, it goes through each of the data product (vectors of stuff) and concatenates them into one big vector and does one put per type into the real event.