Project

General

Profile

Feature #3445

Support for sequences of modules

Added by Rob Kutschke about 8 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Christopher Green
Target version:
art - 1.11.00
Start date:
02/07/2013
Due date:
09/30/2013
% Done:

100%

Estimated time:
24.00 h
Spent time:
40.00 h
Duration: 236

Description

Mu2e has a use case for which we are currently defining trigger paths like:

t1 : [ a, b, c, d, e1]
t2 : [ a, b, c, d, e2]
t3 : [ a, b, c, d, e3]

trigger_paths : [ t1, t2, t3 ]
where the a,b,c,d,e* are module labels.

We would like to be able to define a "shorthand" for the sequence of modules, [a,b,c,d]. Conceptually we would like to be able to say:

sequences : {
    abcd : [ a, b, c, d]
}

t1 : [ abcd, e1 ]
t2 : [ abcd, e2 ]
t3 : [ abcd, e3 ]

trigger_paths : [ t1, t2, t3 ]
The use case is that the maintainer of the sequence abcd may add an other element and the downstream path definitions do not need maintenance.

The above syntax was meant to be illustrative of what we want not a request for exactly that syntax.

We view the notion of a sequence as just a configuration-time shorthand. That is, the data products would still appear to be produced by the individual modules, a, b, c and d, not by the sequence.

If the name sequence already means something else, please suggest a new name.

We would also like sequences to be nestable. The following example is again meant to be illustrative, not a request for a particular syntax:

sequences : {
    abc   : [ a, b, c]
    abcd : [ abc, d]
}
We can probably do some of the above with clever use of @local:: and include. However we feel that our proposal makes the intent of the FHiCL much more clear.

History

#1 Updated by Christopher Green about 8 years ago

  • Subject changed from Support for sequences of models to Support for sequences of modules
  • Description updated (diff)
  • Status changed from New to Accepted

This has been discussed within the art team before, but no-one has previously asked for it.

We had in mind a keyword @expand:

BEGIN_PROLOG
snippet: { a: 6 b: 7 c: 9 }
seq_frag: [ 9, 10, 11 ]
END_PROLOG

set: {
  @expand::snippet
  d: 10
  seq: [ @expand::seq_frag, 12, 13, 14 ]
}

In other words, @expand:: would remove one level of containment and insert the elements in the current context. It would be an error to @expand:: a simple parameter.

Of course, this work would have to be scheduled, but it seems feasible and relatively straightforward.

#2 Updated by Rob Kutschke about 8 years ago

Thanks Chris, Just to be clear, my use case would look like:

 BEGIN_PROLOG
    base_path : [ a, b, c, d ]
 END_PROLOG

 t1 : [ @expand::base_path, e1 ]
 t2 : [ @expand::base_path, e2 ]
 t3 : [ @expand::base_path, e3 ]

 trigger_paths : [ t1, t2, t3]

Do I understand correctly?

If so, I'm not in love with the visual impact of @expand:: but it does do the job and it is probably more teachable than my proposed syntax ( precisely because of the visual impact ). I do appreciate that this confines the solution to FHiCL and does not require a change in art.

#3 Updated by Christopher Green over 7 years ago

  • Tracker changed from Feature to Bug

#4 Updated by Christopher Green over 7 years ago

  • Status changed from Accepted to Feedback

We propose then, to implement the fhicl-only system, but we think that providing two context-specific keywords (say, @table and @seq) instead of the single @expand would make for simpler implementation and better error checking and notification. Would this be acceptable?

#5 Updated by Christopher Green over 7 years ago

  • Tracker changed from Bug to Feature
  • Due date set to 09/30/2013
  • Category set to Infrastructure
  • Target version set to 1.09.00
  • Estimated time set to 24.00 h
  • Scope set to Internal
  • Experiment - added
  • SSI Package fhicl-cpp added
  • SSI Package deleted ()

#6 Updated by Rob Kutschke over 7 years ago

Yes that would be acceptable.

#7 Updated by Christopher Green about 7 years ago

  • Target version changed from 1.09.00 to 521

#8 Updated by David Brown almost 7 years ago

I'm wondering what the status is on this feature. I have a situation currently in Mu2e where deploying a minor change in module order forces me to make a disruptive change, exposed to all users. Sequence functionality would avoid this disruption.

Also, as for names, my preference is for @sequence: that would make reading scripts easy for humans.

#9 Updated by Rob Kutschke almost 7 years ago

We spoke earlier today about getting @sequence:: into the late-June/early-July 2014 release.

I would like to make it clear that we need both @table:: and @sequence:: to do our job. Is this in the plan?

To be clear, here is what we think you mean by @table::

BEGIN_PROLOG
  trackingModuleDefs : {
        tm1 : { ... }
        tm2 : { ....}
        tm3 : {.... } 
  }

  calorimeterModuleDefs : {
       cm1 : { .... }
       cm2 : { .... }
  }

  trackingModuleList      : [ tm1, tm2, tm3 ]
  calorimeterModuleList : [ cm1, cm2, cm3 ]

END_PROLOG

physics : {

   producers : {

      @table::trackingModuleDefs
      @table::calorimeterModuleDefs

      x1: { .... }
      x2: { .... }
   }

   t1 : [ @sequence::trackingModuleDefs, @sequence::calorimeterModuleDefs, x1, x2 ]
   trigger_paths :  [t1]
}

Do we understand correctly?

With these tools the calorimeter and tracking keep people can add, subtract and change modules as they need and user .fcl files will not need maintenance.

#10 Updated by Christopher Green over 6 years ago

  • Status changed from Feedback to Resolved
  • Assignee set to Christopher Green
  • Target version changed from 521 to 1.11.00
  • % Done changed from 0 to 100
  • Experiment Mu2e added
  • Experiment deleted (-)

#11 Updated by Christopher Green over 6 years ago

  • Project changed from art to fhicl-cpp
  • Category deleted (Infrastructure)

#12 Updated by Christopher Green over 6 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF