Project

General

Profile

Feature #23008

It would be helpful to be able to be able to mark configurations as "invalid"

Added by Kurt Biery 4 months ago. Updated 15 days ago.

Status:
Closed
Priority:
Normal
Start date:
07/29/2019
Due date:
% Done:

100%

Estimated time:
Duration:

Description

I'm filing this issue for completeness. Gennadiy, John, and I talked about it several weeks ago, and I imagine that there have been follow-up discussions.

One scenario that is relevant to this issue is the following: on protoDUNE, we have several different configurations that are at different points in their lifetime and that have had different levels of good housekeeping to keep them up-to-date. Let's say that we make an important change to the N configurations that are used most often. (An important change might be one that is associated with a non-backward-compatible code change or one that reflects a change in the detector running condition.) Let's also say that there are configurations that haven't been run in many weeks, but are known to detector experts as their standard. It's possible that configuration experts might not know to create a new version of that config with the important change and detector experts might use such an obsolete configuration without understanding that it is obsolete.

Given this, it would be nice to be able to mark Configurations as "invalid".

In our discussion, we identified several aspects to this:
  • in order to provide support for a "valid/invalid" flag and additional flags that we can't predict, Gennadiy suggested a flags.fcl whose entries would follow a proscribed pattern, and whose values could be interrogated by experiment software to make use of the values.
  • the ability to provide conditions to DB queries to include or exclude particular configurations based on the values of a particular flag. The use case here is that it would be nice to only fetch "valid" configurations when generating a list of available configs to present to users.
  • the ability to update an existing configuration with new, or modified, flag values.

History

#1 Updated by Gennadiy Lukhanin 3 months ago

  • % Done changed from 0 to 90
  • Assignee set to Gennadiy Lukhanin
  • Status changed from New to Work in progress

Please follow the steps below to test the configuration flags management feature.

  1. Install the e17:s82:prof variant of artdaq_demo v3_06_00.
  2. Download and install artdaq_database v1_4_75 with the same qualifiers from scisoft.
    1. https://scisoft.fnal.gov/scisoft/packages/artdaq_database/v1_04_75/
  3. Open two shells and setup artdaq_database in both of them.
    1. source [your-artdaq-products-area]/setup
    2. setup artdaq_database v1_04_75 -q e17:prof:s82
    3. export ARTDAQ_DATABASE_URI=mongodb://127.0.0.1/artdaqtest_db
  4. In the first shell start mongodb.
    1. mkdir -p $HOME/database/mongodb
    2. mongod --dbpath=$HOME/database/mongodb --bind_ip=127.0.0.1
    3. leave mongod running in the foreground
  5. Switch to the second shell, which will be used for running conftool.py and confirm that the test database is accessible.
    1. conftool.py readDatabaseInfo
  6. Create a working directory for configuration files and copy a test configuration into it.
    1. mkdir -p $HOME/database/work-dir/aa/
    2. cp -r [your-test-configuration] $HOME/database/work-dir/aa/
    3. cd $HOME/database/work-dir/aa/
    4. add mappings for “Flags” and “Hashes” to the system_layout sequence in schema.fcl
      system_layout: [{
      
      collection: SystemLayout
      
      pattern: "^(?!.*common_code)(.*)(schema)(\.fcl$)" 
      
      }, {
      
      collection: Flags
      
      pattern: "^(?!.*common_code)(.*)(flags)(\.fcl$)" 
      
      }, {
      
      collection: Hashes
      
      pattern: "^(?!.*common_code)(.*)(hashes)(\.fcl$)" 
      
      }]
      
  7. Import three new configurations and list them.
    1. conftool.py importConfiguration mytest
    2. conftool.py importConfiguration mytest00001
    3. conftool.py importConfiguration mytest
    4. conftool.py getListOfAvailableRunConfigurations
    5. the output should list three configurations
  8. Create a new flags.fcl file and add it to one of the existing configurations.
    1. echo "flag_inactive:true" > flags.fcl
    2. conftool.py updateConfigurationFlags mytest00002
  9. List configurations with the “inactive” flag set to true.
    1. conftool.py getListOfMaskedRunConfigurations
  10. List configurations without the “inactive” flag set to true.
    1. conftool.py getListOfAvailableRunConfigurationsSubtractMasked
    2. the output should list two configurations
  11. Re-activate a previously “inactive” configuration.
    1. echo "flag_inactive:false" > flags.fcl
    2. conftool.py updateConfigurationFlags mytest00002
  12. Confirm that the previously “inactive” configuration became “active”.
    1. echo "flag_inactive:true" > flags.fcl
    2. conftool.py getListOfAvailableRunConfigurationsSubtractMasked
    3. the output should list thee configurations
  13. Go back to the fist shell, which runs mongod and do a “Ctrl-c” in it.
  14. Delete all test data rm -rf $HOME/database.

#2 Updated by Gennadiy Lukhanin 3 months ago

  • % Done changed from 90 to 100
  • Status changed from Work in progress to Resolved

#3 Updated by Ron Rechenmacher 3 months ago

  • Status changed from Resolved to Reviewed

I was able to follow the procedure.

#4 Updated by Gennadiy Lukhanin 15 days ago

  • Status changed from Reviewed to Closed


Also available in: Atom PDF