Project

General

Profile

Idea #14457

Switch for pullProducts to "clean" UPS repository

Added by Eric Flumerfelt almost 3 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
11/09/2016
Due date:
% Done:

0%

Estimated time:
Experiment:
-
SSI Package:
Duration:

Description

I was thinking it would be kind of nice to have a switch to pullProducts that would mean "remove all other versions of each product" while processing a bundle. This would be useful to help clear away old and no-longer-used versions of dependent products like boost and root, where it may be difficult to determine otherwise which products no longer need to be kept around.

History

#1 Updated by Lynn Garren almost 3 years ago

  • Status changed from New to Feedback

Cleaning up a product directory is not a simple problem. The proper solution requires a tool that compares dependencies and suggests which products can be safely removed. Such a tool is not trivial to produce and takes noticeable time. Such a tool is not appropriate for use within pullProducts.

If you simply want to remove everything that is not associated with the current release you are installing, a simple solution is to remove everything from your products directory (or rename it) and then run pullProducts. This will give you a clean install for a single release.

However, if you only want to remove some things, then there is a pair of tools in cetpkgsupport: remove_ups_product and list_products. This tool is not perfect, so use with care. You may find it necessary to recreate the database with some frequency.
  • list_products <product_directory>
    • This will produce a temporary database file (e.g., 2016_11_14-list.db)
  • remove_ups_product <product_directory> <product> <version> <temp database file>
    • This will produce a script that will remove appropriate products
    • The generated script does not have the execute bit set. This is deliberate.
    • Check this script carefully
  • chmod +x the script and run the generated script

Do either of these solutions work for you?

Sample output where the product and some dependencies can be removed:

[garren@woof clean]$ remove_ups_product /products uboonecode v06_04_01 2016-11-14-perl.db
checking 4277 uboonecode v06_04_01
consider removing uboonecode v06_04_01
remove_ups_product: dependency art v2_03_00 is used by another product
remove_ups_product: dependency boost v1_61_0 is used by another product
remove_ups_product: dependency canvas v1_04_06 is used by another product
remove_ups_product: dependency cetlib v1_19_01 is used by another product
remove_ups_product: dependency cetpkgsupport v1_11_00 is used by another product
remove_ups_product: dependency clhep v2_3_2_2 is used by another product
remove_ups_product: dependency cppunit v1_12_1d is used by another product
remove_ups_product: dependency cry v1_7f is used by another product
remove_ups_product: dependency fftw v3_3_4 is used by another product
remove_ups_product: dependency fhiclcpp v4_00_01 is used by another product
remove_ups_product: dependency g4abla v3_0 is used by another product
remove_ups_product: dependency g4emlow v6_41 is used by another product
remove_ups_product: dependency g4neutron v4_5 is used by another product
remove_ups_product: dependency g4neutronxs v1_4 is used by another product
remove_ups_product: dependency g4nucleonxs v1_1 is used by another product
remove_ups_product: dependency g4nuclide v1_0 is used by another product
remove_ups_product: dependency g4photon v3_1 is used by another product
remove_ups_product: dependency g4pii v1_3 is used by another product
remove_ups_product: dependency g4radiative v4_2 is used by another product
remove_ups_product: dependency g4surface v1_0 is used by another product
remove_ups_product: dependency gcc v4_9_3a is used by another product
remove_ups_product: dependency geant4 v4_10_1_p03b is used by another product
remove_ups_product: dependency genie_phyopt v2_10_6 is used by another product
remove_ups_product: dependency genie v2_10_10 is used by another product
remove_ups_product: dependency genie_xsec v2_10_6a is used by another product
remove_ups_product: dependency gsl v2_1 is used by another product
remove_ups_product: dependency ifbeam v1_4_21 is used by another product
remove_ups_product: dependency ifdh_art v1_15_05 is used by another product
remove_ups_product: dependency ifdhc v1_8_11 is used by another product
remove_ups_product: dependency larana v06_01_02 is used by another product
remove_ups_product: dependency larbatch v01_23_04 is used by another product
remove_ups_product: dependency larcoreobj v1_04_02 is used by another product
remove_ups_product: dependency larcore v06_02_00 is used by another product
remove_ups_product: dependency lardataobj v1_04_02 is used by another product
remove_ups_product: dependency lardata v06_03_00 is used by another product
remove_ups_product: dependency lareventdisplay v06_01_01 is used by another product
remove_ups_product: dependency larevt v06_01_00 is used by another product
remove_ups_product: dependency larexamples v06_00_05 is used by another product
remove_ups_product: dependency larpandoracontent v02_07_09 is used by another product
remove_ups_product: dependency larpandora v06_00_06 is used by another product
remove_ups_product: dependency larreco v06_03_01 is used by another product
remove_ups_product: dependency larsimobj v1_05_03 is used by another product
remove_ups_product: dependency larsim v06_03_01 is used by another product
remove_ups_product: dependency larsoft_data v1_01_00 is used by another product
consider removing larsoft v06_04_01
remove_ups_product: dependency lhapdf v5_9_1e is used by another product
remove_ups_product: dependency libwda v2_22_2 is used by another product
remove_ups_product: dependency libxml2 v2_9_3 is used by another product
remove_ups_product: dependency log4cpp v1_1_1e is used by another product
remove_ups_product: dependency marley v0_9_5 is used by another product
remove_ups_product: dependency messagefacility v1_17_01 is used by another product
remove_ups_product: dependency mysql_client v5_5_48a is used by another product
remove_ups_product: dependency nucondb v1_4_21 is used by another product
remove_ups_product: dependency nusimdata v1_05_02 is used by another product
remove_ups_product: dependency nutools v2_03_01 is used by another product
remove_ups_product: dependency pandora v02_07_00b is used by another product
remove_ups_product: dependency pdfsets v5_9_1b is used by another product
remove_ups_product: dependency postgresql v9_3_12 is used by another product
remove_ups_product: dependency pythia v6_4_28e is used by another product
remove_ups_product: dependency python v2_7_11 is used by another product
remove_ups_product: dependency root v6_06_04b is used by another product
remove_ups_product: dependency sqlite v3_12_02_00 is used by another product
remove_ups_product: dependency tbb v4_4_3a is used by another product
consider removing uboonedaq_datatypes v6_19_01_e10
remove_ups_product: dependency uboone_data v01_06_00 is used by another product
consider removing ubutil v06_04_01
remove_ups_product: dependency xerces_c v3_1_3 is used by another product
remove_ups_product: dependency xrootd v3_3_4e is used by another product

issue these commands to remove uboonecode v06_04_01 and its dependencies

rm -rf /products/uboonecode/v06_04_01.version /products/uboonecode/v06_04_01
rm -rf /products/larsoft/v06_04_01.version /products/larsoft/v06_04_01
rm -rf /products/uboonedaq_datatypes/v6_19_01_e10.version /products/uboonedaq_datatypes/v6_19_01_e10
rm -rf /products/ubutil/v06_04_01.version /products/ubutil/v06_04_01

Sample output where a product cannot be removed:

[garren@woof clean]$ remove_ups_product /products art v2_03_00 2016-11-14-perl.db
checking 265 art v2_03_00
remove_ups_product: artdaq_core v1_05_05 uses art v2_03_00
remove_ups_product: dunetpc v06_04_00 uses art v2_03_00
remove_ups_product: duneutil v01_23_06 uses art v2_03_00
remove_ups_product: ifdh_art v1_15_05 uses art v2_03_00
remove_ups_product: larana v06_01_01 uses art v2_03_00
remove_ups_product: larana v06_01_02 uses art v2_03_00
remove_ups_product: larana v06_01_03 uses art v2_03_00
remove_ups_product: larana v06_01_04 uses art v2_03_00
...
remove_ups_product: nutools v2_03_01 uses art v2_03_00
remove_ups_product: nutools v2_04_00 uses art v2_03_00
remove_ups_product: nutools v2_04_01 uses art v2_03_00
remove_ups_product: toyExperiment v0_01_03 uses art v2_03_00
remove_ups_product: uboonecode v06_04_00 uses art v2_03_00
remove_ups_product: uboonecode v06_04_01 uses art v2_03_00
remove_ups_product: ubutil v06_04_00 uses art v2_03_00
remove_ups_product: ubutil v06_04_01 uses art v2_03_00

remove_ups_product: art v2_03_00 is used by another product
remove_ups_product: will not remove art v2_03_00

#2 Updated by Eric Flumerfelt almost 3 years ago

  • Status changed from Feedback to Closed

I suppose that's good enough for now, especially if UPS will be replaced by something else in the somewhat-near future...



Also available in: Atom PDF