Project

General

Profile

Idea #2991

Shipping condor config.d files in the tarball

Added by Igor Sfiligoi about 8 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
John Weigand
Category:
-
Target version:
Start date:
10/23/2012
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Stakeholders:
Duration:

Description

Most of the Condor setup we create with the installer is static,
as shown by the RPMs we ship for OSG.
So it seems just natural to allow people installing via the tarball to have the same convenience.

The proposal is to put the condor config files normally distributed in the RPM in a subdirectory of
glideinWMS/install

E.g.
glideinWMS/install/condor_configs

This way they would be distributed as part of the normal tarball.


Subtasks

Feature #3065: Shipping condor config.d files in v2_6_2ClosedParag Mhashilkar


Related issues

Related to GlideinWMS - Bug #3024: RPM improvementsClosed10/18/2012

Related to GlideinWMS - Feature #2192: SImple configurator for RPM installsNew11/23/2011

Related to GlideinWMS - Feature #3100: Dynamically creating the secondary collectors config fileClosed11/01/2012

History

#1 Updated by Igor Sfiligoi about 8 years ago

I would also create subdirectories that mimic the RPMs.

E.g.
install/condor_configs/frontend_minimal
install/condor_configs/frontend_schedd
install/condor_configs/frontend_collector
etc.

#2 Updated by Burt Holzman about 8 years ago

  • Status changed from New to Assigned
  • Assignee changed from Burt Holzman to Douglas Strain
  • Target version set to v2_7_x

This was discussed at the Oct 15 meeting; everyone believes it to be a good idea. Since Doug has done similar packaging already for the RPMs, I assigned this to him. :)

#3 Updated by Igor Sfiligoi about 8 years ago

Just to be more explicit:
The config files would be used by hand by the sysadmins.

Just copied over in the condor installation dir.

I do not envision using an installer for this... way easier to just use "cp".

Igor

#4 Updated by Douglas Strain about 8 years ago

I have added the RPM condor config files into install/templates. No promises on consistency or stability, since they are not coordinated with the installers, and I have not yet integrated Igor's comments on the RPM condor configs, so these will probably change wildly in the next version.

#5 Updated by Douglas Strain about 8 years ago

  • Status changed from Assigned to Feedback
  • Assignee changed from Douglas Strain to John Weigand

I have integrated the ini installer with the templates.

John, you are the ini installer guru. Could you take a look at my changes?
They are in branch_v2plus_static.

#6 Updated by Douglas Strain about 8 years ago

Update. I changed CONDOR_HOST and added an additional file, so please pull these changes in branch_v2plus_static before you review.

#7 Updated by Igor Sfiligoi over 7 years ago

As mentioned in #3100, I think the installer should use the template the same way they are used by the RPM and "manual copy" admins.

I.e., the template should be copied over as they are, and any customization should go in dedicated config files with higher numbers.

#8 Updated by John Weigand over 7 years ago

Proposal for templates and configuration
----------------------------------------
Using the frontend.xml configuration file as an example.
I believe this will easily work with the condor configs as well.

1. The templates (as in source code version of the template) would have a keyword
specifying the attribute that requires changing for a specific implementation.
e.g. GWMS_FACTORY_COLLECTOR_DN

<frontend  ...
  <downtimes/>
  <log_retention ....
  <match .....
     <factory query_expr='((stringListMember("<GWMS_MY_VO>")), ....
       <match_attrs>
       /match_attrs>
       <collectors>
          <collector DN="GWMS_FACTORY_COLLECTOR_DN" ...
                         factory_identity="GWMS_FACTORY_IDENTITY@GWMS_FACTORY_NODE" 
                         my_identity="GWMS_MY_IDENTITY@GWMS_FACTORY_NODE" 
                         node="GWMS_FACTORY_NODE"/>
          :

2. Have a configurator file with the mappings.
The one below would be for a UCSD factory.
There could be one for an OSG factory
And a 3rd for a generic factory with all values reading CHANGE_ME.
There are only about 20 attributes that need changing.
The rationale for having a value of "CHANGE_ME" is that it makes it easier for
the user to find attributes they missed changing.

GWMS_MY_VO                    CHANGE_ME
GWMS_FACTORY_COLLECTOR_DN     /DC=org/DC=doegrids/OU=Services/CN=glidein-1.t2.ucsd.edu
GWMS_FACTORY_IDENTITY         gfactory
GWMS_FACTORY_NODE             glidein-1.t2.ucsd.edu
GWMS_MY_IDENTITY              CHANGE_ME
   :

3. The resulting frontend.xml would look like this:

<frontend  ...
  <downtimes/>
  <log_retention ....
  <match .....
    <factory query_expr='((stringListMember("<CHANGE_ME>" ....
    <match_attrs>
    /match_attrs>
    <collectors>
       <collector DN="/DC=org/DC=doegrids/OU=Services/CN=glidein-1.t2.ucsd.edu" 
                  factory_identity="gfactory@glidein-1.t2.ucsd.edu" 
                  my_identity="CHANGE_ME@glidein-1.t2.ucsd.edu" 
                  node="glidein-1.t2.ucsd.edu"/>
           :

============================================
As I said, there are about 20 attributes that require changing to the templates that I
have found. A sample of a UCSD one is at the back of this update.

Both the ini installer, if we continue support for it, and the rpm installs can use this
type of configuration file.

My view on how it is used, in the rpm arena, is that the user fills in the CHANGE_MEs
for the factory implementation desired (ucsd/osg/generic). Then runs a configuration
script that creates the respective condor/frontend files.

Another approach would be to have the rpm install run a post-install that creates
3 versions of the frontend.xml and the user selects the desired one to use.

Now, another question, would be whether or not we want some form of validation performed
in this configuration script for items like (there aren't too many that we can do) if we
have a configurator.
1. Verification that there are no CHANGE_MEs
2. existence of the certs/proxy files
3. validation on the DNs
4. ?????

Anyway... these were my thoughts on how to make templates more useful.

========= sample UCSD config ===========================

##############################################################
# Configuration locations (non changeable attributes)
##############################################################
FrontendXML        /etc/gwms-frontend//frontend.xml.template
CondorConfig       /etc/condor/config.d/03_gwms_local.config
CondorMapfile      /etc/condor/certs/condor_mapfile

CondorUser                      condor
CondorAdmin                     CHANGE_ME
FrontendUser                    frontend
#############################################################
##############################################################
# Frontend specific attributes
##############################################################
# <match...<factory .... query_expr
GWMS_YOUR_VO                     OIV_VO

# <security
GWMS_FRONTEND_PROXY_FILE         CHANGE_ME
GWMS_FRONTEND_PROXY_DN           YOU_NEED_A_HOST_OR_SERVICE_CERT_TO_RUN 
GWMS_FRONTEND_SECURITY_NAME      CHANGE_ME

# <security.. <proxies.. <proxy
GWMS_FRONTEND_PILOT_PROXY_FILE   CHANGE_ME

##############################################################
# Factory specific attributes
##############################################################
# <match.. <factory.. <collectors.. <collector 
GWMS_FACTORY_HOST                glidein-1.t2.ucsd.edu
GWMS_FACTORY_DN                  /DC=org/DC=doegrids/OU=Services/CN=glidein-1.t2.ucsd.edu
GWMS_FACTORY_IDENTITY            gfactory
GWMS_FACTORY_MY_IDENTITY         CHANGE_ME_WHEN_FACTORY_PROVIDES_VALUE

##############################################################
# Schedd specific attributes
##############################################################
# <match.. <job.. <schedds.. <schedd
GWMS_SCHEDD_HOST                fermicloud068.fnal.gov
GWMS_SCHEDD_DN                  YOU_NEED_A_HOST_OR_SERVICE_CERT_TO_RUN
GWMS_SCHEDD_NAME                CHANGE_ME

##############################################################
# User Collector specific attributes
##############################################################
# <collectors ... <collector
GWMS_USER_COLLECTOR_HOST        fermicloud068.fnal.gov
GWMS_USER_COLLECTOR_DN          YOU_NEED_A_HOST_OR_SERVICE_CERT_TO_RUN

#9 Updated by Parag Mhashilkar almost 7 years ago

  • Status changed from Feedback to Closed
  • Target version changed from v2_7_x to v2_7

Also available in: Atom PDF