Project

General

Profile

Bug #7278

Gridmanager LOG paths are not expanding $(USERNAME) correctly

Added by Burt Holzman about 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Category:
Factory
Target version:
Start date:
11/05/2014
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:

OSG

Duration:

Description

From Jeff Dost:

Hello glideinWMS support,

It looks like the default configuration of the gridmanager log paths aren't working as expected. For example,
SCHEDDGLIDEINS2_ENVIRONMENT = "_CONDOR_GRIDMANAGER_LOG=$(LOG)/GridManagerLog.$(SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME).$(USERNAME)"

You would expect USERNAME to expand to the frontend user (assuming privsep is enabled). Instead, it expands to "root" no matter what the FE user is:
/var/log/condor/GridManagerLog.schedd_glideins2.root

I suspect this may be because USERNAME is expanded before the privsep uid switch. Unfortunately this means all gridmanagers running under different FE users for a given schedd will be writing to the same log, which we don't want.

Note we were able to get around this the old way of configuring schedd's because we would set GRIDMANAGER_LOG explicitly in terms of SCHEDD_NAME, but then override that later for the secondary schedds:

define gridmanager log:
GRIDMANAGER_LOG = $(LOG)/GridmanagerLog.$(SCHEDD_NAME).$(USERNAME)

environment var overriding:
SCHEDDGLIDEINS2_ENVIRONMENT = "_CONDOR_SCHEDD_NAME=schedd_glideins2 ..."

(The above is simplified, just to illustrate how it was done)

In general I prefer the newer method to avoid the environment variable overriding, but it seems _CONDOR_GRIDMANAGER_LOG is being overridden anyway from the rpms because condor doesn't respect something like the following, which I tried but condor ignored:
SCHEDDGLIDEINS2.GRIDMANAGER_LOG = $(LOG)/GridmanagerLog.$(SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME).$(USERNAME)

What do you suggest we do? Is it possible condor code needs to be changed to accommodate this?

History

#1 Updated by Burt Holzman about 5 years ago

  • Status changed from New to Assigned

#2 Updated by Parag Mhashilkar about 5 years ago

  • Assignee changed from Parag Mhashilkar to Marco Mambelli

#3 Updated by Parag Mhashilkar about 5 years ago

On Nov 24, 2014, at 3:42 PM, Jeff Dost wrote:

As an update, we came up with a work around for this.

Instead of setting _CONDOR_GRIDMANAGER_LOG in the schedd environment, define it earlier for all schedds in a generic way:
GRIDMANAGER_LOG = $(GRIDMANAGER_LOG_DIR)/GridManagerLog.$(SCHEDD_NAME).$(USERNAME)

Then for the secondary schedds, override the SCHEDD_NAME in the environment:
SCHEDDGLIDEINS2_ENVIRONMENT = "_CONDOR_SCHEDD_NAME=$(SCHEDD.SCHEDDGLIDEINS2.SCHEDD_NAME)"

When setting it up this way, $USERNAME expands the way we expect it to. This is essentially the old way we did it, but just for the gridmanager log, and nothing else for the secondary schedd configuration.

For completeness, here are the full schedd and secondary schedd configs I've come up with for our production factories:
https://raw.githubusercontent.com/jdost321/factools/dev/etc/condor_config/generic/02_gwms_factory_schedds.config
https://raw.githubusercontent.com/jdost321/factools/dev/etc/condor_config/generic/11_gwms_factory_secondary_schedds.config

Jeff

#4 Updated by Marco Mambelli about 5 years ago

I applied Jeff's suggestions.
I kept a single file. It will be easier to remove secondary schedds and split it in the future.
I opened https://cdcvs.fnal.gov/redmine/issues/7516

This should be tested with glexec

#5 Updated by Marco Mambelli about 5 years ago

  • Status changed from Assigned to Feedback
  • Assignee changed from Marco Mambelli to Parag Mhashilkar

committed (v3/7278) and tested.

#6 Updated by Parag Mhashilkar about 5 years ago

  • Status changed from Feedback to Resolved
  • Assignee changed from Parag Mhashilkar to Marco Mambelli

Merged to branch_v3_2

#7 Updated by Parag Mhashilkar about 5 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF