Project

General

Profile

Feature #20909

Creation of unit tests to cover all the possible configurations when shared_port is or is not enabled

Added by Lorena Lobato Pardavila over 1 year ago. Updated 12 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
09/21/2018
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

We'd like to increase the unit test coverage for shared_port configuration.

Note: The goal of validate_node function (cWDictFile.py) is to accept any valid address with sinful string because if an admin needs to use it (e.g. needs to have other variables) we should not block her/him from doing so. validate_node should make sure that the values in sock=... are correct (because we process them later in the shell script and don't want that to crash). validate_node should ignore other variables because we don't know what could be a valid value and what not.

Although, in the documentation(configuration.html + condor.html)we encourage to the user to use the pattern established for shared_port in the frontend.xml(sock=collectorN-M, no other variables.)

Some examples of good and bad cases:

No shared_port

   <ccbs>
      <ccb DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="default" node="fermicloud131.fnal.gov:9618-9620"/>
   </ccbs>
   <collectors>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud364.fnal.gov" group="default" node="fermicloud364.fnal.gov:9618" secondary="False"/>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud364.fnal.gov" group="default" node="fermicloud364.fnal.gov:9620-9622" secondary="True"/>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="lorena" node="fermicloud131.fnal.gov:9618" secondary="False"/>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="lorena" node="fermicloud131.fnal.gov:9623-9625" secondary="True"/>
   </collectors>

Shared_port enabled

   <ccbs>
      <ccb DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="default" node="fermicloud131.fnal.gov:9618-9620"/>
      <ccb DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="default" node="fermicloud131.fnal.gov:9618?sock=collector10-20"/>
   </ccbs>
   <collectors>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud309.fnal.gov" group="group309" node="fermicloud309.fnal.gov:9618" secondary="False"/>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud309.fnal.gov" group="group309" node="fermicloud309.fnal.gov:9618?sock=collector10-20" secondary="True"/>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="group131" node="fermicloud131.fnal.gov:9618" secondary="False"/>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud132.fnal.gov" group="group131" node="fermicloud131.fnal.gov:9618?sock=collector30-40" secondary="True"/>
   </collectors>

We have tried to cover all these cases to be accepted:

host.domain
host.domain:9618
host.domain:9620-9640
host.domain:9618?sock=collect10
host.domain:9618?sock=collect10-20
host.domain:9618?var1=val1&sock=collect10-20&var2=val2
host.domain:9618?var1=val1&sock=collect10-20&var2=val2-6
host.domain:9618?var1=val1&sock=my5collect30-50

Note: The most important is to not allow port range when shared_port is enabled, for obvious reasons.

Some examples of bad cases:

 <ccbs>
      <ccb DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="default" node="fermicloud131.fnal.gov: 9630-9618"/> (port range bad configured: P1>P2 )
 </ccbs>
  <collectors>
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud131.fnal.gov" group="default" node="fermicloud131.fnal.gov:9618-9620-9630" secondary="True"/> (three ports)
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud309.fnal.gov" group="group309" node="fermicloud309.fnal.gov:9618?sock=collector50-20" secondary="True"/> (N1>N2 from list of socks in shared_port configuration)
      <collector DN="/DC=org/DC=opensciencegrid/O=Open Science Grid/OU=Services/CN=fermicloud132.fnal.gov" group="group131" node="fermicloud131.fnal.gov:9618-9620?sock=collector30-40" secondary="True"/> (port range + list of socks in shared_port configuration)
   </collectors>

To summarize:

Documentation:
- doc/components/condor.html
- doc/frontend/configuration.html

As reference:
- 7341: Switch child collectors to shared_port
- 20871: Sinful strings in the collector cannot have spaces
- 20880: Collector and CCB strings cut at comma. Group separator character cannot be allowed in collector and CCB strings.

History

#1 Updated by Lorena Lobato Pardavila over 1 year ago

  • Description updated (diff)

#2 Updated by Marco Mambelli over 1 year ago

Some examples and a test done manually after importing the validate_node function:

to_accept = """fermicloudui.fnal.gov:9618-9620,fermicloudui.fnal.gov:9618?sock=collector30-40,fermicloudui.fnal.gov:9618-9630,fermicloudui.fnal.gov:9618?sock=collector30-50,fermicloudui.fnal.gov:9618?sock=collector10-20,fermicloudui.fnal.gov:9618?sock=collector,fermicloudui.fnal.gov:9618?sock=collector30-40,fermicloudui.fnal.gov:9618?sock=collector30,fermicloudui.fnal.gov:9618?sock=collector,fermicloudui.fnal.gov:9618?sock=collector&key1=val1,name@fermicloudui.fnal.gov:9618?sock=schedd,fermicloudui.fnal.gov:9618?sock=my5alpha0num,fermicloudui.fnal.gov:9618?key1=val1&sock=collector&key2=val2,fermicloudui.fnal.gov:9618?sock=my5alpha0num,fermicloudui.fnal.gov:9618?val=val&sock=collect10-20&var2=var2""" 

# probably not correct, but not sure
do_not_care = """fermicloud309.fnal.gov:9618?ivar=3=collector2,fermicloud309.fnal.gov:9618?ivar=3=collector2=collector2,fermicloud309.fnal.gov:9618?ivar=3=sock=collector2""" 

to_fail = """fermicloud309.fnal.gov:9618?sock=collector2=sock=collector4,fermicloud309.fnal.gov:9618?sock=sock=collector4,fermicloud309.fnal.gov:9618?sock=3sock=collector4,fermicloud309.fnal.gov:9618?sock=3/collector4,fermicloud309.fnal.gov:9618?sock=3/*collector4,""" 

aa=to_fail.split(',') 

for a in aa:
  print a
  try:
   vv.validate_node(a, True)
   print "OK" 
  except RuntimeError as e:
   print "FAIL: %s" % e

#3 Updated by Dennis Box over 1 year ago

  • Status changed from New to Feedback
  • Assignee changed from Dennis Box to Marco Mambelli

please see branch v34/20909

#4 Updated by Lorena Lobato Pardavila over 1 year ago

  • Assignee changed from Marco Mambelli to Lorena Lobato Pardavila

#5 Updated by Lorena Lobato Pardavila over 1 year ago

  • Description updated (diff)
  • Status changed from Feedback to Assigned
  • Assignee changed from Lorena Lobato Pardavila to Dennis Box

All the tests were OK.

Just for the record, this case is no longer accepted, as we have changed the configuration ("," are separators of elements from a list within a group) due to #20871:

host.domain:9618?var1=val1&sock=collect*$RANDOM_INTEGER*(3 ,9)&var2=val2-6

User is strongly encouraged to use the pattern established in the documentation. Example:

fermicloudui.fnal.gov:9618?sock=collector30-40

#6 Updated by Dennis Box over 1 year ago

  • Status changed from Assigned to Resolved

#7 Updated by Marco Mambelli about 1 year ago

  • Target version changed from v3_5 to v3_4_2

#8 Updated by Marco Mambelli about 1 year ago

  • Target version changed from v3_4_2 to v3_4_3

#9 Updated by Marco Mambelli 12 months ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF