Project

General

Profile

TOC PREV NEXT
Fermilab CD logo Complete Guide and Reference Manual for UPS and UPD

Chapter Contents

Chapter 22: Configuration of the fnkits Product Distribution Node
   22.1 UPS Configuration for KITS Database
   22.2 UPS Configuration for local Product Database
   22.3 UPD Configuration
     22.3.1 updconfig File Organization
     22.3.2 The Recognized Product Categories
     22.3.3 Location and File Name Definitions
     22.3.4 Pre- and Postdeclare ACTIONS
   22.4 fnkits Server Maintenance
     22.4.1 User Accounts and Group Ids
     22.4.2 Database and Configuration File Locations

Chapter 22: Configuration of the fnkits Product Distribution Node

This chapter describes the UPS / UPD configuration on the Computing Division's central product distribution node, fnkits.fnal.gov . Information is provided for both the KITS distribution database and the server's local database.

22.1 UPS Configuration for KITS Database

The KITS database on the fnkits.fnal.gov node has a fairly minimal configuration file, typical for distribution databases:

  • The database is configured to allow all registered nodes to read and use the products in it.
  • Statistics are not collected for any products.
  • Locations are defined for product instances, the UPS initialization files and the UPD configuration file.

For reference, we list the contents of the dbconfig file minus the comments:

FILE = DBCONFIG 

AUTHORIZED_NODES = * 

PROD_DIR_PREFIX = /ftp/products 

STATISTICS =  

SETUPS_DIR = /fnal/etc 

UPD_USERCODE_DIR = /fnal/ups/db/.updfiles

22.2 UPS Configuration for local Product Database

The local database on the fnkits node is maintained at /fnal/ups/db . The dbconfig file for this database is typical for databases on user nodes, where products are unwound and available for use. This file contains all the information that the dbconfig file for the KITS distribution database does, except that the product area, defined by PROD_DIR_PREFIX, is different. In addition to this content, there are definitions of target directories for various product information files (e.g., man pages).

For reference, we list the contents of the dbconfig file for the local database (minus the comments):

FILE = DBCONFIG

AUTHORIZED_NODES = *

PROD_DIR_PREFIX = ${UPS_THIS_DB}/../prd

MAN_TARGET_DIR = ${UPS_THIS_DB}/man
CATMAN_TARGET_DIR = ${UPS_THIS_DB}/catman

SETUPS_DIR = ${UPS_THIS_DB}/etc

UPD_USERCODE_DIR = ${UPS_THIS_DB}/.updfiles

In particular, notice that UPD_USERCODE_DIR is set to the same value in both files. This indicates that the databases share a UPD configuration.

22.3 UPD Configuration

22.3.1 updconfig File Organization

The UPD configuration for both the KITS distribution database ( /ftp/upsdb ) and the database used for locally installed products ( /fnal/ups/db ) is contained in the same file, ${UPD_USERCODE_DIR}/updconfig . ${UPD_USERCODE_DIR} is defined to be /fnal/ups/db/.updfiles in the dbconfig files for both databases.

The dbconfig file includes several stanzas, each of which pertains to a category of product. The product-matching criterion for each stanza is an option which indicates the category. The categories are: default (no option), fermitools , and proprietary . The section is as follows:

group:
     options = "proprietary" 
     options = "fermitools" 

For default, it is empty.

22.3.2 The Recognized Product Categories

default The default category is the most commonly used, and is for regular products added to the KITS database ( /ftp/products or /ftp/KITS ) for distribution to any on-site or  registered % off-site node. The products are set to group %{font-family: monospace}upd , and group-read-only. No option is associated with the default. fermitools  fermitools products are locally-developed and supported software packages (which are not available elsewhere, generally) that we make available to the public via our  FermiTools program % . These products are installed in the %{font-family: monospace}KITS database, are world-readable, and have a symlink hierarchy under /ftp/pub . The /ftp/pub hierarchy has been created with the same structure as /ftp/KITS . proprietary The proprietary category includes products for which Fermilab has a limited number of licenses. These products are installed in the KITS database, and made accessible only to special groups.

22.3.3 Location and File Name Definitions

The following location and file name definitions are shared by the stanzas for the product categories default , proprietary , and fermitools :

UPS_THIS_DB = "/ftp/upsdb" 
UNWIND_PROD_DIR="/ftp/products/${UPS_PROD_NAME}/${UPS_PROD_VERSION}/
  ${UPS_PROD_FLAVOR}/${UPS_PROD_NAME}_${UPS_PROD_VERSION}_${UPS_PROD_FLAVOR}
  ${UPS_PROD_QUALIFIERS}" 
UNWIND_UPS_DIR = "${UPS_PROD_DIR}/ups" 
UNWIND_TABLE_DIR = "${UPS_TABLE_DIR}" 
UPS_TABLE_DIR = "/ftp/products/${UPS_PROD_NAME}/${UPS_PROD_VERSION}/
  ${UPS_PROD_FLAVOR}" 
UNWIND_ARCHIVE_FILE = "${UNWIND_PROD_DIR}.${SUFFIX}" 
UPS_TABLE_FILE="${UPS_PROD_NAME}_${UPS_PROD_VERSION}_${UPS_PROD_FLAVOR}
  ${UPS_PROD_QUALIFIERS}.table" 
UPS_TABLE_DIR = "${UNWIND_TABLE_DIR}" 
UPS_PROD_DIR = "${UNWIND_PROD_DIR}" 
UPS_UPS_DIR = "ups" 

22.3.4 Pre- and Postdeclare ACTIONS

The stanzas include a PREDECLARE and a POSTDECLARE action.

  • In each case, the PREDECLARE action includes a set of execute statements to chmod/chgrp the files to the right group id and permissions.
  • The POSTDECLARE action makes a convenience tar file of the ups directory for users downloading via ftp .

The execute statements in each stanza are similar, but not identical. We first list them for the default case, and then list the differences for the other product categories relative to the default.

 

PREDECLARE Action for default Products

The PREDECLARE action for the default product category fixes group permissions:

action = predeclare
Execute("chgrp upd ${UNWIND_TABLE_DIR}/*", NO_UPS_ENV)
Execute("chmod o-rwx ${UNWIND_TABLE_DIR}/*", NO_UPS_ENV)
Execute("chmod a+r ${UNWIND_TABLE_DIR}/*.table", NO_UPS_ENV)

and makes old- KITS compatible hierarchy files:

Execute("test -d  /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION} || mkdir -p /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}", NO_UPS_ENV)
Execute("cd /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}; rm -f ${UPS_PROD_NAME}_${UPS_PROD_VERSION}_${UPS_PROD_FLAVOR}${UPS_PROD_QUALIFIERS}.*", NO_UPS_ENV)
Execute("cd /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}; /usr/bin/ln -fs ${UNWIND_PROD_DIR}.* . || true",NO_UPS_ENV)

PREDECLARE Action for FermiTools Products

The PREDECLARE action is the same as for the default products except for the changes noted here.

For fermitools , there is no Execute ("chgrp...") command. The first chmod command is o+rx rather than o-rwx :

Execute("chmod o+rx ${UNWIND_TABLE_DIR}/*", NO_UPS_ENV)

POSTDECLARE Action for All Product Categories

The POSTDECLARE action makes a ${UNWIND_PROD_DIR}.ups.tar tar file:

action = postdeclare
Execute("test -d \"${UNWIND_UPS_DIR}\" && cd ${UNWIND_UPS_DIR} && tar cf ${UNWIND_PROD_DIR}.ups.tar . || true", NO_UPS_ENV)

22.4 fnkits Server Maintenance

22.4.1 User Accounts and Group Ids

The fnkits Web and FTP server configuration files are owned by oss .

The Web server runs as updadmin . updadmin owns all of the FTP -served product files and the KITS UPS database, and therefore also the Web server logs.

22.4.2 Database and Configuration File Locations

The KITS distribution database is /ftp/upsdb .

The Web server configuration files are maintained under the /fnal/www directory.

The FTP configuration files live under /etc/ftpd .

"Sanitized" versions of the configuration files (with hostnames and such trimmed) are available at http://ftp.fnal.gov/ .

 

 

TOC PREV NEXT

Last revised on May 2014

TOC PREV NEXT
Fermilab CD logo Complete Guide and Reference Manual for UPS and UPD

Chapter Contents

Chapter 22: Configuration of the fnkits Product Distribution Node
   22.1 UPS Configuration for KITS Database
   22.2 UPS Configuration for local Product Database
   22.3 UPD Configuration
     22.3.1 updconfig File Organization
     22.3.2 The Recognized Product Categories
     22.3.3 Location and File Name Definitions
     22.3.4 Pre- and Postdeclare ACTIONS
   22.4 fnkits Server Maintenance
     22.4.1 User Accounts and Group Ids
     22.4.2 Database and Configuration File Locations

Chapter 22: Configuration of the fnkits Product Distribution Node

This chapter describes the UPS / UPD configuration on the Computing Division's central product distribution node, fnkits.fnal.gov . Information is provided for both the KITS distribution database and the server's local database.

22.1 UPS Configuration for KITS Database

The KITS database on the fnkits.fnal.gov node has a fairly minimal configuration file, typical for distribution databases:

  • The database is configured to allow all registered nodes to read and use the products in it.
  • Statistics are not collected for any products.
  • Locations are defined for product instances, the UPS initialization files and the UPD configuration file.

For reference, we list the contents of the dbconfig file minus the comments:

FILE = DBCONFIG 

AUTHORIZED_NODES = * 

PROD_DIR_PREFIX = /ftp/products 

STATISTICS =  

SETUPS_DIR = /fnal/etc 

UPD_USERCODE_DIR = /fnal/ups/db/.updfiles

22.2 UPS Configuration for local Product Database

The local database on the fnkits node is maintained at /fnal/ups/db . The dbconfig file for this database is typical for databases on user nodes, where products are unwound and available for use. This file contains all the information that the dbconfig file for the KITS distribution database does, except that the product area, defined by PROD_DIR_PREFIX, is different. In addition to this content, there are definitions of target directories for various product information files (e.g., man pages).

For reference, we list the contents of the dbconfig file for the local database (minus the comments):

FILE = DBCONFIG

AUTHORIZED_NODES = *

PROD_DIR_PREFIX = ${UPS_THIS_DB}/../prd

MAN_TARGET_DIR = ${UPS_THIS_DB}/man
CATMAN_TARGET_DIR = ${UPS_THIS_DB}/catman

SETUPS_DIR = ${UPS_THIS_DB}/etc

UPD_USERCODE_DIR = ${UPS_THIS_DB}/.updfiles

In particular, notice that UPD_USERCODE_DIR is set to the same value in both files. This indicates that the databases share a UPD configuration.

22.3 UPD Configuration

22.3.1 updconfig File Organization

The UPD configuration for both the KITS distribution database ( /ftp/upsdb ) and the database used for locally installed products ( /fnal/ups/db ) is contained in the same file, ${UPD_USERCODE_DIR}/updconfig . ${UPD_USERCODE_DIR} is defined to be /fnal/ups/db/.updfiles in the dbconfig files for both databases.

The dbconfig file includes several stanzas, each of which pertains to a category of product. The product-matching criterion for each stanza is an option which indicates the category. The categories are: default (no option), fermitools , and proprietary . The section is as follows:

group:
     options = "proprietary" 
     options = "fermitools" 

For default, it is empty.

22.3.2 The Recognized Product Categories

default The default category is the most commonly used, and is for regular products added to the KITS database ( /ftp/products or /ftp/KITS ) for distribution to any on-site or  registered % off-site node. The products are set to group %{font-family: monospace}upd , and group-read-only. No option is associated with the default. fermitools  fermitools products are locally-developed and supported software packages (which are not available elsewhere, generally) that we make available to the public via our  FermiTools program % . These products are installed in the %{font-family: monospace}KITS database, are world-readable, and have a symlink hierarchy under /ftp/pub . The /ftp/pub hierarchy has been created with the same structure as /ftp/KITS . proprietary The proprietary category includes products for which Fermilab has a limited number of licenses. These products are installed in the KITS database, and made accessible only to special groups.

22.3.3 Location and File Name Definitions

The following location and file name definitions are shared by the stanzas for the product categories default , proprietary , and fermitools :

UPS_THIS_DB = "/ftp/upsdb" 
UNWIND_PROD_DIR="/ftp/products/${UPS_PROD_NAME}/${UPS_PROD_VERSION}/
  ${UPS_PROD_FLAVOR}/${UPS_PROD_NAME}_${UPS_PROD_VERSION}_${UPS_PROD_FLAVOR}
  ${UPS_PROD_QUALIFIERS}" 
UNWIND_UPS_DIR = "${UPS_PROD_DIR}/ups" 
UNWIND_TABLE_DIR = "${UPS_TABLE_DIR}" 
UPS_TABLE_DIR = "/ftp/products/${UPS_PROD_NAME}/${UPS_PROD_VERSION}/
  ${UPS_PROD_FLAVOR}" 
UNWIND_ARCHIVE_FILE = "${UNWIND_PROD_DIR}.${SUFFIX}" 
UPS_TABLE_FILE="${UPS_PROD_NAME}_${UPS_PROD_VERSION}_${UPS_PROD_FLAVOR}
  ${UPS_PROD_QUALIFIERS}.table" 
UPS_TABLE_DIR = "${UNWIND_TABLE_DIR}" 
UPS_PROD_DIR = "${UNWIND_PROD_DIR}" 
UPS_UPS_DIR = "ups" 

22.3.4 Pre- and Postdeclare ACTIONS

The stanzas include a PREDECLARE and a POSTDECLARE action.

  • In each case, the PREDECLARE action includes a set of execute statements to chmod/chgrp the files to the right group id and permissions.
  • The POSTDECLARE action makes a convenience tar file of the ups directory for users downloading via ftp .

The execute statements in each stanza are similar, but not identical. We first list them for the default case, and then list the differences for the other product categories relative to the default.

 

PREDECLARE Action for default Products

The PREDECLARE action for the default product category fixes group permissions:

action = predeclare
Execute("chgrp upd ${UNWIND_TABLE_DIR}/*", NO_UPS_ENV)
Execute("chmod o-rwx ${UNWIND_TABLE_DIR}/*", NO_UPS_ENV)
Execute("chmod a+r ${UNWIND_TABLE_DIR}/*.table", NO_UPS_ENV)

and makes old- KITS compatible hierarchy files:

Execute("test -d  /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION} || mkdir -p /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}", NO_UPS_ENV)
Execute("cd /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}; rm -f ${UPS_PROD_NAME}_${UPS_PROD_VERSION}_${UPS_PROD_FLAVOR}${UPS_PROD_QUALIFIERS}.*", NO_UPS_ENV)
Execute("cd /ftp/KITS/${UPS_BASE_FLAVOR}/${UPS_PROD_NAME}/${UPS_PROD_VERSION}; /usr/bin/ln -fs ${UNWIND_PROD_DIR}.* . || true",NO_UPS_ENV)

PREDECLARE Action for FermiTools Products

The PREDECLARE action is the same as for the default products except for the changes noted here.

For fermitools , there is no Execute ("chgrp...") command. The first chmod command is o+rx rather than o-rwx :

Execute("chmod o+rx ${UNWIND_TABLE_DIR}/*", NO_UPS_ENV)

POSTDECLARE Action for All Product Categories

The POSTDECLARE action makes a ${UNWIND_PROD_DIR}.ups.tar tar file:

action = postdeclare
Execute("test -d \"${UNWIND_UPS_DIR}\" && cd ${UNWIND_UPS_DIR} && tar cf ${UNWIND_PROD_DIR}.ups.tar . || true", NO_UPS_ENV)

22.4 fnkits Server Maintenance

22.4.1 User Accounts and Group Ids

The fnkits Web and FTP server configuration files are owned by oss .

The Web server runs as updadmin . updadmin owns all of the FTP -served product files and the KITS UPS database, and therefore also the Web server logs.

22.4.2 Database and Configuration File Locations

The KITS distribution database is /ftp/upsdb .

The Web server configuration files are maintained under the /fnal/www directory.

The FTP configuration files live under /etc/ftpd .

"Sanitized" versions of the configuration files (with hostnames and such trimmed) are available at http://ftp.fnal.gov/ .

 

 

TOC PREV NEXT

Last revised on May 2014