Project

General

Profile

TOC PREV NEXT
Fermilab CD logo Complete Guide and Reference Manual for
UPS and UPD

Chapter Contents

Chapter 16: UPS Product Development: General
Considerations

   16.1 Product Development
Considerations and Recommendations

     16.1.1 All Products
(Locally Developed and Third Party)

     16.1.2 Products that You
Develop

     16.1.3 Third-Party
Products Requiring a Hard-Coded Path

   16.2 Directory Structure for a UPS
Product Instance

  Chapter 16: UPS Product
Development: General Considerations

  This chapter discusses the UPS product development methodology and tools that can be
used in product development. It also provides recommendations for
organizing the local product development area and the individual
product root directories needed.

  16.1 Product Development
Considerations and Recommendations

  This section provides
guidelines for product development as it affects the product's
inclusion in the UPS framework.

  Simple scripts which run on any
architecture are straightforward to implement under UPS . Products which are obtained from the outside world
(third party) as executable images with no source code are also
generally straightforward. The ones that get complicated are the
products which must be compiled and/or otherwise built for each and
every supported architecture.

  16.1.1 All Products
(Locally Developed and Third Party)

  Shell Independence

  The product should run the same
way under both shell families, sh and csh . If the product requires any actions to take place
before it will run (e.g., its bin directory added to $PATH, or
environment variables set), provide a table file containing
these actions. The UPS environment is described in Chapter 2: Overview of UPS and UPD and table files in Chapter 36: Table Files . The functions supported in table files are
designed to work in a shell-independent manner.

  Flavor Declaration in
UPS

  On your development system, we
recommend declaring products according to the fully specified flavor of the machine on which
they were built. We consider this to be very
important, especially if the target systems contain or will ever
contain mixed OS releases (e.g., Linux+2.2 and Linux+2.5). This will
help to avoid problems when a new OS release doesn't run images
built on an older one, or vice-versa. It is difficult to go
back and try to figure out which OS release a particular product instance
was built for. It is better to be able to tell immediately from looking
at the database. Installers and users also need this information to
facilitate their database maintenance.

  Products which have no
flavor-dependence at all (shell scripts, for instance), should be
declared as NULL to the database (use the "zero" option, -0 ;
see Chapter 25: Generic Command Option Descriptions ). For other
products, include the entire flavor string of the build platform in
the declaration, or the major portion of that string. For example,
if you build on an Linux machine running V2, declare your products
with the flavor Linux64bit+2 or Linux64bit+2.6 rather than just Linux64bit (e.g.,
use the flavor level corresponding to the options -4 or -3 or -2 rather than -1 )

  16.1.2 Products that You
Develop

  If you're writing your own
product for implementation within UPS , you have the luxury (and, we might add, the responsibility ) of creating it such that it exploits the
important features of UPS , thus making it easy for the user to install and run,
and easy for you or another developer to maintain in the future. We
urge you to follow the guidelines we present here.

  Self-Containment and
Location Determination

  First, design the product such
that it is self-contained. It should identify its location and the
location of any required files at run time (as opposed to compile time). You as the product
developer have total control over the structure and contents of the
product root directory, but no control at all over where the
product root directory will reside on a target system.

  If
you write the product such that it calculates its location at
compile time, you'll be putting the hard-coded path to your
development environment into the image -- most likely not the correct path on the user machine.

 

  Reproducible Build
Procedure

  All products should be built
using a build script in order to ensure that the build procedure is
reproducible.

  System Independence

  The various flavors of UNIX
have many differences. You will generally have to release separate
instances of your (compiled) products for the different flavors.
However, the more you are able to insulate your product from
flavor/release dependencies, the easier your product will be to
maintain, and the less rigid it will appear to installers and
users.

  16.1.3 Third-Party
Products Requiring a Hard-Coded Path

  If you're installing a
third-party product, downloaded from the Web or elsewhere, you may
not have the opportunity to code it such that it identifies its
location at run time based on ${UPS_PROD_DIR} or the
$<PRODUCT>_DIR environment variable. Whereas many products
never need to know their location (they only need to be in your
$PATH, for example), many other products do need to know their location in order to locate auxiliary
commands, libraries, utilities, and so on.

  Techniques for
Implementing these Products

  There are two approaches to implement such products:

  • For a product that is
    already set up and which contains a script that requires an
    interpreter, start the script with #!/usr/bin/env <interpreter> (e.g., #!/usr/bin/env
    perl
    ). The env program will run the first copy of the interpreter it
    finds on your command search path, and your script is then
    executable.
  • Create a "wrapper" shell
    script which sets up the UPS environment, sets up your product, and then invokes the
    appropriate commands.

  Examples of Products
Requiring Hard-coded Paths

  Here are examples of situations
in which hard-coded paths are unavoidable:

  • Pre-built products which
    have hard-coded paths.
  • Products that you can
    rebuild, but which were not coded with the idea of calculating
    where the files sit at run time. You need to state where to
    look for files at compile time, and this leads to hard-coded paths
    in the images.
  • Commands that are not
    executed in the context of a shell, but rather as a program. You can't assume
    the program running the command will expand environment variables. If you spell
    out the whole path, you'll have to edit it every time a new version of
    the command is released.
  • cgi scripts, rsh scripts and other situations in which you can't be sure
    that the product will necessarily have been setup when it is called
    by another one, and it needs to work anyway. Frequently you can create
    product configuration scripts that copy or link the product files
    into the correct location on the target node. In some cases for cgi scripts, you can have your Web server setup the product
    and pass the relevant environment variables.

  16.2 Directory Structure
for a UPS Product Instance

  The top level directory of a UPS product instance is called the product root directory , and in general it should contain
files and subdirectories in which almost everything related to the
product instance resides: the executables, the library files, the
documentation, and so on. The ups directory files (i.e., the UPS metadata) and the table file usually reside here, but
are not required to do so.

  UPS is very lenient in the directory structure it allows. Nothing is
required in all situations beyond a product root directory.
Normally product instances have a table file containing actions
that are run during operations like product installation and
setup.

  We recommend that you follow a
few directory structure guidelines simply to conform to a generally
recognized format. This will make it easy for yourself and others
to identify each file and directory later on. The following is a
relatively complete sample directory structure underneath the
product root directory. Most products won't require all of these
elements. On the other hand, you may include other directories
and/or files not listed here.
README text file containing information such as origin of the product
(by whom, from where, etc.), support level, support group/person,
caveats and known bugs (may be contained in the ups directory) bin directory containing the executable(s) ups directory containing metadata files and other executable and
data files used during implementation and invocation; may also
contain INSTALL_NOTE (described below) file and the directories toman , toInfo , tonews and tohtml . Often the table file
resides here. (This directory is no longer a required element of a UPS product.)Default location of the ups directory is directly
underneath the product root, but it may reside anywhere. ups/INSTALL_NOTE text file containing a detailed description of any installation
actions that are more easily performed directly by the installer
rather than by a script (beyond or instead of running configure and/or current ). This should not be a
script. This file is not usually needed. If provided, mention it in
the README file so that
product installers know to run it. lib directory containing libraries src directory containing source code include directory containing include files
p<>.  

TOC PREV NEXT

Last revised May 2014

TOC PREV NEXT
Fermilab CD logo Complete Guide and Reference Manual for
UPS and UPD

Chapter Contents

Chapter 16: UPS Product Development: General
Considerations

   16.1 Product Development
Considerations and Recommendations

     16.1.1 All Products
(Locally Developed and Third Party)

     16.1.2 Products that You
Develop

     16.1.3 Third-Party
Products Requiring a Hard-Coded Path

   16.2 Directory Structure for a UPS
Product Instance

  Chapter 16: UPS Product
Development: General Considerations

  This chapter discusses the UPS product development methodology and tools that can be
used in product development. It also provides recommendations for
organizing the local product development area and the individual
product root directories needed.

  16.1 Product Development
Considerations and Recommendations

  This section provides
guidelines for product development as it affects the product's
inclusion in the UPS framework.

  Simple scripts which run on any
architecture are straightforward to implement under UPS . Products which are obtained from the outside world
(third party) as executable images with no source code are also
generally straightforward. The ones that get complicated are the
products which must be compiled and/or otherwise built for each and
every supported architecture.

  16.1.1 All Products
(Locally Developed and Third Party)

  Shell Independence

  The product should run the same
way under both shell families, sh and csh . If the product requires any actions to take place
before it will run (e.g., its bin directory added to $PATH, or
environment variables set), provide a table file containing
these actions. The UPS environment is described in Chapter 2: Overview of UPS and UPD and table files in Chapter 36: Table Files . The functions supported in table files are
designed to work in a shell-independent manner.

  Flavor Declaration in
UPS

  On your development system, we
recommend declaring products according to the fully specified flavor of the machine on which
they were built. We consider this to be very
important, especially if the target systems contain or will ever
contain mixed OS releases (e.g., Linux+2.2 and Linux+2.5). This will
help to avoid problems when a new OS release doesn't run images
built on an older one, or vice-versa. It is difficult to go
back and try to figure out which OS release a particular product instance
was built for. It is better to be able to tell immediately from looking
at the database. Installers and users also need this information to
facilitate their database maintenance.

  Products which have no
flavor-dependence at all (shell scripts, for instance), should be
declared as NULL to the database (use the "zero" option, -0 ;
see Chapter 25: Generic Command Option Descriptions ). For other
products, include the entire flavor string of the build platform in
the declaration, or the major portion of that string. For example,
if you build on an Linux machine running V2, declare your products
with the flavor Linux64bit+2 or Linux64bit+2.6 rather than just Linux64bit (e.g.,
use the flavor level corresponding to the options -4 or -3 or -2 rather than -1 )

  16.1.2 Products that You
Develop

  If you're writing your own
product for implementation within UPS , you have the luxury (and, we might add, the responsibility ) of creating it such that it exploits the
important features of UPS , thus making it easy for the user to install and run,
and easy for you or another developer to maintain in the future. We
urge you to follow the guidelines we present here.

  Self-Containment and
Location Determination

  First, design the product such
that it is self-contained. It should identify its location and the
location of any required files at run time (as opposed to compile time). You as the product
developer have total control over the structure and contents of the
product root directory, but no control at all over where the
product root directory will reside on a target system.

  If
you write the product such that it calculates its location at
compile time, you'll be putting the hard-coded path to your
development environment into the image -- most likely not the correct path on the user machine.

 

  Reproducible Build
Procedure

  All products should be built
using a build script in order to ensure that the build procedure is
reproducible.

  System Independence

  The various flavors of UNIX
have many differences. You will generally have to release separate
instances of your (compiled) products for the different flavors.
However, the more you are able to insulate your product from
flavor/release dependencies, the easier your product will be to
maintain, and the less rigid it will appear to installers and
users.

  16.1.3 Third-Party
Products Requiring a Hard-Coded Path

  If you're installing a
third-party product, downloaded from the Web or elsewhere, you may
not have the opportunity to code it such that it identifies its
location at run time based on ${UPS_PROD_DIR} or the
$<PRODUCT>_DIR environment variable. Whereas many products
never need to know their location (they only need to be in your
$PATH, for example), many other products do need to know their location in order to locate auxiliary
commands, libraries, utilities, and so on.

  Techniques for
Implementing these Products

  There are two approaches to implement such products:

  • For a product that is
    already set up and which contains a script that requires an
    interpreter, start the script with #!/usr/bin/env <interpreter> (e.g., #!/usr/bin/env
    perl
    ). The env program will run the first copy of the interpreter it
    finds on your command search path, and your script is then
    executable.
  • Create a "wrapper" shell
    script which sets up the UPS environment, sets up your product, and then invokes the
    appropriate commands.

  Examples of Products
Requiring Hard-coded Paths

  Here are examples of situations
in which hard-coded paths are unavoidable:

  • Pre-built products which
    have hard-coded paths.
  • Products that you can
    rebuild, but which were not coded with the idea of calculating
    where the files sit at run time. You need to state where to
    look for files at compile time, and this leads to hard-coded paths
    in the images.
  • Commands that are not
    executed in the context of a shell, but rather as a program. You can't assume
    the program running the command will expand environment variables. If you spell
    out the whole path, you'll have to edit it every time a new version of
    the command is released.
  • cgi scripts, rsh scripts and other situations in which you can't be sure
    that the product will necessarily have been setup when it is called
    by another one, and it needs to work anyway. Frequently you can create
    product configuration scripts that copy or link the product files
    into the correct location on the target node. In some cases for cgi scripts, you can have your Web server setup the product
    and pass the relevant environment variables.

  16.2 Directory Structure
for a UPS Product Instance

  The top level directory of a UPS product instance is called the product root directory , and in general it should contain
files and subdirectories in which almost everything related to the
product instance resides: the executables, the library files, the
documentation, and so on. The ups directory files (i.e., the UPS metadata) and the table file usually reside here, but
are not required to do so.

  UPS is very lenient in the directory structure it allows. Nothing is
required in all situations beyond a product root directory.
Normally product instances have a table file containing actions
that are run during operations like product installation and
setup.

  We recommend that you follow a
few directory structure guidelines simply to conform to a generally
recognized format. This will make it easy for yourself and others
to identify each file and directory later on. The following is a
relatively complete sample directory structure underneath the
product root directory. Most products won't require all of these
elements. On the other hand, you may include other directories
and/or files not listed here.
README text file containing information such as origin of the product
(by whom, from where, etc.), support level, support group/person,
caveats and known bugs (may be contained in the ups directory) bin directory containing the executable(s) ups directory containing metadata files and other executable and
data files used during implementation and invocation; may also
contain INSTALL_NOTE (described below) file and the directories toman , toInfo , tonews and tohtml . Often the table file
resides here. (This directory is no longer a required element of a UPS product.)Default location of the ups directory is directly
underneath the product root, but it may reside anywhere. ups/INSTALL_NOTE text file containing a detailed description of any installation
actions that are more easily performed directly by the installer
rather than by a script (beyond or instead of running configure and/or current ). This should not be a
script. This file is not usually needed. If provided, mention it in
the README file so that
product installers know to run it. lib directory containing libraries src directory containing source code include directory containing include files
p<>.  

TOC PREV NEXT

Last revised May 2014