Project

General

Profile

DES Kentools

Contact: Steve Kent <>
Logo:[image]
Notes on calculation of t_eff: here

What is it?

The "kentools" package is an updated version of a portion of the SDSS software framework. It is installed on the SISPI machine observer2.ctio.noao.edu.

The original kentools has been augmented with code to aid in quick viewing and simple analyses of DECam data. The functions provided include the ability to list an inventory of all exposures from a given night, load the image of one CCD frame from a DECam exposure, measure the seeing from the image of an isolated star, analyze a focus sequence, and compute the true location on the sky of an exposure and determine offsets from where the telescope thinks it is pointed. The goal is simplicity - for example, an exposure can be identified by just the last few digits of the exposure number, without having to specify the subdirectory or the full file name.

Startup

One logs into observer2.ctio.noao.edu using the DECamObserver account. Type:

observer

The environment is setup and the kentools program is launched. The working directory is set to the top of the
DECam data tree. Files are stored under here organized by program ID.

Commands

Core Commands

The following commands are useful for everyone.

data Change directory to the top of the data tree.
This is normally /home3/data_local/images/fits
inventory [n] List frames from n days ago (default 0). n can also be a date (e.g., 20140821)
inv [n] (abbreviation for above)
invPrint [n] Print the inventory listing to a file in home area.
load [expnum] [ccd-id] Launch a ds9 window and display one CCD from an exposure. expnum,
if specified is the exposure number. It is not necessary to give
all 6 digits; the last 3 or 4 should suffice. If expnum is not
specified, the most recent exposure will be used. If ccd-id is
specified, it determines which CCD will be displayed. The default
is S4. Alternatively, one can specify the HDU number, but one
needs to know the mapping between ccd-id and HDU number for this to
be useful. (Note that this is NOT the same as CCDNUM).
bigload [expnum] Display a complete mosaic of an exposure. expnum, if specified
is the exposure number. It is not necessary to give all 6 digits;
the last 3 or 4 should suffice. If expnum is not specified, the
most recent exposure will be used. The image is compressed by
a factor 8 and bias subtracted, so it looks nicer. A crude WCS
is supplied.
center [expnum] [maxoff] Compute RA, DEC and offsets of an image center relative to the
desired target position. The offsets are given as corrections that can
be fed into SISPI in order to center-up an object. expnum, if specified
is the exposure number. It is not necessary to give all 6 digits; the
last 3 or 4 should suffice. If expnum is not specified or entered as
a null, the most recent exposure will be used. if maxoff is specified,
it is the maximum size for the search window to find a match between the
star field and the Nomad catalog. The default is 1024 pixels
(about 269 arcsec). Occassionally the telescope pointing is bad enough
that a bigger window is needed (but the run time is increased.)
seeingall [expnum] Measure the PSF size for stars in a selection of CCDs. Stars are found
automatically. If expnum is specified, it is the exposure number.
It is not necessary to give all 6 digits; the last 3 or 4 should suffice. If
expnum is not specified, the most recent exposure is used.
pdu [expnum] List the content of a FITS file primary header. expnum, if specified is
the exposure number. It is not necessary to give all 6 digits; the last 3 or 4
should suffice. If expnum is not specified, the most recent exposure will be
used. (to exit this command, type q.)
seeing Measure the PSF size of a single star. It is first necessary to display the image
using the load command. The cursor is positioned on a single, isolated,
bright but not saturated star image and left-clicked.
nick [expnum] Calculate the approximate limiting magnitude of an exposure based on image FWHM
and sky background. The limiting magnitude refers to a point source and computed
by a calculation that uses parameters taken from the image header plus a nominal
value for telescope throughput. The calculation is only valid for data taken in
clear conditions. If expnum is not specified, the most recent exposure
will be used.
psc [expnum] This command combined several others in one. It computes PSF size, ellipicity
pointing error, sky brightness (relative to dark sky), a measure of
cloudiness (which, in this context is also referred to as extinction),
and an overall "teff" effective exposure time, normalized to 1 for
good seeing on a dark night pointing at the zenith. Cloudiness is a rough
estimate and derived by comparing stars against either the APASS or NOMAD
star catalogs. Six CCDs are searched for this comparison. A cloudiness of
< 0.2 mag is not significant. If expnum is not specified, the most
recent exposure will be used. More details on teff can be found here

DES Commands

The following commands are specific to DES but may be of more general use as well.

standards [UT] This command only works on observer2.ctio.noao.edu. It runs D. Tucker's scripts to
select a set of 3 standard stars to be observer at time UT. If UT
is not specified, it is assumed that the standards will be observed immediately.
flatCheck [n] Run a quality check on dome flatfields taken n days ago (default 0).
The mean ADU/sec in each filter are plotted along with some nominal values.
Small deviations from nominal are to be expected as the LED lamps age.
noiseCheck [n] Run a quality check on bias frames taken n days ago (default 0).
Statistics of the readout noise are checked to look for "spiky" noise that
has appeared in the past (e.g., once time due to a loose connector). The
noise is common to all CCDs on a single backplane.
ephem [date/mjd] Print out various time-related quantities including sunset, twilight, moon
rise/set, and sunrise. Both local Chilean time and universal time are listed. The
date, if specified, can be either the UT MJD at local midnight or a date in the
form YYYYMMDD of the evening of observation.
pointing [n] This command runs A. Drlica-Wagner's script to show position of DECam exposures.
Use pointing --help to see more options.
More details on the pointing script can be found in Introduction to pointing

Database Commands

The following commands are useful for querying other databases via a call to Firefox.

skyview Display an image from skyview.gsfc.nasa.gov in a web browser that
matches the field of a DES exposure. It is first necessary to display
the image using the load command. The cursor is positioned on the location
of interest and left-clicked. The web browser (firefox) is launched if an
instance is not already running; else a separate tab is launched.
ned Display a list of objects from NED at IPAC in a web browser that matches an object
in a DES exposure. It is first necessary to display the image using the
load command. The cursor is positioned on the object of interest and
left-clicked. The web browser (Firefox) is launched if an instance is not
already running; else a separate tab is launched.
simbad Display a list of objects from the SIMBAD database in a web browser that
matches an object in a DES exposure. It is first necessary to display the image
using the load command. The cursor is positioned on the object of interest
and left-clicked. The web browser (Firefox) is launched if an instance is not
already running; else a separate tab is launched.
NOTE: If you are running kentools by logging in to observer2 from observer3 and want to use the above commands, you must perform the following:
  • Make sure no firefox is running on observer3
  • ssh -X -Y observer2 (so X windows are displayed via the network on observer3)
  • Type "firefox" - it will run on observer2 but display on observer3
  • Now start kentools.

Engineering Commands

The following commands are mainly used for engineering purposes.

tim [expnum] [maxoff] Compute RA, DEC and offsets of an image center relative to the
telescope coordinates telra and teldec.This command gives a measure of the
pointing error of the telescope but is intended for engineering use only.
(One should use the "center" command for actually centering a field.)
expnum, if specified is the exposure number. It is not necessary to give
all 6 digits; the last 3 or 4 should suffice. If expnum is not specified,
the most recent exposure will be used. The offsets are given in the form of
corrections (although one probably wants the negative of these - the errors).
offset [expnum] This command is used for constructing pointing models. A bright star should
fall within CCD N4. One clicks on the star, and the offset from a
fiducial point on the CCD is printed. expnum, if specified is the
exposure number. It is not necessary to give all 6 digits; the last 3 or 4
should suffice. If expnum is not specified, the most recent exposure will be
used.
boreset [ccd-id] This command is used for constructing pointing models. A bright star should
fall within CCD ccd-id. Default is S4. One clicks on the star,
and the offset from the center of the focal plane is printed. The most recent
exposure is always displayed. (Use centeroffset expnum ccd-id to specify a
different exposure, plus ccd-id).
focus Measure star PSF sizes in a focus sequence. It is first necessary to display
the image using the load command. A focus sequence is a series of exposures of
a field with the CCD clocked by a specified number of rows. The first
and second exposures are separated by a double-sized space. The cursor is
positioned on the first star of the sequence and left-clicked. The sequence
should be relative isolated from other stars.

For Experts - Updating Calibration Constants

Several commands (flatCheck, etc) rely on calibration numbers - expected flatfield level, gain, etc. These numbers
are stored in the directory ~DECamObserver/skent/products/run. There is a README file there that describes the files in
more detail (there are 4 types) and how to update them.

Implementation

The SDSS software stack is constructed as a set of layered "products". At the lowest level, "products" might be
libraries - e.g., a set of routines to manipulate FITS files or a plotting library. Higher level products are
integrated data analysis packages, and these can be layered as well. At the top level are complex data analysis
pipelines. "kentools" is one of those packages.

The software "framework" is based on TCL (Tool Command Language) created by John Oosterhout at UC Berkeley. TCL
is a scripting language with a fairly straightward syntax: a line of code consists of a command followed by zero
or more arguments. There are close to 100 commands in the basic TCL language. The command set can be extended
in two ways: either procedures written in TCL itself or in C code with a binding to the TCL interpreter.
(kentools has nearly 1600 commands). Within the software stack, a product provides either C (or in a few
cases, Fortran code) alone, C code with TCL bindings, and/or TCL procedures. The DES version of kentools
actually consists of a product called "des" that adds TCL procedures to the underlying kentools product.

Even though products are layered, the top-level product usually consists of a monolithic binary that includes a
TCL command interpreter linked with binaries from all the underlying products. Complex processes are constructed
using TCL commands that call the underlying C code. TCL code from the layered products is loaded for use at the
top level.

The source code for the software is stored under the directory ~/skent/sdss/. There are about 20 products
total (a couple are not actually used by kentools). The source code is maintained by SVN code management.
The repository is in the "Science Verification" wiki at Fermilab (but note that you need a Fermilab Kerberos
account to get access):

svn+ssh://p-des-sci-verification@cdcvs.fnal.gov/cvs/projects/des-sci-verification/sv_iq/trunk/sdss

The installed binary products are stored under the directory ~/skent/products/. The products are managed by
the EUPS configuration management system. The purpose of EUPS is to provide a mechanism to manage multiple
versions of installed software products such that one can switch from one version to another easily. In a
dynamic system where new versions of software are being developed and installed all the time, it is desirable to
be able to install and test a new version but be able to roll back to an older, stable version if necessary.

EUPS is itself a product in the products area. One normally initializes EUPS by sourcing the file
products/eups/bin/setups.csh (for a tcsh shell). EUPS maintains a database of products in the directory
products/ups_db.

Products are made available to a user by the command:

setup <product>

Setting up a product usually means adding the product's path to the PATH environment variable and defining or
manipulating additional environment variables as well. Additionally, setting up a product usually sets up
dependent products as well. "setup" serves a dual purpose - it makes a product available for inclusion in the
build process of another product, and it also sets up any environment needed at runtime.

TCL code is generally stored in the etc subdirectory of a product.

The interaction of SVN, source tree, products area and EUPS is a follows. One starts by checking out the source
code tree. For a product at the bottom of the stack, one "builds" the code using "make" (possibly with an
initial "configure" step), runs any tests, "installs" the product in the "products" area, "declares" the
products using EUPS, then sets up the product using "setup". The product is then available for inclusion in the
build process for a product in the next level up in the software stack.

By convention, the directory structures in the source code tree and the installed product tree are made the
same. Thus, at runtime, one can select either the installed product or the source code tree (after building) as
the runtime version. The source code tree is generally tagged as the "devel" version of a product, while the
installed version is tagged either with a version number or other mnemonic. Multiple versions can be declared
to EUPS; one of these versions is tagged as the "current" version, which is normally meant to be the latest
stable and tested version.

The main products where code specific to DES is being developed are kentools and des. The code is primarly
TCL. Normally I run with the "devel" version of these products in order to add and test new code incrementally.
Periodically stable versions are installed in the products area as "current" versions.

Export Versions

kentools is reasonably portable, and full source code (with Makefiles) and binaries for 32- and 64-bit Linux
systems can be made available. NOTE: A common desire is to run it to get image centers; HOWEVER, the NOMAD
catalog (supplied separately) is required. At present I do not know of any place that distributes NOMAD online
(compressed size is 30 GB).