First Cut

The initial image processing pipeline at DM is called First Cut. With a lot of help from Brian Yanny and
Robert Gruendl we compiled the following summary of the workflow. Throughout this document we point out
opportunities for SV squad members to contribute to this critical effort.

First Cut Processing

  1. Split and Cross Talk
    The first step in the nightly image processing pipeline is to apply the cross talk correction and split the image into 62 individual files - one for each science CCD [at some point we need to find out what happens to the 8 focus/alignment CCDs]. Work on the cross talk coefficient matrix was started already at Fermilab when the imager was still at Sidet. Completion of this matrix is considered a commissioning task for Juan's CCD team.
  2. Bias and Flats
    The next step is to apply bias and flat field corrections. Master bias and flat field frames have been prepared outside of First Cut by the calibration pipeline.
  3. Illumination Correction
    Remove stray light and pupil ghost.
  4. Astrorefine
    This processing step finds the astrometric solution for the image. The astrorefine block runs two modules: catalog_exposure and scamp_decam. The first module runs the routine, runSExtractor, which in turn actually runs SExtractor (sex) and fwhm to build a catalog of sources in each image. In this pass SExtractor is run without PSF modeling.
    Then scamp_decam uses the scamp program and the results for all 62 CCDs and matches the object positions to a standard catalog (currently USNOB). The output of this step is a reduced image, still split into 62 individual CCD images with updated WCS values. This is called a red image in DM jargon.
  5. Photometry
    A standard photometry module is under development. To reach the best possible precision many exposures need to be analyzed simultaneously and hence this module is not suitable for single epoch images. A simple photometric solution based on the SExtractor fluxes is currently under consideration. It would attempt to match the fluxes to objects in a photometric catalog. This should be fairly straightforward for Stripe 82 but it is not clear if we have a sufficient number of stars with known magnitude everywhere in the survey area.
    SV Squad Contribution: Having this simple photometry available should be very valuable for science verification. Help is needed to develop this module, test it and to integrate it with DM First Cut.
  6. Decision
    One of the requirements for First Cut is to provide a YES/NO decision whether the image is of survey quality or not. This information will then be used to update the survey table in the mountain which is used by obstac to track survey progress and to select which hexes to observe next.
    At the end of the First Cut pipeline we have an astrometric solution that provides us with some chi-square measure about the quality of the fit and the shifts in RA and dec relative to the telescope position reported in the header. From the simple photometry module we have a zeropoint estimate and some measure of the magnitude scatter. This information needs to pre-defined thresholds and a decision has to be reached whether to accept the image or not.
    SV Squad Contribution: Programming the decision logic is a reasonably well defined project for somebody on the SV squad. Bryan coordinate the effort and provide whatever guidance is needed.
The DM workflow provides several options that can be added to the First Cut pipeline.
  1. mask_decam
    This optional step produces a mask to block bright stars, cosmic rays and bleed trails. It needs a catalog for the (bright) star positions (USNOB). Cosmic rays are identified by using SExtractor with a high pass filter.
  2. sky image
    The image is median filtered in coarse bins to produce a background sky image which is useful, for example, to determine the sky level around bright galaxies.
  3. full SExtractor
    Optional step to run SExtractor with will PSF modeling. This step is needed to produce Model Mags (Galaxies) and PSF Mags (stars).
  4. remap
    This step is needed in preparation for co-adds. Exposures are devided into rectangular tiles labeled by CCD number, sky position and exposure number.
    A (DES) tile is a 10,000 pixel by 10,000 pixel area in the DES survey footprint that forms the basis of DES co-adds. swarp is used in this step.