Data Processing Overview

Processing Raw Data

Summary of signals present in the raw data

  • 2-D bias structure (additive)
  • 1-D readout structure or overscan (additive)
  • QE variations between pixels (multiplicative)
  • scattered light (additive, large scale)
  • fringing in z, Y (additive, intermediate scale)
  • sky brightness variations (additive, large scale)

Data Processing Steps

  • Subtract the 2-D bias structure: bias(x,y)
  • Subtract the 1-D readout structure (overscan): over(y)
  • Divide by a corrected flat field: corrflat(x,y)
  • Fit and subtract a fringe frame (z, Y only): fringe(x,y)
  • Fit and subtract a smooth sky model to remove large-scale variations (sky, scattered light): sky(x,y)

Summary of Operations

  • output(x,y) = [raw(x,y) - bias(x,y) - over(y)]/flat(x,y) - fringe(x,y) - sky(x,y)

Creation of Calibration Files

Bias frame

The bias frame may be created as the median of a series of bias frames (typically 10-20). The bias frames should have their overscan removed first so that the median bias frame only captures 2-D structure that is not contained in the overscan region.

Corrected Flat

The corrected flat refers to a flat field that has had the scattered light removed (as well as earlier the bias and overscan) through application of a 'star flat.' The corrected flat field should consequently just represent the QE variations from pixel to pixel for each detector. The starting point is a series of dome flat fields that have had the bias structure and overscan removed. The star flat is then used to remove the large-scale additive structure from the dome flats to create the corrected flat field.

Star Flat (or Stray Light Map)

This is a map of the large-scale, additive component in the data due to ghost pupils and other forms of scattered light, which is then used to remove this additive component from the dome flats. The star flat is created from a series of dithered observations of the sky such that numerous objects are observed at a range positions across the field. The variations in the brightness of these objects at different positions, compared to the variation in the dome flat, is then used to map out the non-uniform, additive contamination in the dome flat due to stray light, ghost pupils, etc.

Fringe frame

Fringing will be important in Y-band, and perhaps in z-band, and will manifest as additional, intermediate-scale variations in the sky background. These variations may be sufficiently close to the scale of the objects that subtraction of a separate fringe frame, rather than subtraction of a smooth background model, will be the most reliable way to remove this effect. The fringe frame may be calculated from the median of a series of dithered observations in the Y-band. The input Y-band observations should have had the bias, overscan, and flat field corrections first. It is advantageous to identify and mask out bright objects on each of the dithered images to insure that the fringe frame is unaffected by them when these data are combined to construct the fringe frame.