Funky columns

See #3500 for the first discussion of this issue.

Basic properties we believe to know

  • Warmer and colder pixels form entire columns that deviate visibly from the mean.
  • Funky columns are present in the raw, but not in bias or flat frames, hence they survive the typical calibrations and show up in firstcut images.
  • They come in pairs or groups of a few, mainly with one very cold, surrounded by warmer columns such that the total deviation from the mean counts is zero (or very close)
  • They are much more abundant on the chip edges (columns 0-200 and 1950-2048) than in the center.
  • They do not depend on filter, and apparently stable in time.

Detection procedure

Take a bunch of images, split by chip number, and compute the deviation of the column median from the median of the entire chip image. Record this deviation for each column and each exposure. This yields images like this (I used 87 exposures here, all from the SV cluster observations of December 7-8, taken in g and r under dark conditions):

Column deviation vs exposure number

In a second step, take the deviation measurements and make the median deviation for all exposures (to eliminate bright structures such as unsaturated stars that affect several columns, but only on one exposure). The resulting plot shows three distinct features:

  • Funky columns: large positive/negative fluctuations
  • Insufficient flatfielding of the firstcut images
  • the A/B jump

Looking closer at the edges of the chip, we can see that the funky columns come in pairs or larger groups that compensate each other perfectly (to the accuracy of the test). This suggests some sort of charge mixing, either when the charges are accumulated are red out. A hardware involvement is further supported by the fact that the cold column is always on the left on the A side of the chip, and always on the right on the B side, showing that the features "knows" the readout direction.

By smoothing the fluctuations on large scales, I could remove the flatfield and A/B problems, so that reporting funky columns amount to simply cutting above a threshold. Visual inspection of the SV cluster images revealed that a deviation of 1 count is invisible in these dark 90 second g- and r-band exposures.

Open questions

  • The fact that the funky columns escaped flat-fielding and gain correction suggests that the amplitude could be illumination-dependent. This could be confirmed with PTC exposures.
  • The presence of the features in exposures from December and from January suggest that they are stable in time. Applying my script on a longer baseline of exposures can easily reveal visually the appearance of the funky columns at some point in time (the first image above is in fact time-ordered).
  • Some columns seem mainly cold, without compensation. Where and how often does that happend?

List of detected columns

The list of columns that deviated by more then 1 count from the December 7 SV cluster coadds: funky_column.lst

Correction scheme

If the neighboring columns in fact compensate each other, at least approximate corrections are feasible:
Assume column i is cold, and column j=i+1 is warm.
In each row, neighboring pixels have true (=non-funky) values v_i and v_j, totaling to V = v_i+v_j.
If V is conserved, we have approximately (1st order in funkyness parameter x, which is of order 1%)
V =~ (1-x)*v_i + (1+x)*v_j = v_i' + v_j' (the primed quantities are then measured)
That gives v_i =~ [V - v_j']/(1-x) and v_j =~ [V - v_i']/(1+x)