Anom-R1 » History » Version 7

Daniel Gruen, 02/25/2013 05:33 PM

1 1 Daniel Gruen
h1. Anom-R1
2 1 Daniel Gruen
3 1 Daniel Gruen
h2. Description
4 1 Daniel Gruen
5 1 Daniel Gruen
From DESSV Plan: Map of bad pixels, defects is stable, as seen from dome flats and median sky flats aka DECam TD-16
6 1 Daniel Gruen
7 1 Daniel Gruen
Explanation: Bad pixels are pixels whose counts are non-linear in the SB of the sky at their position. The hope is that they are stable over time in the sense that a bad pixel will stay "bad" while no new bad pixels appear. Bad pixels are likely to change in "badness" with thermal cycles (Juan Estrada, private comm.), so some degree of monitoring is required lest they contaminate neighboring pixels.
8 1 Daniel Gruen
9 1 Daniel Gruen
h2. Prerequisites
10 1 Daniel Gruen
11 2 Daniel Gruen
from doc-db:722)
12 1 Daniel Gruen
13 3 Daniel Gruen
14 3 Daniel Gruen
  Black Spots: 10 flat exposures were median combined to eliminate cosmic rays. 
15 3 Daniel Gruen
               Any pixel with 20% less response than the median was counted as a black spot
16 3 Daniel Gruen
  Non-linear: two sets of 10 exposures at 50000 e and 80000 e were median combined 
17 3 Daniel Gruen
              and the two resulting images were divided. Any pixel with a response
18 3 Daniel Gruen
              that was different by more than 1% was flagged as non-linear. Require 4 sigma detection. 
19 3 Daniel Gruen
  Hot pixels: 10 dark 400-second exposures were median combined. 
20 3 Daniel Gruen
              Hot pixels were counted as those that had more than 6300 e/pixel/hour.
21 3 Daniel Gruen
  All the above 3 masks are combined by OR to form a final mask. 
22 3 Daniel Gruen
23 1 Daniel Gruen
24 2 Daniel Gruen
We require *sets of dome flat fields with 3 different flux levels* (1) shortest reasonable exposure time where shutter effects are negligible (below 1% level), (2) at approx. 50000 e and (3) at approx 80000 e, to cover non-linearity in the typical low surface-brightness and the almost-saturated regime. Of set (2) and (3) we require 10 flats, of set (1) we require enough frames for the Poisson rms to be below 0.2%. The frames should be *bias subtracted*. They must be *taken at different times of the commissioning procedure* to check for stability over time and, subsequently, after certain intervals. They should *span the DES filters* since one could imagine spectral dependence of non-linearity. Additionally, we require a *"known bad pixel" mask* for each chip from the DECam Imager Tests to compare with.
25 2 Daniel Gruen
26 2 Daniel Gruen
Note that we need sets of dome flats with different exposure times taken *without moving the telescope* or changing the light source in between, so the gradient stays the same in all images. Small changes in light source luminosity will be corrected. At the lower exposure time limit, we might discover non-linearities due to shutter effects, so we should be careful these are below 1% of the flux or else we won't be able to discriminate shutter effects from bad pixels.
27 2 Daniel Gruen
28 2 Daniel Gruen
We also need a set of *10 dark 400-second exposures* for detecting hot pixels independently from their non-linearity.
29 2 Daniel Gruen
30 1 Daniel Gruen
h2. Procedure
31 1 Daniel Gruen
32 2 Daniel Gruen
h3. Non-Linearity
33 2 Daniel Gruen
34 1 Daniel Gruen
for each filter, for each chip:
35 1 Daniel Gruen
* for each day available:
36 2 Daniel Gruen
** for each of the 3 flux levels:
37 1 Daniel Gruen
*** calculate median flux of each of the frames
38 1 Daniel Gruen
*** multiply each frame with the quotient of desired over median flux level to bring them to the same level
39 1 Daniel Gruen
*** median-stack frames from the same chip to generate master flats
40 2 Daniel Gruen
** for each pair of flux levels (1,2), (2,3)
41 1 Daniel Gruen
*** divide the two master stacks
42 1 Daniel Gruen
*** note pixels where the fraction is off the quotient of the desired flux levels (i.e., which show non-linearity) beyond the expected statistical uncertainty of the master flats
43 1 Daniel Gruen
*** compare the list of noted pixels to the list of known bad pixels: are there any new ones?
44 1 Daniel Gruen
*** save the list for later use
45 1 Daniel Gruen
46 1 Daniel Gruen
after a few observations separated by some time in between have been made:
47 2 Daniel Gruen
for each pair of flux levels (1,2), (2,3)
48 1 Daniel Gruen
* compare the number of noted outliers for each date -> is there an increase?
49 1 Daniel Gruen
* compare the bad pixel map created for each date [manually, by blinking the masks] -> is there any visible development?
50 1 Daniel Gruen
51 1 Daniel Gruen
If we want to monitor the change of these bad pixels over time, we could save the FITS image or simply a list of divided flux levels (2,3) for each date; this will parametrize the non-linearity. 
52 1 Daniel Gruen
53 2 Daniel Gruen
h3. Cold pixels
54 2 Daniel Gruen
55 2 Daniel Gruen
Just get a median stack of 10 50000e flats.
56 2 Daniel Gruen
57 2 Daniel Gruen
h3. Hot pixels
58 2 Daniel Gruen
59 2 Daniel Gruen
Just get a median stack of the dark frames.
60 2 Daniel Gruen
61 4 Daniel Gruen
h3. Noise properties
62 4 Daniel Gruen
63 4 Daniel Gruen
Good pixels should have proper Poisson-noise variation from one flat-field exposure to another. This can be tested by calculating the variance between the rescaled counts of pixels in different frames.
64 4 Daniel Gruen
65 1 Daniel Gruen
h2. Verdict
66 1 Daniel Gruen
67 2 Daniel Gruen
h3. Non-Linearity
68 2 Daniel Gruen
69 2 Daniel Gruen
1% deviation from the expected flux level quotients in any of the (1,2), (2,3) flux level quotient frames will flag a pixel as non-linear. Note the statistical uncertainty of the stacks should be of the order of 0.1%.
70 2 Daniel Gruen
71 2 Daniel Gruen
h3. Cold pixels
72 2 Daniel Gruen
73 2 Daniel Gruen
20% less than median response in the stack of 10 50000e flats will flag a pixel as cold.
74 2 Daniel Gruen
75 2 Daniel Gruen
h3. Hot pixels
76 2 Daniel Gruen
77 2 Daniel Gruen
More than 6300 e/pixel/hour in the dark frame stack will flag a pixel as hot.
78 4 Daniel Gruen
79 5 Daniel Gruen
h3. Noise properties
80 4 Daniel Gruen
81 4 Daniel Gruen
If variance is 6sigma off the expected Poisson variance, a pixel should be considered bad. This will produce approx. one false positive over the whole array.
82 2 Daniel Gruen
83 1 Daniel Gruen
h2. Consequences
84 2 Daniel Gruen
85 2 Daniel Gruen
If significant instability is seen (more than 1% of bad pixel count changes over commissioning / SV time; more than 10% change in non-linearity of single pixels), that might be a problem.
86 6 Daniel Gruen
87 6 Daniel Gruen
h2. Results
88 6 Daniel Gruen
89 6 Daniel Gruen
We have verified that:
90 6 Daniel Gruen
91 6 Daniel Gruen
* the cold pixel mask is reasonably stable over time (PASSED)
92 6 Daniel Gruen
* the hot pixel mask shows instability; some columns change even in the bias frames between above and below the threshold at a ~ 100 ADU level. This is something the bad pixel masking is not designed for yet and potentially harmful (*NOT PASS*)
93 6 Daniel Gruen
* there are additional defects of two adjacent rows, one with a several per-cent excess flux and one with a similar flux decrement (see also DES-WL issue 3500) near the edges of many/all chips, dubbed _funky columns_. They do not appear in the bias, flat or dark frames and are therefore currently not masked or corrected for (*NOT PASS*). Their no-show in these calibration frames means that they are flux-dependent (not exposure time dependent, not constant) defects.
94 6 Daniel Gruen
95 7 Daniel Gruen
The upshot therefore is that current bad pixel masking does not catch all present cases of bad pixels, and we need to find software solutions to that. Therefore this requirement will pass once the hot pixel variability and funky columns are accounted for and it is verified, that no other significant bad pixel problems exist.