Image Quality Diagnostics

If you don't see the inline pdf figures on this page, you might need to download them via the links at the bottom of the page.

Image Quality Catalog

For our image quality studies we have combined information from different sources such as the telemetry database, the donut analysis and DES-DM FirstCut results into a catalog.This catalog and more detailed information is available here.

Image Quality Analysis

Using the image quality catalog be developed a number of tests to show the performance of the telescope and camera systems. The first such study was performed before the January 2013 AAS meeting.
A second study was performed in March using SV exposures from February, the last month of the extended SV period. We are also using our online tools to for continuous monitoring during the non-DES period.

IQ Monitoring during Community Time

During the February engineering time some problems with the DEC servo were fixed leading to further improvements in image quality. The first set of plots shows ellipticity, r50 and seeing (FWHM) for exposures taken from March 1 to March 10. The last plot shows a comparison of the whiskers from February and March (1-10) exposures.

Ellipticity (March 1-10 exposures) r50 (March 1-10 exposures) seeing (March 1-10 exposures)
Comparison of whiskers from Feb. and Mar. exposures

March 2013 Image Quality Study

This study is based on the iq_feb_sv.dat catalog.

The bottom line

(to be completed after the study is done...)

Overview Plots

A series of plots showing the flux radius, whisker, Seeing (FWHM derived from r50 using Jiangang's parameterization) and results from the new Image Health algorithms. Only those exposures with FirstCut information (selected by r50>-1000 since -99999 indicates missing information) are used for this section. In particular all filter, sky positions and programs (DES, Supernova and Cosmos) are included.

r50 (First Cut)
r50 r50 for all filters r50 for all regions r50 for all programs
Ellipticity (Image Health)
Ellipticity Ellipticity for all filters Ellipticity for all (dec) regions Ellipticity for all programs
Seeing (Image Health, derived from r50)
IH Seeing (FWHM)

First Cut vs. ImageHealth Comparison

These plots compare r50 and whiskers from IH and First Cut. The match is much improved when IH finds a larger number of stars (nstars)

Guider Jumps and Guider Whiskers

These plots can be directly compared to the January study. Guider whiskers (guider second moment gxx - guider second moment gyy) are much improved. It seems that we still see a few guider jumps. It might be worthwhile to study this some more.

The plots are the guider jump plot, the whiskers vs. the auto correlation variable, the guider whiskers per region and the image health whiskers per region.
Guider Jumps Guider Whiskers vs Auto-Correlation (RA) Guider whiskers for all regions IH Whiskers for all regions

Mirror Temperature and Dome Seeing

In the January study we noticed that the seeing gets worse when the mirror is warmer than the dome. We still see this trend but since the mirror day time cooling system was turned on this has become much less frequent. Most of the time the mirror is now cooler than the dome.
Dome and Mirror Temperatures Dome seeing effects

January 2013 Image Quality Study

The bottom line

Putting the bottom line at the top, the IQ data for 8-24 December suggest that:
  • The large tracking jumps were eliminated by replacement of the MAMAC on the primary mirror.
  • Tracking is still causing significant degradation to 10's of percent of the images north of zenith. Only ~2% of images S of zenith show large RA extension in the tracking errors, and the effect on the remainder of images appears subdominant to alignment issues. The incidence of tracking problems shows no clear time dependence over the month.
  • All images show substantial mean PSF asymmetry (whisker 0.15-0.25") that is not attributable to RA tracking, and may be indicative of ~2 mm hexapod displacements or equivalent WFE in the primary mirror. The whiskers are in excess of requirement, but are coherent in time, dependent on telescope position, suggesting they are correctible. The filter dependence is puzzling.
  • PSF whiskers vary across the focal plane much more than expected. This may be another manifestation of misalignment at the mm level, and another inducement to work on closing the hexapod loop.
  • PSF sizes are larger than survey requirement of median FWHM=0.90 in all bands (this is only a requirement for riz bands), even when tracking faults and northern exposures are excluded. Filter dependence of PSF size is much larger than predicted, although on 27 Dec a series of exposures in good conditions did indicate the expected (weak) wavelength dependence.
  • There are clear indications that mirror seeing is affecting ~30% of images for the month, an indication that with good thermal management, we would be meeting survey FWHM goals in z band and close in i band. r band median FWHM is so far ~0.12" larger than the desired goal even after cuts on dec and mirror temperatures.

Tracking jumps

This plot shows the (x,y) values of the largest jump on every exposure.
The plume to the upper right shows large jumps with a particular orientation, probably the MAMAC primary mirror support issue. If I select all events with (jumpx + jumpy > 1.5") above the blue line, and then see how often they occur on each night, the result is:
The MAMAC was replaced on 17 December, at which point the plot shows these events stopping.

The events below the line are evenly distributed on dates. Note there are a lot of events with large jumps, e.g. about 10-15 per hour with jumpx+jumpy > 0.7 arcsec.

Tracking oscillations

The best indicator I've found (GB 3 Jan) is the asymmetry in the guider traces, gxx-gyy, which is also known as guider whisker 1 = gw1. If all guide star motion were due to seeing and noise we would expect symmetry in x and y, so this quantity should have mean of zero and a symmetric distribution. Here is the histogram of gw1, split into 3 declination zones: N of -20, -20>dec>-40, S of -40. Yellow stars mark the medians of each distribution.
The median of these three distributions and the fraction of exposures with clearly bad tracking are:

North   median = -0.00977  % below -0.05 :  13.2
Zenith  median = -0.00468  % below -0.05 :  7.8
South   median = -0.00318  % below -0.05 :  1.9

Notice that gxx-gyy=0.05 arcsec^2 requires that there is tracking error in y (RA) that has RMS value of >0.22 arcsec. In the N, ~1/7th images have problems at this level and the negative shift of the red histogram indicates that there is significant contribution from tracking errors even for the remainder. In the South, the histogram is skewed negative a bit, so there are still tracking issues present at some level.

The outliers in gw1 are pretty well correlated with exposures having autocory values that are large, suggesting oscillations, but autocor does not seem to be a better discriminant nor have a bigger declination dependence. Here is the fraction of the exposure time in the North on each night that has |gw1|>0.05, indicative of oscillations or other tracking problems. Exposures with jumps > 1.5" are excluded. Only nights with >=300s of exposures N of -20 are plotted. There is no obvious trend.

The plot below shows gw1 (gxx-gyy) versus autocory for all exposures, Alistair's good sample and exposures known to have oscillations (selected based on logbook entries: 161518 - 161530, 549, 550, 551)). This shows that these two variables are good discriminators to remove exposures with this type of tracking problem. It also demonstrates that Alistair's sample is significantly better than the bulk of the exposures.
Guider Whiskers (gxx-gyy) vs. Autocor y (RA) for different samples

Mean PSF whisker

South of zenith, the PSFs are substantially more asymmetric than the guider logs indicate should be attributable to tracking errors, with a typical mean whisker on the FOV of up to 0.15-0.25". This mean is the vectorial average of the 61 functioning CCDs. There is a requirement that whisker by <0.2" in the riz images, so this mean violates the requirement on many images even before we consider the impact of the whisker variation across the FOV. Here is the distribution of whiskers for the 5 filters. In this and other plots, data are restricted to guider jump < 1.5", |gw1|<0.05 to filter out the more severe tracking errors.
The mean whisker shows a perplexing dependence on filter. It also depends on telescope pointing. Here is the same plot restricted to airmass<1.2, dec<-40, with the HA encoded in the point color:
There seems to be some repeatability of this mean whisker with position in the sky in the bluer filters where it is more variable. This diagram plots direction & size of the whisker vs telescope pointing for all of the r-band exposures, with the color coding denoting the day of the month for each exposure. The whiskers are highly coherent on a given night and location in the sky, and exhibit some coherence at a given telescope location on different nights.

  • The mean whisker is not oriented along the RA direction so it is likely that in the S, the tracking errors are not dominating the PSF elongation.
  • The whiskers are coherent within a night, somewhat so across weeks, and dependent on telescope orientation, suggesting that flexure of the primary mirror and/or collimation errors are responsible. Jiangang reports that the optics model requires ~2 mm of hexapod displacement to cause mean whisker length of ~0.2" as seen here.
  • Yet there is also a filter dependence, so it's not just primary mirror figure errors - maybe it's astigmatism combined with different focus positions on each filter? I am a little bit stumped (GB) but feel that the alignment and primary mirror figure must be the leading suspects. The optics models suggest that this level of mean whisker should be accompanied by ~10% degradation of the R50 as well.

PSF whisker variation

This is the histogram of the RMS variation of the whisker across the FOV relative to the mean whisker of each exposure. grizY filters from top to bottom; the short black ticks mark the RMS whisker expected for perfect optical alignment in griz bands. The observed RMS are several times larger than expected for good alignment. Perhaps this is another consequence of the misalignment or primary-mirror figure errors that are indicated by the mean PSF whiskers.

PSF size

Here is the histogram of PSF R50 in each of the 5 filters, again grizY from top to bottom. The survey requirement is to achieve median 0.90" FWHM in riz bands, which corresponds to R50=0.52" as marked by the red vertical line here. The actual median of each band is marked with a black vertical tick. They are well above the requirements in all bands. In this plot, the guider-log cuts have been made, and they are further restricted to dec<-40, airmass<1.4 to avoid poor tracking and the poorest seeing. (a stricter airmass cut makes no difference). We can also ask what fraction of each band's images are below the targeted median:

g N= 315  median R50  0.667  pseudoFWHM  1.20"  % below 0.9  0.0
r N= 253  median R50  0.611  pseudoFWHM  1.08"  % below 0.9  2.8
i N= 148  median R50  0.575  pseudoFWHM  1.01"  % below 0.9  14.2
z N= 139  median R50  0.542  pseudoFWHM  0.94"  % below 0.9  33.8
Y N= 145  median R50  0.546  pseudoFWHM  0.95"  % below 0.9  33.8

One way we can interpret this is to ask what fraction of the exposures would have to be discarded to attain a median of R50=0.52" (FWHM=0.90"). The answer is ~1/3 of the images in z and Y bands, and most of them in r and i bands. The requirement does not apply to g band, where none of the exposures reach 0.9" FWHM - although Alistair obtained an excellent series of images in all filters on 27 Dec.

Here are some histograms showing the PSF r50 distribution for the SPT-E field (exposures taken between December and January 31) for the g, r, i, z, and Y filters.
PSF size for SPT-E exposures from December until January 31, 2013

Mirror temperature: Aaron has noted that images with the mirror temperature msurftemp warmer than the dome temperature domelow have markedly worse seeing. Here are seeing histograms for each filter, with this mirror warmer (red) and cooler (blue) than the dome. We reproduce in all bands the effect of mirror seeing. The stars mark the medians of the warm/cold mirror seeing distributions. The statistics are now

g Nwarm= 106 median R50  0.694  pseudoFWHM  1.26  % below 0.9  0.0
r Nwarm= 106 median R50  0.640  pseudoFWHM  1.14  % below 0.9  0.9
i Nwarm= 60  median R50  0.620  pseudoFWHM  1.10  % below 0.9  8.3
z Nwarm= 42  median R50  0.628  pseudoFWHM  1.12  % below 0.9  11.9
Y Nwarm= 59  median R50  0.600  pseudoFWHM  1.06  % below 0.9  10.2

g Ncold= 208 median R50  0.651  pseudoFWHM  1.17  % below 0.9  0.0
r Ncold= 147 median R50  0.581  pseudoFWHM  1.02  % below 0.9  4.1
i Ncold= 88  median R50  0.563  pseudoFWHM  0.98  % below 0.9  18.2
z Ncold= 92  median R50  0.527  pseudoFWHM  0.91  % below 0.9  43.5
Y Ncold= 86  median R50  0.520  pseudoFWHM  0.89  % below 0.9  50.0

In z and Y bands, the cold-mirror frames achieve the survey requirements, but r and i still exceed the requirement for the nights so far. But this test shows that thermal management should have a substantial impact on the IQ.

This figure summarizes the distribution of R50 in the rizY bands as we make various cuts that lower the median R50 (marked by the stars) from 0.59" to 0.535", the latter corresponding to 0.93" FWHM median seeing. In order, these cuts are to:
  • eliminate r band and look at just izY
  • cut 2% of exposures with large jumps and asymmetries or oscillations in the guider records
  • cut all exposures north of -40 dec
  • cut exposures at airmass > 1.5
  • cut exposures with mirror warmer than the domelow reading.
    The final histogram is approaching what we require, but contains only 1/3 of the exposures we started with and omits all the r-band data and includes Y. It is encouraging, however:
  • The tracking jumps are largely gone already.
  • Improved tracking would make these results applicable to all exposures, not just the southern ones
  • Re-commissioning the daytime mirror cooling and implementing prime-focus thermal mgmt should lead to much improved median IQ.
  • The whisker data indicate significant aberrations from decenter or primary-figure errors, which may be having disproportionate impact on the bluer filters, but which should be amenable to improvement.