FPACKing the Weight Maps Corrupts Coadd Modeling

-Eli Rykoff

UPDATE: The following is a mis-diagnosis of the problem. See FPACKing exonerated as Bugs in swarp and ImageProc Corrupts Modeling for updates as to the true problem!

I have done a deep-dive into the pipeline to figure out where things are going wrong that lead to the crazy DETMODEL zeropoints (and perhaps other issues).

As I explain below, the problem is isolated to the fpacking/funpacking of the weight maps in between the remapping of finalcut images and the coadding of these remapped images.

The Problem

Please see Y1C1 Stellar and Galaxy Modeling Issues and Y1C2 Stellar and Galaxy Modeling Issues for details. The basic problem is that on both the Y1C1 and Y1C2 coadds the DETMODEL zeropoints (relative to aperture magnitudes) are all over the place. What's up with that?

Note that although the DETMODEL zeropoint shifts (sometimes over 10%) are the most obvious manifestation of this issue, I assume that there may be other problems lurking at a lower level.

Test Data

I have downloaded the individual finalcut images for the 6 fields used in the non-homogenized coadd test: DES0410-4957, DES0414-5748, DES0432-5540, DES0441-5957, DES0502-5248, and DES0522-5705. The constituent images were downloaded via the following probably-could-be-simplified database queries (for all g,r,i,z bands):

select distinct(Y.IMAGEID_g) from Y1C2_COADD_PRERELEASE Y where Y.tilename = 'DES0410-4957'

and for each of these imageids:

  I1.ID = S.SRC_IMAGEID and I2.ID = I1.PARENTID and F.ID = I2.PARENTID and C.ID = imageid

In addition, I downloaded the remapped finalcut images for these fields:

  I1.ID = S.SRC_IMAGEID and F.ID = I1.PARENTID and C.ID = imageid

The Tests

I have hacked my own simulacrum of the DES pipeline as a few python scripts that glue together the relevant bits of code. For a given list of input files, it does the following. Note that I do not run the psf homogenization code.

  1. Redo astrometric solution for each CCD using scamp. (optional)
  2. Resample images to coadd plane ("remap") using swarp (RESAMPLE=Y, COMBINE=N).
  3. fpack/funpack the image and/or weight maps. (optional)
  4. Coadd resampled images using swarp (RESAMPLE=N, COMBINE=Y)
  5. Run psfex on coadded images
  6. Run SExtractor using the i-band image as the detection image, using the Sersic galaxy model. (Using a relatively high threshold so that things process more quickly)
  7. Compute the zeropoint offset for DETMODEL colors relative to large aperture colors for isolated stars and galaxies.

When things are working properly, the scatter in the DETMODEL - APER colors is roughly 0.007 mag (for g-r;r-i;i-z colors for all 6 fields.)

I ran several of the tests with both swarp 2.34.0 and 2.35.1 (with the WEIGHT_THRESH bug fixed).

Results - Part 1

Things that make the DETMODEL colors fail:

  1. Using the original Y1C2 coadds
  2. Using the recomputed Y1C2 coadds with no psf homogenization. (So it's not this step!)
  3. Using the remapped finalcut images. (A-ha!)

Things that result in successful DETMODEL colors:

  1. Using the finalcut images without redoing the astrometric solution.
  2. Using the finalcut images with redoing the astrometric solution.
  3. Using the finalcut images, coadding with either version of swarp.

Results - Part 2

The one additional step that is done in the pipeline -- though not recorded in the logs as best as I can find! -- is the fpacking/funpacking of the images and weight maps. Once I had eliminated all other possibilities this is the only one that remained. Furthermore, this was a possible suspect because of the known problem that fpacking weight maps will turn all the 0s into very small numbers with +/-1e-8 or so.

And what I can do to make DETMODEL colors fail:

  1. Run fpack/funpack on the weight map only in between the remapping and coaddition steps.
  2. Running fpack/funpack on the weight map only in between the remapping and coaddition steps and adding WEIGHT_THRESH 0.0005 (using swarp 2.35.1) [I note that this threshold is too high; however, having no threshold and replacing the 0s also triggers the bug.]
  3. Running fpack/funpack on the weight map only in between the remapping and coaddition steps while directly modifying the fits files to be 0s where they were originally 0s.

It really looks like the fpacking of the weight map is the culprit that triggers this bug (and presumably others).

Additional Thoughts

I first note that the Y1C2 coadds were constructed from the same remapped finalcut images as the Y1C1 coadds, only with different calibration. And so we used the same (corrupted) remapped weight maps in the coadds triggering the same problem.

But what I do not have right now is any plausible mechanism for transmission of this bug. In particular:

  1. Why do just DETMODEL zeropoints in the coadds show up with 10% shifts?
  2. Are the single epoch images corrupted too? We need a set of unfpacked finalcut processed images to test this.
  3. What in the fpack procedure on the weight maps is causing these problems? In some tests that I've done the noise introduced seems reasonable and reasonably small.