Dual image catalog association » History » Version 31
Dual image object association¶
work by Eric Suchyta, Eric Huff, Peter Melchior
In Balrog tests, we found a population of objects, for which the simulated and recovered properties are strongly discrepant, e.g. a simulated galaxy with true magnitude 26 that is measured at mag 22. Using the tile viewer, we could confirm that the brighter object actually does exists. However, the positional offsets between the input coordinates of the mock object and those measured by SExctractor had excursions up to dozens of arcmin. This behavior persists in catalogs even when blends are excluded (flags = 0). So the problem is that distinct objects are associated.
For the coadd catalogs, SExtractor runs in dual image mode, using the riz coadd as detection image. This is different from using the association mode, which requires a reference catalog and performs objects matching in catalog space. Instead, dual image mode simply performs the measurements in the analysis image on the set of pixels (for each object) found in the detection image, basically a sort of forced photometry with a fixed aperture. To be abundantly clear: In dual image mode, no cutoff criterion for object association is considered (such as the default 2 pixels centroid offsets), meaning that potentially very distant objects end up being associated with each other.
Here are the plots for positional offsets between g and r, r and i, r and z from DESDM...
Histograms of differences between g- and i-band positions in Y1A1_coadd_objects table. Catalogs are cut to remove anything with flags_g >0 and flags_i > 0. All three histograms are normalized to integrate to unity. The cumulative distribution, right, shows that ~5% of the Y1A1 coadd objects have astrometric g-i offsets greater than 1".
The figures below provide evidence of the effects on the photometry.
Discontinuous segmentation maps¶
But how can such associations be made? A very old bug in SExtractor results in the presence of disjoint pieces of the segmentation map that are assigned to one object. For instance, the red object in the segmentation map has the same ID 234200 for the big object and the tiny speck below it:
riz detection image from the v4 SV pointed cluster coadd of Bullet, which employed a very setup very similar to DESDM
But since they are obviously not the same object, the dual image mode will force the analysis to utilize pixels that actually belong to two independent objects. This will have undesired consequences for photometry (among other parameters), effectively introducing additional scatter from merging two independent objects. The piece of code in SExtractor that causes the merging is in the clean function. Particularly line 121 in clean.c of the v2.18.10 code base states that the spatial distance for merging is set by 10 times the sum of the semi-major axes of both objects, which can be a very large reach for a big galaxy. Since such galaxies are rare, large offsets should be rare.
So, how often does that happen?
The fraction of objects which at least 2 disjoint groups of pixels in the segmentation map is about 10%, some of them have dozens and hundreds of fragments. These extreme cases are all very bright objects that get split into chunks (probably because the local background is elevated), and these junks are often just one pixel large. The weird thing is: 1-pixel objects should not exists due to the MINAREA requirement of several pixels above threshold. Anyway, merging such pixels back with the parent objects doesn't do any harm since they most likely came from the wings of that object anyway.
It gets more problematic for the cases with only a few fragments because they often come from distinct objects (such as the example above with 2 fragments).
Colors vs positional offsets¶
Regardless of mechanism, the cross-band identification failures indicated by these large positional offsets have significant effects on DES photometry. The plots below show correlations between astrometric offset and magnitude measurements. We chose to compare g- and i-band photometry because g is not included in the riz detection image coadd. For all of the plots below, we have used the entire SVA1 coadd object table, excluded only things with SExtractor flags > 0 in the g or i bands.
Joint distribution of g-i position differences in SVA1_coadd_objects table and measured galaxy g- and i-band model, detmodel, and auto magnitudes. Vertical red lines are drawn at 1" offsets. Similar effects appear in the Y1A1 table.
Joint distribution of g-i position differences in SVA1_coadd_objects table and measured galaxy g-i colors, measured using mag_auto. Red line shows the median g-i color for this sample. Note especially the flare outwards in the envelope of the full distribution once astrometric offsets approach 1". Similar effects appear in the Y1A1 table.
Finally, we show here a map of the mean spatial distribution of g-i astrometric offsets in cells, for everything in SVA1_COADD_OBJECTS; the color scale is log10 (Euclidean, L2 norm) g and i separation on the survey footprint.
Examples of 10 arcmin offsets in the coadd catalogs¶
We thought it might be instructive to look at a number of obvious failures by selecting objects whose g and i band centroids (ALPHAWIN/DELTAWIN) are offset by about 10 arcmin (not a typo: it's arcmin).
There's a tiny speck in the z-band at that location that may have been detected, but how SExtractor could have associated these two is really difficult to comprehend.
If we continue to work under the assumption that this is caused by the "disjoint segmentation map bug" then this position actually would only accidentally correspond to an object. Instead, the other fragment from the segmap would move the centroid somewhere along the line between the parent object and the merged fragment. However, this would mean that the fragment needs to be ~20 arcmin away.
I've also seen cases, where the two locations correspond to two real, but certainly distinct objects. Again, without knowing what SExtractor detected, it is not certain what to make of such cases, they could be entirely accidental.There are a few important aspects here:
- It could be that these huge offsets correspond to marginally detected objects only, where SExtractor found an initial set of pixels in the detection image, but in one of several of the analysis images the objects is too faint for a centroid measurement. So, the windowed centroids end up anywhere.
- If the flux is determined only from the pixels found in the detection image without considering an updated centroid (this is how AUTO and DETMODEL fluxes should do it), then this is basically forced photometry, and no real problem occurs. Fluxes are OK, only single-filter positions would be off. If, on the other hand the flux is measured at the new (measured) centroid, then things will get nasty. MODEL magnitudes could be affected by this problem, but we don't know what centroids are used for each of the filters (they are not stored in the DB).
- Given that we detection image is an riz coadd, any single filter (even r,i,z) might have a drop-out at the location of the riz-detected object. This means that the coadd table should store the centroid of the riz image as the "official" coordinates of the object to maintain consistency. We do not think this is done because there is no detection catalog stored anywhere. Instead, Eli mentioned that the final centroid coordinate could come from the i-band image only.
Eli asked a good question, which is whether position offsets we're seeing are actually affecting the magnitude measurements. To test this, I cut to objects with g- and i-band magnitudes both brighter than 22.5 (none of these should be marginal detections). The plot below shows the joint magnitude-offset distribution of objects in this category.
Note that the tail to very high offsets persists.
Eric Suchyta's taken a closer look at Balrog catalogs run on SVA1 data; note that these are catalogs generated using the same SExtractor config files, in the same mode, and with the same images, as the DES SVA1 coadd data. With Balrog, we know the true Sersic parameters for each object, so we can see whether position offsets are always associated with magnitude errors.