Is something going wrong with errors on MAG AUTO¶
- Diego Capozzi
Following up on the discussion during the last telecon (28/04/14), I have investigated the issue of significant depth differences between detmodel and auto magnitudes.
The initial story¶
I was testing the selection of homogeneous-depth regions using Eli's depth maps. When selecting areas with a given homogeneous 10-sigma depth using different magnitudes (detmodel, auto and aper4), I noticed that the selected regions in the auto magnitude case was significantly smaller than for the remaining two cases. In particular, for an i-band depth of 23, the area selected in the auto magnitude map case is ~1/2 (~97 sq. deg) of the one found for detmodel (~180 sq. deg) and aper4 (~200 sq. deg) magnitude maps. So I became suspicious, as I would have thought that, despite detmodel being an incorrect (i.e., yielding a total magnitude value different from the real one) measure of galaxy total magnitudes, it still is a measure of the integrated light of a galaxy and as such it should give us values more similar to AUTO magnitudes rather than Aper4 ones. So by reasoning in this way I expected that detmodel and auto magnitude depths were similar and so to identify similar areas for a given depth.
The depth maps used are not the latest ones announced by Eli during the telecon. However, the issue identified here and studied below is independent of the maps. In addition, my understanding is that the new maps should be almost identical to the previous ones, but Eli can correct me on this, if need be.
What I have done to investigate this issue¶
1) I checked the definition of detmodel magnitude to understand whether my understanding of what it measures was correct and whether what it measures would fit well into my way of reasoning. I have asked the help of Emmanuel Bertin in order to be sure of the definition of detmodel magnitude.
- Question 1. How is detmodel measured?
- Answer 1. By integrating a model surface-brightness radial profile fitted to that of an observed galaxy.
After establishing a reference detection image (which can be taken in one band or be a linear/non-linear combination of images taken in 2 or more filters) the fitting is carried out on this image using a model (pure De Vaucouleurs, pure exponential, combination of these 2, pure Sersic and so on) decided beforehand. The flux/magnitude estimation in each single-band image of the galaxy under study is then carried out by fitting the observed surface-brightness radial profile with the chosen model surface-brightness radial profile and integrating it. This process makes Detmodel mags not suitable for correctly estimating the real value of total magnitude of a galaxy but makes it more suitable for colour estimation. However it is still a measure of the total light of a galaxy, so it can be defined as a total/integrated magnitude, despite resulting in an estimate different from the real one.
- Question 2. What does detmodel measure?
- Answer 2. The total light of a galaxy. However this estimate should only be used for colour estimation, especially when the reference detection image is a combination of images taken in 2 or more filters, like in our case for SVA1 data, where the detection image is a combination of r, i and z (thanks Nacho for telling me the right filters combination).
- According to this definition it would seem that my reasoning can be considered as sensible, at least from a statistical point of view, that is: a detmodel magnitude distribution should look more similar to a distribution of a total magnitude measurement (like model, auto magnitudes) than to a distribution of an aperture magnitude estimate (like aper3 and aper4 magnitudes).
2) I investigated whether detmodel- and auto-magnitude distributions are indeed statistically similar and yield a similar depth. In order to do this I selected all galaxies from the entire SVA1 GOLD 1.0.2 sample and investigated their number counts for 3 magnitude types: detmodel, auto and aper4. This should give us an independent test for the survey depth, defined as the magnitude value fainter than which the number counts start dropping. Be aware that when the entire catalogue is used, the identified depth is not the real one as number counts are affected by total-exposure-time spatial variation over the entire SVA1 footprint (for instance, SN fields are included). Results are shown in Figure 1.
- Question 3. Are detmodel and auto magnitude distributions statistically similar?
- Answer 3. Yes. By looking at Figure 1, one can clearly see how detmodel and auto magnitude distributions have almost the exact same shape and peak at the same magnitude value. I didn't do a KS test to properly quantify the similarity of these two distributions as it looked already obvious by eye that they can be considered as drawn form the same parent distribution.
- Question 4. Are detmodel-magnitude and auto-magnitude depths similar?
- Answer 4. Yes they are. Their number counts peak at the same value and start dropping immediately fainter than that. In addition, as expected, aper4 magnitude number counts peak at a fainter value and as a consequence start dropping at fainter values as well. Note that probably the reason why there is not a classically sharp drop at the faint end of these distributions is the spatial total-exposure-time inhomogeneity of the survey.
3) Given the results of point 2), why do we identify a significantly smaller area at a minimum depth of i-band mag=23 when using the auto-magnitude depth map compared to when doing the same on detmodel-magnitude depth map? Do we find the same result if we repeat the study described in point number 2) on galaxy samples at 10-sigma level, i.e. made only of galaxies with MAGERR<0.11? Results of this test are shown in Figure 2. I point out that the appropriate MAGERR is used for a given magnitude type.
- Question 5. Are detmodel and auto magnitude distributions still statistically similar?
- Answer 5. No, they aren't. However they should, so why is this happening? Note that the detmodel and aper4 magnitude distribution barely changed (their peaks slightly moved towards brighter magnitude values, as expected). It's the auto magnitude one that changed significantly.
- Question 6. Are detmodel-magnitude and auto-magnitude depths still similar?
- Answer 6. No, they aren't. Auto-magnitude depth is significantly shallower (~0.5 mag) than the detmodel-magnitude one. This confirms the picture highlighted above when using Eli's depth maps.
4) The suspect culprit of the highlighted problem may be the 10-sigma level cut. So I had a look at the MAGERR distributions, shown in Figure 3.
- Question 7. Is the 10-sigma level cut the problem?
- Answer 7. It does seem so. The auto-magnitude error (MAGERR_AUTO) distribution seems odd, with a strange plateau at values<0.15. While detmodel-magnitude (MAGERR_DETMODEL) and aper4-magnitude (MAGERR_APER4) error distributions look exponential-like, the auto-magnitude one does so only out to MAGERR_AUTO~0.15 and plateaus at smaller values. There seem to be a significant lower number of galaxies with MAGERR<0.11 for the auto magnitude case compared to the other two. Figure 4 also shows that auto-magnitude errors are systematically larger than the detmodel-magnitude ones. Why is this happening? It is true that MAG_AUTO is not specifically optimized for SNR estimation, like MAG_DETMODEL for instance, but do we expect such a significant difference in the error values and in their distribution? Are auto-magnitude errors correctly estimated and their characteristics expected to be different (in this case one should know why and take that into account for data selection) or is there a problem in their calculation (they seem to be over-estimated compared to errors on detmodel and aper4 magnitude)?
- Notice the strange importance of the 0.15 error value in Figure 3. This point signs the change for the auto-magnitude error number counts from being the lowest among the three distributions (at MAGERR<0.15) to being the largest ones (at MAGERR>0.15). At the the same time, this point signals the inversion of the detmodel-magnitude error distribution with respect to the aper4-magnitude error one.
- I would expect Aper4 magnitudes to be more precise measurements than detmodel ones as the former are carried out in a small aperture (2") containing the brightest part of a galaxy, so should correspond to a higher SNR level. However I notice that a significantly large amount of galaxies have smaller detmodel-magnitude errors than aper4-magnitude ones (again at error values<0.15). Is that normal?
5) I have made an additional test on point 4) by using again Eli's auto magnitude depth map. I have selected an area in the current SVA1 footprint (excluding SN fields) with an homogeneous i-band depth of 22.5, so taking into account the ~0.5 mag brightward shift of the auto-magnitude depth compared to the detmodel-magnitude one described above. If the 10-sigma level cut is the problem, as pointed out above, we should get an area close to the 180 sq. deg. one, selected from the detmodel-magnitude depth map with i-band depth=23. In fact, by doing this, the area selected as just described from the auto-magnitude depth map totals ~172 sq. deg., clearly close to the 180 sq. deg. area found for the detmodel case at a depth of 23. This result adds evidence to the possibility of problems in the estimation of the errors on auto magnitudes.
MAGERR_AUTO (errors on auto magnitude) should not be used to estimate the depth, nor to produce depth maps, as their use leads to an underestimation of the galaxy (total magnitude) depth of the survey. In order to identify the total magnitude depth of our galaxy sample one should use errors on other total magnitude estimates, like detmodel magnitude or model magnitude (the latter hasn't been tested here). So, if one is interested in characterising the auto-magnitude depth of our galaxy sample and/or producing an auto-magnitude depth map, one should use auto magnitude values in combination with errors on an alternative total magnitude estimator (e.g. detmodel). So, for Eli's maps, for instance, the plot to be used for an auto-magnitude depth map should not be MAGERR_AUTO vs. MAG_AUTO, but MAGERR_OTHER_TOT_MAG_ESTIMATOR vs. MAG_AUTO (for instance MAGERR_DETMODEL vs. MAG_AUTO). This should lead to a proper estimate of the auto-magnitude depth of the survey.
The results found in this test connect with the ones found in the study of the mangle mask depth (Galaxy depth and completeness using Mangle Mask), where the 10-sigma level cut problem was already identified but not connected with the problems on the estimate of auto-magnitude errors highlighted here!
The question remains whether errors on auto magnitudes are reliable or not and this issue should be investigated.
Comments are very welcome!