Agenda And Minutes for 1/22 10am CST¶
1. Dial Toll-Free Number: 866-740-1260 (U.S. & Canada)
2. International participants dial:
Toll Number: 303-248-0285
Or International Toll-Free Number: http://www.readytalk.com/intl
3. Enter your 7-digit access code, followed by “#”
Your access code: 7691719
- Photo-z training plans, and calibration challenges
- Star/galaxy separation challenge updates:
- Using SN C3 for seeing variations - Eli
- TMVA Boosted Decision Trees - Alex
- Thoughts on the first round and plans for the second.
- Catalog validation tests:
1. Huan and Eli discussed the photo-z training. For photo-z training and validation, we need a set of galaxies that has the same calibration as Gold and representative errors. However, the photo-z training fields are deeper than full depth (SN, Cosmos), and the VVDS-14h field is out of the SVA1 footprint. Back in Spring 2013, Huan made full-depth-equivalent coadds . Keith has run his SLR on these fields for Gold appropriate photo-z training.
However, there are some issues. First, there are 5% offsets in the total magnitudes between Gold and Huan's recalibrated mags (though colors seem fine). The origin of this offset is still under investigation. Huan has also noticed a different magnitude (not color) offset in COSMOS relative to SDSS DR8 compared to the other fields with DR8 overlap. This is most likely caused by the trouble getting a good set of 2MASS stars in COSMOS for absolute calibration (due to saturation in the long exposures with good seeing.)
The good news is that in Huan's tests (griz, no Y) the photo-z performance is basically equivalent (small changes in scatter) when training on any one catalog (Huan's galaxy locus; recalibrated SLR; Gold 1.0). The current plan is that Huan will continue with the SLR recalibrated coadds, griz only. (He wants to make another couple of tests). COSMOS will not be used for training because it's too deep, though the full deep catalogs will be used for testing.
Finally, when a couple more tests are complete Huan will make the recalibrated training files (with spec-zs) available for all the photo-z code owners to run on the full SVA1.
2. Nacho gave a good update on the star/galaxy separation challenge . Per Alex's suggestion, the FOM has been changed to a plot of True Positive vs. False Positive rate. So far only a couple of entries have been submitted, with more on the way. The Modest classifier is performing surprisingly well.
We discussed how best to test the s/g separation over the full SVA1. LSS plans to measure cross-correlation functions between the stars and seeing, airmass, galaxies, etc to correct for this. The cross-correlations can also be used to compare different s/g separators, and we will investigate this. In particular, the stellar density map should not be correlated with the seeing map in an ideal scenario. Eli guesses that something like CLASS_STAR will be highly correlated with the seeing map, and this will allow us to test if any s/g separator suffers from overtraining (or overly ideal training.)
Alex also described a clever technique of looking at stellar densities in rich cluster fields (see above).
We also discussed Round 2, with perhaps more fields. The problem with the spectroscopic star fields is that most of these are (a) sparse and (b) strongly biased, due to the selection effects involved. (These stars were originally confused with galaxies or quasars or they wouldn't have been targeted in the first place.) The exception is the VVDS-DEEP spectra (in SN X3) which did not have a star/gal selection applied a priori. There is also a 'Spectra of Everything' (shallow) program in SDSS Stripe 82 we can use in the future, and Alex pointed out the CANDELS UDS field has 40 ACS fields with HST imaging that may be useful.
Finally, we discussed what validation should be done for an SVA1 Gold paper. The sense of the group was that this could easily snowball out of control. Diego suggested we concentrate on four main topics, eg: star/galaxy separation; depth; photo-z quality (based on the photoz paper); and mask maps. Discussion here will be ongoing.
Scott also asked when the Gold 1.0 catalog will be "frozen" for the paper. At the moment, there have been no dealbreaker complaints that require an update. However, many people have not used the catalog yet because the photo-zs are not available yet. Once these are available, if there are no complaints about the underlying galaxy catalog, Eli expects that there won't be any changes, only additions (esp. depth maps when the mangle masks are complete.)