Results
This pages contains the results of various tests we have performed to determine the reliability and accuracy of the results output by the LSS. Most of these tests are based on the analysis of simulated images as they allow us to compare the outputs against known inputs.
One of the primary goals of GOTO is to accurately measure absolute and differential photometry. As such, a key test for the LSS is to assess how reliably and accurately it can measure source photometry. The LSS provides a number of methods to measure photometry. These include:
- Aperture photometry in a range of circular apertures;
- PSF photometry using the PSF characterised (by the LSS) from selected point sources distributed around the image;
- Kron photometry;
- Gaussian fit photometry, in which a 2D-Gaussian is fit to the source;
- CModel photometry, in which a linear combination of a deVaucouliers and exponential light profile is fit to the source;
While the first two (aperture and PSF) work well for point sources, they typically fail to give reliable results for extended objects. The latter three (Kron, Gaussian, and CModel) have been developed to overcome this problem.
The following plots show the input vs. output magnitudes obtain by running LSS on the simulated image: GOTO_01_20170525_0001_01.fits using v13.0 of the stack. These are the output photometry obtained by running singleFrameDriver.py with commit 7c8419b of obs_goto.
First, aperture photometry using a 12 pixel diameter (i.e., ~15 arcsec diameter) aperture:
There's clearly something odd going on around 16th mag; we've not determined what's going on here. It's possible that for bright sources, some of the flux extends outside the aperture, leading to flux loss, but I don't know why this would like to such an abrupt break. We can investigate this by considering the aperture corrections that the LSS spits out. Another possibility is that those are saturated sources; we can check this using the flags that are output by the stack.
Next, PSF photometry:
This works well for stars (blue points in all plots), but a lot less well for galaxies (red points). Of course, this is somewhat to be expected. Again, there seems to be a problem with sources brighter than (input) 16th mag. It's hard to tell from this image, but it seems that not all of these brightest sources have a measured PSF mag in the LSS output.
Finally (for now) CModel:
We think this produces impressively good magnitudes for both stars and galaxies, with only a handful of outliers at (input) magnitudes fainter than about 16.5 mags. Most of these outliers are galaxies, and for most of them the LSS measures brighter sources than the input. Again, we have the problem of spurious measured magnitudes for sources brighter than 16th mag.
In all of the above cases, there is a systematic offset between the input and measured mags. We need to investigate this further. One possibility is incorrect colour terms in our config files, which would be fairly straightfoward to check and change.
We are currently working on getting the LSS working with real data. On doing so, we'll be able to run the same tests comparing measured mags against archival data.
As of August, 2017, we've started to process and analyse data from the real images. The following is for the image r0002279_UT1, taken on the 26th July, 2017. We queried the SDSS for sources brighter than 21st mag (g-band) within the region covered by the GOTO observation. We then matched GOTO sources to SDSS sources to within 1 arcsec (see note below) and plotted the resulting GOTO L-band PSF photometry vs. SDSS g-band photometry (note to self: we need to check whether SDSS is PSF or aperture photometry; update: SDSS photometry is model mags, but we get similar results with SDSS PSF photometry):
There's clearly an offset (colour correction?), but we're getting a promising one-to-one correlation. We're aware that the scatter is quite large (around 1 mag) and we're working to improve this. We're not sure how much of this scatter is due to different colour corrections for different stellar types. Further, the standards used for photometric calibration are in g-band, so their colours will also be another source of uncertainty.
In the meantime, we can determine the accuracy of the photometry independently of colour corrections by comparing GOTO observations of the same patch of sky on two different occasions. In the following figures, we compare L-band PSF photometry from r0002279_UT1 against that from r0002280_UT1 (both taken on 2017-07-26). We use a simple 1-arcsec matching radius to match sources, which results in 6495 of the 10460 sources in r0002279_UT1 being matched (the low matching fraction is due to the elongation of the sources in the outer regions, which causes the stack to detect two separate sources; all the 1-arcsec matches are within the central few degrees). PSF photometry vs. PSF photometry plot below:
As you can see, a lot of the scatter that arises from comparing to SDSS has gone, which is consistent with (but not yet confirmation of) a lot of the scatter in the former being due to unknown colour corrections. To give a better sense of the size of the photometric uncertainties, we plot the histogram of magnitude differences between the two frames below:
We find that 80.3% of sources lie within +/- 0.2 mags.
With the current poor focus toward the edges of the image, we decided to investigate whether the scatter in the measured mags between two observations differs as a function of position of source. For this, we compare sources within the central region (shown as blue points in the following image) against those in the outskirts (red points in the following image):
The corresponding PSF mag vs. PSF mag plot for those (colour-coded as above) is shown below:
It's clear from this that there is far more scatter among sources in the outskirts of the image compared to those in the centre. We'll try to see whether this can be improved via the software, although I suspect that the only was we'll be able to improve this significantly is via a hardware fix.
Update 13 October, 2017
Following the success we had focussing on the central regions, we've made a few adjustments -- specifically on detection and deblending -- to see whether things can be improved further. The results are shown below for LSST PSF photometry (green), LSST aperture photometry (blue) and gotopho (red):
The scatter around the 1 to 1 line appear to be the same for all three at brighter mags. LSST PSF photometry has the tightest relation at mags fainter than about 16. It's not clear in this plot, but the gotophot points stop at around 18th mag, whereas the LSST stack measures to about 20th mag (albeit with quite large scatter); is this due to a high sigma cut in gotophot? We used a 5-sigma cut in the LSS. Finally, although not shown in this plot, the LSS is now measuring mags for far brighter sources than previously; down to around 9.5 mags.
One of the most effective means of identifying variable and transient sources in imaging data is via the technique of Difference Imaging. This involves subtracting a reference image from the image in which one is attempting to identify variable or transient sources. The LSS has a difference imaging module that aligns, PSF-convolves and flux-matches the reference image, subtracts it, and performs positive and negative source detection and characterisation.
While not really a test, I thought it may be insightful to give an example of a difference image produced by the LSS. The following shows a simulated image (GOTO_01_20170803_0001.fits) on the left and the resulting difference image (produced by subtracting GOTO_01_20170803_0001.fits, which has the same PSF, but shifted astrometrically) on the right. For the record, the latter was produced using commit 7c8419b of obs_goto. Green and red regions indicate the positions of variable and transient sources, respectively.
Drag and drop the image to your machine for a larger view.