ING Banner
Home > Astronomy > Site Quality > Atmospheric Seeing at ING > Calibration of RoboDIMM


Results from the Calibration of ING's RoboDIMM

Neil O'Mahony, updated March 2005


Summary

ING needs accurate and reliable measurements of the seeing to provide useful input into observing programmes, especially those involving Adaptive Optics and Queue Scheduling. RoboDIMM was set up to provide automatic measurements of the seeing with the intention of calibrating these afterwards. The first comparisons with the IAC DIMM, using data from Oct 2002, indicated close agreement between the two, as previously publicized on the 5th floor notice board and the Web. From April 2003, following RoboDIMM technical downtime in Winter 2002/03, data from these two DIMMs were no longer in agreement. RoboDIMM monthly averages were now a factor of 1.4 to 1.6 higher than corrresponding IAC DIMM values.

It was assumed that software changes made to RoboDIMM during downtime were now causing RoboDIMM to overestimate seeing relative to the IAC measurement. Besides this, RoboDIMM measurements below 0.5 arcseconds in the summers of 2003 and 2004 are relatively scarce in comparison with their frequency in the 1995-98 ING DIMM and JOSE campaigns. However, when RoboDIMM data were compared with Richard Wilson's 16" Slodar 'scope on the one hand, and with the IAC Scidar on the JKT on the other, it showed agreement to within 10%. This refers to over a one hour of simultaneous data from August in 0.8 arcseconds seeing. More data, taken simultaneously  with Slodar or Scidar, would be required to clarify the question of RoboDIMM accuracy when seeing is better than about 0.6 arcsec.

Methods

A DIMM monitor is a kind of wavefront sensor, measuring the statistics of the wavefront distortion caused by the atmospheric turbulence. This measurement of distortion is then converted into an estimate of the image size (seeing FWHM) it would correspond to on a large telescope, using the Kolmogorov-Fried model. The accuracy of the calculation depends mostly on the accuracy of the optical parameters of the monitor and the errors in measureing the centroids of the wavefront sensor spots.

The relevant optical parameters include focal length, size and separation of sub- apertures, which are well known, but one parameter needs to be measured on sky: the image scale at the detector. Inaccuracy in optical parameters will lead to a scale error in the seeing estimate, that is the seeing FWHM will be wrong by a fixed factor.

The main sources of error in measuring spot centroids on a CCD image are readout noise, image thresholding and window size. Readout noise and window size tend to increase centroiding error, contributing to centroid variance which systematically increases the seeing estimate, while thresholding tends to decrease it. These sources of error therefore cause a more or less constant additive bias in DIMM seeing FWHM estimates, typically of about 10% of the seeing FWHM, and can be distinguished from scale errors by comparison with another seeing monitor.

To gain confidence in RoboDIMM's accuracy then, it seemed logical to first calibrate image scale and other optical parameters of the instrument, then compare with another monitor in order to detect what  kinds of errors might be present, and finally calibrate the smaller effects of centroiding errors.

Seeing monitors that are not located in the same place or not looking along the same column of atmosphere cannot be expected to measure the same seeing from moment to moment. Only a statistical comparison is feasible, using samples of at least 15 minutes approximately, in stable conditions.

Statistical comparison with another monitor can been carried out at 3 levels: at the most superficial level, comparison between statistics of large samples (median, mean and variance of one month or more); taking more care, one needs to select and comapre statistics of only simultaneous data from each monitor, filtering out hours or even nights when one monitor or the other shows rather variable seeing (which implies local turbulence effects). Finally, if from such a selection, a number of samples in a range of seeing conditions can be obtained, their median values can be plotted in a scatter diagram and the agreement and correlation between the two monitors  can be estimated. This provides more detail on the performance of the monitors, e.g. if one of them shows progressive overestimation as seeing improves etc., but it is always just a comparison between two monitors and the actual causes of inaccuracies need to be investigated in the monitors individually.


Work completed and incomplete

Image scale was measured as recently as June this year and found to agree to within 0.2% with the scale measured over a year previously. The measured value is 0.72 arcseconds per pixel, obtained on each occasion using a binary star of known and fixed separation. It is not expected that image scale can vary during the night, except through thermal expansion of the telescope structure, which would be minimal. The focus of the telescope is automatically controlled throughout the night and no dependence of the seeing measurement on the focus position has yet been evident.

The IAC DIMM monitor has since the beginning been the only seeing monitor available for comparison tests with the RoboDIMM. Data from NAOMI wavefront sensor has so far remained inaccessible within the encoded GP logs recorded on navis. The WHT-Slodar instrument is relatively recent and has not yet been satisfactorily calibrated. Scidar on the Mercator is greatly affected by Dome Seeing there, and needs to be reprocessed. Data from Scidar on the JKT has not been yet made available. Recently, the 15-inch Slodar monitor set up by Richard Wilson outside the JKT has provided a decent sample of data which is useful for comparison with RoboDIMM.

No calibration of centroiding errors has yet been carried out on RoboDIMM.

Comparison with IAC DIMM:

First 3-4 months of data (before December 2002): measured a correlation strength of 92% between IAC DIMM and RoboDIMM over a range of 0.63 arcsec to 1.6 arcsec. Close agreement between nightly medians when seeing is stable (periods of highly variable seeing rejected). The frequency distributions in each month largely coincide and overlap. See Appendix.

Data from 2003 - comparison of monthly averages using only simultaneous data from nights with stable conditions. Also a comparison of frequency distributions for these simultaneous samples, typically 5-10 nights in the same month:

histograms for 2 monitors, Jul-Aug 03
Normalized frequency distributions for 9 night samples in July-August 2003: RoboDIMM (barely noticeable dots - the upper curve) and IAC DIMM (solid - lower curve). The scale of the x-axis is log(FWHM), without which it would be much more difficult to show up the differences between the histograms.


A similar graph for April 2003 shows that the difference between IAC DIMM and RoboDIMM existed as soon as RoboDIMM resumed monitoring after technical downtime in Winter 02/03.

Result: The frequency distributions for each monitor are clearly displaced and do not coincide, RoboDIMM peaking at a larger value than the IAC DIMM. The plot in logarithmic scale shows that the displacement is significant, in fact the peak values differ by a factor consistently in the region of 1.4 to 1.6, depending on the samples compared.

Data from 2004: RoboDIMM still showed a factor 1.5 offset from IAC DIMM in monthly averages.


SLODAR Comparison:


Data from 18th August 2004: Comparison with simultaneous data from Richard Wilson's 16inch Meade SLODAR monitor. A sample of 80 minutes of data shows very good agreement (RoboDIMM just 6% larger in the median) in conditions of 0.8 arcsec onaverage. The IAC Scidar working on the JKT measures 0.75 median for the same period.

On several short-lived occasions the RoboDIMM measurement goes down to about 0.45 arcseconds, showing that the monitro is capable of registering such stable conditions. It would be good to check how often these measurements appear in the Slodar sample.

Comparison with ING DIMM (1994-96):

histograms of 3 years comapred.
Comparison of RoboDIMM sample of summer 2003 (April  - October) with old ING DIMM samples of 1994/95 and 1995/96 (Oct - Sept).

The peak values and the upper half of the distributions overlap, indicating some degree of agreement between the two instruments about the frequency of poorer seeing in the 3 years compared. However, it is clear that seeing better than 0.5 arcsec (log FWHM -0.25) is much less common in the 2003 sample than in 1994-96. The RoboDIMM distribution is not symmetric when plotted in the logarithmic scale, much less so than the old ING DIMM distributions. The shape of the RoboDIMM distributions suggests the monitor may be increasingly insensitive as seeing becomes smaller than the size of one pixel (0.76 arcsec). There is a possibility however that good seeing is genuinely absent in Summer 2003, and there is some anecdotal evidence from the TNG to suggest that seeing in recent years has been worse than previously.

However, to return to the question of whether RoboDIMM accuracy changed some time between December 2003 and April 2003, as comparisons with IAC DIMM would seem to imply. In 2002, RoboDIMM and IAC DIMM are in agreement, but from April 2003 on, the  RoboDIMM average is about 1.5 times greater than that of IAC DIMM. Is this difference visible in the RoboDIMM data from those two periods?  The following graphs shows this comparison:

Sep-Nov RoboDIMM comparison  

In 2002, only data from September, October and November is available. Comparing this with the same period in 2003 yields the above graph, in which it is clear that the distributions largely overlap. What can be said with  confidence from this graph is that there is no large factorial change in RoboDIMM seeing measurements in 2003 (dashed line) with respect to 2002 (solid). The probability that these samples were to coincide by pure chance is very small. We conclude that RoboDIMM shows no change in accuracy following the last technical downtime and software changes.

Conclusions

Following a period of technical downtime (December 2002 - March 2003), RoboDIMM and IAC DIMM are no longer in agreement. Instead the mean seeing from RoboDIMM is about 1.5 times larger than that from IAC DIMM. Nevertheless, there no evidence of such a large change in RoboDIMM data when samples from the same season in 2002 and 2003 are compared. 

The main changes made in software during the winter 02/03 were related to the rejection of elongated images and are not therefore expected to cause a major scale change in the seeing FWHM estimate such as observed relative to the IAC DIMM estimate in 2003.

Further confidence is added by the fact that RoboDIMM seeing FWHM agreed to within 6% of the JKT Scidar measurement on August 18th 2004. Although the simultaneous sample only lasts 01:20 hours, it is unlikely to be a coincidence and is a strong indication that RoboDIMM is providing reliable seeing estimates, at least in the conditions then pertaining, around 0.8 arcsec.

The median seeing on ORM in 2003 was, according to RoboDIMM, about 0.9 arcseconds. This is some 30% worse than the median seeing measured by ING DIMM in 1994-97 and corroborated by the JOSE campaign of 1996-1998. Closer examination of the seeing frequency distribution shows that poorer seeing is equally common in both campaigns whereas seeing better than 0.5 arcseconds is rare in RoboDIMM in 2003 in comparison with the 1994-98 samples. This would account for the median seeing in RoboDIMM being greater than in the old ING DIMM. The peak values, the most commonly measured values, of the two samples actually coincide (at about 0.6 arcsec), suggesting that the seeing on ORM has not significantly worse in 2003 than in the 1994-97 period.
   
While we no longer believe that RoboDIMM has changed significantly since 2002, and have seen that it is in agreement with Slodar and Scidar, there is some evidence that it does not register good seeing as often as expected. Further comparison with Scidar in seeing < 0.6 arcsec will be required to clear up this remaining doubt.

This question might also be clarified by calibrating the centroiding errors in the RoboDIMM software.


Appendix 1:

RoboDIMM and IAC DIMM seeing measurements were in good agreement in 2002. large correlation graph
The IAC DIMM measurement of the seeing FWHM, plotted against the ING RoboDIMM measurement.

The best fit line superimposed in solid black and the large correlation coefficient (0.90) illustrate the close dependence of these two variables. Each of the 9 points represents the median seeing from a single night, using simultaneous samples of several hours length. A continuous period of stable seeing was selected to make up the samples, all of which come from October 2002.

The standard error on these points is too small to appear on the graph. The use of a logarithmic scale is necessary to convert FWHM into an approx.  normally distributed and 'equal intervals' variable (see graph on next page). Median seeing ranges from 0.65 to 1.6 arcseconds. Note log FWHM = 0 is 1.0 arcsecond seeing. The points are scattered around the y = x line (dashed) by amounts varying between 3% and 15% on any given nightly sample, or 8% on average. This is no larger than the internal discrepancies between independent seeing estimates by each instrument.

This is strong evidence that the ING RoboDIMM is providing reliable measurements of the seeing FWHM over the range of seeing involved (0.63 to 1.6 arcsec) and is impressive in view of the large separation between the points of measurement, the different designs of the two instruments and the factor 10 difference in their sampling frequencies. The 'wild' point (close to the -0.1 point of the x-axis) may indicate local seeing effects on the ING RoboDIMM. Such effects were avoided in forming these samples, but this was sometimes difficult. Local effects were also observed in the IAC DIMM measurements.

Appendix 2 (overleaf):


histogram
Log-normal characteristics of RoboDIMM seeing (not really relevant to calibration).

This graph illustrates the approximately normal behaviour of seeing expressed as log(FWHM). Seeing FWHM is expected to be log-normal because it is a random variable that ranges from 0 to infinity. The distributions both include data from 2002 and 2003 (till June). No data was taken between mid December and April 1 2003.

The Longitudinal (L) estimates from the horizontal and vertical pairs are in close agreement, but both are slightly skewed to larger values, in comparison with the fitted normal distribution.  This happens too with the data from the old ING DIMM, but the comparison with RoboDIMM above shows good data is relatively lacking in RoboDIMM, something that is not apparent from the  graph on the left.

The label for the solid black line indicates that it is a Gaussian distribution,  centred on 0.76 arcseconds mean value, with a standard deviation of 0.145 (log-arcsec). This translates to saying that 2/3 of measurements are within 40% of the mean (peak), a similar spread to that on the old DIMM data. This Normal distribution is fit 'by eye' to the flanks of the distribution but not very well fit to the peak. (It is constrained have the same area under the curve as the data histograms, out to the points drawn).

The "fit" parameters are comparable to those obtained for the site using data from the old ING DIMM and JOSE.



Top | Back

Contact:  (RoboDIMM Project Scientist)
Last modified: 18 December 2010