Observing overheads at the WHT

The analysis below was made in 1998.

Summary

Use of scheduled observing time on the William Herschel Telescope breaks down as follows:

Lost to bad weather	24%
Lost to technical problems	4%
Observing overheads	22%	(31% of usable night)
Integrating on targets	50%	(69% of usable night)

The 31% overheads include: telescope slews (2%); object acquisition (4%); issuing commands to the system (3%); mechanism movements (2%); CCD readouts (6%); and image inspection / planning next observation (5%). Substantial savings (~ factor of 2) should be possible by improving the interface between the observer and the control system. Total observing overheads for other large telescopes, are similar, ~ 30%.

Observing overheads at the WHT

Observing time on the Isaac Newton Group's William Herschel Telescope costs society ~ 10000 pounds per night (capital cost / life + running cost). The time is oversubscribed by a factor ~ 4. Here I investigate the efficiency with which the telescope is used for observing, and identify areas where improvements are possible.

About 300 nights per year are scheduled for observing at the WHT, the remainder being used for engineering, commissioning and quality-control work. 24% of the scheduled observing time is lost to bad weather (plot). 4% is lost to technical problems (plot).

The fraction of the night lost to observing overheads, f(over), was defined to be the fraction of usable astronomical night (i.e. the clear-sky time between astronomical twilights) during which no detector on the telescope was exposing.

f(over) was determined from the oberving logs for a sample of 462 nights between 1 April 1992 (when electronic logging began) and 18 December 1994. The sample excludes those nights used for commissioning or quality control work, for service observing or for observing with an `own' instrument. The instruments in use on these 462 nights were the Cassegrain intermediate-dispersion spectrograph ISIS (65% of the nights), the Nasmyth high-resolution Utrecht Echelle Spectrograph UES (14%), the Cassegrain Low Dispersion Survey Spectrograph LDSS (11%) and others (10%).

The median value of f(over) is 31%, with two thirds of the values lying in the range 17% < f(over) < 54% (i.e. a slightly skewed distribution). The medians for nights using ISIS, UES and LDSS are 30 +- 1%, 37 +- 2%, 28 +- 2%, the high value for UES probably reflecting the greater difficulty of object acquisition (at least until November 1994 when a more efficient acquisition system was installed). The quoted errors are (estimated rms of nightly distribution)/sqrt(number of nights). The median value of f(over) for ISIS showed no significant change over the period 1992 - 4. The median value of f(over) for first nights of ISIS runs (31 +- 1%) does not differ significantly from that for later nights, suggesting that little observing time is lost while observers gain experience with the system.

Where the observing overheads go

The use to which the 30% ISIS observing overheads are put was investigated by inspecting the intervals between exposures on astronomical targets during 123 hours observing on 15 randomly-selected ISIS nights. There were 270 such intervals on these nights (totalling 30.4 hours, or 25% of these nights), which could be broken down into 5 categories according to the action identified in the observing log:

Table 1 - Intervals between CCD exposures

Action                         N  mean  median  range  median - CCD  Total
                                  mins   mins    mins      mins      hours
Telescope move                107   8     6     4 - 13       5        14
Arc exposure (1 pair)          41   5     4     2 -  9       2         4
Arc exposure (> 1 pair)        22  13    13     8 - 18      10         5
Telescope move + arc           21  13    10     6 - 27       8         4
No action                      79   3     2     2 -  5       1         4
                              ---                                     ----
                              270                                     30.4

`Telescope move' (identified by change of coordinates in the log) includes inspection of previous frame, telescope slew, checking the field on the TV, acquisition of the target on the spectrograph slit, finding a guide star and closing the autoguider loop, checking the instrument and detector configuration, and issuing the command to expose.

`Arc exposure' (identified by title in log) includes inspection of previous frame, configuration of the comparison-lamp system, issuing command to expose, and inspection of the result. Usually, one pair of (simultaneous) exposures is taken on the red and blue arms of the spectrograph. More rarely, two or more pairs are taken, with intervening changes of wavelength setting; these intervals are much longer, probably because of the extra reconfiguration and image-inspection overheads.

`Telescope move + arc' refers to intervals during which the telescope has been moved and arc exposures have been taken.

`No action' refers to an interval in which there has been neither telescope slew nor arc exposure.

N is the number of events. The mean and median durations are given in the third and fourth columns column and the range including 2/3 N of the events is given in the fifth column. The sixth column gives median duration minus the time taken for CCD readout pairs during the interval (1 pair of readouts on target and 0, 1 or >~ 2 pairs of readouts on an arc lamp). CCD readout takes ~ 100 microsec per pixel, and the size read out is typically wavelength range ~ 1100 pixels, 200 < spatial range < 800 pixels, mean spatial range = 550 pixels, mean readout duration 60 sec (in parallel on the two arms of ISIS).

The duration of overheads other than CCD readout during each of the types of interval given in Table 1 were measured by timing activities during scheduled observing (usually with ISIS) on random nights. Given the 30% mean ISIS overheads deduced from the logs, the mean duration of each type of interval given in Table 1, and the breakdown of activities timed within each type of interval, it was possible to deduce the following approximate breakdown of observing overheads:
Table 2 - Breakdown of observing overheads for WHT/ISIS

Overhead                         typical duration   % of night     
Slewing telescope to target          60 sec            2.3           
Acquisition of target on slit        100 sec           3.8           
Acquiring guide star                                   0.8           
Checking spectrograph configuration  30 sec            2.8            
Typing command to expose             20 sec            2.4             
Moving comparison mirror (in + out)  60 sec            1.5             
CCD readout                          60 sec            6.0            
Image inspection, thinking what to do next 
                                     80 sec            4.9            
Other                               >60 sec            5.5            
                                                                      
Total                                                 30.0

`Other' includes mistakes by the observer (e.g. wrong coordinates, inadequate finding charts, sending mechanisms to the wrong position, forgetting that an exposure has finished), technical problems which don't get recorded as such (e.g. needing to fill a cryostat in the middle of the night) and poor weather which isn't recorded as such (e.g. passing cloud, outbreaks of bad seeing). These `other' interrupts are infrequent and usually of long duration, and it is difficult to quantify their relative contributions to the observing overheads. Other configuration changes e.g. of diffraction grating, dichroic, central wavelength, contribute insignificantly to the overheads. Experience suggests that the breakdown of overheads for other WHT instruments is similar.

For the WHT, and probably for most other telescopes, the overheads fall into two groups: those determined by engineering limitations (telescope slew, mechanism moves, CCD readouts) and those due to interactions between the observer and the system.

Engineering overheads
The time taken for the WHT to slew is typically dominated by the azimuth or Cassegrain rotator slew, rate 1 deg per sec (elevation slews are short, with 90% of observations carried out at elevation > 50 deg). The slew speed cannot be changed without major engineering work. The movement of the comparison mirror will probably be substantially speeded up as part of a package of work shortly to be carried out on the acquisition and guidance unit. CCD readout times are currently 100 microsec per pixel, but these could improve by a substantial factor e.g. to 10 microsec per pixel, within a year or two. This increase in readout speed will more than offset likely increases in the area of CCD to be read out. Reducing overheads is only one of the motivations for speeding up CCD readouts; they can be the limiting factor in time-resolved observations of variable objects, and also in obtaining adequate flat fields in the brief opportunity during twilight (Tyson et al 1993).

Observer-system interaction overheads
Most of the remaining overheads are due to the time needed for the observer to interact with the system. Three examples follow. (1) The overhead of longest duration is typically acquisition of the target object on the spectrograph slit. This usually includes relating the appearance of the TV acquisition image to that of a finding chart. The TV has no automatic orientation marker, and observers' finding charts often leave a lot (e.g. scale, orientation and correct parity) to be desired. This leads to much juggling with the scale and lookup table of the TV image, and time-consuming reintegrations (up to 60 sec?) on the TV. An ideal intelligent system, with access to a sky-survey database, should be able to verify the field and centre the target on the slit in a few seconds. (2) Image evaluation imposes another large overhead. The observer typically wants to know, before deciding what to observe next, the signal-to-noise of the spectrum in a given region, or whether a particular spectral line has been detected. Facilities for obtaining quick-look sky-subtracted spectra have only just been introduced at the WHT. They represent only a first step, but a substantial saving in overheads is anticipated. (3) The command interface is another area where substantial reduction of overheads should be possible. Typing in the line of characters required to start an exposure typically takes 20 sec. Remembering the command names and the sequence of parameters required is clearly a burden on observers and mistakes are frequent. One of the solutions being investigated is a graphical user interface. These three examples highlight the importance of minimising the number of unnecessary decisions by the observer. This lesson has been taken to heart in both space and radio astronomy, where it is common for observing to be driven by a command file prepared well in advance; it is the direction in which ground-based optical observing should be heading.

Reducing observing overheads is of course just one of several approaches to squeezing more out of the available observing time. Other current programmes aimed at improving the efficiency of the WHT operation include: reduction of dome seeing, which could have a dramatic effect on the exposure time needed to reach a given signal-to-noise on an unresolved object; investigation of queued observing, in which programmes are scheduled in service mode, according to the sky conditions prevailing; mirror cleaning with CO2 snow as a possible alternative to realuminising, which could eliminate the need for the annual 3-day standdown; multiplexing of observations (e.g. with the WYFFOS wide-field fibre-fed spectrograph); and improvement of the efficiency of instruments and detectors (e.g. of the UV response of CCDs).

Comparison with other telescopes

The other two telescopes of the Isaac Newton Group on La Palma are the Isaac Newton (INT, 2.5-m) and the Jacobus Kapteyn (JKT, 1.0-m) reflectors. Like the WHT, both are driven by command-line interfaces, but differ substantially from the WHT in their construction and instrumentation and in their electronic and computing systems. The median observing overheads for INT Cassegrain spectroscopy during 1994 were 29 +- 1%, very similar to those for the WHT. For imaging with the INT (prime-focus) and JKT (Cassegrain), the overheads are 42 +- 2% and 44 +- 1% of the night respectively. The much higher overhead for imaging on these telescopes is due to the use of larger areas (windows) on the CCDs, slow readout (2 - 3 minutes) through the old Perkin-Elmer computers, smaller mean exposure time, and the many telescope slews associated with imaging surveys. The measured INT overheads are consistent with those implied in the analysis carried out by Benn & Martin (1987) for this telescope during the period 1984 - 1986.

Enquiries of staff at other optical telescopes (Sinnott and Nyren 1993 give a list of 30 with primary-mirror diameters > 2.3 m) yielded the downtime and overheads statistics in Table 3 (data for the JCMT are also included).

Table 3 - Comparison with other large optical telescopes

Telescope                    % lost            % lost        % observing*   
                          to tech. problems  to bad weather    overheads  
WHT 4-m La Palma                 4                 23             31   
INT 2.5-m La Palma spec.         3                 24             29  
INT 2.5-m La Palma imaging       3                 24             42  
JKT 1.0-m La Palma imaging       4                 27             44  
&&&  
Keck 9.8-m Hawaii                8                 19             33  
MMT 4.5-m Arizona                4                 33
CTIO 4.0-m Cerro Tololo          3                 18   
AAT 3.9-m Australia              3                 33        20 - 30  
KPNO 3.8-m Kitt Peak             4                 26   
UKIRT 3.8-m Hawaii               3                 23  
CFHT 3.6-m Hawaii (MOS)                                           40  
NTT 3.5-m La Silla             1-4                               ~30  
McDonald 2.7-m Texas             3                 38  
Shajn 2.6-m Crimea               1                 33            ~20   
Hiltner 2.4-m Kitt Peak                                          ~33  
Calar Alto 2.2-m                                                 ~20  

JCMT 15-m (mm-wave) Hawaii       3                 20
                            
Median                           3                 25             32

* Observing overheads = fraction of night when the sky is clear and there are no technical problems, but the CCDs are not integrating. `~' indicates that the figure is based on only a small sample of observing.

These statistics may be defined slightly differently at other observatories, but the remarkably small range of observing overheads confirms that such differences are small. The similarity of the overheads (~ 30%) then suggests either that optical observers everywhere face very similar overheads, or that pressure from observers for overheads to be reduced relaxes once they fall below a given level. Either way, the analysis of the previous section suggests that substantial savings are possible.

The observing overheads reported for the CFHT (Glaspey 1995) are for the most heavily-used instrument, the Multi-Object Spectrograph (MOS). Glaspey found that the overheads for all CFHT instruments dropped (typically by a third) after the first night's observing. As mentioned above, no such change is noted for ISIS observers, perhaps reflecting a higher level of observer support at the WHT.

Conclusions

Observing overheads ~ 30% of clear time are typical for large telescopes. They dwarf the oft-quoted fraction of time lost to technical problems. These large overheads must be taken into account when applying for, and allocating, observing time for particular projects.

The bulk of the overheads are due not to engineering limitations (telescope slew, CCD readout etc.) but to the time needed for the observer to interact with the observing system. They can be substantially reduced by improving the user interface.

Acknowledgments

I am grateful to Dietrich Baade, Bob Fosbury, Tom Geballe, Peter Gillingham, John Glaspey, Ulrich Hopp, Phil Kelton, Steve Lee, Caty Pilachowski, Peter Petrov, Ian Robson and Gary Wegner for providing operational statistics for various telescopes.

References

Benn C.R. and Martin R., 1987, QJRAS, 28, 481
Glaspey J., 1995, CFHT Bulletin No. 32, p.23
Sinnott R.W. and Nyren K., 1993, Sky and Telescope, July 1993, p.27
Tyson N.D. and Gal R.R., 1993, AJ, 105, 1206

1998 May 1, minor changes 2011 Feb 26