1. 'WARNING: Header collection failed' message appears in the system window
when performing any form of run.
This indicates that UltraDAS cannot connect to the Header
Collection Task (HCT), the TCS, or one of the associated FITS header
packet colletor tasks. The HCT is restarted by issuing the startobssys
command in the system window.
2. The camera mimic disappears or stops updating and all runs fail with
the message 'The camera could not be contacted - please ensure that the camera is running on the DAS machine' .
This indicates that the system computer has lost its connection to
the DAS camera server. This is due either to a communication failure to
the DAS computer, or the camera server running on the DAS computer has
If this is the case, restart the DAS as shown below in the section
below "Stop and restart the DAS".
Resetting the DAS
Occasionally, the data-acquisition sub-system in UltraDAS (i.e. the software
on the DAS computers) misbehaves and needs to be reset. Sometimes it's
an internal problem, sometimes the trouble starts in the the detector controller,
which the DAS software talks to. These notes describe a set of procedures
for recovering the fault with minimal disruption to the rest of the observing
What not to do
It is usually a bad idea to stop and restart the whole observing system,
at least until you have tried the other procedures listed below. Why is
this bad? Firstly, because it takes much longer than the procedures below
(it is a superset of those procedures). Secondly, because it increases
the chance of hitting other problems elsewhere in the system. In particular,
the shutter and filter controls on the INT WFC will be disrupted by a full
restart and these do not always restart properly on the first try.
If you are using the INT WFC and you get problems with the shutter or
filter wheel (or with the task MCA that manages them), please note that
this is nothing to do with the DAS. The shutter of the INT WFC is controlled
by the ICS and resetting the DAS won't solve any ICS problems.
What to do
There are four procedures; try them in order until you get satisfaction.
Always remember to restore your binning, readout speed and readout window
settings after resetting the DAS.
Use the dasreset command. This is quick - it takes less than
a minute - and non-disruptive. No need to call the duty engineer
if this works.
Stop and restart the DAS, but not the other sub-systems.
This takes longer still and disrupts software on other computers.
Power-cycle the SDSU detector controller, then use dasreset. This
takes longer because you need to slew the telescope to get at the controller.
If all else fails, but not before, stop and restart all Unix sub-systems.
Type dasreset <camera-name> at the SYS> prompt: e.g.
dasreset WFC or dasreset RED. Case is significant: the command
must be in lower case and the name of the camera in upper case.
dasreset returns the prompt as soon as the command is accepted.
That is, it does not wait for the command to take effect.
Watch the talker: you should see the camera first announce that it is going
off-line and later announce that it is back on-line.
If the camera fails to come back on-line, then this procedure hasn't worked,
so go on to the next procedure.
If the camera comes on-line you are ready to go again except that
you will need first to reselect your binning, readout speed and readout
Stop and restart the DAS
Log on to the DAS computer. An orange login window should be available
on the system computer. If more than one orange window is visible (ie.
with ISIS at the WHT), check the CCD whiteboard to determine the relevant
DAS computer for the camera needing to be restarted.
Run the obssys command on the DAS computer.
Give the command shutdownobssys to the DAS computer, at the DAS>
prompt.. (Be careful not to give the command at the SYS> prompt
by mistake: the idea here is not to shut down the ICS.)
Wait for the DAS software to die.
Give the command startobssys to the DAS computer, at the DAS>
prompt (see notes below).
Wait for the camera to come back on-line: it will anounce this in the talker and in the DAS window.
Wait for the camera mimics to re-connect.
If you have no outstanding errors, you are ready to observe. Otherwise
go on to the next procedure.
Controller reset followed by dasreset
Slew the telescope so that you can reach the SDSU detector-controller:
see the notes below. If you don't know for certain what or where this device
is, call the duty engineer now.
Power-cycle the SDSU controller.
Do the dasreset thing from the previous procedure.
If dasreset works, you are ready to observe. If not, go on to the
Stop and restart both the DAS and the software on the system computer
At the SYS> prompt on the system computer, give the command shutdownobssys.
Wait for the system-computer software to go away.
At the DAS> prompt on the DAS computer(s), give the command shutdownobssys.
Wait for the DAS software to go away.
Log off the DAS computer(s).
Log off the system computer.
Slew the telescope to get at the SDSU detector-controller (see notes below).
Power-cycle the SDSU detector controller (see notes below).
Log on to the system computer as you would at the start of the night.
From the system computer log on to the DAS computer. (The system should
automatically log on your behalf. Look for orange terminal windows with
the DAS> prompt.
Run the obssys command on the DAS computer(s) at the DAS> prompt.
Run the startobssys command on the DAS computer(s) at the DAS> prompt.
Run the obssys command on the system computer at the SYS> prompt.
Run the startobssys command on the system computer at the SYS> prompt.
You should now be ready to observe.
These instructions apply to observing-systems at the INT and at the WHT, those being the systems currently using
UltraDAS components. The procedures do not apply to other DAS variants.
The telescope position for access to the SDSU detector-controller varies
from instrument to instrument.
The detector controller is in two parts: An aluminum box for the main electronics
and a grey box for the power-supply. The on-off switch is on the power-supply
For INT prime, slew to access park.
For WHT prime, slew to access park.
For WHT cass, slew to zenith.
For IDS, slew to zenith.
The system typically gives you an terminal-window to the DAS computer
when you log on to the system computer. This window may go away if not
used (Unix logs out the terminal session), but this does not mean that
the DAS has crashed. The DAS doesn't need the terminal window to be
there. Don't reset just for this.
The obssys command selects and enables certain Unix commands
that you need to work on the system; if you miss out obssys, the
computer will not know about commands such as startobssys and shutdownobssys.Obssys
is an idempotent command: it doesn't matter if you do it twice in the same
When startobssys completes successfully on the DAS computer,
the DAS is on-line, but is not yet connected to the system computer because
the latter has to make the connection. At this stage, observing commands
like run will not work. When you then do startobssys on the
system computer, you make the connections to the DAS computer that allow
the observing commands to work properly.
At the WHT, startobssys at the system computer also starts the Header Collection Task (HCT) on
the system computer.
There is no reason to stop and restart either the TCS or the autoguider
as part of these procedures.
Guy Rixon (firstname.lastname@example.org) 2000-05-02, Richard Bassom (email@example.com)