| Home > Astronomy > Wide Field Camera > Observing Guide > Troubleshooter |
| Home > Astronomy > Wide Field Camera > Observing Guide > Troubleshooter |
WFC TroubleshooterContents
1. Problems with subsystems
1.1 Dome
If the dome does not rotate when the telescope is preset to a target, then the DOME SHTRS
button may still be pressed on the Engineering console. Depress it to
set the dome back to computing mode. Alternatively, the dome might be
erroneously left in local (engineering) mode. Turn the key in the dome control panel located inside the dome to set it back to remote (computing mode).
1.2 WFC shutter
If, when you start an exposure, you get the message The shutter failed to open at the start of the exposure accompanied by red/blinking/beeping, quit the alarm and repeat the last command. Elapsed time is counted in a blue box on the WFC Graphical User Interface (GUI) while the shutter is open. After read-out has completed, the box turns to green. If the box turns orange with time elapsed = 0, then a fault has occurred. This exposure will then either be logged with a 0 second exposure time, or not quoted at all, in the observations log and in the headers. The fault is a result of a problem with the MCA controller (the electronics rack mounted on the WFC which drives the shutter and filter wheel). To recover type: SYS> cmd MCA RESET With the next
exposure, the exposure time should be recorded correctly again. If this
does not fix the issue, bring the telescope down to access park and
power cycle the MCA controller (this is the controller on the bottom of
the WFC assembly [rotator at 180 deg.]), followed by: SYS> cmd MCA RESET
1.3 Filter wheel
On WFC start-up, it can be the case that the MCA controller has
problems identifying which filters are loaded in the wheel. In this
case, it may be necessary to send the command: SYS> cmd MCA RESET
When changing between filters in the wheel, you might get the message The filter wheel detent pin is malfunctioning accompanied by red/blinking/beeping, or Filter wheel movement timed out.
If this happens, quit the alarm and try to change filters again. If
this does not help, bring the telescope down to access park and power
cycle the MCA controller, followed by: SYS> cmd MCA RESET
1.4 Data acquisition
Data acquisition term refers to the management of an exposed
image, reading of data from the CCDs and saving to disk. This may fail
in one of several stages, for example:
SYS> dasreset WFC When this completes,
try a new run and see if the above fault reoccurs. If it does, then a
lower-level reset of the CCD controller may be necessary. In this case,
move the telescope down to access park and power cycle the unit labeled
SDSU CCD controller (located to the left of the cryostat), followed by:
SYS> dasreset WFC
Note: The above procedures deal with CCD controller problems.
Procedures on how to carry out a full restart of the DAS (Data
Acquisition System) can be found in section 2.
1.5 Meteorological Data System (MDS)
Occasionally the measurements from one, two, or all three of the
meteorological stations will fail to reach the MDS, leaving all the
windows red. If this continues for more than a few minutes the MDS can
be restarted. To close it type Alt+F4 on the attached keyboard, and
reopen by double clicking on the MDS icon near the middle of the
screen. After a minute this should update and work as normal. If this
does not not help, restart the computer with username: skyview and entering the password found on the MDS monitor.
1.6 Bad Pointing
First, check target positions and epoch are correct. Second, check focal station.
If these are OK, next repeat ZEROSET at zenith (see section 6 of the INT/WFC Operation Manual). Note that in HA there are two positions separated by only 13 arc minutes. Make sure to take the one closest to the indicating arrow. Continue with: USER> cal last
1.6.1 Checking pointing with the autoguider
Firstly, check the pointing with a pointing grid star and the autoguider: USER> gocat [object name]
USER> aper 5
Mark the autoguider window centre (SETUP/TOGGLE CENTER CROSS)
SYS> agwin acq pos 1032 512
SYS> autotv on
If the star comes up near the centre, the pointing is fine. If the star does not appear near the centre of the autoguider, then repeat the ZEROSET at zenith (see section 6 of the INT/WFC Operation Manual) and check that the apertures are defined correctly (see section 1.7.4). USER> cal default
1.6.2 Repeat single star calibration
If pointing does not improve after re-doing the zeroset, perform another single star calibration (see section 14 of the INT/WFC Operation Manual).
1.6.3 Seven star calibration
For the ING SA supporting the first S/D night: the seven star calibration should be done always during the first WFC D-night.
Please write down the resulting rms and send the value to the WFC instrument specialist. For the observer: In case the single star calibration does not improve the pointing, then perform a manual 7 star calibration, here is the procedure: SYS> agwin acq pos 1032 512
SYS> autotv on
USER> enter aperture 0 1065 -42
(redefines aperture 0, i.e. the default pointing position, to the centre of the autoguider chip)
USER> cal faint (this runs the 7-star calibration script)
The method to follow is exactly the same as that of a single star
calibration (centering the star on the crosshairs, etc). However,
instead of breaking out of the script using ctrl+z once the first star
has been found, simply position it accordingly and press the HANDSET
button to automatically slew the telescope to the next star. In case one star does not show up
in the field (possible closer to the North pole where the pointing could be worse), then it could
be ignored by typing "I". The script will then end after 7 stars have been centered.
The resulted RMS should be smaller than 10 arcsec for a normal pointing model.
You will then be asked to accept the result. Follow with: USER> next
in order to re-centre the star using the updated pointing model. Prepare the telescope pointing back to the centre of chip 4: USER> enter aperture 0 -251 326
SYS> autotv off
1.6.4 Check focus and aperture
Check that the focus is set to the correct plane (WFC = prime focus): USER> show focal
Check if apertures have been defined and have been defined correctly: USER> show aper
ap 0 -251 326 nominal aperture, object centred on CCD#4 ap 1 704 5 centre object on CCD#1 ap 2 357 -1043 centre object on CCD#2 ap 3 -701 -10 centre object on CCD#3 ap 4 0 0 nominal aperture, object in centre of mosaic camera (shifted from centre of CCD#4) ap 5 1316 -368 centre object on AG-CCD (X=1032, Y=512) ap 6 251 -326 centre object on RC (rotator centre) If the apertures are not defined, the observing system (ICS) must be restarted. See Section 2.2 for instructions on how to do this. After the system returns, re-issue the following commands: USER> show aper
USER> show cal
USER> cal def
and redo a single star calibration. Regarding other possible focus problems, the SA or OSA can check the focus encoders using following engineering "INT Sony incremental focus transducer" document related to the absolute and incremental focus encoders. 1.6.5 No stars in the autoguider window
Check if the automatic procedure for finding guide stars is activated,
using the command:
SYS> cmd INFODISP PREDICT ON
This enables the automatic star acquisition. It predicts the x,y
pixel coordinates at which suitable guide stars are expected to appear in
the autoguider CCD at a given telescope pointing. The prediction is
activated with every gocat command. If no stars are visible in the autoguider window, although the focus and pointing have been checked or/and they are assumed to be fine, then check the camera shutter. First, select a broadband filter (preferably G or R), otherwise a narrow band filter that is centered in the range 500-700 nm. If you still don't see a bright star in the autoguider, then check on the GUI whether the camera shutter is open. This image shows the two possible states of the shutter. Make sure that it is in the OPEN position. If the shutter is closed, then click on the image on the WFC mimic window to open it. NOTE (FR 24392): When the connection to the TCS is lost, then the INFODISP task that does the prediction exits. The recommended solution is to start the ICS, which then starts the INFODISP task and turns the predictions on. Nevertheless, it is possible to run the task without starting the whole ICS:
SYS> online TCS && ( infodisp >/dev/nul & )
SYS> cmd INFODISP PREDICT ON
In case of a further connection problem, the INFODISP task will exit. Once the problem is resolved
then repeat last two instructions.
1.6.6 Check RA/DEC encoders
If you experience possible tracking problems while observing (e.g. the stars appear elongated in the images)
it is important to save the logs of the encoders that describe the position and movement of the telescope. This
information allows to do a better diagnostic of the problems and helps to find a possible solution. To save
the encoders' logs there are the two TCS commands (so they have to be typed in the TCS computer) described below.
USER> log enc keep 30 Then in the TCS it will be displayed the name of the encoder log file just created, which will look as: USER> SYS$SYSDEVICE:[INT.DATA.ENCODER]ENC151027.DAT;1 Sometimes a work around can be, if possible, to switch to observe another object in a different sky position and come back to the previous object later in the night, being in another sky position as the time has passed may not be a problem anymore. 1.7 TCS disconnects
Check that the TCS USER window is responsive. If it is not, it is
possible that the TCS TELD task has died. To restart, go to the
x-terminal where the TCS user interface and info display are running
and type: intobs@inttv> telnet lpas2Then, log onto the computer named lpas2 (username: int_login, password can be found on the lpas2 monitor), and select the option > TELD This will re-establish contact between the observing system and the TCS. When contact is re-established, the TCS icon at the top of the WFC GUI will turn green. 1.8 TCS communication hangs
In the case of loss of communication between the TCS and the user interface, i.e. if either the TCS USER window stops responding, or the TCS DISPLAY window stops updating, try one of the following solutions: S1) Try to restart the TCS by opening the INT TCS terminal on the TCS computer and logging in with the INT TCS username and password. S2) If this does not work, then clearing the communications may recover the system: In the blue band at top of the window, click on S3) In case this still does not work, please contact the WHT OSA asking him/her to restart the TCS display using the following steps: Log in again to lpas2 using the usual userid and password. Select the option REOPEN from menu as shown below:
This is the INT TELESCOPE CONTROL SYSTEM
The TCS is running, using version I30-0-2
Please enter an option -
STOP - Stop the TCS
TELD - Start the TELD task
REOPEN - Open the TCS user and display windows
Info display on inticsdisplay.ing.iac.es
USER window on inticsdisplay.ing.iac.es
DISPLAY - Select different display devices
NFSMOUNT - Mount the ICS Unix partition
EXIT - Exit this menu and log out
NOTE: It is NOT necessary to do a Zeroset of the telescope after reopening the windows using the REOPEN.
1.9 WFC disconnects
If WFC disconnects, as shown by the orange warning-label on the mimic,
the DAS has most likely crashed. Check the orange telnet window that is
logged onto the DAS. If it is not available, you have to start a new
telnet connection from the background. First run obssys with the option shown on the whiteboard first in order to get the DAS> prompt (there is also an option to start up a new DAS window found by right-clicking on the desktop). Press the return key to see if there is a termination message for udas_server (the WFC task is an instance of the udas_server program). If udas_server has stopped running, do the following:
1.10 The wfc_ag_sequencer window missing
In case the small wfc_ag_sequencer windows is closed, it can be activated using the command:
SYS> wfc_ag_sequencer&
1.11 The autoguider window missing
In case the WFC autoguider window is missing, first you should check in the bottom
programs bar, maybe it was minimised by another user. Search for the "ING Autoguider" button
showing a small telescope icon (usually the first from the left in the bottom screen programs bar)
and click on it to restore the window. In case the window was indeed closed, then you can start it again by typing in the pink window: SYS> agstart prime
To establish a connection, choose "File" botton, then "Connect", and confirm a message
which appears in a separate window.
2. Problems with the instrument control system (ICS)
For any issues which appear to affect the instrument, including control mechanism errors,
communication errors, or observing system problems, try the following actions in order
until the problem is resolved
2.1 Reset the Data Acquisition System (DAS)
This can also be important when solving issues with high read-out noise.
SYS> dasreset WFC
2.2 Problems with the instrument control system (ICS)
This takes about 5 minutes, and will clear most communication errors. 2.2.1 Logout from the ICS
In the pink window type: SYS> shutdownobssys
SYS> exit
2.2.2 Logout from the DAS
On the orange DAS window for WFC (triton), type:
DAS4@triton> shutdownobssysAre you sure you want to shut down the DAS? y/n? > y DAS4@triton> exit intobs@proteo> exit 2.2.3 Start up the DAS
Single left-click on the DAS term icon, then type:
intobs@triton> obssysfollowed by the number of the observing station you need (4). Choose your station> 4Then, to begin, type: DAS4@triton> startobssys 2.2.4 Start up the ICS
Once the DAS is running, repeat the same steps in the pink INTICS window.
If INTICS window is not displayed already, just single left-click on the ICS icon.
Then type:
intobs@lpss7> obssysfollowed by the number of the observing station you need (4). Choose your station> 4Then, to begin, type: SYS> startobssys
Then, wait for the following windows to appear.
Check the filter wheel can be rotated and if the shutter works. If one or the other does not work, type: SYS> cmd MCA RESET
In case your observations do not need guiding, check and eventually shutdown the wfc_ag_sequencer, according to the
"Not Guiding" section
of the WFC Operation Manual. In case you need guiding, in the autoguider window, overlay the cross in the centre by selecting TOGGLE CENTRE CROSS from the SETUP menu. 2.3 Restart of individual system controllers
If you are not 100% sure how to power cycle any system, do not
attempt to do this without first contacting your support astronomer or
the WHT Telescope Operator (WHT OSA)! The next stage requires the telescope
to be parked and brought down to access park.
* If you have problems with the filter wheel or camera shutter, you may need to power cycle the MCA controller. This is located below the cryostat, as seen in this photo. Turn off the controller, wait for one minute, then restart. After this you may need to complete a DAS reset or ICS restart as described above. * If you have problems with the autoguiding system, you may need to power cycle the AG controller. This is located to the right of the cryostat, as seen in this photo. Turn off the controller, wait for one minute, then restart. After this you may need to complete an ICS restart as described above. 3. Problems with the telescope control system (TCS)
If you encounter
problems with the telescope system, focussing, pointing or tracking,
you may need to restart the TCS. Shut down the observing system and
log out of the ICS as described above. Be sure to follow this order:
3.1 Log out of the TCS
To log out type:
USER> tcsexit
Then log out of the machine by right-clicking on Desktop, then choose Leave, then Log out.
3.2 Starting up the TCS
Remember to do a ZEROSET whenever the TCS is restarted! After the TCS is restarted, check that the telescope focus value is what you need. If not, check the focal station. Then you can restart the ICS following the instructions in the Section 2.2. 3.3 DEC synchro arrows jumping
4. Data reduction PC
4.1 Log out
In the IRAF window, at the prompt type:
ecl> logout
then close any remaining windows and exit the system as previously shown.
4.2 Log in
To log in type:
username: intguest and the password can be found on the intdrpc
monitor (below). Then click on the xterm icon to open up a xterm
window. Here you should type:
> irafFollowed by: ecl> wfc_ql
Check what directory you are in by typing:
ecl> pwd
The correct directory is displayed on the WFC GUI. If you are not in the correct directory, type:
ecl> cd /obsdata/inta/yyyymmdd
where yyyymmdd is your observing date, e.g., 20110321 for the night started in 21 of March 2011.
Reporting/Searching faults at ING Fault Management System
|
| Top | Back |
|
