|Home > Astronomy > Wide Field Camera > Observing Guide > Troubleshooter|
Elapsed time is counted in a blue box on the WFC Graphical User Interface (GUI) while the shutter is open. After read-out has completed, the box turns to green. If the box turns orange with time elapsed = 0, then a fault has occurred. This exposure will then either be logged with a 0 second exposure time, or not quoted at all, in the observations log and in the headers. The fault is a result of a problem with the MCA controller (the electronics rack mounted on the WFC which drives the shutter and filter wheel). To recover type:
SYS> cmd MCA RESETWith the next exposure, the exposure time should be recorded correctly again. If this does not fix the issue, bring the telescope down to access park and power cycle the MCA controller (this is the controller on the bottom of the WFC assembly [rotator at 180 deg.]), followed by:
SYS> cmd MCA RESET
SYS> cmd MCA RESETWhen changing between filters in the wheel, you might get the message The filter wheel detent pin is malfunctioning accompanied by red/blinking/beeping, or Filter wheel movement timed out. If this happens, quit the alarm and try to change filters again. If this does not help, bring the telescope down to access park and power cycle the MCA controller, followed by:
SYS> cmd MCA RESET
SYS> dasreset WFCWhen this completes, try a new run and see if the above fault reoccurs. If it does, then a lower-level reset of the CCD controller may be necessary. In this case, move the telescope down to access park and power cycle the unit labeled SDSU CCD controller (located to the left of the cryostat), followed by:
SYS> dasreset WFCNote: The above procedures deal with CCD controller problems. Procedures on how to carry out a full restart of the DAS (Data Acquisition System) can be found in section 2.
If these are OK, next repeat ZEROSET at zenith (see section 6 of the INT/WFC Operation Manual). Note that in HA there are two positions separated by only 13 arc minutes. Make sure to take the one closest to the indicating arrow. Continue with:
USER> cal last
USER> gocat [object name]
USER> aper 5Mark the autoguider window centre (SETUP/TOGGLE CENTER CROSS)
SYS> agwin acq pos 1032 512
SYS> autotv onIf the star comes up near the centre, the pointing is fine.
If the star does not appear near the centre of the autoguider, then repeat the ZEROSET at zenith (see section 6 of the INT/WFC Operation Manual) and check that the apertures are defined correctly (see section 1.7.4).
USER> cal defaultsection 14 of the INT/WFC Operation Manual).
For the observer: In case the single star calibration does not improve the pointing, then perform a manual 7 star calibration, here is the procedure:
SYS> agwin acq pos 1032 512
SYS> autotv on
USER> enter aperture 0 1065 -42(redefines aperture 0, i.e. the default pointing position, to the centre of the autoguider chip)
USER> cal faint(this runs the 7-star calibration script) The method to follow is exactly the same as that of a single star calibration (centering the star on the crosshairs, etc). However, instead of breaking out of the script using ctrl+z once the first star has been found, simply position it accordingly and press the HANDSET button to automatically slew the telescope to the next star. In case one star does not show up in the field (possible closer to the North pole where the pointing could be worse), then it could be ignored by typing "I". The script will then end after 7 stars have been centered. The resulted RMS should be smaller than 10 arcsec for a normal pointing model.
You will then be asked to accept the result. Follow with:
USER> nextin order to re-centre the star using the updated pointing model.
Prepare the telescope pointing back to the centre of chip 4:
USER> enter aperture 0 -251 326
SYS> autotv off
USER> show focalCheck if apertures have been defined and have been defined correctly:
USER> show aperap 0 -251 326 nominal aperture, object centred on CCD#4
ap 1 704 5 centre object on CCD#1
ap 2 357 -1043 centre object on CCD#2
ap 3 -701 -10 centre object on CCD#3
ap 4 0 0 nominal aperture, object in centre of mosaic camera
(shifted from centre of CCD#4)
ap 5 1316 -368 centre object on AG-CCD (X=1032, Y=512)
ap 6 251 -326 centre object on RC (rotator centre)
If the apertures are not defined, the observing system (ICS) must be restarted. See Section 2.2 for instructions on how to do this.
After the system returns, re-issue the following commands:
USER> show aper
USER> show cal
USER> cal defand redo a single star calibration.
Regarding other possible focus problems, the SA or OSA can check the focus encoders using following engineering "INT Sony incremental focus transducer" document related to the absolute and incremental focus encoders.
SYS> cmd INFODISP PREDICT ONThis enables the automatic star acquisition. It predicts the x,y pixel coordinates at which suitable guide stars are expected to appear in the autoguider CCD at a given telescope pointing. The prediction is activated with every gocat command.
If no stars are visible in the autoguider window, although the focus and pointing have been checked or/and they are assumed to be fine, then check the camera shutter.
First, select a broadband filter (preferably G or R), otherwise a narrow band filter that is centered in the range 500-700 nm.
If you still don't see a bright star in the autoguider, then check on the GUI whether the camera shutter is open. This image shows the two possible states of the shutter. Make sure that it is in the OPEN position. If the shutter is closed, then click on the image on the WFC mimic window to open it.
NOTE (FR 24392): When the connection to the TCS is lost, then the INFODISP task that does the prediction exits. The recommended solution is to start the ICS, which then starts the INFODISP task and turns the predictions on. Nevertheless, it is possible to run the task without starting the whole ICS:
SYS> online TCS && ( infodisp >/dev/nul & )
SYS> cmd INFODISP PREDICT ONIn case of a further connection problem, the INFODISP task will exit. Once the problem is resolved then repeat last two instructions.
USER> log enc keep 30
Then in the TCS it will be displayed the name of the encoder log file just created, which will look as:
Sometimes a work around can be, if possible, to switch to observe another object in a different sky position and come back to the previous object later in the night, being in another sky position as the time has passed may not be a problem anymore.
intobs@inttv> telnet lpas2Then, log onto the computer named lpas2 (username: int_login, password can be found on the lpas2 monitor), and select the option > TELD This will re-establish contact between the observing system and the TCS. When contact is re-established, the TCS icon at the top of the WFC GUI will turn green. TCS USER window stops responding, or the TCS DISPLAY window stops updating, try one of the following solutions:
S1) Try to restart the TCS by opening the INT TCS terminal on the TCS computer and logging in with the INT TCS username and password.
S2) If this does not work, then clearing the communications may recover the system:
In the blue band at top of the window, click on
S3) In case this still does not work, please contact the WHT OSA asking him/her to restart the TCS display using the following steps:
Log in again to lpas2 using the usual userid and password. Select the option REOPEN from menu as shown below:
This is the INT TELESCOPE CONTROL SYSTEM The TCS is running, using version I30-0-2 Please enter an option - STOP - Stop the TCS TELD - Start the TELD task REOPEN - Open the TCS user and display windows Info display on inticsdisplay.ing.iac.es USER window on inticsdisplay.ing.iac.es DISPLAY - Select different display devices NFSMOUNT - Mount the ICS Unix partition EXIT - Exit this menu and log outNOTE: It is NOT necessary to do a Zeroset of the telescope after reopening the windows using the REOPEN.
First run obssys with the option shown on the whiteboard first in order to get the DAS> prompt (there is also an option to start up a new DAS window found by right-clicking on the desktop). Press the return key to see if there is a termination message for udas_server (the WFC task is an instance of the udas_server program). If udas_server has stopped running, do the following:
In case the window was indeed closed, then you can start it again by typing in the pink window:
SYS> agstart primeTo establish a connection, choose "File" botton, then "Connect", and confirm a message which appears in a separate window.
SYS> dasreset WFC
DAS4@triton> shutdownobssysAre you sure you want to shut down the DAS? y/n? > y
intobs@triton> obssysfollowed by the number of the observing station you need (4).
Choose your station> 4Then, to begin, type:
intobs@lpss7> obssysfollowed by the number of the observing station you need (4).
Choose your station> 4Then, to begin, type:
SYS> startobssysThen, wait for the following windows to appear.
Check the filter wheel can be rotated and if the shutter works. If one or the other does not work, type:
SYS> cmd MCA RESETIn case your observations do not need guiding, check and eventually shutdown the wfc_ag_sequencer, according to the "Not Guiding" section of the WFC Operation Manual.
In case you need guiding, in the autoguider window, overlay the cross in the centre by selecting TOGGLE CENTRE CROSS from the SETUP menu.
* If you have problems with the filter wheel or camera shutter, you may need to power cycle the MCA controller. This is located below the cryostat, as seen in this photo. Turn off the controller, wait for one minute, then restart. After this you may need to complete a DAS reset or ICS restart as described above.
* If you have problems with the autoguiding system, you may need to power cycle the AG controller. This is located to the right of the cryostat, as seen in this photo. Turn off the controller, wait for one minute, then restart. After this you may need to complete an ICS restart as described above.
USER> tcsexitThen log out of the machine by right-clicking on Desktop, then choose Leave, then Log out.
Remember to do a ZEROSET whenever the TCS is restarted!
After the TCS is restarted, check that the telescope focus value is what you need. If not, check the focal station.
Then you can restart the ICS following the instructions in the Section 2.2.
ecl> logoutthen close any remaining windows and exit the system as previously shown.
> irafFollowed by:
ecl> wfc_qlCheck what directory you are in by typing:
ecl> pwdThe correct directory is displayed on the WFC GUI. If you are not in the correct directory, type:
ecl> cd /obsdata/inta/yyyymmddwhere yyyymmdd is your observing date, e.g., 20110321 for the night started in 21 of March 2011.
The ING has a Fault Management System to be used by ING staff and visiting observers mainly to record faults that occur during the operation of the telescopes and to review previous faults and their possible workaround. The INT-WFC observers may use the system by "Login as guest" being able to create and review fault reports.
Reporting a fault in the ING Fault Management System is a way of communication between visitor observers and the ING staff. Therefore, the faults reported can be of diverse nature: instrument problems, telescope problems, INT car issues, etc. The ING Fault Management System is reviewed daily in the morning by Telescope and Instrument Engineers, Operations Team and Instrument Specialists, who will address the reported situations and update the Fault Report with the 'Workaround' or any other relevant information related to the report.
Searching for fault reports can be useful to identify similar problems that occurred in the past and help the WHT operator to find a possible solution. INT observers should not try actions described in the fault reports without the explicit authorization and supervision of the WHT operator, unless they have done it before and they fully understand what are they doing.To look for a fault report in the database proceed as follow: Go to the ING Fault Management System and chick on "Login as guest". It will open a window with three tabs on the top.
Click on 'Search' and there, the best option is to select the 'Advanced' tab. A window with a space to write 'keywords' and a series of options for several searching parameters will appear.
Write down the searching keywords and open the menu in the relevant parameters for your search to choose the correct option then click on the left side box to select the parameter. Note that it is not necessary to select all the parameters available (but check for example 'INT' option under 'Site' parameter to constrain the search).
After you have made your choices go to the 'Search' button. This will produce a list of previous fault reports that match your requested keywords and parameters. The list contains basic information about the report. To access to the complete Fault Report (which includes the 'Fault Description' and the 'Workaround' or an updated status of the report) click on the 'ID' number in the first column (it will open the complete report in a new tab in your browser) or click on the Fault Report title (it will open the complete report in the same tab).
Reporting a fault in the ING Fault Management System it is the way to communicate to the ING staff the problems you had during the night. It is recommended to report most of the faults whether solved or not (to keep track of the system failures in the former case and to allow ING staff handle the problem during the day in the later case). Minor issues as a single "read time out" message after a command in the ICS do not need to be reported unless it keeps on repeating several times.It is important that the Fault Report has all the relevant information related to the problem described, written in a concise and ordered way. Find below a guideline for writing a Fault Report:
Go to the ING Fault Management System and chick on "Login as guest". It will open a window with three tabs on the top. You will already be in the tab for writing the fault report.
The 'Title' should shortly describe the problem. It will appear in the Fault Report list when doing a search for previous fault reports. In the main window below the 'Title' you have to describe the problem. The basic information there should be:
If there is an alarm different from the ones described above, or you are doubting or the alarm is persisting please call the WHT OSA to inform the situation and look for advice.
common use web addresses for observers at INT to copy-and-paste the address in the browser in case links from other pages do not work (and whenever you have time make a Fault Report of the problem so it can be solved as soon as possible).
There is a printed copy of this page in the INT control room, in the same folder as the IDS Operational Manual and Troubleshooter.
|Top | Back|