A diagnostic guide for the JKT A&G box (ver 2.4)
Occasionally the JAG fails due to power cuts and other (unknown) phenomena and this guide should hopefully enable the engineer to get the instrument back on line as quickly as possible. It cannot cover EVERY problem that may happen, but now being the last person still around to have worked in the team at RGO, Herstmonceux that built the JAG back in 1986... There ain't much I don't know about this instrument :-)
In the many years that I have been involved with the JAG, experience has shown that most of the problems can be solved by following the instructions below. The first version of "Taming the JAG" I wrote back in 1990, but the JKT ICS has changed considerably over the years and this document reflects these changes.
I have attempted to make this version of the document an 'expert' procedure? Just click on the links below and follow the steps in a logical order. Clicking on the links within the sections will take you to the to the next stage and you will have the JAG back on the sky within a short time. Any problems with this document or the JAG in general then drop an email to: ejm@ing.iac.es
For more information refer to manuals :
ER0 45 The Local Control System for the 1m CCD A&
G Box
ER0 46 4MS Local Controller Software Manual
These documents and a full set of circuit diagrams (Document folder No 23) can be found in the JKT control room .
I have broken down this document into six main parts:
JAG can you
hear me?
Analysis
Diagnosis
and cure
Lost
movement in autoguider probes
Problems
with filter wheels
Final checkout
with the Sparc based ICS
Appendix
The status
word explained
Important
notes
Rule 1... As "The Hitch Hiker's Guide to the Galaxy" states:
All these checks and recovery procedures below will be done from the JAG ENGINEERING VT220 TERMINAL which sits on the table beside the met station PC. This must be connected AT ALL TIMES. If the Engineer's terminal is not connected to the 4MS controller, it has been found that 4MS can 'lock up' due to pick-up on the RS232 cable running from the telescope to the control room.
n.b. You will be entering instructions to the FORTH interpreter thus ALL commands must be typed in: UPPER CASE.
RGO 4MS.FORTH V.2 hi
Mechanism | Status | Step count | Comments |
%X | 1004 | 005740 | Autoguider X probe (at pos 5740) |
%N | 1104 | 1 | Autoguider ND filter (pos 1) |
%G | ******** | ****** | Red and blue grisms (removed) |
%Y | 1104 | 010000 | Autoguider Y probe (at pos 10000) |
%C | 1000 | 3 | Autoguider Colour filter (pos 3) |
%F | 1104 | O | TV flat mirror (out) |
%S | 1004 | C | TV shutter (closed) |
%T | **** | ****** | Drift scan table (not used now) |
%D | 1104 | 1 | Main CCD filter wheel (pos 1) |
%B | 0002 | Overall box status added by 4MS |
The thing to look for is that the most significant bit in the status line (bit 12 of the status word) is set to 1. This means that the mechanism has been (or still is) initialised. The overall box status %B never has the high order bits set so ignore this.
n.b. This does NOT apply now to mechanisms %G (grisms) or %T (table) as these have been physically removed or de-commissioned.
THE BOTTOM LINE!
Command | Mechanism names | Comment |
SETUP mech_name | XPROBE, ND, YPROBE, COLOUR, FLAT, SHUTTER, CCD | Reload mechanism parameters |
ZEROSET mech_name | XPROBE, ND, YPROBE, COLOUR, FLAT, SHUTTER, CCD | Re-initialise by moving the mechanism to datum switch at engineering velocity |
SETUP mech_name | XPROBE, ND, YPROBE, COLOUR, FLAT, SHUTTER, CCD | Restore normal velocity by reloading parameters |
e.g. To re-initialise the autoguider Y probe:
SETUP YPROBE
ZEROSET YPROBE
SETUP YPROBE
See Note 5
Now enter the ALL? NOW command.
The data block below shows what the status should be for
any mechanism after a successfull initialisation. In practice, only
the re-initialised mechanism will show the status as shown in this table.
%X | 1304 | 000005 |
%N | 1104 | U |
%G | ******** | ****** |
%Y | 1304 | 000005 |
%C | 1104 | U |
%F | 1104 | U |
%S | 1104 | U |
%T | **** | ****** |
%D | 1104 | U |
%B | 0002 |
Undefined Mechanisms
Some mechanisms will show a U value. This means
although the mechanism was initialised successfully, the actual position
of a particular mechanism is Undefined. Driving a filter wheel
or the TV mirror or shutter to a defined position will clear this. Moving
a mechanism from ICS will also clear these U flags. In engineering
mode, the commands and example values to do this are :
Command | Valid positions |
15.000 XPROBE | 5 to 29200 steps |
3. ND | Valid position are 1 to 8 |
15.000 YPROBE | 5 to 29200 steps |
4. COLOUR | Valid positions are 1 to 8 |
IN FLAT | IN or OUT |
CLOSE SHUTTER | OPEN or CLOSE |
3. CCD | Valid positions are 1 to 6 |
IMPORTANT
An integer value as given on an X, Y, ND, COLOUR or CCD
move must contain a period '.' This can be placed anywhere in the
number i.e. 15.000 is the same as 1.5000, 150.00, 15000. etc.
If the mechanism(s) work ok in engineering mode, proceed to: Final checkout
n.b. The CCD, ND and COLOUR filter wheels do NOT have associated limit switches
See Notes 3 and 4
If the X PROBE is in a limit then enter the
commands :
LIMITOVERRIDE XPROBE | override limit switch |
ZEROSET XPROBE | move mechanism to datum switches at engineering speed |
SETUP XPROBE | restore normal speed by re-loading mechansim parameters |
Check the mechanism has been initialised correctly by typing: X? Status returned must be %1304
If the Y PROBE is in a limit then enter the
commands :
LIMITOVERRIDE YPROBE | override limit switch |
ZEROSET YPROBE | move mechanism to datum switches at engineering speed |
SETUP YPROBE | restore normal speed by re-loading mechansim parameters |
Check the mechanism has been initialised correctly by typing: Y? Status returned must be %1304
If the TV FLAT is in a limit then type the commands
:
LIMITOVERRIDE FLAT | override limit switch |
ZEROSET FLAT | move mechanism to datum switch at engineering speed |
SETUP FLAT | restore normal speed by re-loading mechanism parameters |
Check the mechanism has been initialised correctly by typing: F? Status returned must be %1104
If the TV SHUTTER is in a limit then type
the commands :
LIMITOVERRIDE SHUTTER | override limit switch |
ZEROSET SHUTTER | move mechanism to datum switch at engineering speed |
SETUP SHUTTER | restore normal speed by re-loading mechanism parameters |
Check the mechanism has been initialised correctly by typing: S? Status returned must be %1104
Verify that all mechanisms have been successfully initialised by typing the ALL? NOW command.
Some mechanisms will show a U status after initialisation. Follow the steps for an Undefined mechanism to clear this flag.
If the mechanism(s) work ok in engineering mode, proceed to: Final checkout
Box number | Channel 1 | Channel 2 | Channel 3 | Channel 4 |
SMDM 1 | AG X probe | AG ND filter | Red Grism (n.a) | Blue Grism (na.) |
SMDM 2 | AG Y probe | AG COLOUR filter | TV flat mirror | TV shutter |
SMDM 3 | Table (n.a) | CCD filter wheel | Not used | Not used |
n.a. Not Applicable. These mechanisms have been removed (Grisms) or are de-commissioned (Drift scan table)
See Note 2 for more information.
If any of the values in the data block display random ascii values then the battery backed ram is corrupted and will need to be cleared and restored. Use the ALL? NOW command to determine which SMDM is corrupt. Follow the instructions below to restore the corrupted SMDM.
e.g. If SMDM 1 is corrupted, you will see rubbish like this !
%X 5tY!J8*3?dh
%N 3g$w4DT8>e
%G 1[/r %3hvR6s
If SMDM 1 is corrupt then :
RESTORERAM XPROBE | clear out contents of SMDM 1 RAM |
SETUP XPROBE | download X probe parameters |
SETUP ND | download AGND parameters) |
ZEROSET XPROBE | move X probe to datum switches |
ZEROSET ND | move AGND filter to datum switch |
SETUP XPROBE | restore normal mechanism velocity |
SETUP ND | restore normal mechanism velocity |
If SMDM 2 is corrupted then :
RESTORERAM YPROBE | clear out contents of SMDM 2 RAM |
SETUP YPROBE | download Y probe parameters |
SETUP COLOUR | download AGCOL parameters |
SETUP FLAT | download TV flat parameters |
SETUP SHUTTER | download TV shutter parameters |
ZEROSET YPROBE | move Y probe to datum switches |
ZEROSET COLOUR | move AGCOL to datum switch |
ZEROSET FLAT | move TV flat to datum switch |
ZEROSET SHUTTER | move TV shutter to datum switch |
SETUP YPROBE | restore normal mechanism velocity |
SETUP COLOUR | restore normal mechanism velocity |
SETUP FLAT | restore normal mechanism velocity |
SETUP SHUTTER | restore normal mechanism velocity |
If SMDM 3 is corrupted then :
RESTORERAM CCD | clear out contents of SMDM 3 RAM |
SETUP CCD | download CCD filter wheel parameters |
ZEROSET CCD | move filter wheel to datum switch |
SETUP CCD | restore normal mechanism velocity |
Verify that all mechanisms have successfully initialised by typing the ALL? NOW command.
Some mechanisms will show a U status after initialisation. Follow the steps for an Undefined mechanism to clear this flag.
If the mechanism(s) work ok in engineering mode, proceed to: Final checkout
The 24V supply for the SMDM's is derived from two 12V Vero PSU working in series. It has been known to happen that one of these PSU's can crowbar or fail. Drop the front cover of the JAG 4MS controller and check the health of these PSU's. Measuring and seeing 24V across the pins on the SMDM power connector is useful, but is not the ultimate test as a PSU could be breaking down under load.
LOSS OF COMMUNICATIONS
Generally this is due to a loose connection with the DIL ribbon cable connectors going from the pair of triple ACIA boards in the 4MS crate to the 4 pin connectors on the back panel. Drop the front panel of the 4MS controller and make sure that these connectors are seated firmly down on the ACIA boards. The other possibility of course is a broken lead between an SMDM and the 4MS controller.
A more serious problem has been the RS422 line driver and receiver chips blowing up! These are located on both the ACIA boards in the 4MS crate and on the CPU card within the SMDM. If the 24V power to the SMDM's is ok, replace (or swop in the first instance) the RS422 chips on the ACIA board. These are marked as such.
n.b. Not all the chips on the ACIA boards use RS422 tx/rx chips. Some are for RS232 usage so BEWARE!
If the problem still persists, then change the RS422 tx/rx chips on the SMDM cpu board. Usually its best to change the entire board. There are spares available. It is quite simple to remove the 4 front plate screws with the SMDM box still in-situ and remove the CPU board. If this is done, you will need to go through the steps of restoring the on board ram and mechanism setup as described above.
The ABSOLUTE position for the autoguider probes can only be read from the engineering mimic which shows the status bits. From the ENGINEERING TERMINAL enter the following:
15.000 XPROBE 15.000 YPROBE
This will CENTRE the probes. Check that the engineering mimic status is: %1004 15000 for both probes. Now enter:
5. XPROBE 5. YPROBE
This will PARK the probes. Check the mimic that the status returned is: %1304 000005 for both probes
See Notes 3 and 4
n.b. If only the mechanism step count values are
shown and NOT the status information on the engineering mimic, type:
ON
DETAIL
There is no need to go into IC mode for this operation.
If the X or Y probe does not return a status of %1304 at a step count of 5 then re-initialise it as described in: Restoring a mechanism
If the mechanism(s) work ok in engineering mode, proceed to: Final checkout
You will notice checking the engineering mimic that whenever the filter wheel moves to position 1, the datum switch flag bit is set. This is a good test for lost movement. i.e. If a filter is suspected of vignetting the beam, move it to position 1 and check for the %1104 status.
n.b. If only the mechanism step count values are
shown and NOT the status information on the engineering mimic, type:
ON
DETAIL
There is no need to go into IC mode for this operation.
If a wheel shows a %1004 status at position 1 it is misaligned and will need to be re-initialised as described in: Restoring a mechanism
It has also been found that the locking screw that holds the main CCD filter wheel can work loose causing the wheel to slip. If the wheel is suspected of losing position, check that it is inserted correctly, is in mesh with the sprocket and that the locking screw is firmly (but not over) tightened before re-initialising.
It should also be noted that a microswitch operates when the door to the filter wheel compartment is opened. Although the flag set (bit 11) by this switch is not read by 4MS, its status is detected at ICS level. Thus a status of %1504 means that although the wheel has been zeroset the door is open. This will prevent a move of the filter wheel from the ICS.
If the mechanism(s) work ok in engineering mode, proceed to: Final checkout
The switch on this break-out box MUST be set to the 'SPARC' position. The other position allows for example; a laptop PC to be connected for line diagnosis or monitoring.
When you are satisfied that the JAG works ok under engineering control proceed as follows:
ICS Command | Operations | Engineering mimic status |
SYS> tv on | Moves the TV flat mirror into the beam and opens the TV shutter | F=%1004 S=%1104 (datum active) Note |
SYS> tv off | Removes the TV flat mirror from the beam and closes the TV shutter | F=%1104 (datum active) S=%1004 |
SYS> autocentre | Moves the autoguider X and Y probes to centre of field (15000 steps) | X=%1004 Y=%1004 |
SYS> autopark | Parks the autoguider probes (pos X=5 Y=5) | X=%1304 Y=%1304 (both datums active) |
SYS> autocol 4 | Moves the autoguider colour filter wheel to position 4 | %1004 |
SYS> autocol 1 | Moves the autoguider colour filter wheel to position 1 | %1104 (datum active) |
SYS> autond 4 | Moves the autoguider ND filter wheel to position 4 | %1004 |
SYS> autond 1 | Moves the autoguider ND filter wheel to position 1 | %1104 (datum active) |
SYS> filter 3 | Moves the main CCD filter wheel to position 3 | %1004 |
SYS> filter 1 | Moves the main CCD filter wheel to position 1 | %1104 (datum active) |
Note
After the TV shutter opens and closes a few times, the
datum switch is not always activated when the shutter is open? This is
due to a small amount of hysterisis with this mechanism. This is not important.
If however no light is entering the TV camera (and the camera appears to
be working ok) it would be advisible to re-initialise the TV shutter.
THAT'S IT !
If the JAG still doesn't work, there is a REAL hardware problem that will need to be investigated.
The example shown below would be valid status for mechanisms Xprobe or Yprobe after initialisation.
i.e. %1304
Bits generated from SMDM | Bits added by 4MS | |||||||||||||||
Bit | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 |
Value | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
Function | lower
limit switch active |
upper
limit switch active |
stepper
motor moving |
mech
init'sed ok |
datum
switch 4 active |
datum
switch 3 active |
datum
switch 2 active |
datum
switch 1 active |
not
used |
not
used |
datum
failure |
engr's
control on |
monitor
mode invalid bit |
last
comm compl'd bit |
not
used |
mech
busy |
Notes | 1 | 1 | 2 | 3 | 4 |
1. Datum switch 3 is not used. Datum
switch 4 is used only for the filterwheel door (open or closed status).
2. A LIMITOVERRIDE forces this to
Bit=1
3. Bit=1 Last command completed
Bit=0 Command in progress or failed to complete
4. Bit=1 Mechanism busy
Bit=0 Mechanism not busy
Note 2
Parameter values for each mechanism are stored in EPROM
in the 4MS controller. These include things like:
Note 3
When doing a ZEROSET, a mechanism
moves at
engineering velocity. This is very slow and can take sometime to complete
especially in the case of the autoguider X and Y probes if they happen
to be a long way off from the datums. n.b. If a mechanism is ALREADY
at the datum and a ZEROSET is entered, it will move out of the switch
a small distance at normal velocity then re-seek the datum switch at engineering
speed.
For reasons unknown, sometimes the command does not always complete and the mechanism stops short of the datum(s). If this is the case, type ZEROSET again. If you are using the JAG A&G box engineering terminal in the control room, listening to mechanisms moving over the loudspeaker gives confidence that at least something is happening!
Note 4
You may see a B (busy) status returned if you
enter a command more than once. This is sometimes the case if the autoguider
probes have not initialised at the first attempt and you re-enter the command.
If this happens, type RESET which will reboot the 4MS and try again.
You will have to re-type IC so that the ALL? NOW command
shows all mechanism status. If the terminal happens to lock up when typing
RESET,
you will have to power cycle the 4MS crate in the telescope rack.
Note 5
Once a mechanism is successfully initialised
the normal
velocity must be restored. If this is not done, the ICS software will
generate a timeout message as it expects an operation to be completed within
a defined time period.
i.e. You MUST enter a
further SETUP command AFTER doing a ZEROSET to restore the mechanism's
correct operating speed.
Document history
Ver 2.0 This re-written document first issued :
25th May 2001 ejm
Ver 2.1 Filterwheel door switch info added : 12th
Sept 2001 ejm
Ver 2.2 The autoguider probe positions using the
AUTOPARK and AUTOCENTRE commands from the ICS are now in agreement on both
the observers and engineering mimic displays. Removed temporary notes :
13th Dec 2001 ejm
Ver 2.3 4MS PSU info added : 29th July 2002 ejm
Ver 2.4 ON DETAIL command added in Analysis section
: 19th Sept 2002 ejm
This file: http://www.ing.iac.es/~eng/ops/jkt/taming_the_jag.html