From jmb@ing.iac.esThu Aug 28 19:40:50 1997 Date: Tue, 26 Aug 1997 19:57:43 +0100 (BST) From: Jonathan Burch To: Frank Gribbin Subject: WHIRCAM Software Info Dear Frank, There follows some information on the WHIRCAM software and related software ... Cheers, Jonathan 1) UT The WHIRCAM data-acquisition software gets the time for the headers from the VAX system clock. At UKIRT they don't appear to change the VAX system clock to account for a change to summer time or anything like that - perhaps they don't have daylight-saving time on Hawaii. Anyway, I modified the software so that you can specify the difference between the VAX system clock time and Universal Time using a logical name, so that the VAX clock can be changed to summer time and the correct UT will still get put in the headers. The logical name is LOCAL_TIME_OFFSET, and it is set in the WHIRCAM account's LOGIN.COM. When the IRCMVX system clock is changed from UT to BST, LOGIN.COM should be modified to set LOCAL_TIME_OFFSET to 1 (instead of 0), and it should be set to 0 when the clocks are changed back to UT in the autumn. 2) Log files Unlike the normal optical observing system, WHIRCAM has no on-line log program. The headers for the night log must be generated at the end of the night using a program which extracts the relevant information from the data file headers. Details of how to do this are included in the WHIRCAM software instructions, and are familiar to TOs. The software involved consists of a couple of command procedures and a couple of executable images. The command procedure [LPLOGS.LOGDATA]WHIRCAMLIST.COM does a bit of parameter-validation and string-handling, then it sets a couple of logical names before executing [LPLOGS.WHIRCAM]WHIRCAMLIST.EXE. This runs through the data files for the given date, extracting the required information from the headers and writing a text log in the file yymmdd.LOG. I have modified WHIRCAMLIST.COM to cope with finding WHIRCAM files on either the new data disk (DISK$WHTDATA2) or the old data disk (DISK$IRCAM) or both. The TO then has to include the header list in the night log in the normal way. Finally, the TO should use WHIRCAMWHT.COM to generate the finished article - WHIRCAMWHT.COM invokes a modified version of CRB's LOGFORMAT.EXE, which is necessary to have the correct column headings for the WHIRCAM information in the log file. However, sometime during WHIRCAM's period of inoperability a new lplogs account was set up on lpss1, which the TOs use in preference to the VAX account. Currently they must use the VAX account to generate the list of header information, but for the last couple of runs they have chosen not to use it for generating the finished article - they have simply ftped the header list to lpss1 and used the standard procedure for generating the latex file. This means that the column headings in the final log file are not correct for the WHIRCAM information, but I can't get too worked up about this. Various improvements could be made here - provide some kind of on-line logging program for WHIRCAM; provide a sparc-based program to generate the list of header information at the end of the night; get CRB to modify his software on lpss1 to cater for WHIRCAM logs. At the moment it works adequately, however, and I believe that the first of these is definitely not worth the effort, the second might be worth it if were not too much effort, and the third probably would be a good idea since it's someone else's effort (it would be up to CRB). 3) AUTOFITS AUTOFITS should not be used for WHIRCAM - some time ago Vik discovered that it slowed down data-taking enormously, presumably because it is continually accessing the disk and therefore slowing down the data writing. This might be improved if the logical name defining the data files were made more specific - I think a major contributing factor might be that it is defined to be something along the lines of [...IDIR]I%%%%%%_*_1.SDF, so that AUTOFITS is continually looking in the directories of all previous runs that are still on disk. Things might be better if it were tied to the current date, which would require some way of allowing this to be specified. In any case, I've got a feeling that, for a large number of WHIRCAM files (several hundred per night is quite common, more than a thousand is not uncommon), AUTOFITS would not finish the tape-writing before WRITE_FITS even if AUTOFITS were started at the beginning of the night and WRITE_FITS at the end. 4) WRITE_FITS I have provided software in the TPCOPY account for writing WHIRCAM files. You simply enter SELECT_WHIRCAM before starting WRITE_FITS, as explained in the login message screen. The SELECT_WHIRCAM command procedure sets the data directory logical name to pick up WHIRCAM data, and after changing the data disk I modified it to cope with WHIRCAM files being on either the new disk (DISK$WHTDATA2) or the old disk (DISK$IRCAM) or both. The WRITE_FITS command procedure has been modified to cope with WHIRCAM data files. 5) TAG Observing with WHIRCAM usually involves offsetting the telescope a great deal, and for this the TAG task is used. The general idea here was pretty well described in my recent message about the "LOST GUIDING" fault report. The current operational version of TAG is V0-4-4, and this is not built from a CMS class. A while ago I consolidated the software into a new V0-5 CMS class, rebuilt TAG from that class and tested it (I checked that the WHOFFSET action behaved identically to V0-4-4). There is no reason not to make V0-5 the operational version as far as WHIRCAM is concerned, but I didn't want to do this so soon before my departure since there was no intervening WHIRCAM run to throw up any problems with it. To get the TAG subsystem loaded it is necessary to include the TAG version of TEL at the beginning of the WHT_TEL_DIR logical name path, as is done at present. When the next major system release is prepared, TAG should be put in as a subsystem in its own right and the TAG version of TEL can then be dispensed with. There is also a version of TAG called WHIRCAM-AT-CASS in the quickfix directory. This contains the software I had just about got working when WHIRCAM blew up last year. It might be useful if WHIRCAM is ever mounted at Cass again (which it has been agreed would not be a good idea if no explanation for the mishap can be found), and might contain a couple of useful bits of information for TAG's planned CLOFFSET (closed-loop offset) action. In the long run, offsets for WHIRCAM at Cass would be handled by this CLOFFSET action anyway (although this needs a bit of thought since people don't always want to do closed-loop offsets but in some cases would want to leave the autoguider probe where it is and do an open-loop offset instead). 6) Data disks Two logical names must be changed in order to switch the data disk - they are DISK$DATA and DISK$UKIRTDATA. They are defined in the file UKIRTCOM_DIR:SYSTARTUP.COM, which is invoked by the VMS SYSTARTUP procedure. The LPLOGS log-file software, and TPCOPY tape-writing software, have been modified to cope with the data files being on either or both of these disks, so they don't need to be worried about when/if the data disk is changed. 7) File protection The top-level WHIRCAM data directory on DISK$WHTDATA2 has been set up so that WHIRCAM data files are created with ACLs which allow anything with the WHT_USER identifier CONTROL (plus READ, WRITE and EXECUTE, but not DELETE) access to them. I have set up a symbol in OBSERVER's LOGIN.COM, called UNPROTECT, which grants DELETE access and allows the files to be deleted. It's not recommended to delete the files from the WHIRCAM account since rights identifiers don't seem to work on IRCMVX, and currently the WHIRCAM account can delete the files with no safeguards. PVDV hasn't managed to solve the problem of rights identifiers on IRCMVX, but he suspects that it might be because the RIGHTSLIST.DAT was generated by a different version of VMS. 8) Disk space I notice that there are nearly two million blocks of WHIRCAM data on DISK$WHTDATA2. My software news note was intended to make people (especially VBF and SAs) aware that WHIRCAM data is now on that disk, so that it is properly dealt with, but perhaps the message hasn't got through? 9) Contacts I used to contact Alan Bridger (AB) at JACH quite a lot, but he has now returned to ROE. His replacement is Nick Rees (NPR), with whom I communicated only a couple of times (in order to get a copy of the latest IRCAM software at the time) before WHIRCAM blew up, and I've not been in contact since. Alan Pickup (DAP) at ROE is very helpful and I contacted him many times. Pat Roche (PFR) at Oxford was in charge of the project initially. He is an extremely pleasant chap, and I have contacted him a few times about the filter and focus mechanisms and their software, which he was involved with. Chris Mayer originally modified the software for the WHT. Joel Sylvester (ROE) is the ALICE expert. Magnus Paterson was involved in the attempt to install snapshot mode, along with Alan Pickup and Archie Smyllie (who I believe has now left). 10) ADAMNET I think you know about ADAMNET. 11) Software Here is a brief description of the most important tasks. AIM - handles execution of EXECs; ALD - ALice D-task, handles coordination of actions for observing; ALF - ALice Filing task, handles writing of data files; VA - Vax Alice task, handles communication with ALICE; WHIRCAM - new d-task, written by CJM, for controlling the filter and focus mechanisms. The MOTORCODE software (see below) lives with the WHIRCAM task. The tasks are documented in more detail in the paper-copy programmers' guides I gave you. Other relevant pieces of software are as follows - CAL - library of subprograms for handling dates and times; COM - contains a command procedure, IRCAM.COM, which is involved in loading and running the system. This sets simulation flags which it is sometimes useful to modify for testing purposes; CONFIG - subsystem containing a data file of filter names, positions etc; GETDAY - program which sets up parameters concerned with the date; PRC - contains an ICL procedure, IRCAM.ICL, which is involved in loading and initializing the system; SCT - Screen Control Tables, defines menus and procedures associated with the control system interface; TELU - library of subroutines for the AIM task which communicate with the telescope control software. Modifications for the WHT are documented in an accompanying message. Use of the UKIRT software planes is described in the accompanying message from CJM. 12) Motor control code The memory in the motor controller cards seems to be rather less non-volatile than it claims to be - the software must be reloaded whenever the motor controllers have been powered off. This is usually a task to be done once as part of setting up WHIRCAM. It can be done when the control system is not running by entering the command MOTORCODE at the DCL prompt; or when the control system is running by pressing SELECT on the SMS screen to get an ICL prompt, spawning out of ICL, and entering MOTORCODE at the DCL prompt. The motorcode program displays each line of control code on the terminal as it downloads it through the serial port to the controllers, and issues error messages if the serial port appears to be disconnected or unavailable. The motorcode software lives in the WHIRCAM subsystem. 13) Testing A quick and easy check that things are working is to execute the ARRAY_TESTS system EXEC and to analyze the results with the ARRAY_TESTS data reduction procedure - CRB and AWR, amongst others, know how to do this. Another confidence-building test is to perform a mosaic with the TCS simulating and the autoguider centroiding, and check that the STRED data reduction procedure can reduce the resulting frames. In my EXEC directory (DISK$OPENVMS060:[WHIRCAM.DATA.JMB.IRCAMEXEC]) are various test EXECs which do mosaics, filter moves and so on. Checking the headers is the same as doing so for optical files, I guess - I use KAPPA FITSLIST. I also used KAPPA for checking the fidelity of the data arrays read back from tape. 14) Mechanisms The filter and focus mechanisms are not very reliable. The control system used to try to datum them on startup but, since they often failed to datum which then messed up the control system startup, I removed the automatic datum attempt and it is now necessary to do it manually after startup. Some time ago, after the wheels had been particularly unreliable for a while, it was found that some of the filters had come loose from the wheels and were clashing when the wheels were moved (there is very little clearance). Judging from recent fault reports this might have happened again, but I expect Kevin is planning to look at this before the next WHIRCAM run (it might be a good idea to carefully check that the correct filters are being put in the beam for each of the possible menu options too). 15) Common problems and solutions Filter/focus move failures - 1) try again, 2) datum them and then try again, 3) reload the motor control software and then datum them and try again. Problems with ALICE (a common one is "timeout occurred on tag receive", which has nothing to do with the TAG task) - 1) close down and restart the control system, 2) close down the control system, power-cycle ALICE (not forgetting to disable the array before powering-off and reenable it after powering on), then restart. Task load failures - this is usually caused by the good old ADAM problem of a corrupt HDS file, and is cured by deleting the offending task.SDF file. Such files are to be found in [WHIRCAM.ADAM] or in [WHIRCAM.DATA..IRCAMCONFIG]. ALICE data doesn't make sense - sometimes this can be caused by saturation, sometimes because the array wasn't enabled before it was used - it seems that if you try to use the array when it is disabled, it will give nonsense data thereafter, even if you subsequently enable it, and the only solution is to power-cycle it. 16) Known problem The data-taking is handled by several queues, which buffer the data until it has been written to disk. This all works fine for full-frame images but there is a problem for rapid sub-frame readouts. Running out of space in the ALICE memory is handled cleanly, but running out of "image slots" isn't. The problem with rapid sub-frame readouts is that you run out of "image slots" before you run out of memory. The problem only arises if you set the readout area to something smaller than 256x256 and then try to take data at a greater rate than the disk-writing can keep up with (which is about 4 seconds per frame), and it was only a problem during the Neptune occultation run last September. It's known about by JACH/ROE, and is probably fixed in their software by now.