INS-DAS-22

Interfaces for collection of FITS-header packets

Issue 4.2, 2001-05-07 ;

Guy Rixon, UK Astronomical Technology Centre (gtr@mrao.cam.ac.uk)

Introduction

Purpose of this document

Interfaces are described which transfer FITS-header data between the parts of the observing system. This document defines those interfaces, and authors of UltraDAS software should write their programmes to conform to the current issue of this document. If the implemented interfaces are later found to differ from the document then the software should be changed.

History of this document

Issue 1.1, 1998-11-03

The original issue: an unreviewed draft of the interface.

Issue 2.1, 1999-02-22

"PKTS" is renamed "HCT". The naming of header-packet files is revised (includes run-number; ends in .pkt; all in lower case). The rules for configuring the system are simplified (no "CONFIGURE" actions; instrument name is passed with ARCHIVE_B/HCT, ARCHIVE_E/HCT and ARCHIVE/CAM). The run-number is an argument to ARCHIVE_B/ARCHIVE_E. The format of the packet-list file is changed (length of packets is not stated).

Issue 3.1, 1999-03-22

The argument lists for ARCHIVE_B/HCT and ARCHIVE_E/HCT are revised to include essential information missed out in the previous issue. The responsibility for deleting old packet files us moved from the HCT to the camera server. The section about using the VAX-ADAM HCT is added.

Issue 4.1, 1999-07-07

The packet files are no longer required to be in $OBSSYS/etc. The final form of UltraDAS' configuration files, defined in INS-DAS-25, is specified as the way of choosing packets.

Issue 4.2, 2001-05-07

Updated links to other HTML documentation.

Scope of the interfaces

This specification applies to UltraDAS. The Data-Cell DAS on the INT and JKT has similar interfaces that differ in detail.

Header packets are generated by the TCS, ICS and CIA, passed around the system as disc files, and consumed by the DAS. The production and consumption of packets is controlled by a set of DRAMA action on server programmes of UltraDAS. The exact set of packets to be collected is set by a configuration file "owned" by UltraDAS.

The context of packet collection is explained in the architecture document [1].

This text defines the action interface, the format, location and naming of the packet files, the format of the configuration file and the overall sequence of the operation. The contents of the packets are defined in documents INT-DAS-2 [3] and JKT-DAS-2 [4] .

References

Architecture for the UltraDAS

INS-DAS-23

Definition of the Flexible Image Transport System (FITS)

http://www.gsfc.nasa.gov/astro/fits/fits_home.html

FITS headers for INT observations

INT-DAS-2

FITS headers for JKT observations

JKT-DAS-2

Configuration files for UltraDAS

INS-DAS-25

Use case: collect FITS-header packets

The packet-collection interfaces are designed to be used in a single, specific sequence. This section defines the sequence.

Purpose of use case

To pass header information from the TCS, ICS and CIA into the DAS for inclusion in FITS files.

Actors

Client programme; agent programme HCT on system computer; camera-server programme "CAM", say, on DAS computer; various D-tasks D1, D2,..Dn, say, in the ICS and TCS. (That is, the name "HCT" is a fixed part of the architecture, but all the other task-names vary from configuration to configuration.)

Outline

The client managing the observation signals HCT to collect packets. HCT signals a list of D-tasks in the ICS and TCS to make their packets. The client signals the camera server to collect the packets; the server copies the packet files into the FITS header.

Typical course of events

	Client	HCT	Dn	CAM
0.	Use case begins when the client is managing an observation. It gets the name of the instrument and the name of the camera from its environment variables.
1.	Invokes RUN on CAM.			Allocates the run number and returns it to the client.
2.	Waits for start of integration.
3.	Invokes ARCHIVE_B on HCT, passing instrument name, camera name and run number.
4.		Reads list of packets; invokes ARCHIVE_B on Dn as necessary, passing the packet name, camera name and run number.
5.			Starts collecting data for packet; returns good status to HCT.
6.		Returns good status to client.
7.	Writes "observation" packet; waits for end of integration.
8.	Invokes ARCHIVE_E on HCT, with same arguments as ARCHIVE_B/HCT.
9.		Reads list of packets; invokes ARCHIVE_E on Dn as necessary, with same arguments as ARCHIVE_B/Dn.
10.			Completes collection of packet data; writes packet file on disk; returns good status to HCT.
11.		Returns good status to client.
12.	Invokes ARCHIVE on CAM.			Creates FITS file; reads packet list and copies packets into FITS header. Deletes the packet files after they are used. Returns good status to client.

Variations

In step 5 or 10, one or more of Dn does not respond or returns bad status for the action. HCT aborts the transaction with that task when the bad status is received or after 10s if there is no response. HCT logs an error explaining that a packet could not be collected. Transactions on other Dn are not affected. HCT returns bad status to the client, but the client ignores this.
In step 5 or step 10, Dn cannot collect or write to disk the packet data. Dn logs an error explaining the failure and aborts the action with a bad status.
In step 12, one or more of the packet files is not there. CAM replaces the packet with a FITS comment explaining what is missing. CAM does not log an error as the failure to produce the packet is assumed to have been reported at an earlier step. CAM does not abort the ARCHIVE action, but eventually returns a bad status for it; the client ignores this status.
In Step 4, 6, 9 or 11, HCT does not respond to the client. The client aborts the observation.

Notes

"Reading the packet list" means parsing the list of header packets described below.
CAM will have to read the size of the packets from the packet list at the start of the observation, in order to work out where the image starts in the FITS file.
The bad stati returned to the client by the DAS servers are meant to be ignored in normal running; the task that should have generated the packet will have reported the error and it is almost always better to press on with too few packets than to abort. The status codes are included to aid debugging.

The action interface

Command	Task	Action
ARCHIVE_B	HCT	Starts collection of all packets
ARCHIVE_B	TCS or ICS task	Starts collection of one packet
ARCHIVE_E	HCT	Completes collection of packets
ARCHIVE_E	TCS or ICS task	Completes collection of one packet
ARCHIVE	camera server	Adds packets to FITS header

Command ARCHIVE_B on task HCT

Purpose: start generation of packets at the start of an observation.

Messages: command, status; action has multiple phases, is spawnable, but may not be kicked.

Arguments with command:

Argument1: name of instrument (as given in the configuration file described below).
Argument2: name of camera.
Argument3: run number.

Arguments with status: none.

Action:

Invoke ARCHIVE_B on all tasks listed in the configuration.
Wait for all subordinate actions to complete, fail or time out (see use case, above).

Outcomes:

STATUS-S-OK

the packet-generation activity has started.

UDAS-E-PKTSMISSING

some or all of the packet collection could not be done.

Command ARCHIVE_E on task HCT

Purpose: complete generation of packets at the end of an observation.

Messages: command, status; action has multiple phases, is spawnable, but may not be kicked.

Arguments with command:

Argument1: name of instrument (as given in the configuration file described below).
Argument2: name of camera.
Argument3: run number.

Arguments with status: none.

Action:

Invoke ARCHIVE_E on all tasks listed in the configuration.
Wait for all subordinate actions to complete, fail or time out (see use case, above).

Outcomes:

STATUS-S-OK

the packet-generation activity is over; the desired packets now exist on disc.

UDAS-E-PKTSMISSING

some or all of the packets were not created.

Command ARCHIVE_B on a TASK in the ICS or TCS

Purpose: start generation of one packet at the start of an observation.

Messages: command, status; action has one phase.

Arguments with command:

Argument1: name of packet, e.g. "observation", "tcs".
Argument2: name of camera that needs the packet, e.g. "TEK3", "WFC", "INGRID".
Argument3: number of run.

Arguments with status: none.

Action:

Start recording data for the packet. (The exact activity varies from task to task.)
Return within 0.5s.

Outcomes:

STATUS-S-OK

the packet data are being collected.

UDAS-E-BADPKTNAME

the name of the packet is invalid (typically too long).

UDAS-E-BADCAMNAME

the name of the camera is invalid.

UDAS-E-PKTDATAFAIL

the packet data cannot be collected (task also logs an error-message explaining the failure).

Command ARCHIVE_E on a TASK in the ICS or TCS

Purpose: complete generation of one packet at the end of an observation.

Messages: command, status; action has one phase.

Arguments with command:

Argument1: name of packet, e.g. "observation", "tcs".
Argument2: name of camera that needs the packet, e.g. "TEK3", "WFC", "INGRID".
Argument3: run number.

Arguments with status: none.

Action:

Finish recording data for the packet. (The exact activity varies from task to task.)
Write the packet file to disc.
Return within 0.5s.

Outcomes:

STATUS-S-OK

the packet file is complete on disc.

UDAS-E-BADPKTNAME

the name of the packet is invalid (typically too long).

UDAS-E-BADCAMNAME

the name of the camera is invalid.

UDAS-E-PKTDATAFAIL

the packet data cannot be collected (task also logs an error-message explaining the failure).

UDAS-E-PKTFILEFAIL

the packet file could not be written (task also logs an error-message explaining the failure).

Command ARCHIVE on a camera server.

Purpose: read header packets and add to FITS header.

Messages: command, status; action has one phase.

Arguments with command:

Argument1: name of instrument, as given in list of packets, below.

Arguments with status: none.

Action:

Copy the packets listed in the configuration from their files into the current FITS header.
If any packets are missing, write FITS comments noting this into the header.
Add to the header FITS data that is available inside the DAS.

STATUS-S-OK

packet copying has finished; the all data are correctly in the header.

UDAS-E-NOTARMED

the camera is not "armed": i.e. there is no observation in progress to receive the packet data

UDAS-E-PKTSMISSING

some or all of the packets could not be copied into the header .

The packet files

Packet files contain FITS descriptors, also known as "card images". The descriptor format is defined in the FITS standard. The files are text files, and hence can be inspected using "cat" and "more", but the FITS format does not allow line-break characters so the formatting may be poor. A packet file should not contain partial descriptors but it may contain partial FITS-blocks. It is valid for a packet file to be empty and consuming programmes should not complain about this.

The packet files are named packet_camera_run-number.pkt, e.g. observation_tek3_100042.pkt or cagb_ingrid_100056.pkt. Since the files are used by a Unix file system, case is significant. Since some of the files have to be shared with VMS systems via NFS, and since VMS finds it difficult to deal with mixed-case names, the names of packet files are entirely in lower case. The camera name is the name by which the camera server logs in to the DRAMA message net, converted to lower case.

The files can be written on any disc that is available to the DAS computer for both reading and writing, and different packets can be in different directories if required. The configuration files for UltraDAS define where the DAS should look for each packet. The ideal is to write all packets in the sub-directory var of the observing-software tree for the telescope: i.e. /int/var, /jkt/var or /wht/var.

Packet files are created with read and write access for all users. This ensures that undeletable files from a previous session can never jam the packet-collection facility.

Because UltraDAS may be archiving one run while generating another, it is necessary to include the run-number in the file-name. Having done this, the camera name isn't strictly necessary to distinguish the files, since a single run applies only to one instrument. However, it is useful to include it for debugging purposes.

The configuration files

UltraDAS' configuration files are defined in detail in INS-DAS-25 [ 5 ]. The definition there supersedes the specification of configuration files in previous issues of this document. The configuration method is outlined here for information.

Only the packets statements in the files affect packet collection. This is a possible packets statement for IDS:

0 packets  IDS  observation:given:/int/var  tcs:TCS:/int/var/tcs  agb:AGB/int/var  ids:IDS:/int/var

This says that UltraDAS must collect four packets per run. The first, called observation, is given gratis to UltraDAS and does not need to be invoked by the HCT (in fact, this packet is provided by the client-programme that is sequencing the run). The second packet, called tcs, is provided by the TCS D-task and arrives in the directory /int/var/tcs (the D-task TCS is a VMS task; the directory is assumed to be NFS-mounted by the DAS computer from the telescope computer). The HCT has to order the production of this packet by invoking the ARCHIVE_B and ARCHIVE_E actions on TCS. The third and fourth packets are provided by the tasks AGB and IDS respectively; the HCT has to order these packets too.

Use with the VAX-ADAM system on the WHT

The text above assumes that HCT is a programme provided as part of the CIA running on a Unix system-computer. On the WHT, where UltraDAS is sometimes interfaced to the VAX-ADAM ICS, this is not necessarily true.

The ADAM system has its own HCT, running on the VAX system-computer, which replaces the Unix HCT. The interface is the same, but the following specializations apply.

The HCT on Unix is not used and must not be started.
The instrument/configuration name used by UltraDAS matches the "channel name" used by the ADAM system. That is, UltraDAS' configuration file is set up to match existing conventions of the ADAM system. The name is typically the name of the instrument, e.g. "UES".
The VAX HCT produces a single packet-file which typically has packets from several devices concatenated within it. UltraDAS treats this as one packet. The ADAM system fixes the name of this packet and UltraDAS' configuration file must be set up to match that name.
The contents of the single packet file - that is, the actual list of packets collected - is set by the configuration files that the VAX HCT has traditionally used. UltraDAS has no access to these files.
In this mixed configuration, the HCT on VMS does not and may not read UltraDAS' configuration files.

This is part of a possible configuration file for UES the WHT:

0 packets  UES  observation:given/wht/var  ues:given:/wht/var/ics

This implies that the observation packet comes from the Unix side (from the client programme, as normal), and that the rest of the packets, include the TCS data, are concatenated in the UES packet-file. The latter is packed up from a sub-directory of /wht/var that is assumed to be mounted from the VAX ICS.