Isaac Newton Group of Telescopes
 | Home | Search | About ING | Astronomy | Public Information | Engineering |
¦ CSG Home Page ¦ Software Documentation ¦ Software Search ¦ External Software ¦ Catalogue ¦ 

EPICS: Debugging & Troubleshooting

  1. Overview

    EPICS is a software system for distributed control systems which uses a database resident in the VME crate to control instrument mechanisms and acquire status from the hardware. EPICS runs on top of the real-time operating system called vxWorks (from Wind River Systems Inc). Tools are available on the Suns to provide status from an instrument as well as a control interface. The primary displays are graphical user interfaces but other utilities are available on the Suns as well as directly on the VME crate to obtain further detailed information.

  2. Other EPICS Documentation

    Extensive documentation on the EPICS systems here at ING are available on the local Intranet. Follow the links

    Software group-> Software documentation-> EPICS

    A particularly useful page for troubleshooting purposes is the page showing the boot parameters for that instrument's crate. Each instrument section in the EPICS documentation has a page called `Bootconfig'.

    For the full worldwide picture, see the main EPICS web site at http://www.aps.anl.gov/epics

  3. EPICS systems

    EPICS runs on VME crates connected to the network which load vxWorks, EPICS software and data files remotely from a Sun host over the network. Currently all ING EPICS systems run on a Motorola CPU card (either the MV147 or MV167, both based on the 68000 series). The instruments using the EPICS software system at ING are (as of October 2000):

    Instrument VME Crate full host name IP address Sun Host
    WYFFOS wyffos.ing.iac.es 161.72.6.16 taurus
    ISISP isisp.roque.ing.iac.es 161.72.6.190 taurus
    INTEGRAL integral.roque.ing.iac.es 161.72.6.192 taurus
    INGRID ingrid.ing.iac.es 161.72.6.194 crater
    NAOMIOMC naomiomc.ing.iac.es 161.72.6.129 navis
    NAOMISDSU naomisdsu.ing.iac.es 161.72.6.128 navis
    OSCA oscaioc.ing.iac.es 161.72.6.134 taurus
    LIRIS (not yet commissioned) lirisioc.ing.iac.es 161.72.6.87 taurus
    USP usp.ing.iac.es 161.72.6.185 taurus

    The "Sun host" listed above is the computer from which the vxWorks and EPICS system software is loaded for that crate. This is subject to change (many Suns have the same set of files available using shared disk access).

  4. Access to the EPICS crates

    While a system is operational it is possible to log in to an EPICS crate remotely over the network, without having to use a directly connected terminal. Each crate has a login username that is the same as the crate name (e.g. user name `wyffos' to log in to crate wyffos etc).
    All VME crates should have a connection to a portserver which will allow a tip session from the whtics console. The information is in the document COMMS-PORT-1: Portserver Configurations, which describes the tip sesion name to use for connection.
    If the portserver connection is not available then crates may have a boot console connected directly to the crate's serial port. When troubleshooting with suspected network problems or the system does not boot properly, it may be necessary to set up a suitable console.

  5. Troubleshooting Hints

    Logging in to the Sun for troubleshooting

    A suitable account to be used when troubleshooting EPICS is whtmimic. This account is configured to be able to easily run up the instrument EPICS engineering screens as well as other EPICS utilities. It is recommended that the whtics or whtics1 machines should be used for this purpose.

    Testing a network connection

    If there appears to be no response from the EPICS system a likely cause is problems with the network or the EPICS crate's connections to the network. To check whether the crate is on the network use the `ping' command from any Sun terminal. e.g. if the wyffos crate is online then the command issued from the Sun and its (quick) response would be as follows:

    > ping wyffos

    wyffos.ing.iac.es is alive

    If the crate is not able to respond over the network there would instead be a pause (20 seconds or so) after the ping command: eventually the response would be :

    > ping wyffos

    no answer from wyffos.ing.iac.es

    Other EPICS test commands: from the Sun

    There are several commands available to examine or alter the values in the EPICS control database. Changing a value should only be done in the full knowledge of its effect! In practice, these commands are only useful when used in conjunction with the EPICS software source or a list of the database records. Ideally, a list needs to be made available to show the most important EPICS records in a system, particularly those which interact directly with the instrument hardware.

    Commands available from the Sun include:

    caget record.FIELD

    Display the value of the FIELD in the named EPICS record.

    caput record.FIELD value

    Set the specified value in the FIELD of the named EPICS record.

    cau

    An interactive utility to get, set or monitor changes in a specified EPICS record. Type help at the cau: prompt for further details.

    probe record.FIELD

    An interactive GUI utility to get, set or monitor changes in a specified EPICS record.

    Other EPICS test commands: from the EPICS console

    Unfortunately, the " " (quotes) shown below surrounding all symbols are necessary when working at the vxWorks prompt.

    dbl

    List all the EPICS records on this crate

    dbgrep "*string*"

    Search for records specified by the quoted string (analogous to grep on the Sun)

    dbgf "record.FIELD"

    Display the value of the FIELD in the named EPICS record.

    dbpf "record.FIELD", "value"

    Set the specified value in the FIELD of the named EPICS record.

    dbpr "record","level"

    List all the fields of a record along with their values. Level 1 is basic, level 3 or 4 shows almost all record fields.

    dbior "drivername","level"

    Print data about the specified I/O card and its data signals. Drivers available in ING systems include: "drvXy220", "drvXy240", "drvOms" and "drvXy566".

  6. Boot Parameters

    The EPICS system runs with the vxWorks operating system. vxWorks uses the following boot parameters. The example below shows the actual boot parameters used for booting the INGRID crate, see the Intranet pages for each instrument's Bootconfig.

    Parameter Comment
    boot device : ln Network device: do not change
    processor number : 0 do not change
    host name : crater Name of Sun host to load vxWorks from
    file name : /home/epicssm/ingrid/bin/mv147/vxWorks vxWorks boot filename
    inet on ethernet (e) : 161.72.6.194:ffffff00 This crate's IP address and netmask
    inet on backplane (b): (Not used)
    host inet (h) : 161.72.6.51 IP address of host (crater)
    gateway inet (g) : Gateway if required
    user (u) : epicssm Username for accessing the Sun host
    ftp password (pw) (blank = use rsh): Normally left blank
    flags (f) : 0x8 Boot flag (0x8 = fast boot)
    target name (tn) : ingrid Name of this crate on the network
    startup script (s) : /home/epicssm/ingrid/bin/mv147/st.cmd Startup script file for running EPICS
    other (o) (Not used)

    Normally there should be no need to change boot parameters. They should only be changed in the event of the host name being changed (e.g. the usual Sun host is unavailable) or if for some reason the boot parameters were corrupted (e.g. accidentally or due to a VME hardware fault).

    Setting Boot Parameters: the bootChange command

    From the EPICS console (directly or via remote login) the command bootChange should be used. This will prompt for a new value for each parameter. Simply hitting <Return> will preserve the current value. Special characters available include: "." (set the parameter value to blank) and "-" (step back to the previous boot parameter).


EPICS Software Manager
Last modified: Wed Aug 20 18:48:16 WEST 2003