Data Reduction Pipeline

Download

The latest version of the IDL-based (v3.0 from September 2015, optimised for IDL version 7 and tested on versions 8.1 and 8.3) AF2/WYFFOS data reduction pipeline can be downloaded from the following link:

pipeline-current_v3.tar.gz

If using wget, first change the directory where you want to instal the pipeline and type:

wget http://www.ing.iac.es/astronomy/instruments/af2/pipeline/pipeline-current_v3.tar.gz

Version 3.0 includes some enhancements over the previous version, e.g. enhanced sky subtraction method, improvements in the wavelenght calibration routines, and in the flat-fielding algorithm, fixing of some issues related to data sampling, restructuring of the parameters in the configuration file, new procedure to check output spectra from pipeline, new routine to check the pipeline installation, etc.

Please contact Alireza Molaeinezhad (molaeiipm.ir) and Lilian Domínguez (ldping.iac.es) for further help on this version.

Installation

Full documentation for the pipeline is contained in the subdirectory /docs, including installation of the software.

Basically, the directory structure required to run the pipeline is shown in the figure below (extracted from the user manual v1). The pipeline directory contains the IDL-based AF2 pipeline procedures and some third-party IDL routines which are modified or customized to be used by the pipeline. In addition to pipeline directory, user must install the ASTRON (version 61: Aug. 2013 or later) and Coyote IDL libraries, in addition to built-in and default IDL libraries which are usually in IDL installation directory.

Below you will find the installation instructions and examples:

1. Untar and uncompress the downloaded file:

   tar zxvf <pipeline.tar.gz>

   Files will be untarred to a directory called pipeline_v#, where # is the version number. Directory pipeline underneath contains all the IDL procedures.

2. If not installed on your system, download and install the Coyote library to the pipeline directory:

   wget http://www.idlcoyote.com/programs/zip_files/coyoteprograms.zip

   unzip coyoteprograms.zip

3. If not installed on your system, download and install the ASTRON library to the pipeline directory:

   wget http://idlastro.gsfc.nasa.gov/ftp/astron.tar.gz

   tar zxvf astron.tar.gz

4. Download the Craig Markwardt's cmps_form.pro file for setting postscript parameters, to the pipeline directory:

   wget http://cow.physics.wisc.edu/%7Ecraigm/idl/down/cmps_form.pro

5. Check the enviromental variable IDL_PATH.
   • If you are using a c-shell terminal (csh, tcsh):

     printenv IDL_PATH

     If IDL_PATH doesn't exist, then set it to the directory containing the pipeline IDL routines, for example:

     setenv IDL_PATH "<IDL_DEFAULT>:+/home/user/pipeline_directory/pipeline"

     Otherwise, add the directory name to the existing IDL_PATH variable:

     setenv IDL_PATH "${IDL_PATH}:+/home/user/pipeline_directory/pipeline"

     If installed elsewhere, add also the ASTRON and Coyote IDL libraries to your IDL_PATH.

   • If you are using a bourne shell terminal (sh, ksh, bash):

     echo $IDL_PATH

     If IDL_PATH doesn't exist, then set it to the directory containing the pipeline IDL routines, for example:

     IDL_PATH=<IDL_DEFAULT>:/home/user/pipeline_directory/pipeline

     Otherwise, add the directory name to the existing IDL_PATH variable:

     IDL_PATH=$IDL_PATH:/home/user/pipeline_directory/pipeline

     If necessary, add also ASTRON and Coyote IDL libraries to your IDL_PATH. More about adding entries to your IDL_PATH variable can be seen here.

   We strongly recommend you to set your IDL_PATH as above in your startup file (.cshrc, .bashrc, .bash_profile...). Then reset your terminal, or source your startup file, to make sure that all changes are implemented correctly.

6. Change to your project directory and copy a suitable control data file from the uncompressed project directory (project/control_data_WHTWFC.txt or project/control_data_RED+4.txt, for data taken with the old WHTWFC or the new RED+4 detectors, respectively) to ./control_data.txt and edit accordingly.

   If working on an ING Linux computer, you can find a template of the control file for each detector used with AF2 at /usr/local/af2_pipeline/project/. Copy the respective template to ./control_data.txt file.

   Edit the line:

   ./                    ;   PDIR  Add project directory here

   accordingly, and remove "Add project directory here".

7. Change to your working directory (referred in the documentation as the project directory). Both the control_data.txt and the data directory, named as such, must be at your project directory. If your data directory is located elsewhere, update the PDATA entry in the control_data.txt file.

8. Start IDL:

   >idl

   and run the AF2 pipeline procedures following the instructions in the User Manual (it can be found in the uncompressed directory docs/):

   IDL>gui

Troubleshooter

X11 window display not properly working

The Coyote library might issue an error if IDL and X11 window manager are not handling the backing-store graphic mechanism properly (read more here). The problem can be fixed if you type the following at the IDL prompt:

IDL>Device,RETAIN=2

This can be added to the user's IDL_STARTUP file, if using the c-shell:

setenv IDL_STARTUP ~/mystartup.pro

And add Device,RETAIN=2 to the file mystartup.pro.

Cursor routine not properly working on Mac IDL installations

The cursor is not able to pick up spectra properly, for instance. See these remedies.

PDIR keyword not working properly

When starting up, you get the following error:

IDL> gui
% Compiled module: GUI.
% Compiled module: XREGISTERED.
% Compiled module: READ_CONTROL.
% Compiled module: READCOL_AF2.
% Compiled module: REMCHAR.
% Compiled module: GETTOK.
% Compiled module: STRSPLIT.
FATAL ERROR: PDIR keyword not found or duplicate entries found in the Configuration file

Check the entry PDIR at the beging of the control_data.txt file. It should point to your project directory. Don't forget to remove the sentence "Add project directory here".

No control_data.txt found

You get the following error when starting the gui:

IDL> gui
% Compiled module: GUI.
% Compiled module: XREGISTERED.
% Compiled module: READ_CONTROL.
% Compiled module: READCOL_AF2.
% FILE_LINES: Error opening file. File: control_data.txt
  No such file or directory

Check that the control_data.txt file, named as such, is placed in the same directory from where you run IDL.

No data found

When starting the gui you get the following error message:

IDL> gui
% Compiled module: GUI.
% Compiled module: XREGISTERED.
% Compiled module: READ_CONTROL.
% Compiled module: READCOL_AF2.
% Compiled module: REMCHAR.
% Compiled module: GETTOK.
% Compiled module: STRSPLIT.
% Compiled module: TAG_EXIST.
Fatal Error: There is no data within ./data/ directory

Check the entry PDATA in your control_data.txt file. If your data are not saved to the ./data directory, edit PDATA accordingly.