General info about DPI pipeline (beta)

General info about DPE pipeline (beta):

• The DPE (du Pont echelle) pipeline is based on, and runs similarly to, the MIKE pipeline in terms of the commands needed to set up, as well as, the keywords associated with the various frames used in the pipeline.

  (MIKE pipeline documentation is available online at http://obs.carnegiescience.edu/Code/mike)

STEP 1: Setting up the database file

• First, you need to create a database file from a directory with DPE data in it. This database file will contain important properties gathered from the headers of the files in the input directory. To create this file for the directory /data1/vulcan/edwardv/dupont/may2011, for example, you can run the following command:

  $ DPEdb -d /data1/vulcan/edwardv/dupont/may2011

• Then, once you have this database file (named <directory>DPE.db by default), you should inspect the file to make sure there aren't any unwanted files in there. In the above case, it would be called "may2011DPE.db"

  Tips on editing the database file:

  1. Adding the word "ignore" to the end of the OBJECT name (last column) will tell the setup script to ignore that frame when setting up the pipeline.
  2. The keywords the setup script looks for in the OBJECT column are "thar" for the lamp frame (used for wavelength calibration), "milky" for the milky flat (used for flat-fielding), and "sky" for the twilight sky frames (used to find the order edges).
  3. Any objects with names other than the ones listed above will be treated as science targets, unless you specify additional arguments with the setup script (see below).

STEP 2: Generating the Pipelines

• As an initial check of your database file, you can run DPEsetup, with only the .db file as an argument to list the targets that it will generate pipelines for:

  $ DPEsetup -db may2011DPE.db

  This will print out a list of the objects that pipelines will be generated for. If there are any objects in there that you don't want pipelines made for, you can do any of the following:

  1. Delete the line from your .db file
  2. Add "ignore" to the OBJECT name
  3. Run DPEsetup with additional arguments, denoting the keywords to look for (see below)

• If you see nothing wrong with this list, add "-all" or <object_name> to the command to generate the Makefile for all objects or for a specific object, respectively:

  $ DPEsetup -db may2011DPE.db -all $ DPEsetup -db may2011DPE.db hd1234

• As mentioned previously, you can specify additional arguments with DPEsetup to denote other keywords for a particular frame. For example, if you denoted the milky flats and lamp frames with the keywords "flat" and "lamp" instead of "milky" and "thar", then you would type:

  $ DPEsetup -db may2011DPE.db -all -flatkey flat -lampkey lamp

• By default, the above will generate a top- level Makefile called "Makefile." If you would like to call the Makefile something else, you would type the following, for exmaple:

  $ DPEsetup -db may2011DPE.db -all -mk may2011.make

STEP 3: Time to reduce your data!

• Once you have the pipeline generated, you can type one of the following commands to start reducing your data:

  $ make # for the default "Makefile" name $ make -f may2011.make # to run a specific

  System Message: ERROR/3 (<string>, line 114)

  Unexpected indentation.

  Makefile

• If you would like to take advantage of multiple processors on your machine, you can reduce several objects in parallel by adding "-j <# of processes>" to the "make" command. Note that the parallel processing begins only after the calibration stages have finished, since those are dependent on each other. For example, the following will run 4 objects at a time:

  $ make -j 4

• If you would like to regenerate the pipelines for whatever reason, you can type the following:

  $ make regenerate

• The above command will only remove the symbolic links to the flat.fits, lamp.fits, etc. files and regenerate the pipelines using the original parameters you used with magesetup before. If you would like to remove intermediate files created by the pipeline, you can type:

  $ make clean

• The first pipelines that will be run are the calibration pipelines (flat, slit, lamp). Once these are finished, the science pipelines will be run. As an alternative to running just "make", you can run the calibration and science pipelines separately by typing the following.

  $ make calib $ make science

• Each time an object is finished running through the pipeline, a dummy file called "targ_<object_name>" will be created. If you would like to rerun the pipeline for a particular object, hd1234, for example, you'll need to delete this file and then run the following:

  $ make targ_<object_name>

• Some useful summary plots of the results can be made that show ordinary and cumulative histograms of the median S/N around the Mg-triplet, as well as, the number of orders extracted per object. Two files will be generated using the command below. SUMMARY.txt lists each object with the number of orders extracted for that object, while <object_name>.png contains the plots. Both of these files will be in the Final-Products directory. SUMMARY.txt also lists the objects which had less than 60 orders extracted or failed at some point during the pipeline.

  $ make summary

• To delete all these dummy files and rerun the pipeline from scratch, type:

  $ make veryclean