The following describes some procedures for handling data reduction tasks. For detailed information on each of the programs the reader should refer to the web pages for that program.
Let's assume that you have a mask named Mymask
, and you have obtained the
following set of observations with IMACS using the short camera in Nod&Shuffle
orientation, and the 200 line grism
Frame | Exposure |
---|---|
ccd001 | bias |
ccd002 | direct mask image |
ccd003 | spectroscopic flat |
ccd004 | comparison arc |
ccd005 | spectrum |
ccd006 | comparison arc |
ccd007 | spectrum |
ccd008 | comparison arc |
ccd009 | spectroscopic flat |
COSMOS assumes that all the data files except the FITS image files are in the
current working directory, but that the FITS files are in the directory
pointed to by the environment variable COSMOS_IMAGE_DIR. Everybody has their
own preferences for arranging files; one convenient way is to have all the data
files for an observing run in a directory dir
, with subdirectories n1, n2, ...
for each night's FITS files. You should cd
to the directory in which you will
keep and use the non-image files, then use setcosdir to specify the image
directory. With the above suggested directory structure, to work with image
files in the n1
subdirectory, you could either define COSMOS_IMAGE_DIR as ".",
(i.e. the current directory) and refer to image files as n1/filename
, or define
COSMOS_IMAGE_DIR as "/home/joe-users/.../dir/n1"
, and refer to image files as
filename
.
We need to set up an obsdef file for the spectroscopic observations. It also doesn't hurt to first check the direct image of the mask, just to ensure that there are no major problems. For this we need a direct image obsdef file as well.
We invoke defineobs and insert the following data for the direct image:
Instrument: | IMACS |
Mask: | Mymask |
Dewar Offset File | SCdirect_NS |
Camera: | SHORT f/2 |
Mode: | DIRECT |
Observation Definition File | Mymask-direct |
and for the spectroscopic observations, the following:
Instrument: | IMACS |
Mask: | Mymask |
Dewar Offset File | SC200g_NS |
Camera: | SHORT f/2 |
Mode: | SPECTROSCOPIC |
Grism: | 200l |
Observation Definition File | Mymask |
Note that we have specified the instrument as "IMACS". This is correct for data taken before August 1, 2005, when chip 6 was replaced in the SITE dewar. After August 1, the instrument should be identified as "IMACS2".
Because we have not yet used align-mask on any images, we have used the default
dewar offset files, which are located in $COSMOS_HOME/examples/dewoff
. When
finished, we now have two obsdef files: Mymask-direct.obsdef
, and
Mymask.obsdef
.
One important step to take before doing any wavelength calibration is to check (and possibly adjust) your initial dewar offset file, as the initial offset may be too large for align-mask to detect:
spectral-lines -o Mymask -l HNA.dat -b 1 -eThe above command will generate a file called
Mymask.xy
. The -e
flag is a powerful option because it will map the ends of the slit for each feature in the line list.
display8 ccd006 1 z1=0 z2=whateverThis will display a mosaic of your lamp frames and mark the slit ends for each of the features in your line list file.
tvmark 1 Mymask.xy
SEARCHBOX
specified in the align-mask parameter file.
D_ALIGNROT
) in the obsdef file using the COSMOS routine defineobs or your favorite text editor (see the defineobs page for more details about D_ALIGNROT
).$COSMOS_HOME/examples/linelists/HeNeAr_short.dat
) to make sure that there is no large offset or rotation. Don't worry about getting the features to match exactly, as align-mask is designed to get you an approximate initial map; adjust-map will do the fine-tuning, as far as wavelength calibration is concerned.It's usually not even necessary to look at the direct mask images, but let's do that anyway just to make sure everything is OK (useful to do, for example, at the beginning of a run). First, we set the parameters in align-mask
editpar align-mask type q to end SEARCHBOX 10 half width of search range MAGFACTOR 100 magnification factor for offset vectors LAMFILE HNA.dat line list file NAVER 1 number of neighboring points to average is plot Change parameter:
A SEARCHBOX
of 10 is usually sufficient; if most apertures cannot be found,
increase it to 20. The LAMFILE
is only needed for the spectroscopic image
alignment.
Now, run align-mask to check and correct the alignment.
align-mask -o Mymask-direct -f ccd002
After the first iteration, assuming that it is successful, rerun defineobs and insert ccd002 as the name of the dewar offset file. The second iteration should show quite good agreement.
Assuming that this worked well, we are ready to align the spectroscopic images.
If it did not work well, something may be very wrong. In that case we should generate a list of predicted positions using apertures and overlay it in IRAF with the mosaiced image.
apertures -o Mymask-direct -b 1
(assuming you used 1x1 binning)
In IRAF:
display8 ccd001 1 z1=0 z2=whatever tvmark 1 Mymask.xy
A comparison of predicted and observed apertures positions should allow you to understand the problem.
Now we need to align the spectroscopic mask. We only do this once, using one of
the exposures, for a set of spectroscopic exposures that we want to reduce as a
group and combine (even though flexure may result in slight shifts from one
exposure to the next.) The reason for this will be explained later. We need to
specify a line list file in the align-mask parameter file. This list should
consist of a small number (no more than 1 to 3)of clean, well-isolated comparison lines. In this
example, we have called it HNA.dat
, and specified that in the align-mask
parameter file. Having done so, we can execute align-mask:
align-mask -o Mymask -f ccd006
Again, change the name of the dewar offset file in Mymask.obsdef
to ccd006
after the first iteration. The second iteration should look pretty good, like
this:
The typical residual is about a pixel, and is dominated, as is obvious, by systematic errors. These are due to imperfectly-mapped distortions in the IMACS optics. These small errors will be removed later when we construct and adjust the spectral map.
The spectral map is a file which contains the information necessary to transform from CCD coordinates to the space of wavelength vs slit position into which we want to transform our data. The information necessary to construct a first, approximate map (which should be good to a few pixels) is all contained in the observation definition file. The map is constructed using the program map-spectra. So:
map-spectra Mymask
We will do this only once for each set of data which we wish to combine. Because this map is approximate, we use the comparison arc exposures to adjust it in the most suitable way for each science spectral exposure, using adjust-map. We can use a single arc exposure, or interpolate between two. adjust-map works best with bias-subtracted frames; In our example, each science exposure is bracketed by two arc exposures, so we can do:
biasflat ccd004 ccd006
then
adjust-map -m Mymask -f ccd004_b ccd006_b
which will produce a new map file ccd004-6_b.map
to use with the science
exposure ccd005, and so forth. To repeat what was stated earlier, if we wish to
later combine ccd005, and ccd007, with cosmic ray rejection, we should start
with the same basic map file, Mymask.map
, created from one observation
definition file, Mymask.obsdef
.
Care must be taken to pick a good list of comparison lines. Make sure that all
chosen lines are clean and well-isolated, lines near to stronger lines are
particularly to be avoided. Particularly when using a new line list, or a new
instrumental setup, it is wise to first run adjust-map in debugging mode (add the -d
flag and a slit number), so
that you can see how the dispersion fits are working: you may find that one
line is consistently not behaving well, or that the order of the adjustment is
too high or low. It's also not a bad idea to check the map file output created
by adjust-map, by using spectral-map to create a list of expected spectral line
positions, then overlaying them on the image of the arc frame.
spectral-map -m ccd004-6_b -l linelist.dat -b 1
where linelist.dat
is a file containing a comparison arc line list. Then, in
IRAF:
display8 ccd004 1 z1=0 z1=whatever tvmark 1 ccd004-6_b.xy
The marked positions should align virtually perfectly with the slit centers (to within the pixelization). If they're still off, try repeating adjust-map:
adjust-map -m ccd004-6_b -f ccd004 ccd006
That should do it quite well. If it doesn't, it's probably due to a bad line
fit. adjust-map produces a file called adjust-map.dat
, which contains details
of the line offset determinations. See the web page for adjust-map for details.
Look at the values of delta(x) for the spectrum which is giving problems. There
will probably be one discrepant value. Chances are that inspection will show
that this line is being influenced by a neighboring line; if so, delete it from
the line list and repeat the fit.
In this case, adjust-map produces a postscript file called ccd004-6_b.ps
. A quicker, alternative way to check the results of adjust-map is by looking at the plots in this file. The first plot in each row is the dispersion fit for that particular slit, and can tell you if there were any problems with any of the features.
We now need to do bias subtraction and flat-fielding of the science exposures, using biasflat. Since we will use the same bias frame for all science exposures, we can identify it in the biasflat parameter file, but we will need to specify different flat-field frames on the command line for each science exposure. We create spectroscopic flat fields for the science exposures using Sflats, in our case:
Sflats -m ccd004-6_b -f ccd003
will create a set of image files ccd003_flat which we will use with ccd005. Sflats normally uses the standard bad pixel file for each dewar, to mask out bad regions of the chips. If you have created your own custom bad pixel file for a mask, for example to mask out zero order grism images, you would say:
Sflats -m ccd004-6_b -f ccd003 -z mybadfile
Here's what a section of one of these frames looks like:
Now we are ready to biasflat ccd005:
biasflat -f ccd003_flat ccd005
The output will be a set of images files ccd005_f. Here's what the same section of a flattened spectrum frame looks like:
The next step is to subtract the sky using the routine subsky.
subsky -m ccd004-6_b -f ccd005_f
or
subsky -m ccd004-6_b -f ccd005_f -z mybadfile
which produces a set of image files ccd005_s, a section of which looks like this:
If the comparison arcs are well-matched to the object frames, the sky subtraction should usually be this clean, but it might not for several reasons:
This seldom occurs with IMACS, but does seem to be more common with LDSS3. In this case, switching to a 2-d fit should solve the problem.
Occasionally, the spline fit for a spectrum does not converge. In this case, you will get a message like this:
Problem fitting sky of slit 2, chip 6 0 (1)and this spectrum will be set to zeroes.
We are now ready to extract the spectra. You have three choices:
Method 1 is the most robust at the present time, so we'll do that.
First, we extract each object exposure
extract-2dspec -m ccd004-6_b -f ccd005_s extract-2dspec -m ccd006-8_b -f ccd007_s
If the search parameter has been set to a non-zero value, extract-2dspec will present various plots of spectrum shape and offset.
Now, having turned on cosmic-ray rejection in the sumspec parameter file, we do:
sumspec -o Mymask_2spec ccd005 ccd007
Note that we don't specify any spectrum type (like _2spec) for the input files, but must for the output file.
A bad pixel map exists for each dewar, in the directory $COSMOS_HOME/sdata
,
with the name dewar.badpix
. This file maps the location of known bad features
on each chip. These files contain, the coordinates of bad pixel groups, one per
line. The format is:
chip xmin xmax ymin ymax
Unless another bad pixel map is specified on the command line, Sflats and subsky use the standard files, however the user can construct and use his/her own bad pixel file, either from scratch, or using the standard files as a starting point.
In addition to bad pixels on the chips, it is often useful to block regions where unwanted spectral orders overlie the desired spectrum. This can be done by constructing a new bad pixel file for each observing setup, using the program badorders. badorders requires, as input, an observation definition file specific to each spectral order to be blocked. These files will differ from the standard observation definition file for the observing setup in the spectral order specified, and probably also in the dewar offset file, since there are usually small order-to-order differences offsets between the true positions of spectral features and those predicted by the optical model.
Constructing an appropriate dewar offset file for each order may take some work. Since the orders that one wants to block are those which lie on top of the desired order, align-mask may or may not be successful in determining a correct alignment because of the confused image. If it does not work well, it will be necessary to create one for the order by adjusting the dewar offset file created for the primary order, using adjust-offset.
Don't forget:
Although the COSMOS routines handle most aspects of Nod&Shuffle data automatically, there are some aspects of N&S data reduction that must be noted:
Data taking with the IMACS and the MOE echelle grating is reduced like any
other IMACS data. To set up MOE reductions, in defineobs select MOE (for old-
MOE data) or MOE2 (for new-MOE data) from the list of gratings, and select the
spectral order to be reduced. (Note that each MOE order must be reduced
separately.) The grating angle should be set to zero. Template dewar offset
files for MOE and MOE2 can be found in the $COSMOS_HOME/examples/dewoff
directory.
The only complication in reducing MOE data is that the high density of thorium-argon comparison lines plus the as-yet-imperfect mapping of the MOE optical distortions complicate the tasks of align-mask and adjust-map: the danger of zeroing in on the wrong line is significant. To deal with this align-mask should be run using a small number of isolated comparison lines. Note that the dewar offset file produced by this process should work equally well for at least several adjacent spectral orders.
adjust-map should be run in two steps. In the first step, the same short list
of isolated comparison lines should be used and the ORD_DISP
and ORD_SAG
parameters should be set to 1. In the second step, the map produced by the
first step can be refined using a more complete list of thorium-argon lines,
with ORD_DISP
and ORD_SAG
set to appropriately higher values.
The most important advice for reducing MOE data is to check every step of the process: do not assume that MOE data can be reduced in a hands-off, pipelined manner. Use the position lists produced by spectral-lines and spectral-map, together with the IRAF routine tvmark, to check the accuracy of the predictions of the dewoff and map files at each stage of the reductions. Check the graph plots produced by adjust-map to ensure that the fit to the comparison arcs is reliable. Inspect the sky-subtracted files to ensure that this step went well, etc.
Reducing LDSS3 data is very similar to IMACS with a few minor changes:
DEWAR
needs to be changed to "LDSS3", since stitch takes care of all the gains and biases.
NOISE
and GAIN
values, which can be found here.
Longslit data can be taken for IMACS or LDSS3. COSMOS can correct for line-curvature to produce rectified longslit data.
fit_curv
parameter in the adjust-map parameter file in order to enable fitting for the curvature.
Once all this becomes routine, i.e. you know how all the programs behave and how your data behaves, you can put everything from the map-spectra procedure onward into a script, using process-2spec. Note the restrictions listed in the web page before running.
process-2spec
Spectrum set: Mymask_night1
Associated obsdef file: Mymask
Bad pixel file: Mybadfile
Science frame # 005
Bias frame: bias
Comparison arcs: 004 006
Flat frames: 005
Science frame # 007
Bias frame: bias
Comparison arcs: 006 008
Flat frames: 009 010 011
Science frame #
Spectrum reduction makefile Mymask_night1.make
created
The script Mymask_night1.make
can be executed using the standard UNIX command
make.
make -f Mymask_night1.make