COPIPE Process Description
Author: Brian Pohl
The COPIPE reduction stage computes the camera rotator angles for all GONG network sites. CAMOFFSET uses these angles to align the image orientation. The COPIPE module itself is a script that calls several other modules. The script automates the reduction process, but a few things need to be set properly first. First the input data needs to be available. The user must then set the software parameters correctly and finally enter in drift scan information if necessary. This document covers these points in detail and conclude with instructions on operation itself.
COPIPE is very flexible, designed to run on any machine and data set. This document discusses the routine operation on hohokam, but these are by no means the required operating instructions. The reduction processes mentioned in this document should be considered examples.
1. Setting up the input data.
COPIPE uses two types of data to compute the final camera angles (the mrcoyYYMMDD files used by CAMOFFSET.) First and foremost are the input images. The second are the accumulated history files (mrhiyYYMMDD) produced by previous COPIPE sessions. These files contain the camera rotator equations for that run and are used as the initial guess for the next session. Finally, COPIPE interpolates and smoothes these files over time to produce the final output file.
1.1 Input Images
COPIPE can use either velocity, modulation or intensity images as long as all the input data is of the same type. Every tenth image taken at the 0 minute over the day (ie. bbvzi961101t2140, bbvzi961101t2150etc...) is needed as input. The latest version of READTAPE contains a parameter for extracting only data necessary for COPIPE. COPIPE will run on a complete set of images by selecting only every tenth, but staging the unused images wastes disk space. The images from all sites should be placed in the same directory. In routine operation, the images are transferred online by VMICAL operators to the directory /ana1/OFFSET on anasazi.
1.2 The hiy files
COPIPE generates the hiy files that contain the polynomial coefficients for each site and are used as an initial guess for the next data set. In routine operation, COPIPE is run over a seven day long data set using the hiy file for the first day as the initial guess and generating a new hiy file for the center date. For example, the input range is 961101 to 961105, mrhiy961101 is the initial guess and mrhiy961103 is the produced file. All hiy files are stored and written to a single directory set by a parameter in COPIPE. Once 15 days worth of hiy files are accumulated, COPIPE automatically calls a module called CAMINTERP to smooth and interpolate the coefficients and produce the final mrcoyYYMMDD for the center date. For this reason, the coy files produced from an input range trail the center date by about seven to eight days. In other words, when mrhiy961118 is created, mrcoy961110 is produced. Therefore, in addition to the initial guess file, the previous fifteen day's worth of hiy files are required. For routine operation, all hiy files produced are stored in the same directory and never deleted.
2. Setting up the software parameters
The following parameters are used to run COPIPE on hohokam.
dn> lpar copipe input = "in.dat" File containing list of path names to images start_date = "970608" Starting date YYMMDD? end_date = "970614" Ending date YYMMDD? (tsample = 10) Time sample spacing (minutes)? (tbl_path = "/ana1/CAMOFFSET") Path to offset tables (dev = "stdplot") Graphics output device (data_type = "vzi") Image data type (vzi, izi, etc...) (image_type = "fits.gz") Image type (imh, fits, etc...) (pm_tbl = "/ana1/CAMOFFSET/pm_data") Full name & path of PM data table (use_tc = no) Use TC data if available? input: This is an ascii text file containing the path where the input images are stored. The file "in.dat" listed above simply contains /ana1/OFFSET. When staging the data on your computer, simply create this file and have it list all the directories that contain input data. Multiple directories can be listed, but it is recommended that everything be placed in the same directory. start_date and end_date: Simply the input date range. Standard processing uses a 7 calendar day range (three days before and after the center date.) The next date range will begin on the previous center date. For example, after this input range ran, the next would be 970611 970617. tsample: This indicates the time interval of the input images. Standard processing uses a 10 minute interval, but that can be changed for special tasks. tbl_path: This parameter indicates the directory that contains the hiy files and consequently where the coy files are written. It is very important to set this to the right directory. dev: Output device for the qa plots, default is stdgraph. data_type: Velocity images are normally used for input, but it also supports modulation and intensity. All input must be the same type. image_type: Implemented for the fits kernel. All the input on anasazi is in fits format and uses the fits kernel. pm_tbl: This lists the full path to the table containing PM driftscan results. This is discussed in detail in the next section. use_tc: Added to support the Tucson test site.
The most common problems in running COPIPE are associated with the parameters tbl_path and pm_tbl. If set improperly, the software terminates with a mysterious error. COPIPE passes all parameters to the various modules it calls. The user never needs to adjust the parameters in these modules.
3. Handling PM drift scans.
Drift scans are taken during routine preventive maintenance, about one site per month. Since these produce an "observed" table of polynomial coefficients, they are assumed to be correct and COPIPE sets the other sites to match it. Once the results from the drift scan have been obtained (these are posted on the gong_ops mailing list and in the service logs on mojave,) they need to be entered into a file. The one on hohokam is called /hoh0/CAMOFFSET/pm_data.
dn> tail /hoh0/CAMOFFSET/pm_data 1996/03/06 BB 4 -0.02727 -0.1527 0.03963 0.07591 1996/04/21 UD 2 -0.02860 0.046457 0.00000 0.00000 1996/05/28 CT 3 0.07990 0.3164 0.09956 0.00000 1996/06/28 ML 0 ***** Site coordinates updated ***** 1996/08/06 TD 0 ***** No Drift Scan Obtained ***** 1996/08/26 LE 3 0.02126 -0.00928 -0.05652 0.00000 1996/10/11 UD 3 0.01075 -0.02767 0.01745 0.00000 1996/11/16 ML 0 ***** No Drift Scan Obtained ***** 1997/02/05 BB 0 ***** No Drift Scan Obtained ***** 1997/03/12 CT 4 -0.00370 0.43320 0.01410 -0.13554 1997/04/23 LE 4 0.00070 0.08090 -0.01250 -0.10080 1997/05/30 UD 4 -0.04970 -0.05345 -0.03701 0.02036
The first column is the date the drift scan was performed, the second is the site. The third column contains the number of coefficients (the order of the polynomial.) Note that it isn't always 4 and an entry of 0 will ignore the rest of the line. You can enter in comments when a driftscan wasn't taken, there is no specific format for commenting on a 0 line. The remaining columns are the coefficients (0 term, 1st term ... 3rd term.) The table does need to be in chronological order, new entries must go on the last line.