User instructions for reducing COPIPE Angles

Updated 08/19/2002 for GONG+.
By Katrina Gressett

Space Requirements: Approximately 5 Gb to store the images (every 10th) for one GONG+ month.

Time Requirements: 1-2 weeks to run 1 GONG+ month.

1) Maintain the following directory structure:


2) Obtain the current driftscan pm_data or drift.dat file. Copy the file to CAMOFFSET/pm_data.

3) Examine the vmi status in http://dsdsom1/VMICAL.backlog.html to verify that the vzi's have been calibrated through the required date range. Then, examine the OFFSET directory, to make sure that the directories for each day you need are there.

4)Make a file called Source.txt containing the path to each directory containing vzi's that need to be processed and copy the file to CAMOFFSET/. A command such as:

>ls -d OFFSET/[B-U]*.gz/ >Source.txt

will list every site day directory in the OFFSET directory. If there are other site days that need to be processed that are not located in the OFFSET directory, you will need to add them to the Source.txt file. Make sure you complete this step before each processing run.

Note: Steps 5 and 6 are now automatically handled if the operator uses the script called "". The instructions are left in to clarify what that script does and in the event that that script becomes lost or outdated. This script handles the areas around the pm's nearly eliminating any operator error associated with them. This script simply accepts a start date, a stop date (both in the form "YYMMDD"), and a pmdata table (typically CAMOFFSET/pm_data).

5)Make a copipe list. This list should be a file called, say,copipe.list, containing lines of the form copipe Source.txt start stop dev=lw20 where start and stop are dates in the YYMMDD format. These dates should span about 7 days, and should be offset, in general, by three days. For example, covering the thirteen day stretch from 970205 to 970217 would require three copipe lines:

copipe Source.txt 970205 970211 dev=lw20
copipe Source.txt 970208 970214 dev=lw20
copipe Source.txt 970211 970217 dev=lw20

However, any hard pm's must abut the edges of these ranges, and not be included in them. Had there been a driftscan on 970212,then the copipe lines would look like this:

copipe Source.txt 970205 970211 dev=lw20
copipe Source.txt 970208 970212 dev=lw20
copipe Source.txt 970212 970218 dev=lw20

To determine if there were any hard pm's, check for rows ending in an 'H' in the pm_data file mentioned above.

Your copipe parameters should look like this (when you lpar copipe):

cl>lpar copipe
PACKAGE = pipeline
TASK = copipe
input= Source.txt File containing list of path names to images
start_da= 010727 Starting date YYMMDD?
end_date= 010802 Ending date YYMMDD?
(tsample= 10) Time sample spacing (minutes)?
(nannuli= 12) No. of annuli (10 for Classic, 12 for Plus)
(nsector= 1024) No. of sectors/annulus (512 for Classic, 1024 f
(tbl_pat= CAMOFFSET) Path to offset tables
(dev = stdgraph) Graphics output device
(data_ty= vzi) Image data type (vzi, izi, etc...)
(image_t = fits.gz) Image type (imh, fits, etc...)
(pm_tbl = CAMOFFSET/pm_data) Full name & path of PM data table
(compres= gzip) Compression: none|gzip|compress
(use_tc = no) Use TC data if available?
(tlist = tmp$tmp4746vrd)
(mode = q)

if you lpa caminterp, your parameters need to match these:

cl>lpar caminterp
PACKAGE = pipeline
TASK = caminterp
mrhiy_pa= CAMOFFSET/) Path to mrhiy files
(filter_= 15) Filter length (days)
(plot_ty= monthly) Type of output plot (monthly|full)
(device = lw20) Graphics output device
(pm_tabl= CAMOFFSET/pm_data) Full path & name of PM data table
(mode = ql)

xoffsets parameters look like this:

cl>lpar xoffset

PACKAGE = pipeline
TASK = xoffset
input = input images of unknown camera offset angle
referenc= Reference images of known camera offset angle
(geometr= hgeom) Use geometry fndlmb|hgeom
(ref_off= 90.) Offset angle (degrees) of ref. images if not in
(directi= clock) Direction of positive rotation (clock|counter)
(nbins = 1024) Number of angular bins (must be even)
(fint_fa= 1) Fourier interpolate cross-correlation by 2**fin
(nannuli= 12) Number of annuli to cross-correlate
(rmin = 0.7) Fractional radius at inner edge of inner-most a
(width = 5 ) Width of zone to search for peak around lag=0
(updhdr = no) Update input image headers with new offset angl
(output = yes) Create output file?
(outname= ) Output data file name
(compres= gzip) Compression type |none|gzip|compress|
(use_mrh= no) Use mrhiy file for initial guess?
(mrhiy_t= ) Full path & name of mrhiy file
(use_PM = no) Use recent pm/drift scan data?
(pm_info= ) Full path & name of file with recent drift scan
(verbose= no) Operate in verbose mode?
(mode = ql)

NOTE: If you copy mrhiy files over without copying the mrcoy files, COPIPE will create ALL of the mrcoy files over again, which can be quite time-consuming. If you don't copy the *.plt files over, then one plot will be printed for EACH SITE for EVERY MONTH which has mrhiy files, but is missing the .plt file. So save some trees and copy over the .plt files!

6) Run the list :

cl < copipe.list

7) Now the mrcoy and mrhiy files should be in the CAMOFFSET directory, as well as the mrmsy and mrqay files. Briefly examine the mrmsy message files to see if they contain anything. If they have a file size of 0 then there were no system messages. To return the file sizes the command:

> ls -ltr mrmsy* is useful.

This will list all the mrmsy files in reverse (-r) order according to the time (-t) and it prints out the long (-l) list of file attributes. This command returns information in this form:

-rw-r--r-- 1 gressett staff 0 Jun 14 22:00 mrmsy020201
-rw-r--r-- 1 gressett staff 62 Jun 15 11:09 mrmsy020207
-rw-r--r-- 1 gressett staff 62 Jun 20 21:42 mrmsy020210
-rw-r--r-- 1 gressett staff 0 Jun 21 07:31 mrmsy020213

There will also be several plots in the printer. These are weekly plots that correspond to the dates that each execution ofcopipe was run over. (See step 6 for more explanation, but one execution of copipe would be :

cl> copipe Source.txt 970212 970217 dev=lw20

When you have processed 8 days past the end of each GONG month, there will be a monthly plot produced for each site. Examine these for anomalies. If none are found, file these into the COPIPE Output Plots binder and turn in the mrhiy and mrcoy files to the DSDS.

8) Now, make a toc file for the mrcoy files that you just produced.

>ls mrcoy* > coy.toc

edit the coy.toc so that every line is followed by a unique number. Make sure that no file in this list exists in the database already.

9)Next, make a toc file for the mrhiy files.

ls mrhiy* > hiy.toc

Edit the hiy.toc file so that every line is followed by a unique number. Make sure that no file in this list exists in the database already.

10) Do an online transfer of the hiy.toc and coy.toc files through http://dsdsom1/pipe.html.

11) Wait overnight, to allow the copipe angles to feed through the DSDS system, before processing these days through dnspipe.

Once these steps are completed, the vzi and bzi files need to be staged and apodized, and the camera offset angles must be inserted into the FITS headers. For instructions click here.

Revision: $Id: copipe_operator.html,v 1.2 2004/09/03 23:10:32 khanna Exp $