VMBICAL algorithm hierarchy
Program Directory ContentsAfter the vmbical_soft.YYYYMMDD.tar.Z file has been uncompressed and untarred, you should end up with the structure shown here:
VMBICAL_SOFT |-- bin_c: All compiled C-executables as well as older versions of executables. | |-- bin_sh: All shell scripts as well as older versions of shell scripts. | |-- bin_spp: IRAF packages, par files and tables that VMBICAL requires for processing. | |-- iraf: DNSPIPE and VELCAL IRAF packages. | |-- iraf_cl: All IRAF CL scripts with accompanying par files. | |-- params: Individual site directories for each GONG site contain parameter files needed | for UCOR, GEOM, HGEOM, EPHEM and other processing. | |-- src_c: All source code for C-executables as well as older versions of the source. | |-- wa0: Contains a sample r_all script that can be modified/customized to begin VMBICAL processing. | |-- wa3: All IRAF CL scripts, shell scripts and older versions of scripts required for | for running a modified version of DNSPIPE to create l-nu spectra for | a data quality check.
R_ALL -> GET_SITE_DAYS -> GETRANGE | | | -> DAYBREAk | | | -> CHECK_RAW_LIST | | | -> PLOT_NET_RAD.CL | | | -> PREPDISK -> GETDATE | | | -> TRANSFER_CAL | -> PROC_SITE_DAYS -> TAPE2STZ | | | -> STZ2RAW -> UNRICE | | | -> RUN_UCOR (x_images.e, x_velcal.e, x_ephem.e) -> UCOR | | | | | -> UCOR_STATS | | | | | -> VMBIGAPS (x_images.e) | | | | | -> EPHEMINTERP | | | -> RUN_GEOM (x_images.e, x_gongcor.e, x_suntrans.e) | | | | | -> GPFLAT | | | | | -> GEOM | | | | | -> GEOM_STATS (x_images.e) | | | | | -> HGEOM | | | | | -> HGEOM_STATS (x_images.e) | | | -> GEOMPLT | -> GETQA.CL -> GETQA1.CL -> GETQA2 -> VELFIT | -> PUT_SITE_DAYS -> EDIT_MTF (x_images.e) | | | -> SPLIT_VMBI (x_images.e, x_dataio.e, x_copipe.e) | | | -> MK_ANNULI (x_images.e, x_dataio.e, x_copipe.e) | -> PREP_ANCI | -> VMBI2TAPE
Output hierarchyThe output generated by VMBICAL, when run according to the operating instructions is organized as follows:
WORKING DIRECTORY (wa0.siteday): R_ALL is run from this directory | |--Siteday Directories: control directories for each siteday that is being processed that contain all | log files, calibration images and statistics. | |--VSN Directory: contains all files associated with reading the raw field tape created by FTR. | |--Statistic Directories (A_???): contains mtf, qac and other statistic files for the sitedays processed. WORKING DIRECTORY 1 (wa1.siteday): all raw and intermediate data directories | |--Siteday.stz: all stz files read from the raw field tape (see TAPE2STZ) | |--Siteday.raw: all unriced raw fits files (see STZ2RAW) | |--Siteday.geom: all calibrated fits images to be QA'ed | |--Siteday.vmbi: all QA'ed calibrated fits files ready to be split (see SPLIT_VMBI) WORKING DIRECTORY 2 (wa2.siteday): all calibrated data products | |--Siteday.int: all intensity images | |--Siteday.mag: all magnetograms | |--Siteday.mod: all calibrated modulation images | |--Siteday.vel: all calibrated velocity images | |--Siteday.vel.gz: every tenth velocity image for COPIPE |
Program descriptionR_ALL (SH): Command script that runs the VMBICAL processing. Prepares the siteday control directories using GET_SITE_DAYS. Transfers the necessary calibration images used for processing each siteday through UCOR. Reads raw stz files from field tapes using TAPE2STZ and unrices them using STZ2RAW. Calibrates images using PROC_SITE_DAYS and then QA's the calibrated images using GETQA.CL. Calibrated and QA'ed images are split into the 4-planes (modulation, intensity, magnetic and velocity) by SPLIT_VMBI. Ancillary files (mtfs, qacs, etc.) are created using PREP_ANCI and then all ancillary and split-calibrated images are written to tape by VMBI2TAPE.
GET_SITE_DAYS (SH): Creates the VSN control directory and calls GETRANGE to read the FTR rhd and hif files to
determine how the sitedays on the tape should be broken up. Calls CHECK_RAW_LIST to edit the raw list of files
to run through the calibration process. PLOT_NET_RAD.CL is called to display the net radiation data for each
siteday and bad net radiation sitedays can then be removed from the siteday list to be processed. Creates working
directories 1 (wa1.siteday) and 2 (wa2.siteday) for the sitedays on the siteday list to be processed. Calibration
images for each siteday are copied to the siteday control directories using TRANSFER_CAL.
GETRANGE (C): Reads the FTR rhd and hif files to determine the number of files on the field tape and calls DAYBREAK
to break the data into sitedays. Creates siteday directories for the sitedays to be read off of the field tape
and copies raw_list, guider, net_rad, phase, itime and hour_ang files specific for each siteday.
CHECK_RAW_LIST (SH): Checks the raw list of images on the field tape and edits the raw list to remove files when the file name does not end in 16, duplicate images, images where the guider was not on, images where the net radiation value is less than the minimum net radiation value for that site, images that do not have an integration time of 60 seconds and images that have a larger phase plate rotation error than the maximum value allowed for the site.
PREPDISK (C): Checks for the existence of the siteday control directories inside of the main working directory as well as checking for the existence of working directories 1 (wa1) and 2 (wa2). If they do not exist they are created. Inside of working directory 1 (wa1) stz, raw, geom and vmbi directories are created for each siteday. Inside of working directory 2 (wa2) int, mag, mod, vel and vel.gz directories are created for each siteday.
PROC_SITE_DAYS (SH): Creates the VMBICAL log to begin processing. TAPE2STZ is called to read all files in the raw list off of the field tape. Each file read off of the tape is then unriced using STZ2RAW. Images are then calibrated by running RUN_UCOR and RUN_GEOM.
TAPE2STZ (SH): Reads files in the raw list off of tape into the siteday.stz directory using the UNIX dd command, or, if the FTR function has already placed the images on a disk, creates a link to those images. As it reads the first and last file it verifies that the IRAF name is correct.
RUN_UCOR (SH): Uses the adk, wlm and nbc calibration images to create the cal calibration image. The integration time for the calibration is checked and is scaled to 60 seconds if it is not already 60 seconds. Calls the IRAF task UCOR to begin the calibration process. Calls UCOR_STATS to create calibration statistic files.
UCOR_STATS (SH): Compares the intensity, modulation, velocity and phase values for the calibrated images with the minimum and maximum values set in the site specific parameter files. Creates a status.dat file that gives an index of why an image has the FILLED keyword in its header after the calibration process.
VMBIGAPS (SH): Checks the list of calibrated images to determine if any are missing. If images are missing the previous image is copied and the DATE-OBS, TIME-OBS and FILLED keywords in the image header are updated to accommodate the missing image's time.
RUN_GEOM (SH): Checks the integration time of the flt calibration image and scales it to 60 seconds if it is not already 60 seconds. Copies images into the geom directory and runs IRAF tasks GPFLAT and GEOM. GEOM_STATS is run to extract statistics from the calibrated images. Adds the MTF_FILE keyword into the images headers and then runs the IRAF task HGEOM.
GETQA2 (SH): Creates limb statistic files and calls VELFIT to calculate the VEL_SCALE and VEL_BIAS values as well as the residual velocities.
SPLIT_VMBI (SH): For each plane of the calibrated intensity images, copies the plane of each intensity image in the geom directory into the appropriate siteday directory (int, mag, vel or mod), apodizes the image, adds the VEL_BIAS and VEL_SCALE values into the image header, removes all ADC and U_[A-Z] values from the image header, edits the DTYPE keyword in the image header, adds the keyword FILLED to the image header if necessary, writes the image into a 16-bit FITS file and compresses the images using GZIP.
MK_ANNULI (SH): Copies every tenth image from the vel directory into the vel.gz directory, apodizes the velocity images into an annulus, writes the image into a 16-bit FITS file and compresses them using GZIP.
PREP_ANCI (SH): Creates A_rhd, A_hif, A_hic, A_mtf, A_cal, and A_qac directories for ancillary files. Copies rhd and hif files from the VSN directory into their new respective A_* directories. Copies adk, flt, wlm, nbc and mtf files from the siteday directory into their new respective A_* directories. Creates qac files from geometry files in the siteday directory. Compresses log files to create hic file.
Revision: $Id: vmbical_algorithm_hierarchy.html,v 1.3 2004/10/08 22:27:52 khanna Exp $