#### UNIX Script Documentation Block # # Script name: bufr_dump_obs.sh Driver for the obs. data dump util # # Author: D.A. Keyser Org: NP22 Date: 2007-05-25 # # Abstract: This script is a driver for the all-purpose BUFR # observation database tank dump utility, dumpjb. It's main purpose # is to output BUFR data dumps to the appropriate /com directory # and path file naming structure used by the operational NCEP # networks. The dumping of any number of BUFR data types can thus be # accomplished for all NCEP networks via a single line in the # analyses scripts which executes this driver script. A secondary # purpose (if so requested) is to: 1) run the prepssmi program which # will read SSM/I data from one of three possible just-generated BUFR # dump sources [FNOC "operational" products, NCEP Neural Net-3 # products, or brightness temperatures (the latter will be used to # generate products in-line in prepssmi)], reprocess the data (most # likely generating superobs), and output the reprocessed data into a # BUFR file ("spssmi"); 2) run a series of three programs (dcodcloc, # dataqc, datasort) which will read ERS scatterometer data from the # just-generated ERS BUFR dump ("erswnd"), do some special reprocessing # of these data, and output the reprocessed data into a BUFR file # ("erscat"); 3) run the dcodquikscat program which will read Quikscat # scatterometer data from the just-generated Quikscat BUFR dump ("qkscat"), # do some special reprocessing of these data (most likely generating # superobs), and output the reprocessed data into a BUFR file ("qkswnd"); # 4) run the supertmi program which will read TRMM TMI data from the just- # generated TRMM TMI BUFR dump ("trmm") and superob these data and output # the reprocessed data into a BUFR file ("sptrmm"); 4) run the dcodwindsat # program which will read WindSat scatterometer data from the just-generated # WindSat BUFR dump ("wndsat"), do some special reprocessing of these data # (including possibly generating superobs), and output the reprocessed data # into a BUFR file ("wdsatr"). All output SSM/I, ERS, Quikscat, TRMM TMI and # WindSat files are written to the same /com directory and path file naming # structure as the original BUFR data dump files; also all five are then # ready to be input to subseqeunt programs (the first 3 and the fifth to # PREPOBS_PREPDATA and the fourth one to GLOBAL_SSI). Upon completion of the # dumping of requested data types, and the completion of the prepssmi, # ERS, Quikscat, TRMM TMI and/or WindSat reprocessing programs (if # requested), a status file can be generated (by request) and written to the # same path as the dump files. The ASCII status file contains a summary of # report counts for all requested subtypes that have been dumped. It also # contains information on the return code for each data group dumped. # # It should be noted that this script can output data dumps to any # file path location specified by the imported variable "$COMSP". It # is not tied to the /com structure, and is thus suitable for # non-operational "over-the-counter" type runs. # # Script history log: # 1996-09-27 D.A. Keyser -- Original version for implementation # 1997-02-19 D.A. Keyser -- A condition code of 4 out of prepssmi # will no longer cause a fatal error in this script # 1997-04-14 D.A. Keyser -- Added optional ERS scatterometer data # processing section to this script # 1999-02-03 D.A. Keyser -- Total y2k compliance as central date (input # script positional parameter 1) is now ALWAYS expected to # contain a 4-digit year # 1999-03-08 D.A. Keyser -- The ucl for the datasort program modified # since this program now genrates an NCEP-format BUFR # output file (copied to /com for IBM PREPDATA program) # as well as the previous WMO BUFR file (still used by # operational PREPDATA program on Cray); qsub added to # execute new version of prepssmi which will generate an # NCEP-BUFR spssmi file and copy it to /com for IBM # PREPDATA program # 1999-06-01 D.A. Keyser -- The ucl for the prepssmi program modified # since this program now genrates an NCEP-format BUFR output # file (copied to /com for PREPDATA program); the ucl for # the datasort program modified since this program now # generates only an NCEP-format BUFR output file (copied # to /com for PREPDATA program); modified to run on IBM SP # 2000-02-25 D.A. Keyser -- Modified to run on the IBM-SP # 2000-05-16 D.A. Keyser -- Added hybrid processing option in SSM/I # processing (PREPSSMI_PRODTYPE=COMBINATION) # 2000-06-26 D.A. Keyser -- Added optional Quikscat scatterometer data # processing section to this script; input positional # parameter "ntype" ($3) is no longer used - all outputs # are tied to the imported variable "$COMSP" (ntype remains # as a dummy variable) # 2001-11-02 D.A. Keyser -- Added optional TRMM TMI data processing # section to this script # 2003-08-13 D.A. Keyser -- Checks to see if a dump file qualifies as a # "RESTRICTED" data type - if so, changes group to "rstprod" # and permission to "640" (if CHGRP_RSTPROD=YES); set-up to list # counts by satellite id in output status file copied to /com # if they are present in output file that this script greps; # method for greping out data counts from dump output files # modified such that if dump file in $COMSP is missing or empty, # the status file will always stamp out ZERO reports regardless # of what was in the dump output files # 2004-01-15 D.A. Keyser -- Compares AFWA vs. ARINC ACARS report counts in # AIRCAR dump and generates a flag file (*aircar_status_flag*) # in $COMSP path indicating which type should be processed in # subsequent PREPBUFR steps - this allows AFWA ACARS to serve # as a backup when ARINC ACARS is down # 2004-01-20 D.A. Keyser -- Removed all references to "RUC2B" (folding in its # 00 and 12Z processing with RUC2A) # 2004-02-20 D.A. Keyser -- Removed logic that handled split QuikSCAT dump # processing (qksca1 --> qkswn1 for first 1/2 of dump period, # and qksca2 --> qkswn2 for second 1/2 of dump period) - this # was originally done to speed up dumps on slower platforms, # but this is no longer necessary - assumes only qkscat is # dumped and generates reprocessed dump qkswnd from it # 2006-03-02 D.A. Keyser -- Added logic to grep out the number of Level 2 # radial wind and reflectivity replications for each subtype # from the dump output files and write to the dump status file; # imports new script variable "JOB_NUMBER" {currently either 1, # 2, or null (default)} which is appended to the name of the # dump status file in /com - this allows two unique status files # to be written from two different dump jobs for the same # network and cycle time (these are later combined into a single # status file in the dump post job) # 2006-06-06 D.A. Keyser -- Added optional WindSat scatterometer data # processing section to this script; added new network option # "rtma" (Real-Time Mesoscale Analysis); now looks first for # external data cards in optional QuikScat, TRMM TMI and WindSat # processing and only if not found uses hardwired "herefiles" in # this script as the data cards (adds more fexibility) # 2007-05-25 D.A. Keyser -- Set-up parm cards for bufr_dcodwindsat to superob # on 1 degree lat/lon boxes (had been set to not superob); # missing "airsev" dump r.c. reduced from 22 to 5 for all NAM # and tm03 NDAS network runs; missing MODIS (Aqua/Terra) IR/WV # satwnd dump r.c. reduced from 11 to 4 for all NAM and tm03 # NDAS network runs # # # Usage: bufr_dump_obs.sh yyyymmddhh hh<.hh> ntype dgrp1 dgrp2 ... dgrpN # # Input script positional parameters: # yyyymmddhh<.hh> $1 - center of time window for dump # hh<.hh> $2 - radius of time window for dump # ntype $3 - reserved, currently has no meaning # dgrp1 $4 - data group 1 - see dumpjb for valid data group list # dgrp2 - data group 2 # ... ... # dgrpN $# - data group N # NOTE: If data group 1 is set to the string 'null', no data dumps # will be performed regardless of the remaining data groups # listed. However, if "STATUS=YES" (see below) a status # file containing reports counts/info from all previous runs # of this script by a particular parent script will be # generated and all outputs from previous runs of this script # in the various $DATA/job${DUMP_NUMBER} directories will be # combined into the $DATA directory. This is currently used # when running this script in simultaneous background jobs - # after all of the background jobs have completed, a final # job is run with this string set to null. # # Imported Shell Variables: # (NOTE: These do not include most of the variables listed in the # dumpjb docblock that are specific to dumpjb - see dumpjb script # for a list of the dumpjb variables) # # These must ALWAYS be exported from the parent script -- # # DUMP_NUMBER - string indicating the number associated with this # particular run of this script (needed when this script is # run in simultaneous background jobs) # DATA - string indicating the initial (temporary) directory path # to output data destination # (NOTE: Each particular run of this script will write # output data to $DATA/job$DUMP_NUMBER directory; # only the final run with STATUS=YES will combine # outputs for individual runs into $DATA directory. # This allows this script to run in simultaneous # background jobs.) # NET - string indicating system network (i.e., "arc", "gfs", # "nam", "gdas", "ruc", "rucs", "dump", "rtma") -- points to # proper data card file for programs executed here and to # echo in dump status file # NOTE: NET is changed to gdas in the parent Job script for # the gdas1 RUN (was gfs - NET remains gfs for gfs # RUN) # COMSP - string indicating the final directory/filename path to # output data destination # (e.g., "/com/nam/prod/ndas.20000102/ndas.t12z.") # {NOTE: If the imported variable "SENDCOM" (see below) # is "NO", then COMSP is hardwired to the string # "$DATA/"} # tmmark - string indicating hour for center time for dump # relative to the analysis time, it is a qualifier in the # output data dump file names (e.g., "tm12", "tm09", # "tm06", "tm03", "tm00") # # These will be set to their default value in this script if not exported # from the parent script -- # # JOB_NUMBER - string indicating the number associated with this # particular dump job (needed when the overall network dump # is split into more than one dump job) # Default is "" (null) # STATUS - string: if = 'YES' will generate a status file which is # written to the same path as the dump files. This is # written out after all dump files have been created, and # contains both report counts (by subtype) and return code # information for each data group dumped. # Default is "YES" # NOTE: If this script is executed more than one time by # the parent script, "STATUS" should be imported as # "NO" unless this is the FINAL execution of this # script. The status file will always contain # information from all previous runs of this # script by a particular parent script. The last # execution of this script MUST ALWAYS set STATUS=YES # since this combines output from all previous # executions of this script which write to the # directory $DATA/job${DUMP_NUMBER} . # envir - string indicating environment under which job runs # (either "prod" or "test") # Default is "prod" # DUMP - string indicating path for dump script file # Default is "/nw${envir}/ush/dumpjb" # LOUD - string: if != 'off' will turn on script trace (set -x) # in dump script ($DUMP) execution (does NOT apply to # this script) # Default is "off" # jlogfile - string indicating path for sms joblog file # Default is null string of length zero # job - string indicating the job name of the parent script # (i.e., "nam_dump_12") # Default is "j????" # cycle - string indicating the analysis cycle time (i.e., "t00z" # or "t18z") # Default is "t??z" # (Note: This is not critical to processing, it is used # only for generating a header to the dump status # file) # SENDCOM - string: if = 'NO' will redefine "COMSP" variable to be # "$DATA/", regardless of its imported value - this has # the effect of preventing any files from going to an # operational /com directory path, and instead sending # them to the "DATA" directory # Default is "YES" # pgmout - string indicating path for standard output file (output # always contatenated onto this file) # Default is "$DATA/allout" # prepssmi - string: if = 'YES' will run prepobs_prepssmi to reprocess # SSM/I data (most likely generating superobs) into BUFR # file copied to the designated path to output data # destination; if 'NO' will not run prepobs_prepssmi program # Default is "YES" # prepersd - string: if = 'YES' will reprocess ERS scatterometer wind # data by running three separate programs (dcodcloc, dataqc, # datasort) to ultimately generate a quality controlled and # sorted ERS data file in BUFR copied to the designated path # to output data destination; if 'NO' will not reprocess ERS # scatterometer data by running these three programs # Default is "YES" # prepqksd - string: if = 'YES' will run wave_dcodquikscat to reprocess # Quikscat scatterometer wind data (most likely generating # superobs) into BUFR file copied to the designated path to # output data destination; if 'NO' will not run # wave_dcodquikscat program # Default is "YES" # preptrmm - string: if = 'YES' will run bufr_supertmi to reprocess # (superob) TRMM TMI data into BUFR file copied to the # designated path to output data destination; if 'NO' will # not run bufr_supertmi program # Default is "YES" # prepwindsat - string: if = 'YES' will run bufr_dcodwindsat to reprocess # WindSat scatterometer data into BUFR file copied to the # designated path to output data destination; if 'NO' will # not run bufr_dcodwindsat program # Default is "YES" # CHGRP_RSTPROD - string: if = 'YES' will check to see if a dump file # qualifies as a "RESTRICTED" data type and if so, will # change the file's group to "rstprod" and its permission to # "640" so that it can only be read by users in the rstprod # group # Default is "YES" # # These apply ONLY for imported shell variable "prepssmi" set to YES # and will be set to their default value in this script if not exported # from the parent script -- # # EXECPREP - string indicating directory path for prepobs_prepssmi # executable # Default is "/nw${envir}/exec" # FIXPREP - string indicating directory path for prepobs_prepssmi bufr # mnemonic table (this is a partial table, ush # prepobs_prepssmi.sh generates the complete table based on # namelist switches in the prepobs_prepssmi data cards) # Default is "/nw${envir}/fix" # PARMPREP - string indicating directory path for prepobs_prepssmi # data cards # Default is "/nw${envir}/parm" # utilparm - string indicating directory path for prepobs_prepssmi # NESDIS land/sea tag (mask) fixed file (only needed if file # input to prepobs_prepssmi contains brightness temperatures # and products are being caluclated in-line) # Default is "/nw${envir}/util/parm" # EXGRBIX - string indicating directory path for grib file # processing executables # Default is "/nw${envir}/util/exec" # NOTE: Also applies for prepersd=YES (see below) # USHPMI - string indicating directory path for prepobs_prepssmi # ush script prepobs_prepssmi.sh # Default is "/nw${envir}/ush" # PREPSSMI_PROD_TYPE - string: if = 'GOODBERLET' will input "ssmip" data # dump to prepobs_prepssmi program (FNOC "operational" # derived products); if = 'NEURAL_NET3' will input "ssmipn" # data dump to prepobs_prepssmi program (NCEP Neural Net-3 # derived products); if = 'COMBINATION' will input "ssmip" # and "ssmipn" data dumps each into separate executions of # prepobs_prepssmi script and program and then combine the # output spssmi BUFR files into a single file for input to # the prepobs_prepdata program (this is used when it is # necessary to reprocess different variables with different # algorithms; e.g., precipitable water with NN3 algorithm # and wind speed with FNOC algorithm - here the data cards # determine which variable uses which algorithm) # NOTE 1: For "PREPSSMI_PROD_TYPE" = 'COMBINATION', BOTH # "ssmip" and "ssmipn" must appear in the SAME data # group list (i.e., they can't appear in separate # executions of this script in the same job) - their # location in the data group list is not important, # however. # NOTE 2: The value for this variable is overridden if the # prepobs_prepssmi data cards contain IALG=1,2,..,98 # which means input the brightness temperature # (ssmit) data dump and let prepobs_prepssmi generate # the products in-line (product type is dependent # upon the value of IALG in data cards). # Default is "GOODBERLET" # PMIX - string indicating executable path for prepobs_prepssmi # program # Default is "$EXECPREP/prepobs_prepssmi # PMIC - string indicating data card path for prepobs_prepssmi # program # Default is "$PARMPREP/prepobs_prepssmi.${NET}.parm" # PMIT - string indicating mnemonic bufrtable file path for # prepobs_prepssmi program (this is a partial table, ush # prepobs_prepssmi.sh generates the complete table based on # namelist switches in the prepobs_prepssmi data cards) # Default is "$FIXPREP/prepobs_prepssmi.bufrtable # LANDC - string indicating NESDIS land/sea tag (mask) fixed file # path for prepobs_prepssmi program (only needed if file # input to prepobs_prepssmi contains brightness temperatures # and products are being caluclated in-line) # Default is "$utilparm/nesdis.lstags" # # These apply ONLY for imported shell variable "prepersd" set to YES # and will be set to their default value in this script if not exported # from the parent script -- # # EXGRBIX - string indicating directory path for grib file # processing executables # Default is "/nw${envir}/util/exec" # NOTE: Also applies for prepssmi=YES (see above) # EXECWAVE - string indicating directory path for ERS scatterometer # data reprocessing executables (wave_dcodcloc,wave_dataqc, # wave_datasort) # Default is "/nw${envir}/exec" # NOTE: Also applies for prepqksd=YES (see below) # FIXWAVE - string indicating directory path for ERS scatterometer # data reprocessing fixed fields (including mnemonic # bufrtable file read by wave_datasort) # Default is "/nw${envir}/fix" # NOTE: Also applies for prepqksd=YES (see below) # DCLX - string indicating executable path for wave_dcodcloc # program # Default is "$EXECWAVE/wave_dcodcloc" # DQCX - string indicating executable path for wave_dataqc # program # Default is "$EXECWAVE/wave_dataqc" # DSRX - string indicating executable path for wave_datasort # program # Default is "$EXECWAVE/wave_datasort" # DSRT - string indicating mnemonic bufrtable file path for # wave_datasort program # Default is "$FIXWAVE/wave_bufrtab.ersscat" # # These apply ONLY for imported shell variable "prepqksd" set to YES # and will be set to their default value in this script if not exported # from the parent script -- # # EXECWAVE - string indicating directory path for wave_dcodquikscat # executable # Default is "/nw${envir}/exec" # NOTE: Also applies for preperds=YES (see above) # FIXWAVE - string indicating directory path for wave_dcodquikscat # fixed fields {mnemonic bufrtable file and 0.5 x 0.5 # degree land/sea tag (mask) file} # Default is "/nw${envir}/fix" # PARMWAVE - string indicating directory path for wave_dcodquikscat # data cards # Default is "/nw${envir}/parm" # DQKX - string indicating executable path for wave_dcodquikscat # program # Default is "$EXECWAVE/wave_dcodquikscat" # DQKC - string indicating data card path for wave_dcodquikscat # program # Default is "$PARMbufr/wave_dcodquikscat.${NET}.parm" # {If $DQKC not found, reverts to using internal "herefile"} # DQKT - string indicating mnemonic bufrtable file path for # wave_dcodquikscat program # Default is "$FIXWAVE/wave_bufrtab.quikscat" # LANDC_DQK - string indicating 0.5 x 0.5 degree land/sea tag (mask) # fixed file path for wave_dcodquikscat program # Default is "$FIXWAVE/wave_landchxh" # # These apply ONLY for imported shell variable "preptrmm" set to YES # and will be set to their default value in this script if not exported # from the parent script -- # # EXECbufr - string indicating directory path for bufr_supertmi # executable # Default is "/nw${envir}/exec" # FIXbufr - string indicating directory path for bufr_supertmi fixed # fields (mnemonic bufrtable file) # Default is "/nw${envir}/fix" # PARMbufr - string indicating directory path for bufr_supertmi data # cards # Default is "/nw${envir}/parm" # DTMX - string indicating executable path for bufr_supertmi # program # Default is "$EXECbufr/bufr_supertmi" # DTMC - string indicating data card path for bufr_supertmi # program # Default is "$PARMbufr/bufr_supertmi.${NET}.parm" # {If $DTMC not found, reverts to using internal "herefile"} # DTMT - string indicating mnemonic bufrtable file path for # bufr_supertmi program # Default is "$FIXbufr/bufr_bufrtab.sptrmm" # # These apply ONLY for imported shell variable "prepwindsat" set to YES # and will be set to their default value in this script if not exported # from the parent script -- # # EXECbufr - string indicating directory path for bufr_dcodwindsat # executable # Default is "/nw${envir}/exec" # FIXbufr - string indicating directory path for bufr_dcodwindsat fixed # fields (mnemonic bufrtable file) # Default is "/nw${envir}/fix" # PARMbufr - string indicating directory path for bufr_dcodwindsat data # cards # Default is "/nw${envir}/parm" # DWSX - string indicating executable path for bufr_dcodwindsat # program # Default is "$EXECbufr/bufr_dcodwindsat" # DWSC - string indicating data card path for bufr_dcodwindsat # program # Default is "$PARMbufr/bufr_dcodwindsat.${NET}.parm" # {If $DWSC not found, reverts to using internal "herefile"} # DWST - string indicating mnemonic bufrtable file path for # bufr_dcodwindsat program # Default is "$FIXbufr/bufr_bufrtab.windsat" # LANDC_DWS - string indicating 0.5 x 0.5 degree land/sea tag (mask) # fixed file path for bufr_dcodwindsat program # Default is "$FIXbufr/wave_landchxh" # # # Modules and files referenced: # scripts : /nwprod/util/ush/setup.sh # $DUMP # $USHPMI/prepobs_prepssmi.sh # /nwprod/util/ush/getges.sh # executables: $PMIX # $EXGRBIX/grbindex # $DCLX # $DQCX # $DSRX # $DQKX # $DTMX # $DWSX # data cards : $PMIC ($PMIX) # data cards : $DQKC ($DQKX) # $DTMC ($DTMX) # $DWSC ($DWSX) # fixed fields: $FIXWAVE/wave_bufrtab.erscat ($DSRX) # $FIXWAVE/wave_CMOD_DBLUT ($DSRX) # $FIXWAVE/wave_CMOD_QSLUT ($DSRX) # $FIXWAVE/wave_zmask1x1 ($DCLX) # $LANDC_DQK ($DQKX) # $DQKT ($DQKX) # $PMIT ($PMIX) # $DTMT ($DTMX) # $LANDC_DWS ($DWSX) # $DWST ($DWSX) # utility parm: $LANDC ($PMIX) # # Remarks: # # Output dump data set will be in the form (see - @): # ${COMSP}dgrpX.${tmmark}.bufr_d # (See above for usual string appearance for $COMSP and $tmmark; dgrpX # is input positional parameter mnemonic associated with a particular # data group) # # prepobs_prepssmi may or may not be run (depending on imported shell # variable "prepssmi") # # ERS scatterometer data reprocessing may or may not occur (depending # on imported shell variable "prepersd") # # wave_dcodquikscat may or may not be run (depending on imported shell # variable "prepqksd") # # bufr_supertmi may or may not be run (depending on imported shell # variable "preptrmm") # # bufr_dcodwindsat may or may not be run (depending on imported shell # variable "prepwindsat") # # @ - If imported shell variable "SENDCOM" is set to 'NO', then "COMSP" # above is hardwired to be "$DATA/" and output dump data set will be # in the form: ${DATA}/dgrpX.${tmmark}.bufr_d # # Condition codes # dumpjb -- # 0 - all requested data group subtypes found and dumped # 4 - one or more data group subtype dumps contain zero reports - all # of these subtypes are currently EXPECTED to be missing or dump # zero reports SOME or ALL of the time for particular data group # at the dump cycle time; does not indicate a problem # 5 - all data group subtype dumps contain zero reports - this data # group is currently EXPECTED to be missing or dump zero reports # SOME or ALL of the time at the dump cycle time; does not indicate # a problem # 11 - one or more data group subtype dumps contain zero reports - at # least one of these subtypes is currently EXPECTED to dump reports # ALL of the time for particular data group at the dump cycle time; # indicates a problem # 22 - all data group subtype dumps contain zero reports - at least one # of the subtypes in this data group is currently EXPECTED to dump # reports ALL of the time at the dump cycle time; indicates a # problem # 99 - catastrophic problem -- script aborted # # prepobs_prepssmi -- # 0 - program completed successfully # < 51 - some problem encountered (see prepobs_prepssmi docblock) # (non-fatal) # > 50 - catastrophic problem -- script aborted # # wave_dcodcloc -- # 0 - program completed successfully # > 0 - some problem encountered (see wave_dcodcloc docblock) # (non-fatal) # # wave_dataqc -- # 0 - program completed successfully # > 0 - some problem encountered (see wave_dataqc docblock) # (non-fatal) # # wave_datasort -- # 0 - program completed successfully # > 0 - some problem encountered (see wave_datasort docblock) # (non-fatal) # # wave_dcodquikscat -- # 0 - program completed successfully # > 0 - some problem encountered (see wave_dcodquikscat docblock) # (non-fatal) # # bufr_supertmi -- # 0 - program completed successfully # > 0 - some problem encountered (see bufr_supertmi docblock) # (non-fatal) # # bufr_dcodwindsat -- # 0 - program completed successfully # > 0 - some problem encountered (see wave_dcodwindsat docblock) # (non-fatal) # # # NOTE: Be careful, if this script ever needs to obtain the file ncepdate # from the working directory (and it is written there by parent # script), make sure it is correct! (Right now, this script does # not need to obtain the file ncepdate.) # #### set -aux mkdir -p $DATA/job${DUMP_NUMBER} cd $DATA/job${DUMP_NUMBER} sh /nwprod/util/ush/setup.sh ############################################################################# ############################################################################# # EXECUTE THE DUMP SCRIPT ############################################################################# ############################################################################# #============================================ cendat=$1 timwin=$2 ntype=$3 job=${job:-j????} cycle=${cycle:-t??z} set +x echo echo "job = " $job echo "cycle = " $cycle echo set -x #============================================ shift 3 FORM=ibm #--------------------------------------------------------------------------- # The arguments for the DUMP script are described below. #--------------------------------------------------------------------------- JOB_NUMBER=${JOB_NUMBER:-""} echo echo "JOB_NUMBER = " $JOB_NUMBER echo STATUS=${STATUS:-YES} echo echo "STATUS = " $STATUS echo envir=${envir:-prod} DUMP=${DUMP:-/nw${envir}/ush/dumpjb} LOUD=${LOUD:-off} if [ "$1" = 'null' ];then errdmp=0 else #--------------------------------------------------------------------------- # The DUMP script The center of The radius of Data Group Names: # accepts these the time the time # arguments: window: window: # # Example. yyyymmddhh<.hh> hh<.hh> dgrp1 dgrp2 ... dgrpN #--------------------------------------------------------------------------- $DUMP $cendat $timwin $* > outout 2>errfile errdmp=$? fi #....................................................................... errt=0 typeset -Z2 chr chr=`expr $cendat % 100` # Note: The following subtypes are EXPECTED to OFTEN be missing regardless # of the center dump time or the data dump network - if only these types # are missing the return code of "11" is reduced to "4" (see "for" loop # below) rm pattern # Mobil land raob, drops, reccos, INSAT satwnds and GMS vis satwnds cat <<\EOFp > pattern 002.002 002.004 004.005 005.021 005.022 005.023 005.042 005.045 EOFp if [[ $job = nam_dump_?? ]];then # The pattern is expanded to include GMS IR, WV and MODIS (Aqua/Terra) IR, WV # satwnds and wind profilers orig. from PILOT (pibal) bulletins for Job # nam_dump at all center dump times cat <<\EOFp1 >> pattern 005.041 005.043 005.070 005.071 002.009 EOFp1 if [ "$chr" = '00' -o "$chr" = '06' ];then # The pattern is expanded to include EUMETSAT vis satwnds for Job nam_dump # at center dump times 00 and 06Z (nighttime) cat <<\EOFp1p2 >> pattern 005.065 EOFp1p2 fi elif [[ $job = gfs_dump_?? ]];then # The pattern is expanded to include MODIS (Aqua/Terra) IR and WV satwnds for # JOB gfs_dump at all center times cat <<\EOFp1gfs >> pattern 005.070 005.071 EOFp1gfs elif [[ $job = ndas_dump_tm??_?? ]];then # The pattern is expanded to include wind profilers from PILOT (pibal) # bulletins for Job ndas_dump at all center dump times cat <<\EOFp1p1 >> pattern 002.009 EOFp1p1 if [ "$chr" = '00' -o "$chr" = '03' -o "$chr" = '06' ];then # The pattern is expanded to include EUMETSAT vis satwnds for Job ndas_dump # at center dump times 00, 03 and 06Z (nighttime) cat <<\EOFp1p3 >> pattern 005.065 EOFp1p3 fi if [[ $job = ndas_dump_tm03_?? ]];then # The pattern is expanded to include MODIS (Aqua/Terra) IR and WV satwnds for # Job ndas_dump_tm03 at all center dump times cat <<\EOFp1p4 >> pattern 005.070 005.071 EOFp1p4 fi elif [[ $job = ruc2?_dump_?? ]];then # The pattern is expanded to include ship raobs for Job ruc2a_dump at all # center dump times (the latter was added only because RUC data is no longer # dumped globally) # The pattern is also expanded to include pibals for Job ruc2a_dump at all # center dump times (this was added only because RUC data is no longer # dumped globally - actually pibals are already excluded at times other than # 00 or 12Z via logic further below which was not related to change from # global dumping - this had to be added to account for 00 and 12Z missing # pibals) # The pattern is also expanded to include automated tide gauge reports for # Job ruc2a_dump at all center dump times (this was added because since # 4/11/2001 these data almost always arrive too late for use by these # networks) cat <<\EOFp2 >> pattern 002.003 002.005 001.005 EOFp2 if [[ $job = ruc2a_dump_?? ]];then if [ "$chr" != '00' -a "$chr" != '12' ];then # The pattern is expanded to surface marine ships, drifting buoys and APG # MESONETs for Job ruc2a_dump at all center dump times except 00 and 12Z cat <<\EOFp3p1 >> pattern 001.001 001.002 255.011 EOFp3p1 fi fi elif [[ $job = sruc_dump_???? ]];then # The pattern is expanded to include surface marine drifting buoys, C-MAN # platforms, automated tide gauge reports and APG MESONETs for Job sruc_dump # (both T+0:05 and T+0:22) at all center dump times cat <<\EOFp3p3 >> pattern 001.002 001.004 001.005 255.011 EOFp3p3 if [[ $job = sruc_dump_??05 ]];then # The pattern is expanded to include all MESONET subtypes except for 012-014 # and 030 for Job sruc_dump at T+0:05 (only) at all center dump times cat <<\EOFp3p4 >> pattern 255.001 255.002 255.003 255.004 255.005 255.006 255.007 255.008 255.009 255.010 EOFp3p4 fi fi if [ "$chr" -ge '5' -a "$chr" -le '11' ];then # The pattern is expanded to include GOES high-density visible satwnds for # all jobs centered on "night-time" center dump times # Summer nighttime: 5Z - 11Z, inclusive cat <<\EOFp5 >> pattern 005.012 EOFp5 elif [ "$job" = 'nam_dump_18' ];then # The pattern is expanded to include GOES high-density visible satwnds for # Job nam_dump at center dump time 18Z cat <<\EOFp6 >> pattern 005.012 EOFp6 fi if [ "$1" != 'null' ];then # Loop through all groups being dumped - assign proper return code to # each dumped data group cat <> status1.out ------------------------------------------------------------------------------- Dump Script run for data groups: $* EOFds1 for n do echo " -- Data Group $n --" >> status1.out grep "DESCRIPTOR" $DATA/${n}.out >> status1.out grep ">>>" $DATA/${n}.out > grep1.out errgrep=$? if [ $errgrep -eq '0' ];then nn=`head -n 1 grep1.out` msg="**WARNING: $n dump incomplete -- $nn" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u echo "$msg" >> outout echo "$msg" >> status1.out echo " " >> outout echo " " >> status1.out fi rm grep1.out errn=`grep "Dump Status: $n " outout | cut -f2 -d=` if [ "$errn" -eq '0' ];then msg="$n dump SUCCESSFUL, all subtypes contain reports (RC=$errn)" elif [ "$errn" -gt '22' ];then msg="**FATAL ERROR in $n dump, NO REPORTS DUMPED (RC=$errn)" elif [ "$errn" -eq '22' ];then msg="**WARNING: $n dump EMPTY, all subtypes missing (RC=$errn)" if [[ $job = ruc2a_dump_?? && $n = satwnd ]];then # Note: For Job ruc2a_dump and SATWND at all center dump times except 00, 01, # 04, 07, 10, 12, 13, 16, 19, and 22Z, all subtypes are expected to # OFTEN be missing - the return code of "22" is reduced to "5" here [ "$chr" != '00' -a "$chr" != '01' -a "$chr" != '04' -a \ "$chr" != '07' -a "$chr" != '10' -a "$chr" != '12' -a \ "$chr" != '13' -a "$chr" != '16' -a "$chr" != '19' -a \ "$chr" != '22' ] && errn=5 elif [[ $job = ruc2a_dump_?? && ( $n = ssmip || $n = ssmipn || \ $n = ssmit ) ]];then # Note: For Job ruc2a_dump and SSMIP, SSMIPN, or SSMIT at all center dump # times except 00 and 12Z; all subtypes are expected to OFTEN be missing # - the return code of "22" is reduced to "5" here [ "$chr" != '00' -a "$chr" != '12' ] && errn=5 elif [[ $job = sruc_dump_??05 && $n = proflr ]];then # Note: For Job sruc_dump at T+0:05 (only) at all center dump times, PROFLR is # expected to ALWAYS be missing - the return code of "22" is reduced to # "5" here errn=5 else # Note: For ADPUPA or SFCBOG at all center dump times other than 00 or 12Z, # all subtypes are expected to OFTEN (ADPUPA) or ALWAYS (SFCBOG) be # missing - the return code of "22" is reduced to "5" here if [ "$chr" != '00' -a "$chr" != '12' ];then [ "$n" = 'adpupa' ] && errn=5 [ "$n" = 'sfcbog' ] && errn=5 else # Note: For Job gfs_dump and SFCBOG at center dump times 00 or 12Z; all # subtypes are expected to ALWAYS be missing - the return code of "22" # is reduced to "5" here [[ $job = gfs_dump_?? && $n = sfcbog ]] && errn=5 fi if [[ $job = nam_dump_?? || $job = ndas_dump_tm03_?? ]];then if [ $n = airsev ];then # Note: For Job nam_dump and tm03 ndas_dump at all cycle times, AIRSEV data are # expected to OFTEN be missing - the return code of "22" is reduced to # "5" here errn=5 fi fi if [ $job = nam_dump_06 ];then if [ $n = 1bamub -o $n = 1bhrs3 ];then # Note: For Job nam_dump at 06Z cycle time, 1BAMUB and 1BHRS3 data are expected # to OFTEN be missing - the return code of "22" is reduced to # "5" here errn=5 fi if [ "$chr" != '12' ];then if [ $n = 1bmsu -o $n = 1bhrs2 ];then # Note: For Job nam_dump at all cycle times except 12Z, MSU and HIRS2 (NOAA 14) # 1B data are expected to OFTEN be missing # - the return code of "22" is reduced to "5" here errn=5 fi fi fi fi elif [ "$errn" -eq '11' ];then grep " HAS 0" $DATA/${n}.out | cut -c1-7 | \ grep -v -f pattern > list_11 errgrep=$? grep " HAS 0" $DATA/${n}.out | cut -c1-7 | \ grep -f pattern > list_04 # Note: A non-zero "errgrep" mean all missing subtypes in this group are # EXPECTED to OFTEN be missing regardless of the center dump time - the # return code of "11" is reduced to "4" here (see "pattern" above) if [ "$errgrep" -ne '0' ];then errn=4 else if [[ $job = ruc2a_dump_?? && $n = satwnd ]];then # Note: For Job ruc2a_dump and SATWND at all center dump times other than 00, # 01, 04, 07, 10, 12, 13, 16, 19 or 22Z, one or more subtypes are # expected to OFTEN be missing - the return code of "11" is reduced to # "4" here # (Note: After all is said and done, the only times when a ruc2a dump # could return an 11 for GOES visible winds are 00?, 01, 04, 12?, # 13, 16, 19, 22Z) [ "$chr" != '00' -a "$chr" != '01' -a "$chr" != '04' -a \ "$chr" != '07' -a "$chr" != '10' -a "$chr" != '12' -a \ "$chr" != '13' -a "$chr" != '16' -a "$chr" != '19' -a \ "$chr" != '22' ] && errn=4 else # Note: For all jobs dumping ADPUPA at all center dump times other than 00 or # 12Z, and for all jobs EXCEPT job ruc2a_dump dumping SATWND at center # dump times other than 00, 03, 06, 09, 12, 15, 18, or 21Z; one or more # subtypes are expected to OFTEN (ADPUPA) or ALWAYS (SATWND) be # missing - the return code of "11" is reduced to "4" here # {Note 1: At 03, 09, 15, and 21Z only NDAS jobs dump SATWNDs here and # these include only GOES and EUMETSAT SATWNDs (no other # foreign winds)} # (Note 2: Missing GOES visible SATWNDS at 06 and 09Z due to nighttime # have already been accounted for by this script) if [ "$chr" != '00' -a "$chr" != '12' ];then [ "$n" = 'adpupa' ] && errn=4 if [ "$chr" != '03' -a "$chr" != '06' -a "$chr" != '09' -a \ "$chr" != '15' -a "$chr" != '18' -a "$chr" != '21' ];then [ "$n" = 'satwnd' ] && errn=4 fi fi fi [ "$errn" -eq '4' ] && cat list_11 >> list_04 fi else msg="**WARNING: $n dump RETURN CODE NOT RECOGNIZED, status unknown (RC=$errn)" fi [ "$errt" -lt "$errn" ] && errt=$errn if [ "$errn" -eq '5' ];then msg="$n dump EMPTY, all subtypes missing as expected at this time (RC=$errn)" msg45="[For $n, original R.C. 22 reduced to 5, all subtypes expected to be missing at this time]" fi if [ "$errn" -eq '4' ];then msg45="[For $n, original R.C. 11 reduced to 4, missing data of these subtypes are expected at this time]" nindx=`wc list_04 | awk '{ print $1 }'` mindx=0 until [ "$mindx" -eq "$nindx" ] do mindx=`expr $mindx + 1` subt=`head -n${mindx} list_04 | tail -n1` msg="$n dump SUCCESSFUL, subtype $subt missing as expected at this \ time (RC=$errn)" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u done msg="$n dump SUCCESSFUL, some subtypes missing as expected at this time \ (RC=$errn)" elif [ "$errn" -eq '11' ];then nindx=`wc list_11 | awk '{ print $1 }'` mindx=0 until [ "$mindx" -eq "$nindx" ] do mindx=`expr $mindx + 1` subt=`head -n${mindx} list_11 | tail -n1` msg="**WARNING: $n dump INCOMPLETE, subtype $subt missing (RC=$errn)" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u done nindx=`wc list_04 | awk '{ print $1 }'` mindx=0 until [ "$mindx" -eq "$nindx" ] do mindx=`expr $mindx + 1` subt=`head -n${mindx} list_04 | tail -n1` msg="... even though $n dump incomplete, subtype $subt missing as \ expected at this time" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u done msg="**WARNING: $n dump INCOMPLETE, 1 or more subtypes missing (RC=$errn)" else set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u fi echo "$msg" >> outout echo "$msg" >> status1.out if [ "$errn" -eq '4' -o "$errn" -eq '5' ];then echo "$msg45" >> outout echo "$msg45" >> status1.out fi echo " " >> outout echo " " >> status1.out set +x echo echo "The foreground exit status for dumpjb, $n only, is " $errn echo set -x done if [ "$errdmp" -eq '11' -a "$errt" -eq '4' ];then msg1="==> The final overall return code for this run of dumpjb is 4, \ reduced from original value of 11 for reasons noted just above" elif [ "$errdmp" -eq '22' -a "$errt" -eq '5' ];then msg1="==> The final overall return code for this run of dumpjb is 5, \ reduced from original value of 22 for reasons noted just above" else msg1="==> The final overall return code for this run of dumpjb remains $errt" fi echo "$msg1" >> outout echo " " >> outout echo "$msg1" >> status1.out echo " " >> status1.out errdmp=$errt rm pattern #....................................................................... if [ "$LOUD" != 'off' ];then set +x echo echo "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" echo "------------------------------------------------------------------------" echo " LOUD != off - below is script trace for this run of dumpjb" echo "------------------------------------------------------------------------" echo cat errfile echo echo "------------------------------------------------------------------------" echo " end of script trace for this run of dumpjb" echo "------------------------------------------------------------------------" echo "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" echo set -x else cat errfile >> outout cat errfile >> status1.out fi rm errfile set +x echo echo "---------------------------------------------------" echo "************ COMPLETED SCRIPT dumpjb ************" echo "** foreground exit status for ALL GROUPS is $errdmp **" echo "---------------------------------------------------" echo set -x #fi for $1!=null test fi # Move output files to proper naming structure # -------------------------------------------- errmv=0 pgmout_this=$DATA/job${DUMP_NUMBER}/allout #============================================ SENDCOM=${SENDCOM:-YES} [ "$SENDCOM" = 'NO' ] && COMSP=$DATA/ pgmout=${pgmout:-$DATA/allout} prepssmi=${prepssmi:-YES} prepersd=${prepersd:-YES} prepqksd=${prepqksd:-YES} preptrmm=${preptrmm:-YES} prepwindsat=${prepwindsat:-YES} CHGRP_RSTPROD=${CHGRP_RSTPROD:-YES} set +x echo echo "SENDCOM = " $SENDCOM echo echo "pgmout = " $pgmout echo echo "pgmout_this = " $pgmout_this echo echo "prepssmi = " $prepssmi echo echo "prepersd = " $prepersd echo echo "prepqksd = " $prepqksd echo echo "preptrmm = " $preptrmm echo echo "prepwindsat = " $prepwindsat echo echo "CHGRP_RSTPROD = " $CHGRP_RSTPROD echo set -x #============================================ for n do [ "$1" = 'null' ] && break cat $DATA/${n}.out >> outout rm $DATA/${n}.out if [ -s $DATA/${n}.${FORM} ];then cp $DATA/${n}.${FORM} ${COMSP}${n}.${tmmark}.bufr_d errmvt=$? rm $DATA/${n}.${FORM} errmvl=$errmv errmv=`expr $errmvl + $errmvt` else [ "$SENDCOM" != 'NO' ] && cp /dev/null ${COMSP}${n}.${tmmark}.bufr_d fi if [ "$CHGRP_RSTPROD" = 'YES' ]; then if [ $n = adpsfc -o $n = aircar -o $n = aircft -o $n = msonet -o \ $n = sfcshp ]; then chgrp rstprod ${COMSP}${n}.${tmmark}.bufr_d errch=$? if [ $errch -eq 0 ]; then chmod 640 ${COMSP}${n}.${tmmark}.bufr_d msg="NOTE: $n dump contains RESTRICTED data, only users in \ rstprod group have read permission" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u echo "$msg" >> outout echo "$msg" >> status1.out echo " " >> outout echo " " >> status1.out else cp /dev/null ${COMSP}${n}.${tmmark}.bufr_d msg="**WARNING: $n dump contains RESTRICTED data, since user \ $USER is not in rstprod group a null file is copied in its place" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u echo "$msg" >> outout echo "$msg" >> status1.out echo " " >> outout echo " " >> status1.out fi fi fi done if [ -f outout ];then cat outout >> dumpjb.out cat outout >> $pgmout_this rm outout fi ############################################################################# ############################################################################# # EXECUTE THE PREPOBS_PREPSSMI PROGRAM (IF REQUESTED) ############################################################################# ############################################################################# PREPSSMI_PROD_TYPE=${PREPSSMI_PROD_TYPE:-GOODBERLET} kindx=1 ssmip_flag=0 ssmipn_flag=0 ssmit_flag=0 [ "$PREPSSMI_PROD_TYPE" = 'COMBINATION' ] && ssmit_flag=1 > ssmi_count until [ $kindx -eq 3 ] do kindx=`expr $kindx + 1` ssmif=ssmip kflag=0 for nnn do if [[ $nnn = ssmi* ]];then eval iflag=\$${nnn}_flag [ "$iflag" -eq '1' ] && continue if [ "$kindx" -eq '2' ];then # set-up variables and file paths to run prepobs_prepssmi # ------------------------------------------------------- #============================================ EXECPREP=${EXECPREP:-/nw${envir}/exec} PARMPREP=${PARMPREP:-/nw${envir}/parm} FIXPREP=${FIXPREP:-/nw${envir}/fix} utilparm=${utilparm:-/nw${envir}/util/parm} EXGRBIX=${EXGRBIX:-/nw${envir}/util/exec} USHPMI=${USHPMI:-/nw${envir}/ush} PMIT=${PMIT:-$FIXPREP/prepobs_prepssmi.bufrtable} PMIX=${PMIX:-$EXECPREP/prepobs_prepssmi} PMIC=${PMIC:-$PARMPREP/prepobs_prepssmi.${NET}.parm} [ -s $PMIC ] || break 2 pmic=$PMIC LANDC=${LANDC:-$utilparm/nesdis.lstags} set +x echo echo "EXECPREP = " $EXECPREP echo echo "PARMPREP = " $PARMPREP echo echo "utilparm = " $utilparm echo echo "FIXPREP = " $FIXPREP echo echo "EXGRBIX = " $EXGRBIX echo echo "PREPSSMI_PROD_TYPE = " $PREPSSMI_PROD_TYPE echo set -x fi if [ "$PREPSSMI_PROD_TYPE" = 'COMBINATION' ];then grep ${nnn}: $PMIC | awk -F: '{print $2}' > insert mcount=`grep -n IVAR $PMIC | grep = | cut -f1 -d: | head -n 1` mcount=`expr $mcount - 1` mtotal=`cat <$PMIC | wc -l` head -n $mcount $PMIC > top_part mcount=`expr $mtotal - $mcount - 1` tail -n $mcount $PMIC > bottom_part cat top_part insert bottom_part > prepobs_prepssmi.cards.$nnn pmic=$DATA/job${DUMP_NUMBER}/prepobs_prepssmi.cards.$nnn rm top_part bottom_part insert eval ${nnn}_flag=1 else kindx=3 fi # Grep out value for IALG switch in the data cards, as this helps determine # which BUFR SSM/I data dump file is to be input to prepobs_prepssmi # ------------------------------------------------------------------------- IALG=`grep IALG $pmic | awk -F, \ '{print $1; print $2; print $3; print $4; print$5}' | grep IALG | \ awk -F= '{print $2}'` if [ "$IALG" -gt '0' ];then ssmif=ssmit else if [ "$PREPSSMI_PROD_TYPE" = 'NEURAL_NET3' ];then ssmif=ssmipn elif [ "$PREPSSMI_PROD_TYPE" = 'COMBINATION' ];then ssmif=$nnn else ssmif=ssmip fi fi set +x echo echo "ssmif = " $ssmif echo set -x kflag=1 break #============================================ fi done [ "$kflag" -eq '0' ] && kindx=3 cp /dev/null sstgrb cp /dev/null sstgrb.index errsmi=0 msg1=NO for nnn do [ "$nnn" = $ssmif -a "$SENDCOM" != 'NO' ] && \ cp /dev/null ${COMSP}spssmi.${tmmark}.bufr_d if [ "$nnn" = $ssmif -a "$prepssmi" = 'YES' ];then set +x echo "****************************************************************\ ***************" >> status1.out msg="PROGRAM prepobs_prepssmi will run from input $ssmif dump" echo " " echo "##################################################################" echo "$msg" echo "##################################################################" echo " " for OUT in prepssmi.out $pgmout_this status1.out do echo " " >> $OUT [ "$OUT" != 'status1.out' ] && echo "#############################\ #####################################" >> $OUT echo "$msg" >> $OUT [ "$OUT" != 'status1.out' ] && echo "#############################\ #####################################" >> $OUT [ "$OUT" != 'status1.out' ] && echo " " >> $OUT done set -x set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u if [ -s ${COMSP}$ssmif.${tmmark}.bufr_d ];then # more variables and file paths to run prepobs_prepssmi # ----------------------------------------------------- cd $DATA/job${DUMP_NUMBER} cp ${COMSP}$ssmif.${tmmark}.bufr_d ssmif_in SSMI_IN=ssmif_in ssmiOU=${COMSP}spssmi.${tmmark}.bufr_d if [ "$PREPSSMI_PROD_TYPE" = 'COMBINATION' ];then ssmiOU_save=$ssmiOU ssmiOU=$ssmiOU_save.$ssmif cp /dev/null $ssmiOU fi DATA_save=$DATA DATA=$DATA/job${DUMP_NUMBER} pgmout_save=$pgmout pgmout=$pgmout_this PMIC_save=$PMIC PMIC=$pmic timex $USHPMI/prepobs_prepssmi.sh $cendat errsmi=$? if [ "$errsmi" -eq 0 -a -s grep.count.ssmi ];then echo " " >> status1.out if [ "$PREPSSMI_PROD_TYPE" = 'COMBINATION' ];then cut -c46- grep.count.ssmi | tr 'A-Z' 'a-z' | tr -d '<<<' \ >>status1.out else cut -c41- grep.count.ssmi | tr -d '<<<' >> status1.out fi rm grep.count.ssmi fi DATA=$DATA_save pgmout=$pgmout_save PMIC=$PMIC_save [ -s msg1_file ] && msg1="`cat msg1_file` from input $ssmif dump" [ -s msg2_file ] && msg2="`cat msg2_file` from input $ssmif dump" cp ssmi.bufr $ssmiOU errmvt=$? errmvl=$errmv errmv=`expr $errmvl + $errmvt` rm ssmif_in ssmi.bufr else msg2="**PROGRAM PREPOBS_PREPSSMI NOT RUN BECAUSE $ssmif dump \ file not found" fi set +u if [ -n "$jlogfile" ];then [ "$msg1" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg1" $DATA/postmsg "$jlogfile" "$msg2" fi set -u set +x echo echo "************************************************************************" [ "$msg1" != 'NO' ] && echo "$msg1" echo "$msg2" echo "************************************************************************" echo set -x for OUT in prepssmi.out $pgmout_this status1.out do echo " " >> $OUT [ "$msg1" != 'NO' ] && echo "$msg1" >> $OUT if [ "$OUT" = 'status1.out' ];then echo $msg2 | grep "generated" errgrep=$? if [ "$errgrep" -eq '0' ]; then echo "$msg2" | cut -c1-66 >> $OUT else echo "$msg2" >> $OUT fi else echo "$msg2" >> $OUT fi echo " " >> $OUT done echo "**********************************************************************\ *********" >> status1.out [ "$errsmi" -gt '50' ] && exit $errsmi break fi done done set +u if [ "$PREPSSMI_PROD_TYPE" = 'COMBINATION' -a -n "$ssmiOU_save" ];then set -u # if reprocessed ssmipn and ssmip dumps exist, run combfr to combine # them into reprocessed spssmi file # ------------------------------------------------------------------ if [ -s $ssmiOU_save.ssmipn -a -s $ssmiOU_save.ssmip ];then cp $ssmiOU_save.ssmipn bufr1 cp $ssmiOU_save.ssmip bufr2 cat < cards bufr1 bufr2 EOFd pgm=`basename /nwprod/exec/bufr_combfr` if [ -s $DATA/prep_step ];then set +u . $DATA/prep_step set -u else [ -f errfile ] && rm errfile export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u fi XLFUNIT_50=combfr.spssmi timex /nwprod/exec/bufr_combfr outout 2> errfile err=$? ######cat errfile cat errfile >> outout rm errfile cat outout >> combfr.out cat outout >> $pgmout_this set +x echo echo "The foreground exit status for BUFR_COMBFR to make combined \ spssmi file is " $err echo set -x rm bufr1 bufr2 if [ "$err" -gt '0' ];then msg="**FATAL ERROR PROGRAM BUFR_COMBFR (making combined spssmi \ file) RETURN CODE $err" else set +x echo echo "--------------------------------------------------------------------" echo "**** COMPLETED PROGRAM bufr_combfr to make combined spssmi file ****" echo "--------------------------------------------------------------------" echo set -x msg="program BUFR_COMBFR completed normally - combined spssmi file \ generated" fi rm outout set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msg" set -u echo " " >> $pgmout_this echo "$msg" >> $pgmout_this echo " " >> $pgmout_this [ "$err" -gt '0' ] && exit 99 elif [ -s $ssmiOU_save.ssmipn ];then cp $ssmiOU_save.ssmipn combfr.spssmi err=$? elif [ -s $ssmiOU_save.ssmip ];then cp $ssmiOU_save.ssmip combfr.spssmi err=$? else cp /dev/null combfr.spssmi err=$? fi rm $ssmiOU_save.ssmipn $ssmiOU_save.ssmip if [ $err -eq 0 ];then cp combfr.spssmi $ssmiOU_save if [ -s ssmi_count ];then n1=`head -n1 ssmi_count` n2=`tail -n1 ssmi_count` nt=`expr $n1 + $n2` rm ssmi_count else nt="?????" fi cat << EOF_C >> status1.out ******************************************************************************* spssmi dump file generated by combining reprocessed ssmipn & ssmip dump files >>> COMBINED TOTAL FOR ALL DATA TYPES ... $nt reprocessed ssmipn and ssmip dump files removed ******************************************************************************* EOF_C fi ssmiOU=$ssmiOU_save fi ############################################################################# ############################################################################# # EXECUTE ERS SCATTEROMETER REPROCESSING (IF REQUESTED) ############################################################################# ############################################################################# for nnn do if [ "$nnn" = 'erswnd' ];then msg1=NO msg2="**NO ERS SCATTEROMETER DATA reprocessed due to above PGM FAIL \ --> non-fatal" [ "$SENDCOM" != 'NO' ] && cp /dev/null ${COMSP}erscat.${tmmark}.bufr_d if [ "$prepersd" = 'YES' ];then echo "*******************************************************************\ ************" >> status1.out if [ -s ${COMSP}erswnd.${tmmark}.bufr_d ];then # set-up variables and file paths # ------------------------------- cd $DATA/job${DUMP_NUMBER} cp ${COMSP}erswnd.${tmmark}.bufr_d erswnd_bufr ersIN=erswnd_bufr ersOU=${COMSP}erscat.${tmmark}.bufr_d #============================================ EXGRBIX=${EXGRBIX:-/nw${envir}/util/exec} EXECWAVE=${EXECWAVE:-/nw${envir}/exec} FIXWAVE=${FIXWAVE:-/nw${envir}/fix} DCLX=${DCLX:-$EXECWAVE/wave_dcodcloc} DQCX=${DQCX:-$EXECWAVE/wave_dataqc} DSRT=${DSRT:-$FIXWAVE/wave_bufrtab.erscat} DSRX=${DSRX:-$EXECWAVE/wave_datasort} set +x echo echo "EXGRBIX = " $EXGRBIX echo echo "EXECWAVE = " $EXECWAVE echo echo "FIXWAVE = " $FIXWAVE echo set -x #============================================ # get the "best" pressure grib file valid at cycle time and create # grib index file here to ensure that it's from the same network # (10 meter temperature and wind fields are needed) # ---------------------------------------------------------------- set +x echo echo "VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV" echo " Get pressure grib file valid at center dump time" echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" echo set -x /nwprod/util/ush/getges.sh -t pgbcur -v $cendat pgrb err1=$? set +x echo echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" echo set -x err2=99 if [ "$err1" -eq '0' ];then rm errfile $EXGRBIX/grbindex pgrb pgrb.index 2>errfile err2=$? [ "$err2" -ne '0' ] && cat errfile rm errfile fi # if not already available, get the most recent daily 1 deg. X 1 deg. # GRIB sst data set (unblocked) - create grib index file here to ensure # that it's from the same network and time # (NOTE: sstgrb available at 00, 06, 12, and 18Z from GDAS, but # the index file only available at 00Z each day) # ----------------------------------------------------------------------- if [ -s sstgrb -a -s sstgrb.index ];then err3=0 err4=0 else set +x echo echo "VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV" echo " Get sst grib file valid at center dump time" echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" echo set -x /nwprod/util/ush/getges.sh -t sstgrb -v $cendat sstgrb err3=$? set +x echo echo "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" echo set -x err4=99 if [ "$err3" -eq '0' ];then rm errfile $EXGRBIX/grbindex sstgrb sstgrb.index 2> errfile err4=$? [ "$err4" -ne '0' ] && cat errfile rm errfile fi fi toterr=`expr $err1 + $err2 + $err3 + $err4` if [ "$toterr" -eq '0' ];then # run dcodcloc (colocation) # ------------------------- cat << EOFdat2 > datefile $cendat EOFdat2 typeset -Z2 dumhr dumhr=`expr $cendat % 100` echo $dumhr > coloc.parm cat datefile cat coloc.parm export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u XLFUNIT_11=datefile XLFUNIT_12=$ersIN XLFUNIT_13=pgrb XLFUNIT_14=pgrb.index XLFUNIT_15=sstgrb XLFUNIT_16=sstgrb.index XLFUNIT_19=$FIXWAVE/wave_zmask1x1 XLFUNIT_52=ers2dat.in timex $DCLX outout 2>errfile errers=$? cat errfile >>outout rm errfile cat outout >> ersproc.out cat outout >> $pgmout_this rm outout set +x echo echo "The foreground exit status for WAVE_DCODCLOC is " $errers echo set -x if [ "$errers" -ne '0' ];then msg1="**NON-FATAL ERROR PROGRAM WAVE_DCODCLOC RETURN CODE $errers" else set +x echo echo "------------------------------------------------------------" echo "************ COMPLETED PROGRAM wave_dcodcloc ************" echo "------------------------------------------------------------" echo set -x # run dataqc (quality control and reprocessing of ERS data) # --------------------------------------------------------- export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u XLFUNIT_11=ers2dat.in XLFUNIT_51=ers2.94 XLFUNIT_90=datefile XLFUNIT_94=ers2.9998 XLFUNIT_95=iang8 XLFUNIT_96=err8x timex $DQCX >outout 2> errfile errers=$? cat errfile >>outout rm errfile cat outout >> ersproc.out cat outout >> $pgmout_this rm outout set +x echo echo "The foreground exit status for WAVE_DATAQC is " $errers echo set -x if [ "$errers" -ne '0' ];then msg1="**NON-FATAL ERROR PROGRAM WAVE_DATAQC RETURN CODE $errers" else set +x echo echo "----------------------------------------------------------" echo "************ COMPLETED PROGRAM wave_dataqc ************" echo "----------------------------------------------------------" echo set -x # run datasort (time sorting, minimization and ambiguity removal) # --------------------------------------------------------------- export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u XLFUNIT_12=ers2.94 XLFUNIT_20=$DSRT XLFUNIT_30=$ersIN XLFUNIT_40=$FIXWAVE/wave_CMOD_DBLUT XLFUNIT_42=$FIXWAVE/wave_CMOD_QSLUT XLFUNIT_60=erscat_bufr XLFUNIT_91=ers2.so XLFUNIT_92=ers2.in XLFUNIT_93=ers2.94er timex $DSRX >outout 2> errfile errers=$? cat errfile >>outout rm errfile cat outout >> ersproc.out cat outout >> $pgmout_this set +x echo echo "The foreground exit status for WAVE_DATASORT is " $errers echo set -x if [ "$errers" -ne '0' ];then msg1="**NON-FATAL ERROR PROGRAM WAVE_DATASORT RETURN CODE $errers" else grep -e ">>> TOTAL NUMBER OF REPROCESSED ERS WIND \ REPORTS WRITTEN ..." outout > grep.count.ers errgrep=$? if [ "$errgrep" -eq '0' ];then echo " " >> status1.out cut -c2- grep.count.ers | tr -d '<<<' >> status1.out fi rm grep.count.ers set +x echo echo "------------------------------------------------------------" echo "************ COMPLETED PROGRAM wave_datasort ************" echo "------------------------------------------------------------" echo set -x msg1="ERS programs WAVE_DCODCLOC, WAVE_DATAQC & \ WAVE_DATASORT all completed normally" ########################cp ers2.so ${COMSP}ers2.sort cp erscat_bufr $ersOU errcpy=$? if [ "$errcpy" -eq '0' ];then msg2="ERS scatterometer data reprocessed successfully into NCEP BUFR file" else msg2="**NO ERS SCATTEROMETER DATA reprocessed \ into NCEP BUFR file; copy to /com failed --> non-fatal" fi fi rm outout rm erscat_bufr ers2.so ers2.94er ers2.in fi rm ers2.94 ers2.9998 iang8 err8x fi rm erswnd_bufr datefile ers2dat.in coloc.parm else msg2="**NO ERS SCATTEROMETER DATA reprocessed; grib files \ not found --> non-fatal" fi rm pgrb pgrb.index else msg2="**NO ERS SCATTEROMETER DATA reprocessed; erswnd dump file \ not found --> non-fatal" fi set +u if [ -n "$jlogfile" ];then [ "$msg1" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg1" $DATA/postmsg "$jlogfile" "$msg2" fi set -u set +x echo echo "************************************************************************" [ "$msg1" != 'NO' ] && echo "$msg1" echo "$msg2" echo "************************************************************************" echo set -x for OUT in ersproc.out $pgmout_this status1.out do echo " " >> $OUT [ "$msg1" != 'NO' ] && echo "$msg1" >> $OUT echo "$msg2" >> $OUT echo " " >> $OUT done echo "*******************************************************************\ ************" >> status1.out fi break fi done rm sstgrb sstgrb.index ############################################################################# ############################################################################# # EXECUTE QUIKSCAT SCATTEROMETER REPROCESSING (IF REQUESTED) ############################################################################# ############################################################################# for nnn do if [ $nnn = qkscat ];then msg1=NO msg2="**NO QUIKSCAT SCATTEROMETER DATA reprocessed (from input \ qkscat dump) due to above PGM FAIL --> non-fatal" [ "$SENDCOM" != 'NO' ] && cp /dev/null ${COMSP}qkswnd.${tmmark}.bufr_d if [ "$prepqksd" = 'YES' ];then echo "**************************************************************\ *****************" >> status1.out echo " " >> status1.out echo "PROGRAM wave_dcodquikscat will run from input qkscat dump" \ >> status1.out if [ -s ${COMSP}qkscat.${tmmark}.bufr_d ];then # set-up variables and file paths # ------------------------------- cd $DATA/job${DUMP_NUMBER} cp ${COMSP}qkscat.${tmmark}.bufr_d qkscat_bufr qksIN=qkscat_bufr qksOU=${COMSP}qkswnd.${tmmark}.bufr_d #============================================ EXECWAVE=${EXECWAVE:-/nw${envir}/exec} FIXWAVE=${FIXWAVE:-/nw${envir}/fix} PARMWAVE=${PARMWAVE:-/nw${envir}/parm} DQKT=${DQKT:-$FIXWAVE/wave_bufrtab.quikscat} DQKX=${DQKX:-$EXECWAVE/wave_dcodquikscat} DQKC=${DQKC:-$PARMWAVE/wave_dcodquikscat.${NET}.parm} LANDC_DQK=${LANDC_DQK:-$FIXWAVE/wave_landchxh} set +x echo echo "EXECWAVE = " $EXECWAVE echo echo "FIXWAVE = " $FIXWAVE echo echo "PARMWAVE = " $PARMWAVE echo set -x #============================================ # run dcodquikscat (internal date checking, overland check, wind # vector selection, q.c. checks maybe, superobing maybe, reprocess # into BUFR file for PREPOBS_PREPDATA program) # ----------------------------------------------------------------- if [ -s $DQKC ]; then cp $DQKC dcodquikscat.parm else cat < dcodquikscat.parm &RDATA IPRINT = 0, ! Toggle (0=off,1=on) for text listing of all reprocessed ! reports in unit 51 ISUPOB = 1, ! Superob the reports DELAT = 0.5, ! - latitude spacing (degrees) of superob grid box DELON = 0.5, ! - longitude spacing (degrees) of superob grid box LIMCNT = 2, ! - minimum number of reports per superob box required to ! make superob IQCPOR = 1, ! Perform probability of rain q.c. PORLIM = 0.10, ! - all reports with p.o.r. .gt. 10% are tossed IQCEDG = 1, ! Perform orbital swath edge q.c. IEDLLM = 8, ! - all reports with cell # .le. 8 are tossed IEDULM = 64, ! - all reports with cell # .ge. 64 are tossed LATS =-90, ! Southernmost lat boundary (deg) for accepting rpts (N+; S-) LATN =+90, ! Northernmost lat boundary (deg) for accepting rpts (N+; S-) LONW =360, ! Westernmost lon boundary (deg) for accepting rpts (0-360W) LONE = 0 ! Easternmost lon boundary (deg) for accepting rpts (0-360W) / EOFd fi export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u XLFUNIT_11=$qksIN XLFUNIT_19=$LANDC_DQK XLFUNIT_20=$DQKT XLFUNIT_51=qkswnd_listing XLFUNIT_52=qkswnd_bufr timex $DQKX >outout < dcodquikscat.parm 2> errfile errqks=$? cat errfile >>outout rm errfile cat outout >> dcodquikscat.out cat outout >> $pgmout_this set +x echo echo "The foreground exit status for WAVE_DCODQUIKSCAT is " $errqks echo set -x if [ "$errqks" -ne '0' ];then msg1="**NON-FATAL ERROR PROGRAM WAVE_DCODQUIKSCAT \ (making qkswnd file) RETURN CODE $errqks" else grep -e ">>> TOTAL NUMBER OF REPROCESSED QUIKSCAT WIND \ REPORTS WRITTEN ..." outout > $DATA/grep_quikd.count errgrep=$? if [ "$errgrep" -eq '0' ];then echo " " >> status1.out cut -c2- $DATA/grep_quikd.count | tr -d '<<<' >> status1.out fi rm $DATA/grep_quikd.count set +x echo echo "------------------------------------------------------------" echo "********** COMPLETED PROGRAM wave_dcodquikscat ***********" echo "------------------------------------------------------------" echo set -x msg1="program WAVE_DCODQUIKSCAT completed normally, qkswnd \ file generated from input qkscat dump" cp qkswnd_bufr $qksOU errcpy=$? msg2=NO [ "$errcpy" -ne '0' ] && msg2="**NO QUIKSCAT SCATTEROMETER \ DATA reprocessed into NCEP BUFR file (qkswnd); copy to /com failed --> \ non-fatal" fi rm qkscat_bufr qkswnd_bufr outout else msg2="**NO QUIKSCAT SCATTEROMETER DATA reprocessed; qkscat dump \ file not found --> non-fatal" fi set +u if [ -n "$jlogfile" ];then [ "$msg1" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg1" [ "$msg2" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg2" fi set -u set +x echo echo "************************************************************************" [ "$msg1" != 'NO' ] && echo "$msg1" [ "$msg2" != 'NO' ] && echo "$msg2" echo "************************************************************************" echo set -x for OUT in dcodquikscat.out $pgmout_this status1.out do echo " " >> $OUT if [ "$msg1" != 'NO' ];then if [ "$OUT" = 'status1.out' ];then echo $msg1 | grep "generated" errgrep=$? if [ "$errgrep" -eq '0' ]; then echo "$msg1" | cut -c1-67 >> $OUT else echo "$msg1" >> $OUT fi else echo "$msg1" >> $OUT fi fi [ "$msg2" != 'NO' ] && echo "$msg2" >> $OUT echo " " >> $OUT done echo "*******************************************************************\ ************" >> status1.out fi break fi done ############################################################################# ############################################################################# # EXECUTE TRMM TMI DATA REPROCESSING (IF REQUESTED) ############################################################################# ############################################################################# for nnn do if [ "$nnn" = 'trmm' ];then msg1=NO msg2="**NO TRMM TMI DATA reprocessed (from input trmm dump) due to \ above PGM FAIL --> non-fatal" [ "$SENDCOM" != 'NO' ] && cp /dev/null ${COMSP}sptrmm.${tmmark}.bufr_d if [ "$preptrmm" = 'YES' ];then echo "**************************************************************\ *****************" >> status1.out echo " " >> status1.out echo "PROGRAM bufr_supertmi will run from input trmm dump" >> \ status1.out if [ -s ${COMSP}trmm.${tmmark}.bufr_d ];then # set-up variables and file paths # ------------------------------- cd $DATA/job${DUMP_NUMBER} cp ${COMSP}trmm.${tmmark}.bufr_d trmm_bufr tmiIN=trmm_bufr tmiOU=${COMSP}sptrmm.${tmmark}.bufr_d #============================================ EXECbufr=${EXECbufr:-/nw${envir}/exec} FIXbufr=${FIXbufr:-/nw${envir}/fix} PARMbufr=${PARMbufr:-/nw${envir}/parm} DTMT=${DTMT:-$FIXbufr/bufr_bufrtab.sptrmm} DTMX=${DTMX:-$EXECbufr/bufr_supertmi} DTMC=${DTMC:-$PARMbufr/bufr_supertmi.${NET}.parm} set +x echo echo "EXECbufr = " $EXECbufr echo echo "FIXbufr = " $FIXbufr echo echo "PARMbufr = " $PARMbufr echo set -x #============================================ # run supertmi to superob TRMM TMI data and write reprocessed data # into BUFR file for subsequent SSI analysis program) # ----------------------------------------------------------------- if [ -s $DTMC ]; then cp $DTMC supertmi.parm else cat < supertmi.parm &INPUT IPRINT = 0, ! Toggle (0=off,1=on) for text listing of all reprocessed ! reports in unit 51 DELAT = 1.0, ! Latitude spacing (degrees) of superob grid box DELON = 1.0, ! Longitude spacing (degrees) of superob grid box LIMCNT = 6, ! Minimum number of reports per superob box required to make ! superob LATS =-90, ! Southernmost lat boundary (deg) for accepting rpts (N+; S-) LATN =+90, ! Northernmost lat boundary (deg) for accepting rpts (N+; S-) LONW =360, ! Westernmost lon boundary (deg) for accepting rpts (0-360W) LONE = 0 ! Easternmost lon boundary (deg) for accepting rpts (0-360W) / EOFd fi export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u XLFUNIT_11=$tmiIN XLFUNIT_20=$DTMT XLFUNIT_51=sptrmm_listing XLFUNIT_52=sptrmm_bufr timex $DTMX >outout < supertmi.parm 2> errfile errtmi=$? cat errfile >>outout rm errfile cat outout >> supertmi.out cat outout >> $pgmout_this set +x echo echo "The foreground exit status for BUFR_SUPERTMI is " $errtmi echo set -x if [ "$errtmi" -ne '0' ];then msg1="**NON-FATAL ERROR PROGRAM BUFR_SUPERTMI (making \ sptrmm file) RETURN CODE $errtmi" else grep -e ">>> TOTAL NUMBER OF REPROCESSED (SUPEROBED) \ TRMM TMI REPORTS WRITTEN .." outout > grep.count.sptrmm errgrep=$? if [ "$errgrep" -eq '0' ];then echo " " >> status1.out cut -c2- grep.count.sptrmm | tr -d '<<<' >> status1.out fi rm grep.count.sptrmm set +x echo echo "------------------------------------------------------------" echo "************ COMPLETED PROGRAM bufr_supertmi *************" echo "------------------------------------------------------------" echo set -x msg1="program BUFR_SUPERTMI completed normally, sptrmm file \ generated from input trmm dump" cp sptrmm_bufr $tmiOU errcpy=$? msg2=NO [ "$errcpy" -ne '0' ] && msg2="**NO TRMM TMI DATA reprocessed \ into NCEP BUFR file (sptrmm); copy to /com failed --> non-fatal" fi rm trmm_bufr sptrmm_bufr outout else msg2="**NO TRMM TMI DATA reprocessed; trmm dump file not found \ --> non-fatal" fi set +u if [ -n "$jlogfile" ];then [ "$msg1" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg1" [ "$msg2" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg2" fi set -u set +x echo echo "************************************************************************" [ "$msg1" != 'NO' ] && echo "$msg1" [ "$msg2" != 'NO' ] && echo "$msg2" echo "************************************************************************" echo set -x for OUT in supertmi.out $pgmout_this status1.out do echo " " >> $OUT if [ "$msg1" != 'NO' ];then if [ "$OUT" = 'status1.out' ];then echo $msg1 | grep "generated" errgrep=$? if [ "$errgrep" -eq '0' ]; then echo "$msg1" | cut -c1-64 >> $OUT else echo "$msg1" >> $OUT fi else echo "$msg1" >> $OUT fi fi [ "$msg2" != 'NO' ] && echo "$msg2" >> $OUT echo " " >> $OUT done echo "*******************************************************************\ ************" >> status1.out fi break fi done ############################################################################# ############################################################################# # EXECUTE WINDSAT SCATTEROMETER REPROCESSING (IF REQUESTED) ############################################################################# ############################################################################# for nnn do if [ $nnn = wndsat ];then msg1=NO msg2="**NO WINDSAT SCATTEROMETER DATA reprocessed (from input wndsat \ dump) due to above PGM FAIL --> non-fatal" [ "$SENDCOM" != 'NO' ] && cp /dev/null ${COMSP}wdsatr.${tmmark}.bufr_d if [ "$prepwindsat" = 'YES' ];then echo "**************************************************************\ *****************" >> status1.out echo " " >> status1.out echo "PROGRAM bufr_dcodwindsat will run from input wndsat dump" \ >> status1.out if [ -s ${COMSP}wndsat.${tmmark}.bufr_d ];then # set-up variables and file paths # ------------------------------- cd $DATA/job${DUMP_NUMBER} cp ${COMSP}wndsat.${tmmark}.bufr_d wndsat_bufr wnsIN=wndsat_bufr wnsOU=${COMSP}wdsatr.${tmmark}.bufr_d #============================================ EXECbufr=${EXECbufr:-/nw${envir}/exec} FIXbufr=${FIXbufr:-/nw${envir}/fix} PARMbufr=${PARMbufr:-/nw${envir}/parm} DWST=${DWST:-$FIXbufr/bufr_bufrtab.windsat} DWSX=${DWSX:-$EXECbufr/bufr_dcodwindsat} DWSC=${DWSC:-$PARMbufr/bufr_dcodwindsat.${NET}.parm} LANDC_DWS=${LANDC_DWS:-$FIXbufr/wave_landchxh} set +x echo echo "EXECbufr = " $EXECbufr echo echo "FIXbufr = " $FIXbufr echo echo "PARMbufr = " $PARMbufr echo set -x #============================================ # run dcodwindsat (internal date checking, overland check, wind # vector selection, superobing maybe, reprocess into BUFR file # for PREPOBS_PREPDATA program) # ---------------------------------------------------------------- if [ -s $DWSC ]; then cp $DWSC dcodwindsat.parm else cat < dcodwindsat.parm &RDATA IPRINT = 0, ! Toggle (0=off,1=on) for text listing of all reprocessed ! reports in unit 51 ISUPOB = 1, ! Toggle (0=off,1=on) superobing of reports DELAT = 1.0, ! - latitude spacing (deg) of superob grid box (ISUPOB=1) DELON = 1.0, ! - longitude spacing (deg) of superob grid box (ISUPOB=1) LIMCNT = 1, ! - minimum number of reports per superob box required to ! make superob LATS =-90, ! Southernmost lat boundary (deg) for accepting rpts (N+; S-) LATN =+90, ! Northernmost lat boundary (deg) for accepting rpts (N+; S-) LONW =360, ! Westernmost lon boundary (deg) for accepting rpts (0-360W) LONE = 0 ! Easternmost lon boundary (deg) for accepting rpts (0-360W) / EOFd fi export XLFUNITS=0 unset `env | grep XLFUNIT | awk -F= '{print $1}'` set +u if [ -z "$XLFRTEOPTS" ];then export XLFRTEOPTS="unit_vars=yes" else export XLFRTEOPTS="${XLFRTEOPTS}:unit_vars=yes" fi set -u XLFUNIT_11=$wnsIN XLFUNIT_19=$LANDC_DWS XLFUNIT_20=$DWST XLFUNIT_51=wdsatr_listing XLFUNIT_52=wdsatr_bufr timex $DWSX >outout < dcodwindsat.parm 2> errfile errwns=$? cat errfile >>outout rm errfile cat outout >> dcodwindsat.out cat outout >> $pgmout_this set +x echo echo "The foreground exit status for BUFR_DCODWINDSAT is " $errwns echo set -x if [ "$errwns" -ne '0' ];then msg1="**NON-FATAL ERROR PROGRAM BUFR_DCODWINDSAT (making \ wdsatr file) RETURN CODE $errwns" else grep -e " >>> TOTAL NUMBER OF REPROCESSED WINDSAT REPORTS \ WRITTEN .." outout > $DATA/grep_wsatd.count errgrep=$? if [ "$errgrep" -eq '0' ];then echo " " >> status1.out cut -c2- $DATA/grep_wsatd.count | tr -d '<<<' >> status1.out fi rm $DATA/grep_wsatd.count set +x echo echo "------------------------------------------------------------" echo "********** COMPLETED PROGRAM bufr_dcodwindsat ***********" echo "------------------------------------------------------------" echo set -x msg1="program BUFR_DCODWINDSAT completed normally, wdsatr file \ generated from input wndsat dump" cp wdsatr_bufr $wnsOU errcpy=$? msg2=NO [ "$errcpy" -ne '0' ] && msg2="**NO WINDSAT SCATTEROMETER \ DATA reprocessed into NCEP BUFR file (wdsatr); copy to /com failed --> \ non-fatal" fi rm wndsat_bufr wdsatr_bufr outout else msg2="**NO WINDSAT SCATTEROMETER DATA reprocessed; wndsat dump \ file not found --> non-fatal" fi set +u if [ -n "$jlogfile" ];then [ "$msg1" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg1" [ "$msg2" != 'NO' ] && $DATA/postmsg "$jlogfile" "$msg2" fi set -u set +x echo echo "************************************************************************" [ "$msg1" != 'NO' ] && echo "$msg1" [ "$msg2" != 'NO' ] && echo "$msg2" echo "************************************************************************" echo set -x for OUT in dcodwindsat.out $pgmout_this status1.out do echo " " >> $OUT if [ "$msg1" != 'NO' ];then if [ "$OUT" = 'status1.out' ];then echo $msg1 | grep "generated" errgrep=$? if [ "$errgrep" -eq '0' ]; then echo "$msg1" | cut -c1-67 >> $OUT else echo "$msg1" >> $OUT fi else echo "$msg1" >> $OUT fi fi [ "$msg2" != 'NO' ] && echo "$msg2" >> $OUT echo " " >> $OUT done echo "*******************************************************************\ ************" >> status1.out fi break fi done ############################################################################# ############################################################################# # GENERATE THE STATUS FILE (IF REQUESTED) ############################################################################# ############################################################################# if [ "$STATUS" = 'YES' ];then cd $DATA [ -f status1.out ] && rm status1.out # Combine outputs from previous (and possibly simultaneous) runs of this # script for n in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 \ 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 do [ ! -d $DATA/job${n} ] && break [ -f $DATA/job${n}/status1.out ] && cat $DATA/job${n}/status1.out \ >> status1.out [ -f $DATA/job${n}/allout ] && cat $DATA/job${n}/allout >> $pgmout done # Generate status file to alert that all dumps for this time have been # produced - also summarizes substype report counts and data group return # code information cat <<\EOFblank > blank EOFblank WALLCLOK=`date -u` if [ "$job" = 'j????' ];then run="???" elif [[ $job = ndas_dump_tm??_?? ]];then run=ndas elif [[ $job = ruc2a_dump_?? ]];then run=ruc2a else run=$NET fi if [ "$run" != '???' ]; then run_uc=$(echo $run | tr [a-z] [A-Z]) else run_uc="???" fi tmmark_uc=$(echo $tmmark | tr [a-z] [A-Z]) cycle_uc=$(echo $cycle | tr [a-z] [A-Z] | cut -c2-) cat < status.out #### INFORMATION REGARDING ${tmmark_uc} OBSERVATIONAL DATA DUMP FOR ${cycle_uc} ${run_uc} RUN #### PARENT SCRIPT IS: $job CENTRAL DATE IS $cendat THIS STATUS FILE CREATED $WALLCLOK THIS CREATION TIME MUST BE EARLIER THAN THE START TIME FOR ANY OPERATIONAL ANALYSES JOBS THAT READ FROM THESE OBSERVATIONAL DATA DUMP FILES (i.e., Jobs NAM_PREP, NDAS_PREP, RUC2A_PREP, GFS_PREP, GDAS_PREP) ******************************************************************************* ++++ NUMBER OF REPORTS DUMPED BY DATA SUBTYPE ++++ TIME WINDOW RADIUS IS DATA GROUP DEPENDENT GEOGRAPHICAL DOMAIN IS DATA GROUP DEPENDENT EOFs1 > updated_counts.out grep -e "Dumping [0-2]" -e "Missing [0-2]" $pgmout | cut -f2- -d" " \ > cutLv.allout grep " HAS " $pgmout | cut -f2- -d" " > cutRv.allout paste -d"\0\n" cutLv.allout cutRv.allout > pasteB.allout cut -c1-64 pasteB.allout > cutLv.allout cut -c67- pasteB.allout > cutRv.allout rm pasteB.allout paste -d"\0" cutLv.allout cutRv.allout > paste.part1 grep -e "Domain for [0-2]" $pgmout > paste.part2 paste -d"\n\n" paste.part1 paste.part2 blank > counts.out nindx=`cat temp1 grep -e "in data group" temp1 | grep -v -e "Domain" err_grep=$? if [ $err_grep -ne 0 ]; then cat temp1 >> updated_counts.out else group=`cut -f5 -d" " temp1` if [ -s ${COMSP}${group}.${tmmark}.bufr_d ]; then cat temp1 >> updated_counts.out else sed "s/HAS....... REPORTS/HAS 0 REPORTS/g" temp1 > temp2 cat temp2 >> updated_counts.out fi fi done cat updated_counts.out >> status.out rm cutLv.allout cutRv.allout paste.part1 paste.part2 blank \ temp1 temp2 counts.out updated_counts.out cat <<\EOFs2 >> status.out ############################################################################### EOFs2 grep -q -e " SAT. ID " -e " %#" $pgmout errgrep=$? if [ $errgrep -eq 0 ]; then cat <<\EOFs3 >> status.out ******************************************************************************* +++ NUMBER OF REPORTS DUMPED BY SATELLITE ID IN EACH SATELLITE DATA SUBTYPE +++ EOFs3 grep -e " SAT. ID " -e " %#" $pgmout | sed "s/%#/ /g" \ >> status.out fi grep -q -e " replicated observations" -e " #%" $pgmout errgrep=$? if [ $errgrep -eq 0 ]; then cat <<\EOFs4 >> status.out ******************************************************************************* +++ TOTAL NUMBER OF REPLICATED OBS IN EACH NEXRAD LEVEL 2 DATA SUBTYPE +++ EOFs4 grep -e " replicated observations" -e " #%" $pgmout | \ sed "s/#%/ /g" >> status.out fi cat <<\EOFs5 >> status.out %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ++++ SPECIFIC INFORMATION ON EACH DATA GROUP DUMP ++++ EOFs5 cat status1.out >> status.out rm status1.out cat status.out > ${COMSP}status${JOB_NUMBER}.${tmmark}.bufr_d errmvt=$? errmvl=$errmv errmv=`expr $errmvl + $errmvt` fi if [ -s ${COMSP}status${JOB_NUMBER}.${tmmark}.bufr_d -a \ -f ${COMSP}aircar.${tmmark}.bufr_d ]; then # Compare AFWA ACARS vs. ARINC ACARS report counts in AIRCAR dump and generate # a flag file (*aircar_status_flag*) in $COMSP path indicating which type # should be processed from the AIRCAR dump in PREPBUFR; this flag file will # later be read by the script prepobs_makeprepbufr.sh - normally there are # more ARINC ACARS (primary) and ONLY this type will be processed from AIRCAR # dump; on rare occasions the ARINC feed may go down and there are more AFWA # ACARS (backup) and then ONLY this type will be processed from AIRCAR dump count_004004=`grep -Fe "004.004" \ ${COMSP}status${JOB_NUMBER}.${tmmark}.bufr_d | awk -F" HAS" '{print$2}' | \ awk -F" REPORTS" '{print$1}'` set +u [ -z "$count_004004" ] && count_004004=0 set -u count_004007=`grep -Fe "004.007" \ ${COMSP}status${JOB_NUMBER}.${tmmark}.bufr_d | awk -F" HAS" '{print$2}' | \ awk -F" REPORTS" '{print$1}'` set +u [ -z "$count_004007" ] && count_004007=0 set -u if [ $count_004004 -ge $count_004007 ]; then msgp="" msg="process only message type 004.004 for AIRCAR data in PREPBUFR" else msgp="***WARNING: ARINC ACARS dump count ($count_004004) < AFWA ACARS \ dump count ($count_004007)" set +u [ -n "$jlogfile" ] && $DATA/postmsg "$jlogfile" "$msgp" set -u msg="process only message type 004.007 for AIRCAR data in PREPBUFR" fi echo $msg > ${COMSP}aircar_status_flag.${tmmark}.bufr_d set +x echo echo $msgp echo $msg echo set -x fi errtot=`expr $errdmp + $errmv` if [ "$errtot" -gt 0 ];then exit $errtot elif [ "$errsmi" -gt 0 ];then errsmi=`expr $errsmi + 20` exit $errsmi elif [ "$errtot" -eq 0 ];then exit fi