#!@DASPERL # # ANALYZER # # Main driver for the fvSSI analysis # # !TODO: # # 1) Need to unwire filenames for "previous day" when running in asynoptic mode # 2) remove iss argument from calling sequence # # !REVISION HISTORY: # # 30Apr2001 da Silva Inspirational fvcqc code. # 06Jun2002 Cruz New implementaion to run Ncep's ssi. # 18Nov2002 Cruz Modified to run under FVSSI # 15Jul2003 Cruz Added new options # 01Aug2003 Todling - Generalized location of perl. # - Added args bkg/sbkf; changed # fv2ss and ss2fv accordingly. # 04Aug2003 Todling - Removed -rs opt from 2nd call to ss2fv # - Added log opt # - Added no_recycle opt # 05Aug2003 Todling - Making SSI run in its own private dir # 05Aug2003 Owens/RT - Added call to obs files handler # 07Aug2003 Todling - Added zeit calls to System # 15Aug2003 Todling - Bug fix: exist chk should be on satbias_in w/o expid # 06Nov2003 Todling - Fixed log when running DAS # - Fixed zeit output location # - Only certain system cmds should use System # 17Nov2003 Todling - Added SAC capability # - Renamed satbias file permanently stored # - Removed reference to satbias_angle from fvInput # 20Nov2003 Todling Modification to accomodate GSI # 21Nov2003 Todling - Added logics to take guess field # - Removed option -res (resolution is automatic) # - phis static removed (using bkg field instead) # 12Dec2003 Todling Fixed ssi so it cats sat file after completion. # 18Dec2003 Todling Calling fv2ss.x w/ -ncep_phis (needs solution). # 15Jan2004 Todling Bug fix; accidentally removed eta_out.hdf from 1st ss2fv.x # 03Feb2004 Todling Fixing pending issue w/ running sac.x # 05Feb2004 Todling Implemented capability to handle asynoptic bkg's # 03Mar2004 Todling Either skip all transf or do them all; also added/fixed # flexibility to run analysis solver only. # 25Mar2004 Kokron pabove/pbelow had reversed defaults. # 29Mar2004 Todling redefined gesfile name. # 30Mar2004 Todling changed for ncep-gsi-1_1: # - added diag_conv for omf's of conv data # - renamed bkg error file(s) # 13Apr2004 Todling Added no_asyn flag to allow skipping asyn calc when bkg # file's less than a 6 hr files. # 15Apr2004 Todling - Added "replay" option; works from spectral files only for now. # - Removed "no_recycle" option. # 06May2004 Treadon - Added global_ozinfo.txt to list of static files. SSI does not # need global_ozinfo.txt, but GSI does # - Expanded list (@satlist) of satellites for radiance diagostfic files # - Included scripting to handle ozone dianostic file (GSI only) # 13May2004 Todling - Regroupped etc dir to look more like NCEP's arrangement # 19Oct2004 WGu/RT - Bug fix: sfcfXX files not provided correctly in ASYN case # 19Oct2004 Todling - Added OIQC capability # 09Nov2004 WGu/RT - Bug fix: satbias/ang initialization was not doing what was meant to do # 24Nov2004 Kokron - Fix to allow tranform only case to run; Only call rep if replay mode # Fix to prevent hangs when instrument diag files are not present # Correct name of prep_bufr file # 13Dec2004 Todling Adding support for surface analysis # 24Feb2005 Todling - Updated fixed files location for latest (current) version of NCEP's GSI # - Updated names of diag_ files from GSI (how about backward compatibility!?) # - Fix related to possibly empty sfcanl file # - Updated stats file to be latest from NCEP # 08Mar2005 Todling Added support for T382L64 analysis # 10Mar2005 Owens Bug fix in lnlist call: strict n/a # 12Apr2005 ELiu/RT Added GMAO buffer table for AIRS # 18Apr2005 Todling Some fixes in preparation for analyzing on model's levels # 20Apr2005 Todling Updated berror to be compatible w/ version ncep-gsi-2005_04 # 02May2005 Todling - Taking satinfo files from fvwork(run) dir; turned on AMSUA(c9/c14) on NOAA-16 # - gsi.rc updated to allow fvsetup to decide on hybrid opt. # 17May2005 Todling Temporarily turned off sat bias angle correction program due to existing bug # 13Jun2005 Todling In antecipation to 1/4 degree analysis reduced output to 32 bits (see ss2fv.x) # 27Jun2005 Todling Renamed files siganl and sfcanl to anl.sig and anl.sfc respectively # 30Jun2005 Todling New filename for berror file (r4 files now) # 28Jul2005 Todling Worked in changes to accommodate new GMAO interface to GSI # 03Aug2005 Todling Skipping fv2ss.x call when doSPC (spectral) not desired # 11Aug2005 Todling Using E. Liu's satangbias file w/ 150 AIRS channels "tuned" # 11Aug2005 Treadon Add gmao_global_convinfo.txt to list of static files # 28Sep2005 Jing Guo . Added option --berror=%s to specify the file # of background error statistics for input. # . Changed the algorithm for the selection of the # default background error statistics file. # . Updated usage() contents. # 05Oct2005 Jing Guo . Fixed HGRD, designed for NLAT not for NLON. # 05Oct2005 Todling Temporarily removed ref to berror native to fall in previous default # 05Oct2005 Stassi Added surface analysis # 19Oct2005 Todling Merged w/ gAdas-1_5beta3p9 version # 11Jan2006 Todling Added errtable to allow bypass of obs-errors in conv-buffer file # 02Feb2006 Todling Redefined namnig gmao convention for berror files # 01Mar2006 Treadon Add CRTM fix files, update satlist and ozlist strings # #----------------------------------------------------------------------------------------------------- use Env; # make env vars readily available use File::Basename; # for basename(), dirname() use Getopt::Long; # load module with GetOptions function use File::Copy "cp"; # for cp() use Time::Local; # time functions use Shell qw(cat rm); # cat and rm commands # Command line options GetOptions ( "d=s" => \$fvInput, "prepqc=s" => \$prepqc, "expid=s" => \$expid, "iss=s" => \$ss_in, "oss=s" => \$ss_out, "ua=i" => \$upa, "lwi=s" => \$lwi_nm, "ts=s" => \$ts_nm, "obs=i" => \$obs, "rc=s" => \$rcname, "t=s" => \$jcap, "y=s" => \$hgrd, "levs=s" => \$nsig, "pa=f" => \$pabove, "pb=f" => \$pbelow, "bkg=s" => \$fvbkg, "sbkg=s" => \$fvsbkg, "ssbkg=s" => \$ssbkg, "sssbkg=s" => \$sssbkg, "log=s" => \$log, "berror=s" => \$berror, "replay", "debug", "no_asyn", "skipSOLVER", "skipTRANSF", "skipSATBIAS", "strict", "h"); usage() if $opt_h; # Parse command line, etc init(); # Run Vortex Relocator vtxrlc ( $nymd, $nhms ) if ( $doANA ); # Run preparatory transforms if ( $doTRANSF ) { # Run fv2ss foreach $sbground (@atbkg) { ($asynymd, $asynhms) = ( tick($asynymd,$asynhms,$dtasyn*3600) ); $asynhh = substr($asynhms,0,2); # $fvbkg = `echorc.x -template $expid $asynymd $asynhms -rc $rcname upper-air_bkg_filename`; chomp($fvbkg); # $fvsbkg = `echorc.x -template $expid $asynymd $asynhms -rc $rcname surface_bkg_filename`; chomp($fvsbkg); $fvbkg = "$expid.bkg.eta.${asynymd}_${asynhh}z.hdf"; $fvsbkg = "$expid.bkg.sfc.${asynymd}_${asynhh}z.hdf"; fv2ss($fvbkg,$fvsbkg,"$asynymd","$asynhms","$dtasyn","$sbground"); } # Run ss2fv (no data) ss2fv("$nymd","$nhms") if ( $doSPC ); } else { Assignfn ( "$fvwork/$ssbkg", "sigf06" ); Assignfn ( "$fvwork/$sssbkg", "sfcf06" ); } # Run Setobs setobs ( $nymd, $hh ) if ( $doANA ); # Run OIQC oiqc ( $nymd, $hh, "$anadir/sigf06" ) if ( $doANA && $doTRANSF ); # Run SSI if ( $doANA ) { if ( $doSPINBIAS ) { ana(); # Spin-up Satellite Bias Correction sac(); # Spin-up Satellite Angular Correction } ana(); # Run SSI sac(); # Run Satellite Angular Correction } elsif ($replay) { rep(); exit($rc); } # Run ss2fv ss2fv("$nymd","$nhms") if ( $doTRANSF && $doSPC ); # Convert diag files to ODS d2ods() if ( $doANA && $doODS ); # Run ana5sfc sfc_ana ($nymd, $nhms) if ( $doANA && $doSFCANA ); # All done if ($rc==0) { print "$0: sucessfully completed.\n\n"; } else { print "$0: trouble.\n\n"; } exit(0); #...................................................................... sub init { if ( $#ARGV < 1 ) { print STDERR " Missing nymd nhms ; see usage:\n"; usage(); } else { # required command line args $nymd = $ARGV[0]; $nhms = sprintf("%6.6d",$ARGV[1]); $yyyy = substr($nymd,0,4); $mm = substr($nymd,4,2); $dd = substr($nymd,6,2); $hh = substr($nhms,0,2); } # process options $rc = 0; # FVROOT is where the binaries have been installed # CAUTION: to be used only as a means of identifying gsi/ssi appls # ---------------------------------------------------------------- $fvroot = $ENV{FVROOT}; $doGSI = 1 if ( -e "$fvroot/bin/gsi.x" ); if ( $doGSI ) { $myexec = "gsi.x"; $myetc = "gsi/etc"; $myrc = "gsi.rc"; $ananm = "gsi"; } else { $myexec = "ssi.x"; $myetc = "ssi/etc"; $myrc = "ssi.rc"; $ananm = "ssi"; } $dyn_a = "dyn.ana" unless $dyn_a; $fvInput = "/share/fvdas/fvInput/Static" unless $fvInput; $doOIQC = $ENV{OIQC}; $doVTXRLC = 1; # $ENV{VTXRELOC}; $doSFCANA = 1; $doANA = 0; $doTRANSF = 0; $doODS = 0; $doSPC = 0; # controls spectral transf (eventually, same as doTRANSF) $doANA = 1 if ( ! $opt_skipSOLVER ); $doTRANSF = 1 if ( ! $opt_skipTRANSF ); $doODS = 1 if ( $ENV{DIAG2ODS}); $doSFCANA = 0 if ( $ENV{NOSFCANA} ); $doSPC = 1 if ( $ENV{DOSPC}); $replay = 1 if ( $ENV{REPLAY} || $opt_replay ); $debug = 1 if ( $ENV{DEBUG} || $opt_debug ); $strict = "-strict" if ( $ENV{STRICT} || $opt_strict ); $prepqc = "prepqc" unless $prepqc; $expid = $ENV{EXPID} unless $expid; if ( $replay ) { $doANA = 0; # make sure analysis is off when replay is on $doOIQC = 0; # make sure OIQC is off when replay is on $doVTXRLC = 0; # make sure vortex rel is off when replay is on } print "skipping VTXRELOC\n" if ( ! $doVTXRLC ); print "skipping OIQC\n" if ( ! $doOIQC ); print "skipping ANALYSIS\n" if ( ! $doANA ); print "skipping TRANSFORMS\n" if ( ! $doTRANSF ); $fvwork = $ENV{FVWORK}; $ssbkg = "none" unless $ssbkg; # spec/sig bkg file $sssbkg = "none" unless $sssbkg; # spec/sfc bkg file $rcname = "$fvwork/$myrc" unless $rcname; $fvbkg = `echorc.x -template $expid $nymd $nhms -rc $rcname upper-air_bkg_filename` unless $fvbkg; chomp($fvbkg); $fvsbkg = `echorc.x -template $expid $nymd $nhms -rc $rcname surface_bkg_filename` unless $fvsbkg; chomp($fvsbkg); $ss_in = "fv2ss_sig_out.dat" unless $ss_in; $ss_out = "ss2fv_eta_out.hdf" unless $ss_out; $upa = "none" unless $upa; $lwi_nm = "none" unless $lwi_nm; $ts_nm = "none" unless $ts_nm; $obs = "1" unless $obs; $jcap = "62" unless $jcap; $nsig = "28" unless $nsig; $hgrd = "96" unless $hgrd; $doASYN = 1 unless $no_asyn; # Define entries for asynoptic setup # ---------------------------------- $timeinc = 360 unless $ENV{TIMEINC}; # time interval between consecutive analyses (minutes) $anafreq = $timeinc / 60; # time interval between consecutive analyses (hours) if ($doASYN) { $dtasyn = `gethdftinc.x $fvwork/$fvbkg`; # time freq of asynoptic interval set from bkg file } else { $dtasyn = 060000; # set regular 6 hrs analysis } $dtasyn = $dtasyn / 10000 ; if($dtasyn==6) { @atbkg = qw(06) }; # synoptic analysis if($dtasyn==3) { @atbkg = qw(03 06 09) }; # 3hr asynoptic analysis ($asynymd, $asynhms) = ( tick($nymd,$nhms,-6*3600) ); # step back to previous synoptic time $log = "1" unless $log; if ($log eq "1") { $log_vtxr = "$fvwork/vtxr.log"; $log_oiqc = "$fvwork/oiqc.log"; $log_fv2ss = "$fvwork/fv2ss.log"; $log_ss2fv1 = "$fvwork/ss2fv1.log"; $log_ss2fv2 = "$fvwork/ss2fv2.log"; $log_ana = "$fvwork/ssi.log"; $log_ods = "$fvwork/ods.log"; $log_ana5sfc = "$fvwork/ana5sfc.log"; } else { $log_vtxr = "$fvwork/$log"; $log_oiqc = "$fvwork/$log"; $log_fv2ss = "$fvwork/$log"; $log_ss2fv1 = "$fvwork/$log"; $log_ss2fv2 = "$fvwork/$log"; $log_ana = "$fvwork/$log"; $log_ods = "$fvwork/ods.log"; $log_ana5sfc = "$fvwork/$log"; } # create SSI date $adate = $yyyy.$mm.$dd.$hh; $hr = substr($adate,8,2); $ghr = "t".$hr."z"; $daily = $yyyy.$mm.$dd; # define blending region @blend = (0.0,0.0); # defaults for no-blending if (($nsig =~ "28" && $jcap =~"62") || ($nsig =~ "42" && $jcap =~ "170")) { @blend = (10.0,30.0); } if (($nsig =~ "64" && $jcap =~ "62") || ($nsig =~ "64" && $jcap =~ "254") || ($nsig =~ "64" && $jcap =~ "382")) { @blend = (0.2,0.4); } if ($nsig =~ "32" ) { @blend = (1.0,5.0); } $pabove = $blend[0] unless $pabove; $pbelow = $blend[1] unless $pbelow; # Get full pathnames # ------------------ if ( $doTRANSF ) { $ss_out = fullpath($ss_out) } # Work in local SSI directory to avoid conflicts # ---------------------------------------------- $tmp = $fvwork; $anadir = "$tmp/ana.$nymd.$nhms"; # SSI working directory $rc = system("/bin/mkdir -p $anadir" ); die ">>> ERROR <<< cannot create $anadir " if ( $rc ); chdir("$anadir"); $rc_ignore = system("/bin/touch $anadir/.no_archiving"); # working cqc dir not to be achived # NOTE: The following links are to be handled by an SSI file handler TBD # Static data needed for transforms and resolution independent static data # ------------------------------------------------------------------------ Assignfn( "$fvInput/$myetc/fv_phis.dat", "fv_phis.dat"); Assignfn( "$fvInput/$myetc/ncep_out.dat", "ncep_out.dat"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/global_spectral_coefs.f77", "spectral_coefficients"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/global_transmittance_coefs.f77", "transmittance_coefficients"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/global_emissivity_coefs.f77", "emissivity_coefficients"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/prepobs_errtable.global", "errtable"); Assignfn( "$fvInput/$myetc/global_eofs.l${nsig}.f77", "eofs"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/CRTM_SpcCoeff.f77", "SpcCoeff.bin"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/CRTM_TauCoeff.f77", "TauCoeff.bin"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/CRTM_EmisCoeff.f77", "EmisCoeff.bin"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/CRTM_AerosolCoeff.f77", "AerosolCoeff.bin"); Assignfn( "$fvInput/$myetc/fix_ncep20051020/CRTM_CldCoeff.f77", "CloudCoeff.bin"); # SSI/GSI static data and resolution dependent static data # NOTE: for a while, when using GSI, will need both because of spectral transforms # -------------------------------------------------------------------------------- if ( $doTRANSF ) { if ($doGSI) { Assignfn( "$fvInput/gsi/etc/newncepsfc.${jcap}", "ncepsfc"); } else { Assignfn( "$fvInput/ssi/etc/T${jcap}L${nsig}/newncepsfc.${jcap}", "ncepsfc"); } } if ( $doGSI ) { Assignfn( "$fvwork/$myrc", "gsiparm.anl"); Assignfn( "$fvwork/gmao_global_pcpinfo.rc", "pcpinfo"); Assignfn( "$fvwork/gmao_global_satinfo.rc", "satinfo"); Assignfn( "$fvwork/gmao_global_ozinfo.rc", "ozinfo"); Assignfn( "$fvwork/gmao_global_convinfo.rc", "convinfo"); Assignfn( "$fvwork/prepobs_prep.bufrtable", "prepobs_prep.bufrtable"); Assignfn( "$fvwork/gmao_airs_bufr.tbl", "airs_bufr.table"); #________________________________ # Choose the file of background error statistics. First, use any user # defined run-time value, from --berror=<> or from environment variable # BERROR. if ( ! "$berror" ) { $berror = $ENV{BERROR} }; # If $berror is undefined, try a list of known historical filenames in # a backward order. if ( ! "$berror" ) { # locally derived file on fv-interpolated Gaussian grid $berror="$fvInput/gsi/gmao08Nov2005/l${nsig}y${hgrd}.berror_stats.bin"; # or a NCEP derived file on a Gaussian grid. $berror="$fvInput/gsi/l${nsig}y${hgrd}_ncep14Apr2005/global_berror.l${nsig}y${hgrd}.sig.sp.f77" unless ( -e "$berror" ); # if none fits, set the name back to none. $berror = "" unless ( -e "$berror" ); } # Check the status if ( ! "$berror" ) { die ">>>> ERROR <<< no berror_stats file defined" } if ( ! -e "$berror" ) { die ">>>> ERROR <<< can not open berror_stats file, \"$berror\"" } Assignfn( "$berror", "berror_stats"); $rc_ignore = system("ls -tl berror_stats"); $rc_ignore = system("ls -tl $berror"); #________________________________ $rc_file = ""; # input file not read from std input } else { Assignfn( "$fvInput/$myetc/global_pcpinfo.txt", "pcpinfo"); Assignfn( "$fvInput/$myetc/fix_ncep20050202/global_satinfo.txt", "satinfo"); Assignfn( "$fvInput/ssi/T${jcap}L${nsig}/tracers23.dat.${jcap}", "tracers23.dat"); Assignfn( "$fvInput/$myetc/global_divten.l${nsig}.f77", "divterrs"); Assignfn( "$fvInput/$myetc/global_vars.l${nsig}.f77", "vars"); $rc_file = "< $rcname"; # read from std input in ssi } # Initialize sat-bias file (link or prepare to create one) # -------------------------------------------------------- $doSPINBIAS = 0; if ( $doANA ) { if ( -e "$fvwork/satbias" ) { Assignfn ( "$fvwork/satbias", "satbias_in" ); } else { print " Generating satbias_in ... \n"; $doSPINBIAS = 1 unless ( $opt_skipSATBIAS ); } if ( -e "$fvwork/satbang" ) { Assignfn ( "$fvwork/satbang", "satbias_angle"); } else { print " Using static satangbias file ... \n"; cp( "$fvInput/$myetc/fix_ncep20050202/global_satangbias_eliu.txt", "satbias_angle"); } if ( -e "$fvwork/pcpbias" ) { Assignfn ( "$fvwork/pcpbias", "pcpbias_in"); } } # Bring in guess field (in case it's available) # --------------------------------------------- if ( -e "$fvwork/gesfile" && $doANA ) { if ( $doGSI ) { cp ( "$fvwork/gesfile", "gesfile_in"); } else { cp ( "$fvwork/gesfile", "ingesfile"); } } # Welcome message print <<"EOF" ; ------------------------------------------------------ Analyzer - Produces atmospheric analysis with SSI/GSI ------------------------------------------------------ Experiment id : $expid fvInput : $fvInput Work dir : $anadir NYMD : $nymd NHMS : $nhms TRANSF : $doTRANSF ANA : $doANA PREPQC name : $prepqc RC name : $rcname Spectral T : $jcap NLAT : $hgrd Sigma levs : $nsig PABOVE : $pabove PBELOW : $pbelow berror : $berror Starting... EOF } #...................................................................... sub fv2ss { my ( $fvbkg, $fvsbkg, $this_nymd, $this_nhms, $fhour, $atbkg ) = @_; Assignfn ( "$fvwork/$fvbkg", "fv-dyn.${atbkg}" ); Assignfn ( "$fvwork/$fvbkg", "$fvbkg" ); Assignfn ( "$fvwork/$fvsbkg", "fv-sfc.${atbkg}" ); Assignfn ( "$fvwork/$fvsbkg", "$fvsbkg" ); Assignfn ( "ncepsfc", "nc-sfc.${atbkg}" ); # this is fake since ncepsfc is static at fixed time if ( $doSPC ) { print " Running fv2ss.x, please wait...\n"; if ( $nsig == 28 || $nsig == 42 || $nsig == 64 ) { $cmd = "fv2ss.x $fvbkg $fvsbkg -jcap $jcap -nsig $nsig -ncep_phis -fhour $fhour -pick $this_nymd $this_nhms -o sigf${atbkg} -os sfcf$atbkg "; } else { $cmd = "fv2ss.x $fvbkg $fvsbkg -jcap $jcap -fhour $fhour -pick $this_nymd $this_nhms -o sigf${atbkg} -os sfcf$atbkg "; } print " $cmd\n"; $rc = System($cmd, "$log_fv2ss",""); print STDOUT " $0: fv2ss.x \$rc = $rc\n" ; die ">>>> ERROR <<< running fv2ss.x" if ( $rc ); if ( "$this_nymd" == "$nymd" && "$this_nhms" == "$nhms" ) { Assignfn ( "sigf${atbkg}", $ss_in ); # Link is needed to allow for no-data/no-harm to work # it links the synoptic hour spectral-bkg to the input # needed by the intermmediate ss2fv call. } my $this_hh = substr($this_nhms,0,2); cp ( "sigf${atbkg}", "$fvwork/$expid.bkg.spcsig.${this_nymd}_${this_hh}z.bin" ); # Make this available for storage if desired } # < doSPC > } #...................................................................... sub vtxrlc { my ( $nymd, $nhms ) = @_; return if ( ! $doVTXRLC ); print " will now run Vortex Relocator ...\n"; # Quality control conventional observations # ----------------------------------------- $cmd = "vtxreloc -overwrite $nymd $nhms $expid" ; print " $cmd\n"; $rc = System($cmd, "$log_vtxr"); print STDOUT " $0: vtxrlc \$rc = $rc\n" ; die ">>>> ERROR <<< running vtxreloc" if ( $rc ); } #...................................................................... sub setobs { my ( $nymd, $hh ) = @_; return if ( $rc!=0 ); print " Setting up observations ...\n"; # Link observation files # ---------------------- $cmd = "lnlist -d $nymd -t $hh -r $fvwork/lnlist.rc -a $fvwork/obsys.acq"; print " $cmd\n"; $rc = system($cmd); print STDOUT " $0: lnlist \$rc = $rc\n" ; die ">>>> ERROR <<< running lnlist" if ( $rc ); } #...................................................................... sub oiqc { my ( $nymd, $hh, $qcbkg ) = @_; return if ( $rc!=0 ); return if ( ! $doOIQC ); print " and preparing to run OIQC ...\n"; $prepqc_new = "$anadir/$prepqc" ; # full filename of prep qc-ed conventional obs file $prepqc_old = "$anadir/prepqc_old" ; # full filename of original prepqc conv. obs file # Rename original prepqc file to serve as input to OIQC # ----------------------------------------------------- cp("$prepqc_new","$prepqc_old"); rm("$prepqc_new"); # Quality control conventional observations # ----------------------------------------- $cmd = "ssoiqc -r $fvwork -o $prepqc_new $nymd $hh $prepqc_old $qcbkg" ; print " $cmd\n"; $rc = System($cmd, "$log_oiqc",""); print STDOUT " $0: ssoiqc \$rc = $rc\n" ; die ">>>> ERROR <<< running ssoiqc" if ( $rc ); # Reserve a copy of quality controled file for archiving purposes # and restablish link to prepqc file actually used by analysis # --------------------------------------------------------------- cp("$prepqc_new","$fvwork/$expid.ana.prepqc.${nymd}_${hh}z.bfr"); Assignfn( "$fvwork/$expid.ana.prepqc.${nymd}_${hh}z.bfr","$prepqc"); } #.................................................................................... sub ana { return if ( $rc!=0 ); print " Running $myexec , please wait...\n"; # Now run NCEP analysis # --------------------- $mpirun_ana = $ENV{MPIRUN_ANA}; if ( $mpirun_ana =~ "mpirun" || $mpirun_ana =~ "prun" ) { $cmd = "${mpirun_ana} $rc_file "; } elsif ( $mpirun_ana =~ "poe" ) { $cmd = "${mpirun_ana} < gsiparm.anl "; # this should be generilzed; gsi should not have wired in filename } else { $npes = $ENV{NCPUS}; if ( $npes > 0 ) { $cmd = "mpirun -prefix \"%g: \" -np $npes $myexec $rc_file "; } else { $cmd = "mpirun -prefix \"%g: \" -np 1 $myexec $rc_file "; } } print " $cmd\n"; $rc = System($cmd, "$log_ana","$myexec"); print " $0: $myexec \$rc = $rc\n"; die ">>>> ERROR <<< running $myexec" if ( $rc ); if ( -e "satbias_out" ) { cp("satbias_out","$fvwork/$expid.ana.satbias.${nymd}_${hh}z.txt"); # satbias for storage cp("satbias_out","$fvwork/satbias"); # satbias to recycle if ( $doSPINBIAS ) { cp("satbias_out","satbias_in"); # copy satbias file for 2nd analysis pass } } if ( -e "pcpbias_out" ) { cp("pcpbias_out","$fvwork/$expid.ana.pcpbias.${nymd}_${hh}z.txt"); # pcpbias for storage cp("pcpbias_out","$fvwork/pcpbias"); # pcpbias to recycle if ( $doSPINBIAS ) { cp("pcpbias_out","pcpbias_in"); # copy pcpbias file for 2nd analysis pass } } if ( $debug ) { $cmd = "cat `ls $anadir/fort.2[01]*`"; print " $cmd\n"; $rc_ignore = system($cmd); } # Cat diag files to proper after-analysis names # --------------------------------------------- lndiag_files("set"); # If analysis completed successfully it can become a guess for the next analysis # ------------------------------------------------------------------------------ if ( -e "gesfile_out" ) { cp("gesfile_out","$fvwork/gesfile") } if ( -e "siganl" ) { cp("siganl" ,"$fvwork/$expid.anl.sig.${nymd}_${hh}z.bin") } if ( -e "an-dyn.99" ) { cp("an-dyn.99" ,"$ss_out") } if (! -z "sfcanl.gsi" ) { cp("sfcanl.gsi" ,"$fvwork/$expid.anl.sfc.${nymd}_${hh}z.bin") } } #.................................................................................... sub lndiag_files { my ( $todo ) = @_; my(@loops, @satlist, @ozlist); @loops = qw(01 03); @satlist = qw(hirs2_n14 msu_n14 sndr_g08_prep sndr_g10_prep sndr_g12_prep sndrd1_g08 sndrd2_g08 sndrd3_g08 sndrd4_g08 sndrd1_g10 sndrd2_g10 sndrd3_g10 sndrd4_g10 sndrd1_g12 sndrd2_g12 sndrd3_g12 sndrd4_g12 hirs3_n15 hirs3_n16 hirs3_n17 amsua_n15 amsua_n16 amsua_n17 amsub_n15 amsub_n16 amsub_n17 hsb_aqua airs_aqua amsua_aqua goesimg_g08 goesimg_g10 goesimg_g12 pcp_ssmi_dmsp pcp_tmi_trmm conv sbuv2_n16 sbuv2_n17 sbuv_n18 omi_aura ssmi_f13 ssmi_f14 ssmi_f15 hirs4_n18 amsua_n18 mhs_n18 amsre_low_aqua amsre_mid_aqua amsre_hig_aqua ssmis_las_f16 ssmis_uas_f16 ssmis_img_f16 ssmis_env_f16); @ozlist = qw(sbvu2_n16 sbuv2_n17 sbuv2_n18 omi_aura); if ( $todo eq "set" ) { # Cat satellite diagnostic files for each processor into single file # NOTE: Need a dynamic way for doing this ... # ------------------------------------------------------------------ foreach $loop (@loops) { foreach $sat (@satlist) { $files_tmp = `ls ${sat}_$loop.*`; chomp($files_tmp); @files = split(/\n/,$files_tmp); $files=@files; # how many files are there if ($files > 0 ) { $cmd = "cat @files > diag_${sat}_$loop.${nymd}${hh}"; print " $cmd\n"; $rc_ignore = system($cmd); if ( -z "diag_${sat}_$loop.${nymd}${hh}" ) { rm("diag_${sat}_$loop.${nymd}${hh}"); } else { if ( "$loop" == "01" ) { cp("diag_${sat}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${sat}_ges.${nymd}${hh}"); Assignfn( "$fvwork/$expid.diag_${sat}_ges.${nymd}${hh}","diag_${sat}.${nymd}${hh}"); } elsif ( "$loop" == "03" ) { cp("diag_${sat}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${sat}_anl.${nymd}${hh}"); } else { cp("diag_${sat}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${sat}_$loop.${nymd}${hh}"); } } } } } # Cat conventional data diagnostic files for each processor into single file # -------------------------------------------------------------------------- @convlist = qw(conv); foreach $loop (@loops) { foreach $cls (@convlist) { $cmd = "cat `ls ${cls}_$loop.*` > diag_${cls}_$loop.${nymd}${hh}"; print " $cmd\n"; $rc_ignore = system($cmd); if ( -z "diag_${cls}_$loop.${nymd}${hh}" ) { rm("diag_${cls}_$loop.${nymd}${hh}"); } else { if ( "$loop" == "01" ) { cp("diag_${cls}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${cls}_ges.${nymd}${hh}"); } elsif ( "$loop" == "03" ) { cp("diag_${cls}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${cls}_anl.${nymd}${hh}"); } else { cp("diag_${cls}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${cls}_$loop.${nymd}${hh}"); } } } } # Cat ozone data diagnostic files for each processor into single file # -------------------------------------------------------------------------- foreach $loop (@loops) { foreach $ols (@ozlist) { $cmd = "cat `ls ${ols}_$loop.*` > diag_${ols}_$loop.${nymd}${hh}"; print " $cmd\n"; $rc_ignore = system($cmd); if ( -z "diag_${ols}_$loop.${nymd}${hh}" ) { rm("diag_${ols}_$loop.${nymd}${hh}"); } else { if ( "$loop" == "01" ) { cp("diag_${ols}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${ols}_ges.${nymd}${hh}"); Assignfn( "$fvwork/$expid.diag_${ols}_ges.${nymd}${hh}","diag_${ols}.${nymd}${hh}"); } elsif ( "$loop" == "03" ) { cp("diag_${ols}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${ols}_anl.${nymd}${hh}"); } else { cp("diag_${ols}_$loop.${nymd}${hh}","$fvwork/$expid.diag_${ols}_$loop.${nymd}${hh}"); } } } } } else { # clean up diag files my(@newlist); @newlist = qw( @satlist @ozlist ); foreach $sat (@newlist) { $files_tmp = `ls diag_${sat}.*`; chomp($files_tmp); @files = split(/\n/,$files_tmp); $files=@files; # how many files are there if ($files > 0 ) { if ( -e "diag_${sat}.${nymd}${hh}" ) { rm("diag_${sat}.${nymd}${hh}") }; } } } } #.................................................................................... sub rep { $rc = 0; if (-e "$fvwork/$expid.anl.sig.${nymd}_${hh}z.bin" ) { cp("$fvwork/$expid.anl.sig.${nymd}_${hh}z.bin","siganl"); print " Successfully copied already existing analysis ...\n"; } else { print " Error: failed copying already existing analysis ...\n"; $rc = 1; } if (-e "$fvwork/$expid.anl.sfc.${nymd}_${hh}z.bin" ) { cp("$fvwork/$expid.anl.sfc.${nymd}_${hh}z.bin","sfcanl.gsi"); print " Successfully copied already existing sfc analysis ...\n"; } else { print " Error: failed copying already existing sfc analysis ...\n"; $rc = 1; } } #.................................................................................... sub sac_namelist { my($ft, $frun); $rc_sac = "$anadir/sac.nl"; $frun = "$fvwork/sac.nl.tmpl"; open(LUN,"$frun") || die "Fail to open sac.nl.tmpl: $!\n"; open(LUN2,">$rc_sac") || die "Fail to open sac.nl: $!\n"; # Change variables to the correct inputs #--------------------------------------- while( defined($rcd = ) ) { chomp($rcd); if($rcd =~ /\@YYYY/) {$rcd=~ s/\@YYYY/$yyyy/; } if($rcd =~ /\@MM/) {$rcd=~ s/\@MM/$mm/; } if($rcd =~ /\@DD/) {$rcd=~ s/\@DD/$dd/; } if($rcd =~ /\@HH/) {$rcd=~ s/\@HH/$hh/; } print(LUN2 "$rcd\n"); } close(LUN); close(LUN2); } #.................................................................................... sub sac { return if ( $rc!=0 ); print " Running sac.x, please wait...\n"; # Fix date/time in namelist # ------------------------- sac_namelist(); # Now run SAC: satellite angular correction # ----------- $mpirun_sac = $ENV{MPIRUN_SAC}; if ( $mpirun_sac =~ "mpirun" || $mpirun_sac =~ "prun" || $mpirun_sac =~ "poe" ) { $cmd = "${mpirun_sac} "; } else { $npes = $ENV{NCPUS}; if ( $npes > 0 ) { $cmd = "mpirun -prefix \"%g: \" -np $npes sac.x "; } else { $cmd = "mpirun -prefix \"%g: \" -np 1 sac.x "; } } print " $cmd\n"; $rc = System($cmd, "$log_ana","sac.x"); die ">>>> ERROR <<< running sac.x" if ( $rc ); print " $0: sac.x \$rc = $rc\n"; cp("satbias_angle.out","$fvwork/$expid.ana.satbang.${nymd}_${hh}z.txt"); # satbang for storage cp("satbias_angle.out","$fvwork/satbang"); # satangbias to recycle if ( $doSPINBIAS ) { cp("satbias_angle.out","satbias_angle"); # cp sat bias ang. correction for 2nd analysis pass } # Get rid of diag files links # --------------------------- lndiag_files("unset"); } #.................................................................................... sub ss2fv { my ( $this_nymd, $this_nhms ) = @_; return if ( $rc!=0 ); print " Running ss2fv.x, please wait...\n"; my $this_hh = substr($this_nhms,0,2); # $fvbkg = `echorc.x -template $expid $this_nymd $this_nhms -rc $myrc upper-air_bkg_filename`; # $fvsbkg = `echorc.x -template $expid $this_nymd $this_nhms -rc $myrc surface_bkg_filename`; $fvbkg = "$expid.bkg.eta.${this_nymd}_${this_hh}z.hdf"; $fvsbkg = "$expid.bkg.sfc.${this_nymd}_${this_hh}z.hdf"; chomp($fvbkg); chomp($fvsbkg); if ( -e "siganl" ) { # check completion of GSI/SSI if ( ! -z "sfcanl.gsi" ) { # check presence of surface analysis file $cmd = "ss2fv.x siganl $fvbkg -is sfcanl.gsi -ua $fvbkg -o $ss_out -pick $this_nymd $this_nhms -pa $pabove -pb $pbelow -prec 32"; } else { $cmd = "ss2fv.x siganl $fvbkg -ua $fvbkg -o $ss_out -pick $this_nymd $this_nhms -pa $pabove -pb $pbelow -prec 32"; } print " $cmd\n"; $rc = System($cmd, "$log_ss2fv2", ""); } else { $cmd = "ss2fv.x $ss_in $fvbkg -is sfcf06 -ua $fvbkg -o eta_out.hdf -pick $this_nymd $this_nhms -pa $pabove -pb $pbelow -prec 32"; print " $cmd\n"; $rc = System($cmd, "$log_ss2fv1", ""); if ( ! $doANA ) { cp("eta_out.hdf","$ss_out") }; } die ">>>> ERROR <<< running ss2fv.x" if ( $rc ); print "$0: ss2fv.x \$rc = $rc\n"; } #...................................................................... sub d2ods { chdir("$fvwork"); $cmd = "diag2ods $nymd $nhms $expid"; print " $cmd\n"; $rc = System($cmd, "$log_ods",""); } #...................................................................... # # Tick - advance date/time by nsecs seconds # # sub tick { my ( $nymd, $nhms, $nsecs ) = @_; if("$nsecs" == "0" ) { return ($nymd, $nhms); } $yyyy1 = substr($nymd,0,4); $mm1 = substr($nymd,4,2); $dd1 = substr($nymd,6,2); $hh1 = 0 unless ( $hh1 = substr($nhms,0,2)); $mn1 = 0 unless ( $mn1 = substr($nhms,2,2)); $ss1 = 0 unless ( $ss1 = substr($nhms,4,2)); $time1 = timegm($ss1,$mn1,$hh1,$dd1,$mm1-1,$yyyy1) + $nsecs; ($ss1,$mn1,$hh1,$dd1,$mm1,$yyyy1,undef,undef,undef) = gmtime($time1); $nymd = (1900+$yyyy1)*10000 + ($mm1+1)*100 + $dd1; $nhms = sprintf("%6.6d",$hh1*10000 + $mn1*100 + $ss1); return ($nymd, $nhms); } #.................................................................................... sub System { my ( $cmd, $logfile, $xname ) = @_; my ( @zname ); open SAVEOUT, ">&STDOUT"; # save stdout open SAVEERR, ">&STDERR"; # save stderr open STDOUT, ">>$logfile" or die "can't redirect stdout"; open STDERR, ">>$logfile" or die "can't redirect stderr"; select STDERR; $| = 1; # make it unbuffered select STDOUT; $| = 1; # make it unbuffered @zname = split(" ", $cmd); if ( "$zname[0]" eq "mpirun" || "$zname[0]" eq "prun" ) { $rc1 = system( "zeit_ci.x -r $fvwork/.zeit $xname"); } else { $rc1 = system( "zeit_ci.x -r $fvwork/.zeit $zname[0]"); } $rc = system ( $cmd ); # run the shell command if ( "$zname[0]" eq "mpirun" || "$zname[0]" eq "prun" ) { $rc2 = system( "zeit_co.x -r $fvwork/.zeit $xname"); } else { $rc2 = system( "zeit_co.x -r $fvwork/.zeit $zname[0]"); } # Bitwise shift returns actual UNIX return code $exit_code = $rc >> 8; close STDOUT; close STDERR; open STDOUT, ">&SAVEOUT" ; # restore stdout open STDERR, ">&SAVEERR" ; # restore stdout return $exit_code; } #...................................................................... sub Assign { my ( $fname, $lu ) = @_; $f77name = "fort.$lu"; unlink($f77name) if ( -e $f77name ) ; symlink("$fname","$f77name"); } sub Assignfn { # Assignfn - assigns fn to given file name fname. # fname = old file # fn = new file (links to old) my ( $fname, $fn ) = @_; unlink($fn) if ( -e $fn ) ; symlink("$fname","$fn"); } sub fullpath { my ( $fname ) = @_; $dirn = dirname("$fname"); chomp($dirn = `pwd`) if ( "$dirn" eq "." ) ; $name = basename("$fname"); $full = "$dirn/$name"; return ($full); } #...................................................................... sub sfc_ana { my ( $nymd, $nhms) = @_; ($hour) = ($nhms =~ /(..)..../); die ">>> ERROR <<< cannot run on non-synoptic hour = $hour" if ($hour % 6 != 0); $prefix = $fvwork . "/" . $expid; #-----------------------# # process synoptic hour # #-----------------------# $stem = $nymd . "_" . $hour . "z.hdf"; $anaETA = $prefix . ".ana.eta." . $stem; $bkgETA = $prefix . ".bkg.eta." . $stem; $bkgSFC = $prefix . ".bkg.sfc." . $stem; $xanaSFC = $prefix . ".xana.sfc." . $stem; $xbkgSFC = $prefix . ".xbkg.sfc." . $stem; #*** create xana.sfc file print " Running ana5sfc.x (xana) for synoptic hour $hour ....\n"; $cmd = "ana5sfc.x $anaETA $bkgETA $bkgSFC -o $xanaSFC"; print " $cmd\n"; $rc = System($cmd, "$log_ana5sfc", ""); print STDOUT " $0: sfc_ana \$rc = $rc\n"; die ">>> ERROR <<< running ana5sfc (xana) for synoptic hour $hour" if ($rc); #*** create xbkg.sfc file print " Running ana5sfc.x (xbkg) for synoptic hour $hour ....\n"; $cmd = "ana5sfc.x $bkgETA $bkgETA $bkgSFC -o $xbkgSFC"; print " $cmd\n"; $rc = System($cmd, "$log_ana5sfc", ""); print STDOUT " $0: sfc_ana \$rc = $rc\n"; die ">>> ERROR <<< running ana5sfc (xbkg) for synoptic hour $hour" if ($rc); #------------------------------------# # process previous non-synoptic hour # #------------------------------------# if ( $doASYN ) { $minus3hrs = -3 * 3600; ($nymd, $nhms) = ( tick($nymd,$nhms,$minus3hrs) ); ($hour) = ($nhms =~ /(..)..../); $stem = $nymd . "_" . $hour . "z.hdf"; $bkgETA = $prefix . ".bkg.eta." . $stem; $bkgSFC = $prefix . ".bkg.sfc." . $stem; $xbkgSFC = $prefix . ".xbkg.sfc." . $stem; #*** create xbkg.sfc file only for non-synoptic hours print " Running ana5sfc.x (xbkg) for hour $hour ....\n"; $cmd = "ana5sfc.x $bkgETA $bkgETA $bkgSFC -o $xbkgSFC"; print " $cmd\n"; $rc = System($cmd, "$log_ana5sfc", ""); print STDOUT " $0: sfc_ana \$rc = $rc\n"; die ">>> ERROR <<< running ana5sfc (xbkg) for hour $hour" if ($rc); } } #...................................................................... sub usage { print <<"EOF"; NAME analyzer - Produces atmospheric analysis SYNOPSIS analyzer [...options...] nymd nhms DESCRIPTION This script is a frontend for the FVSSI system. It drives the analsysis process for a given synoptic time. It executes the following applications: fv2ss.x - eta to spectral transform $myexec (NCEP's analysis) ss2fv.x - spectral to eta transform The following parameters are required nymd Year-month-day, e.g., 2002010 for 10 Jan 2002 nhms Hour-month-sec, e.g., 000000 for 00Z OPTIONS -h : prints this usage notice -d : fvInput directory (default /share/fvdas/fvInput/ssi) -skipOIQC : skips OIQC -skipSOLVER : skips analysis, sets increments to zero -skipTRANSF : skips transforms -berror : file of background error statistics (default is from a list) -prepqc : prepqc file name -expid : experiment id -rc : resource file -t : resolution (default: T62L28) -pa : pabove blending level in mb (default 30) -pb : pbelow blending level in mb (default 10) -bkg : FV background eta file -sbkg : FV background sfc file -log : name of log file (default: three log files) -strict : when specified, will die in case of missing obs files (see ENV) -debug : turns on extra output from SSI (bkg-to-data fit; see ENV) -replay : allows running from already-existing (spectral) analysis (see ENV) The following are used when solver is running alone (no transforms): -ssbkg : spectral sigma background -sssbkg : gaussian NCEP-like surface background The following are used by fv2ss.x : -iss : Filename of NCEP sigma-level analysis in spectral space Input to SSI (default fv2ss_sig_out.dat) The following are used by ss2fv.x : -oss : Filename of NCEP analysis in eta format Output from SSI (default ss2fv_eta_out.hdf) -ua : file name to indicate if upper air blending should be done (default none) -lwi : Filename of fv_phys restart (default none) -ts : Filename of ncep surface temperature datafile (default none) This is an special option to choose specific observations -obs : observations to be used (default 1) 1 = all observations, 2 = no radiances, 3 = by kx (also runs prepbykx.x to generate a special prepbufr file) NECESSARY ENVIRONMENT MPIRUN_SAC env var specifying mpi command line for sac.x MPIRUN_ANA env var specifying mpi command line for ssi.x/gsi.x (e.g., mpirun -np 4 fullpath/$myexec) NCPUS number of procs to use in mpi call to SSI (used only when MPIRUN_ANA not defined) FVWORK location of rc file(s), satbias_in, bkg files, and observations files FVROOT bin directory for run OPTIONAL ENVIRONMENT DEBUG activates debugging (for now, it only controls listing opt) STRICT controls level of acceptance of missing observation files REPLAY allows running from previously existing (spectral) analysis NODIAG2ODS when set, skips conversion of diag output files to ods AUTHOR Carlos Cruz (ccruz\@gmao.gsfc.nasa.gov), NASA/GSFC/GMAO Last modified: 20May2005 by: R. Todling EOF exit(1) }