Release Notes:  HREF v3.1.0 - released to NCO on September 22, 2021.
Transition from WCOSS Cray to WCOSS2.

Re-released September 23, 2021 with updated resources for certain jobs.  Also adds
OMP_NUM_THREADS=1 to certain submission scripts. 

A better formatted version of this document is on google docs at:
https://docs.google.com/document/d/1WRUhYiDUsdCQ4Jt-JzIImuBqLRBZy7wRN3zUta2Ltts/edit?usp=sharing

#######################################
Obtaining and installing
#######################################


git clone -b v3.1_deliver https://github.com/MatthewPyle-NOAA/HREF.git
This will create an HREF/href.v3.1.0 directory.  Going into HREF/href.v3.1.0/sorc/, 
./build_href.sh (to compile)
./install_href.sh (to copy executables to exec/)
./link_href_fix.sh (to populate fix files)

#######################################
Software utilized from outside of vertical structure 
#######################################

PrgEnv-intel/8.1.0
intel/19.1.3.304
craype/2.7.6
cray-mpich/8.1.4

ip_ver=3.3.3
bacio_ver=2.4.1
w3nco_ver=2.4.1
g2_ver=3.4.4
g2tmpl_ver=1.9.1
jasper_ver=2.0.25
libpng_ver=1.6.37
zlib_ver=1.2.11
python_ver=3.8.6
gempak_ver=7.14.0
cfp_ver=2.0.4
prod_util_ver=2.0.8
wgrib2_ver=2.0.8
netcdf_ver=4.7.4
libjpeg_ver=9c



#######################################
Description of code/script changes 
#######################################

ush/:
eas_config.py (splits the 12h and 24 h QPF thresholds differently to speed up the EAS scripts)
make_easfracqpf_combo.py (changes for python3; 12/24h QPF threshold reswizzle)
make_easfracsnow_combo.py (changes for python3) 
href_ensprod.sh (adds error checking on copy of previous FFG cycle)

jobs/:
All J-jobs are modified.  Most are COMIN/COMOUT changes adapting to new filesystem organization, and use of compath.py in definitions.  Also changes from aprun to mpiexec where needed.   Removes RUN_ENVIR usage and definitions of no longer needed variables like COMOUTPERM and COMINpcpanl.
scripts/:
exhref_eas_1.sh.ecf (adopts to the low/medium/high threshold types for 12h/24h QPF probabilities, mpiexec in place of aprun)
exhref_eas_2.sh.ecf (adopts to the low/medium/high threshold types for 12h/24h QPF probabilities, mpiexec in place of aprun )
exhref_ffggen.sh.ecf (modified for the restructured DCOM)

sorc/:

href_get_prcip.fd/href_get_prcip.f (eliminates an undefined variable from several print statements that caused trouble with a debug built code).
href_get_prcip.fd/makefile (changes to allow for various build options - all, debug, install)
href_ensprod.fd/makefile (changes to allow for various build options - all, debug, install)
href_ffg_gen.fd/makefile (changes to allow for various build options - all, debug, install)

All build related scripts (*.sh) in the main sorc/ directory are modified (module file renamed from v3.0.0 to v3.1.0, and is loaded slightly differently on WCOSS2; changes to how makefiles are invoked, and also change to fix file location for WCOSS2):
versions/:
Adds build.ver and run.ver files 
launch/:
New submission scripts for PBS that also contains resource information

#######################################
Product and resource changes
#######################################

Aside from the reorganization on disk (such as href no longer being under hiresw), there are no changes to product names or details.  Also no changes to product dissemination or what is saved to HPSS.
The launch/*.sh_in  (PBS submission template files) and launch/run* jobs that modify and submit the final PBS scripts based on the domain and cycle specified provide resource information. Also, a spreadsheet showing some timing and resource comparisons for the different HREF domains is available in:
https://docs.google.com/spreadsheets/d/1a0vakGsv-HobaMCSRPAujliE8OZQA5RCQ0UxvY_x-7U/edit?usp=sharing
Disk space needed per day should be unchanged - about 225 GB/day (for the href.${PDY} and nwges/href.${PDY} contents).
#######################################
Additional information
#######################################

The upstream and downstream dependencies - should be the same as for HREF v3.0:
https://docs.google.com/document/d/1OyHVfZo9mhQ2SedkcMcGQXwjXgbz3CqSDr-U4qnBeP8/edit?usp=sharing
A copy of the COMOUT directory from final testing: 
/lfs/h2/emc/lam/noscrub/Matthew.Pyle/test/com/href/v3.1 (on Cactus)

A summary document including some graphical examples comparing operational runs from WCOSS-Cray and runs made on WCOSS2:

https://docs.google.com/presentation/d/1E6_XRQUS60DN3JL1ScssE-3zIrqSCaxUi-vrylR3m7g/edit?usp=sharing