Fri Sep 8 02:55:16 UTC 2017 Start Regression test baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_control working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_control Checking test 01 results .... Comparing atmos_4xdaily.tile1.nc .........OK Comparing atmos_4xdaily.tile2.nc .........OK Comparing atmos_4xdaily.tile3.nc .........OK Comparing atmos_4xdaily.tile4.nc .........OK Comparing atmos_4xdaily.tile5.nc .........OK Comparing atmos_4xdaily.tile6.nc .........OK Comparing nggps2d.tile1.nc ............MISSING baseline Comparing nggps2d.tile2.nc ............MISSING baseline Comparing nggps2d.tile3.nc ............MISSING baseline Comparing nggps2d.tile4.nc ............MISSING baseline Comparing nggps2d.tile5.nc ............MISSING baseline Comparing nggps2d.tile6.nc ............MISSING baseline Comparing nggps3d.tile1.nc ............MISSING baseline Comparing nggps3d.tile2.nc ............MISSING baseline Comparing nggps3d.tile3.nc ............MISSING baseline Comparing nggps3d.tile4.nc ............MISSING baseline Comparing nggps3d.tile5.nc ............MISSING baseline Comparing nggps3d.tile6.nc ............MISSING baseline Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.tile1.nc .........OK Comparing RESTART/fv_core.res.tile2.nc .........OK Comparing RESTART/fv_core.res.tile3.nc .........OK Comparing RESTART/fv_core.res.tile4.nc .........OK Comparing RESTART/fv_core.res.tile5.nc .........OK Comparing RESTART/fv_core.res.tile6.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile1.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile2.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile3.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile4.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile5.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile6.nc .........OK Comparing RESTART/fv_tracer.res.tile1.nc .........OK Comparing RESTART/fv_tracer.res.tile2.nc .........OK Comparing RESTART/fv_tracer.res.tile3.nc .........OK Comparing RESTART/fv_tracer.res.tile4.nc .........OK Comparing RESTART/fv_tracer.res.tile5.nc .........OK Comparing RESTART/fv_tracer.res.tile6.nc .........OK Comparing RESTART/sfc_data.tile1.nc .........OK Comparing RESTART/sfc_data.tile2.nc .........OK Comparing RESTART/sfc_data.tile3.nc .........OK Comparing RESTART/sfc_data.tile4.nc .........OK Comparing RESTART/sfc_data.tile5.nc .........OK Comparing RESTART/sfc_data.tile6.nc .........OK Test 01 FAIL baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_control working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_decomp Checking test 02 results .... Comparing atmos_4xdaily.tile1.nc .........OK Comparing atmos_4xdaily.tile2.nc .........OK Comparing atmos_4xdaily.tile3.nc .........OK Comparing atmos_4xdaily.tile4.nc .........OK Comparing atmos_4xdaily.tile5.nc .........OK Comparing atmos_4xdaily.tile6.nc .........OK Comparing nggps2d.tile1.nc ............MISSING baseline Comparing nggps2d.tile2.nc ............MISSING baseline Comparing nggps2d.tile3.nc ............MISSING baseline Comparing nggps2d.tile4.nc ............MISSING baseline Comparing nggps2d.tile5.nc ............MISSING baseline Comparing nggps2d.tile6.nc ............MISSING baseline Comparing nggps3d.tile1.nc ............MISSING baseline Comparing nggps3d.tile2.nc ............MISSING baseline Comparing nggps3d.tile3.nc ............MISSING baseline Comparing nggps3d.tile4.nc ............MISSING baseline Comparing nggps3d.tile5.nc ............MISSING baseline Comparing nggps3d.tile6.nc ............MISSING baseline Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.tile1.nc .........OK Comparing RESTART/fv_core.res.tile2.nc .........OK Comparing RESTART/fv_core.res.tile3.nc .........OK Comparing RESTART/fv_core.res.tile4.nc .........OK Comparing RESTART/fv_core.res.tile5.nc .........OK Comparing RESTART/fv_core.res.tile6.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile1.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile2.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile3.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile4.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile5.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile6.nc .........OK Comparing RESTART/fv_tracer.res.tile1.nc .........OK Comparing RESTART/fv_tracer.res.tile2.nc .........OK Comparing RESTART/fv_tracer.res.tile3.nc .........OK Comparing RESTART/fv_tracer.res.tile4.nc .........OK Comparing RESTART/fv_tracer.res.tile5.nc .........OK Comparing RESTART/fv_tracer.res.tile6.nc .........OK Comparing RESTART/sfc_data.tile1.nc .........OK Comparing RESTART/sfc_data.tile2.nc .........OK Comparing RESTART/sfc_data.tile3.nc .........OK Comparing RESTART/sfc_data.tile4.nc .........OK Comparing RESTART/sfc_data.tile5.nc .........OK Comparing RESTART/sfc_data.tile6.nc .........OK Test 02 FAIL baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_control working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_2threads Checking test 03 results .... Comparing atmos_4xdaily.tile1.nc .........OK Comparing atmos_4xdaily.tile2.nc .........OK Comparing atmos_4xdaily.tile3.nc .........OK Comparing atmos_4xdaily.tile4.nc .........OK Comparing atmos_4xdaily.tile5.nc .........OK Comparing atmos_4xdaily.tile6.nc .........OK Comparing nggps2d.tile1.nc ............MISSING baseline Comparing nggps2d.tile2.nc ............MISSING baseline Comparing nggps2d.tile3.nc ............MISSING baseline Comparing nggps2d.tile4.nc ............MISSING baseline Comparing nggps2d.tile5.nc ............MISSING baseline Comparing nggps2d.tile6.nc ............MISSING baseline Comparing nggps3d.tile1.nc ............MISSING baseline Comparing nggps3d.tile2.nc ............MISSING baseline Comparing nggps3d.tile3.nc ............MISSING baseline Comparing nggps3d.tile4.nc ............MISSING baseline Comparing nggps3d.tile5.nc ............MISSING baseline Comparing nggps3d.tile6.nc ............MISSING baseline Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.tile1.nc .........OK Comparing RESTART/fv_core.res.tile2.nc .........OK Comparing RESTART/fv_core.res.tile3.nc .........OK Comparing RESTART/fv_core.res.tile4.nc .........OK Comparing RESTART/fv_core.res.tile5.nc .........OK Comparing RESTART/fv_core.res.tile6.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile1.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile2.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile3.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile4.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile5.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile6.nc .........OK Comparing RESTART/fv_tracer.res.tile1.nc .........OK Comparing RESTART/fv_tracer.res.tile2.nc .........OK Comparing RESTART/fv_tracer.res.tile3.nc .........OK Comparing RESTART/fv_tracer.res.tile4.nc .........OK Comparing RESTART/fv_tracer.res.tile5.nc .........OK Comparing RESTART/fv_tracer.res.tile6.nc .........OK Comparing RESTART/sfc_data.tile1.nc .........OK Comparing RESTART/sfc_data.tile2.nc .........OK Comparing RESTART/sfc_data.tile3.nc .........OK Comparing RESTART/sfc_data.tile4.nc .........OK Comparing RESTART/sfc_data.tile5.nc .........OK Comparing RESTART/sfc_data.tile6.nc .........OK Test 03 FAIL baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_restart working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_restart Checking test 04 results .... Comparing atmos_4xdaily.tile1.nc .........OK Comparing atmos_4xdaily.tile2.nc .........OK Comparing atmos_4xdaily.tile3.nc .........OK Comparing atmos_4xdaily.tile4.nc .........OK Comparing atmos_4xdaily.tile5.nc .........OK Comparing atmos_4xdaily.tile6.nc .........OK Comparing nggps2d.tile1.nc ............MISSING baseline Comparing nggps2d.tile2.nc ............MISSING baseline Comparing nggps2d.tile3.nc ............MISSING baseline Comparing nggps2d.tile4.nc ............MISSING baseline Comparing nggps2d.tile5.nc ............MISSING baseline Comparing nggps2d.tile6.nc ............MISSING baseline Comparing nggps3d.tile1.nc ............MISSING baseline Comparing nggps3d.tile2.nc ............MISSING baseline Comparing nggps3d.tile3.nc ............MISSING baseline Comparing nggps3d.tile4.nc ............MISSING baseline Comparing nggps3d.tile5.nc ............MISSING baseline Comparing nggps3d.tile6.nc ............MISSING baseline Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.tile1.nc .........OK Comparing RESTART/fv_core.res.tile2.nc .........OK Comparing RESTART/fv_core.res.tile3.nc .........OK Comparing RESTART/fv_core.res.tile4.nc .........OK Comparing RESTART/fv_core.res.tile5.nc .........OK Comparing RESTART/fv_core.res.tile6.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile1.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile2.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile3.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile4.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile5.nc .........OK Comparing RESTART/fv_srf_wnd.res.tile6.nc .........OK Comparing RESTART/fv_tracer.res.tile1.nc .........OK Comparing RESTART/fv_tracer.res.tile2.nc .........OK Comparing RESTART/fv_tracer.res.tile3.nc .........OK Comparing RESTART/fv_tracer.res.tile4.nc .........OK Comparing RESTART/fv_tracer.res.tile5.nc .........OK Comparing RESTART/fv_tracer.res.tile6.nc .........OK Comparing RESTART/sfc_data.tile1.nc .........OK Comparing RESTART/sfc_data.tile2.nc .........OK Comparing RESTART/sfc_data.tile3.nc .........OK Comparing RESTART/sfc_data.tile4.nc .........OK Comparing RESTART/sfc_data.tile5.nc .........OK Comparing RESTART/sfc_data.tile6.nc .........OK Test 04 FAIL baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_read_inc working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_read_inc Checking test 05 results .... Comparing atmos_4xdaily.tile1.nc ............NOT OK Comparing atmos_4xdaily.tile2.nc ............NOT OK Comparing atmos_4xdaily.tile3.nc ............NOT OK Comparing atmos_4xdaily.tile4.nc ............NOT OK Comparing atmos_4xdaily.tile5.nc ............NOT OK Comparing atmos_4xdaily.tile6.nc ............NOT OK Comparing nggps2d.tile1.nc ............MISSING baseline Comparing nggps2d.tile2.nc ............MISSING baseline Comparing nggps2d.tile3.nc ............MISSING baseline Comparing nggps2d.tile4.nc ............MISSING baseline Comparing nggps2d.tile5.nc ............MISSING baseline Comparing nggps2d.tile6.nc ............MISSING baseline Comparing nggps3d.tile1.nc ............MISSING baseline Comparing nggps3d.tile2.nc ............MISSING baseline Comparing nggps3d.tile3.nc ............MISSING baseline Comparing nggps3d.tile4.nc ............MISSING baseline Comparing nggps3d.tile5.nc ............MISSING baseline Comparing nggps3d.tile6.nc ............MISSING baseline Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.tile1.nc ............NOT OK Comparing RESTART/fv_core.res.tile2.nc ............NOT OK Comparing RESTART/fv_core.res.tile3.nc ............NOT OK Comparing RESTART/fv_core.res.tile4.nc ............NOT OK Comparing RESTART/fv_core.res.tile5.nc ............NOT OK Comparing RESTART/fv_core.res.tile6.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile1.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile2.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile3.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile4.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile5.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile6.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile1.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile2.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile3.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile4.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile5.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile6.nc ............NOT OK Comparing RESTART/sfc_data.tile1.nc ............NOT OK Comparing RESTART/sfc_data.tile2.nc ............NOT OK Comparing RESTART/sfc_data.tile3.nc ............NOT OK Comparing RESTART/sfc_data.tile4.nc ............NOT OK Comparing RESTART/sfc_data.tile5.nc ............NOT OK Comparing RESTART/sfc_data.tile6.nc ............NOT OK Test 05 FAIL baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_stretched working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_stretched Checking test 06 results .... Comparing atmos_4xdaily.tile1.nc ............NOT OK Comparing atmos_4xdaily.tile2.nc ............NOT OK Comparing atmos_4xdaily.tile3.nc ............NOT OK Comparing atmos_4xdaily.tile4.nc ............NOT OK Comparing atmos_4xdaily.tile5.nc ............NOT OK Comparing atmos_4xdaily.tile6.nc ............NOT OK Comparing nggps2d.tile1.nc ............NOT OK Comparing nggps2d.tile2.nc ............NOT OK Comparing nggps2d.tile3.nc ............NOT OK Comparing nggps2d.tile4.nc ............NOT OK Comparing nggps2d.tile5.nc ............NOT OK Comparing nggps2d.tile6.nc ............NOT OK Comparing nggps3d_4xdaily.tile1.nc ............NOT OK Comparing nggps3d_4xdaily.tile2.nc ............NOT OK Comparing nggps3d_4xdaily.tile3.nc ............NOT OK Comparing nggps3d_4xdaily.tile4.nc ............NOT OK Comparing nggps3d_4xdaily.tile5.nc ............NOT OK Comparing nggps3d_4xdaily.tile6.nc ............NOT OK Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.tile1.nc ............NOT OK Comparing RESTART/fv_core.res.tile2.nc ............NOT OK Comparing RESTART/fv_core.res.tile3.nc ............NOT OK Comparing RESTART/fv_core.res.tile4.nc ............NOT OK Comparing RESTART/fv_core.res.tile5.nc ............NOT OK Comparing RESTART/fv_core.res.tile6.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile1.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile2.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile3.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile4.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile5.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile6.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile1.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile2.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile3.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile4.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile5.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile6.nc ............NOT OK Comparing RESTART/sfc_data.tile1.nc ............NOT OK Comparing RESTART/sfc_data.tile2.nc ............NOT OK Comparing RESTART/sfc_data.tile3.nc ............NOT OK Comparing RESTART/sfc_data.tile4.nc ............NOT OK Comparing RESTART/sfc_data.tile5.nc ............NOT OK Comparing RESTART/sfc_data.tile6.nc ............NOT OK Test 06 FAIL baseline dir = /scratch4/NCEPDEV/nems/noscrub/emc.nemspara/RT/NEMSfv3gfs/trunk-20170714/fv3_stretched_nest working dir = /scratch4/GFDL/gfdlscr/Rusty.Benson/Rusty_tic20//stmp1/Rusty.Benson/FV3_RT/rt_115549/fv3_stretched_nest Checking test 07 results .... Comparing atmos_4xdaily.tile1.nc ............NOT OK Comparing atmos_4xdaily.tile2.nc ............NOT OK Comparing atmos_4xdaily.tile3.nc ............NOT OK Comparing atmos_4xdaily.tile4.nc ............NOT OK Comparing atmos_4xdaily.tile5.nc ............NOT OK Comparing atmos_4xdaily.tile6.nc ............NOT OK Comparing atmos_4xdaily.nest02.nc ............NOT OK Comparing nggps2d.tile1.nc ............NOT OK Comparing nggps2d.tile2.nc ............NOT OK Comparing nggps2d.tile3.nc ............NOT OK Comparing nggps2d.tile4.nc ............NOT OK Comparing nggps2d.tile5.nc ............NOT OK Comparing nggps2d.tile6.nc ............NOT OK Comparing nggps2d.nest02.nc ............NOT OK Comparing nggps3d_4xdaily.tile1.nc ............NOT OK Comparing nggps3d_4xdaily.tile2.nc ............NOT OK Comparing nggps3d_4xdaily.tile3.nc ............NOT OK Comparing nggps3d_4xdaily.tile4.nc ............NOT OK Comparing nggps3d_4xdaily.tile5.nc ............NOT OK Comparing nggps3d_4xdaily.tile6.nc ............NOT OK Comparing nggps3d_4xdaily.nest02.nc ............NOT OK Comparing RESTART/coupler.res .........OK Comparing RESTART/fv_core.res.nc .........OK Comparing RESTART/fv_core.res.nest02.nc .........OK Comparing RESTART/fv_BC_ne.res.nest02.nc ............NOT OK Comparing RESTART/fv_BC_sw.res.nest02.nc ............NOT OK Comparing RESTART/fv_core.res.tile1.nc ............NOT OK Comparing RESTART/fv_core.res.tile2.nc ............NOT OK Comparing RESTART/fv_core.res.tile3.nc ............NOT OK Comparing RESTART/fv_core.res.tile4.nc ............NOT OK Comparing RESTART/fv_core.res.tile5.nc ............NOT OK Comparing RESTART/fv_core.res.tile6.nc ............NOT OK Comparing RESTART/fv_core.res.nest02.tile7 ............MISSING file Comparing RESTART/fv_srf_wnd.res.tile1.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile2.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile3.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile4.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile5.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.tile6.nc ............NOT OK Comparing RESTART/fv_srf_wnd.res.nest02.tile7.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile1.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile2.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile3.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile4.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile5.nc ............NOT OK Comparing RESTART/fv_tracer.res.tile6.nc ............NOT OK Comparing RESTART/fv_tracer.res.nest02.tile7.nc ............NOT OK Comparing RESTART/phy_data.tile1.nc ............NOT OK Comparing RESTART/phy_data.tile2.nc ............NOT OK Comparing RESTART/phy_data.tile3.nc ............NOT OK Comparing RESTART/phy_data.tile4.nc ............NOT OK Comparing RESTART/phy_data.tile5.nc ............NOT OK Comparing RESTART/phy_data.tile6.nc ............NOT OK Comparing RESTART/phy_data.nest02.tile7.nc ............NOT OK Comparing RESTART/sfc_data.tile1.nc ............NOT OK Comparing RESTART/sfc_data.tile2.nc ............NOT OK Comparing RESTART/sfc_data.tile3.nc ............NOT OK Comparing RESTART/sfc_data.tile4.nc ............NOT OK Comparing RESTART/sfc_data.tile5.nc ............NOT OK Comparing RESTART/sfc_data.tile6.nc ............NOT OK Comparing RESTART/sfc_data.nest02.tile7.nc ............NOT OK Test 07 FAIL FAILED TESTS: Test fv3_control failed Test fv3_decomp failed Test fv3_2threads failed Test fv3_restart failed Test fv3_read_inc failed Test fv3_stretched failed Test fv3_stretched_nest failed REGRESSION TEST FAILED Fri Sep 8 05:20:41 UTC 2017