Fix weights generation for tripole->rectilinear output of ocean and ice

[DeniseWorthen]

DESCRIPTION OF CHANGES:

When generating rectilinear destination grids, use lat_typ=cap in order to obtain grid from -90:90. Also, return an optional RouteHandle (un-used) when generating the ESMF Weights. This creates a weights file which properly generates values along the tripole seam.

TESTS CONDUCTED:

If there are changes to the build or source code, the tests below must be conducted. Contact a repository manager if you need assistance.

• Compile branch on all Tier 1 machines using Intel (Orion, Jet, Hera and WCOSS2).
• Compile branch on Hera using GNU. Done on Hera.
• Compile branch in 'Debug' mode on WCOSS2. Done on Cactus using ef2aae1.
• Run unit tests locally on any Tier 1 machine. Done on Hera using ef2aae1.
• Run relevant consistency tests locally on all Tier 1 machine.
  Describe any additional tests performed.

New Baselines have been generated on Hera,Jet,Orion and WCOSS2. On hera, the new baseline was used for comparison and all tests passed.

ISSUE:

• fixes in cpld_gridgen, fix postwgts generation  #857
• fixes switch MOM6_FIXDIR used by cpld_gridgen utility #864

[DeniseWorthen]

@GeorgeGayno-NOAA I'm not sure why I am failing the CI, since I see no differences with develop (my develop branch also fails CI).

Also, I'm not sure about some of the new testing required. I've run on Hera and the cpld_gridgen tests pass. I can run on the other platforms, but I'm not sure how to do the additional compiles and the unit tests. I thought the unit tests were automatic?

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA I'm not sure why I am failing the CI, since I see no differences with develop (my develop branch also fails CI).

All four tests are passing. I see the green checkmark. What was the error?

Also, I'm not sure about some of the new testing required. I've run on Hera and the cpld_gridgen tests pass. I can run on the other platforms, but I'm not sure how to do the additional compiles and the unit tests. I thought the unit tests were automatic?

Complete as many of the tests as you can (and check off the appropriate 'boxes' above). I can help with the remaining tests.

[DeniseWorthen]

Regarding the error, I see a red X for both the develop branch and this one saying that 'Clang macOS Build and Test / ufs_utils (push) Failing after 3m'. The failure is

/Users/runner/work/UFS_UTILS/UFS_UTILS/ufs_utils/sorc/fre-nctools.fd/shared_lib/create_xgrid.c:1602:6: error: implicit declaration of function 'insertIntersect' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
            insertIntersect(grid1List, intersect[0], intersect[1], intersect[2], 0.0, u2, inbound, p1_1[0], p1_1[1], p1_1[2]);

I'll test on the remaining platforms and make sure all are B4B. Thanks

[DeniseWorthen]

@GeorgeGayno-NOAA I'd like to modify this PR slightly to allow the generation of postweights for the 5-deg tripole (to a 5-deg rectlinear) and then from each of the existing tripoles to the 5-deg rectilinear. This should have no impact on baselines because we do not keep the weights files in the baseline.

We stage the 5-deg ocean fix data only on Hera (at /scratch1/NCEPDEV/global/glopara/fix/mom6/20220805/500/). I'd need that input data across all platforms instead. Would that be OK, and how do I do that?

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA I'd like to modify this PR slightly to allow the generation of postweights for the 5-deg tripole (to a 5-deg rectlinear) and then from each of the existing tripoles to the 5-deg rectilinear. This should have no impact on baselines because we do not keep the weights files in the baseline.

We stage the 5-deg ocean fix data only on Hera (at /scratch1/NCEPDEV/global/glopara/fix/mom6/20220805/500/). I'd need that input data across all platforms instead. Would that be OK, and how do I do that?

We can ask @KateFriedman-NOAA or @WalterKolczynski-NOAA for assistance hosting data to the 'fixed' directory.

[DeniseWorthen]

I've added the code and script changes required for the 5deg case. Once the required files are in place on the remaining platforms, I will be able to generate new baseline files for 500. No other baseline changes but the regression test script will fail with missing files until the 5deg baseline is in place.

[KateFriedman-NOAA]

We stage the 5-deg ocean fix data only on Hera (at /scratch1/NCEPDEV/global/glopara/fix/mom6/20220805/500/). I'd need that input data across all platforms instead. Would that be OK, and how do I do that?

@DeniseWorthen These fix files:

-bash-4.2$ ll /scratch1/NCEPDEV/global/glopara/fix/mom6/20220805/500/
total 148
-rw-r--r-- 1 role.glopara global   113 Jul 14  2022 MOM_override
-rw-r--r-- 1 role.glopara global  6642 Jul 14  2022 hycom1_25.nc
-rw-r--r-- 1 role.glopara global   745 Jul 14  2022 layer_coord25.nc
-rw-r--r-- 1 role.glopara global 74704 Jul 14  2022 ocean_hgrid.nc
-rw-r--r-- 1 role.glopara global  9982 Jul 14  2022 ocean_mask.nc
-rw-r--r-- 1 role.glopara global 15861 Jul 14  2022 ocean_mosaic.nc
-rw-r--r-- 1 role.glopara global 22403 Jul 14  2022 ocean_topog.nc
-rw-r--r-- 1 role.glopara global  1722 Aug  7 19:17 oceanda_zgrid_25L.nc

...are already synced onto the supported platforms within the developmental fix sets we maintain on them. See the following locations on the other platforms that are listed in the global-workflow link script: https://github.com/NOAA-EMC/global-workflow/blob/develop/sorc/link_workflow.sh#L58
Example for WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/FIX/fix/mom6/20220805/500/

Let me know if I'm misunderstanding your statement or if you have new files to get added. If you have new files, please open a "Fix File Update" issue in global-workflow: https://github.com/NOAA-EMC/global-workflow/issues/new/choose

[DeniseWorthen]

Thanks @KateFriedman-NOAA On hera, for example, the regression test script for the UFS-UTILS cpld_gridgen utility points to

/scratch1/NCEPDEV/nems/role.ufsutils/ufs_utils/reg_tests/cpld_gridgen/fix_mom6/

Is the right solution to actually have that MOM6_FIXDIR point to the various glopara/fix directories?

[KateFriedman-NOAA]

Is the right solution to actually have that MOM6_FIXDIR point to the various glopara/fix directories?

If the contents of the glopara/fix mom6 directories matches that ufs_utils set, then yes. Let us know if you need any additional fix files added to the global group space sets, or updated. A "Fix File Update" issue in global-workflow covers adds, removals, changes/updates, etc.".

[DeniseWorthen]

OK, great. I'll check that they're identical (they should be) and make the change in the regression test script.

[DeniseWorthen]

I've created an issue #864. As it turns out, the mom6 fix directory currently used by the ufs-utils regression test does not contain one of the required files. This file is the the All_edits.nc file used by the 1/4deg configuration. Because of this, the ocean depth field in the tripole.mx025.nc file checked by the regression test will not be B4B.

I've added a check for these files existing and confirmed that it catches the missing files in the current mom6-fix directory.

[DeniseWorthen]

I'm going to merge in the fix for the MOM6 Fix directory and generate new baseline on all platforms. The 050 and 100 case will be identical and I will generate a new 500 baseline.

[DeniseWorthen]

@GeorgeGayno-NOAA New baselines have been generated on hera,jet and wcoss2. I will do orion when it is back from maintenance. Here are the locations:

jet: /lfs4/HFIP/h-nems/Denise.Worthen/CPLD_GRIDGEN/BASELINE
hera: /scratch1/NCEPDEV/stmp4/Denise.Worthen/CPLD_GRIDGEN/BASELINE
wcoss2: /lfs/h2/emc/stmp/denise.worthen/CPLD_GRIDGEN/BASELINE

[DeniseWorthen]

Orion baseline is located at /work/noaa/stmp/dworthen/CPLD_GRIDGEN/BASELINE/

[DeniseWorthen]

@GeorgeGayno-NOAA I was able to compile with GNU on hera, but the job failed because it couldn't find ncremap. I compiled by manually setting the compiler in rt.sh

diff --git a/reg_tests/cpld_gridgen/rt.sh b/reg_tests/cpld_gridgen/rt.sh
index 49f35396..1021612f 100755
--- a/reg_tests/cpld_gridgen/rt.sh
+++ b/reg_tests/cpld_gridgen/rt.sh
@@ -114,7 +114,8 @@ MEM_dflt="--mem=12g"
 export MEM=$MEM_dflt

 cd $PATHRT
-export compiler=${compiler:-intel}
+#export compiler=${compiler:-intel}
+export compiler=gnu

[GeorgeGayno-NOAA]

Orion baseline is located at /work/noaa/stmp/dworthen/CPLD_GRIDGEN/BASELINE/

Your new baseline files have been placed on all machines:

orion: /work/noaa/nems/role-nems/ufs_utils/reg_tests/cpld_gridgen/baseline_data
hera: /scratch1/NCEPDEV/nems/role.ufsutils/ufs_utils/reg_tests/cpld_gridgen/baseline_data
jet: /lfs4/HFIP/hfv3gfs/emc.nemspara/role.ufsutils/ufs_utils/reg_tests/cpld_gridgen/baseline_data
wcoss2:/lfs/h2/emc/nems/noscrub/emc.nems/UFS_UTILS/reg_tests/cpld_gridgen/baseline_data

They are indicated by the .new in the directory name.

[DeniseWorthen]

Thanks. What is my next step? Do I need to run the verification on each platform against the .new baseline?

[GeorgeGayno-NOAA]

Thanks. What is my next step? Do I need to run the verification on each platform against the .new baseline?

Yes, please do. Should I link the .new directories to the proper name for your tests? Ex ln -fs 025.new 025

[DeniseWorthen]

I'm able to run against the .new directories by setting in rt.sh

-    BASELINE=$BASELINE_ROOT/$TEST_NAME
+    BASELINE=$BASELINE_ROOT/$TEST_NAME.new

[DeniseWorthen]

@GeorgeGayno-NOAA I've posted the RT logs for all the platforms against the .new directories.

[GeorgeGayno-NOAA]

@DeniseWorthen - I checked off all the test boxes. Can we merge? There are no fixed files that need to be baselined, right?

[DeniseWorthen]

@GeorgeGayno-NOAA I'm not sure what you're asking? The new baselines now contain the ESMF weights for post. I was planning on making a fix-file request to g-w to have those added.

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA I'm not sure what you're asking? The new baselines now contain the ESMF weights for post. I was planning on making a fix-file request to g-w to have those added.

Understood. Will approve and merge.