@JohnHalleyGotway recommenced this bug fix after discussion on webcall. @jprestop

Authors of bug fix: @HathewayWill

Describe the Problem

• With the update of MET v11 and METplus v5 the compile_MET_all.sh script is broken in a clean environment and missing variable exports.
  -- In compile_MET_all.sh the variable LIB_Z is defined at line 811. However, it is called and referenced earlier in the script causing HDF5 & NETCDF to fail building. While HDF5 will compile and install without zlib, NETCDF requires HDF5 to be built with zlib.
  -- ${LIB_Z} is referenced at line numbers 98, 621, 811, 831.

• Library tar files have been updated and old tar file not archived with version numbers so previous compile_MET_all.sh scripts (ie v10.0.2) will not install properly.
  -- With the update of v11 the netcdf library file in the tar_files.tar changed from a .zip file to a tar.gz file. In the compile_MET_all.sh script from v10.0.2 the script utilizes the old naming structure and compression. Therefore the new tar_files will not work with it.

Expected Behavior

Should be able to run compile_MET_all.sh on a clean environment and install all the required libraries and software packages.

Environment

Describe your runtime environment:
*1. Machine: Linux Workstation
*2. OS: Ubuntu 22.04.1
*3.
-hdf5 1.12.2
-libpng 1.6.37
-zlib 1.2.11
-jasper 1.900.1
-netcdf 4.7.4

To Reproduce

Run the below code in clean environment terminal:

sudo apt -y update
sudo apt -y upgrade
sudo apt -y install python3 python3-dev emacs flex bison libpixman-1-dev libjpeg-dev pkg-config libpng-dev unzip python2 python2-dev python3-pip pipenv gcc gfortran g++ libtool automake autoconf make m4 default-jre default-jdk csh ksh git libncurses5 libncurses6 mlocate pkg-config build-essential curl libcurl4-openssl-dev byacc flex

 pip3 install python-dateutil==2.8

mkdir $HOME/WRF
export WRF_FOLDER=$HOME/WRF
 mkdir $WRF_FOLDER/MET-11.0.0
 mkdir $WRF_FOLDER/MET-11.0.0/Downloads
 mkdir $WRF_FOLDER/METplus-5.0.0
 mkdir $WRF_FOLDER/METplus-5.0.0/Downloads

 cd $WRF_FOLDER/MET-11.0.0/Downloads
 wget -c -4 https://raw.githubusercontent.com/dtcenter/MET/main_v11.0/internal/scripts/installation/compile_MET_all.sh
 wget -c -4 https://dtcenter.ucar.edu/dfiles/code/METplus/MET/installation/tar_files.tgz
 wget -c -4 https://github.com/dtcenter/MET/archive/refs/tags/v11.0.0.tar.gz

 cp compile_MET_all.sh $WRF_FOLDER/MET-11.0.0
 tar -xvzf tar_files.tgz -C $WRF_FOLDER/MET-11.0.0
 cp v11.0.0.tar.gz $WRF_FOLDER/MET-11.0.0/tar_files
 cd $WRF_FOLDER/MET-11.0.0

 # Installation of Model Evaluation Tools
 export CC=gcc
 export CXX=g++
 export FC=gfortran
 export F77=gfortran
 export CFLAGS="-fPIC -fPIE -O3"

 cd $WRF_FOLDER/MET-11.0.0
 export GCC_VERSION=$(/usr/bin/gcc -dumpfullversion | awk '{print$1}')
 export GFORTRAN_VERSION=$(/usr/bin/gfortran -dumpfullversion | awk '{print$1}')
 export GPLUSPLUS_VERSION=$(/usr/bin/g++ -dumpfullversion | awk '{print$1}')

 export GCC_VERSION_MAJOR_VERSION=$(echo $GCC_VERSION | awk -F. '{print $1}')
 export GFORTRAN_VERSION_MAJOR_VERSION=$(echo $GFORTRAN_VERSION | awk -F. '{print $1}')
 export GPLUSPLUS_VERSION_MAJOR_VERSION=$(echo $GPLUSPLUS_VERSION | awk -F. '{print $1}')

 export version_10="10"

 if [ $GCC_VERSION_MAJOR_VERSION -lt $version_10 ] || [ $GFORTRAN_VERSION_MAJOR_VERSION -lt $version_10 ] || [ $GPLUSPLUS_VERSION_MAJOR_VERSION -lt $version_10 ]
 then
   sed -i 's/-fno-second-underscore -fallow-argument-mismatch/-fno-second-underscore -Wno-argument-mismatch/g' compile_MET_all.sh
 fi
 export PYTHON_VERSION=$(/usr/bin/python3 -V 2>&1|awk '{print $2}')
 export PYTHON_VERSION_MAJOR_VERSION=$(echo $PYTHON_VERSION | awk -F. '{print $1}')
 export PYTHON_VERSION_MINOR_VERSION=$(echo $PYTHON_VERSION | awk -F. '{print $2}')
 export PYTHON_VERSION_COMBINED=$PYTHON_VERSION_MAJOR_VERSION.$PYTHON_VERSION_MINOR_VERSION

 export FC=/usr/bin/gfortran
 export F77=/usr/bin/gfortran
 export F90=/usr/bin/gfortran
 export gcc_version=$(gcc -dumpfullversion)
 export TEST_BASE=$WRF_FOLDER/MET-11.0.0
 export COMPILER=gnu_$gcc_version
 export MET_SUBDIR=${TEST_BASE}
 export MET_TARBALL=v11.0.0.tar.gz
 export USE_MODULES=FALSE
 export MET_PYTHON=/usr
 export MET_PYTHON_CC=-I${MET_PYTHON}/include/python${PYTHON_VERSION_COMBINED}
 export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python${PYTHON_VERSION_COMBINED}/config-${PYTHON_VERSION_COMBINED}-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython${PYTHON_VERSION_COMBINED}\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lm
 export SET_D64BIT=FALSE

 chmod 775 compile_MET_all.sh
./compile_MET_all.sh |& tee compile_MET_all.log

These errors occur and we have attached the relevant log files:

compile_MET_all.log
zlib_make.log
zlib_configure.log
zlib_make_install.log
HDF5_config.log
HDF5_configure.log
HDF5_make_install.log
NETCDF_config.log
NETCDF_configure.log

Tar file issue:

Looking at the source link for the tar files (https://dtcenter.ucar.edu/dfiles/code/METplus/MET/installation/) the files were updated on 2022-12-21 17:42 and there is no archive for the old files.

Comparing the two compile_MET_all files (below) and looking at the commits in the change log ( 21d54bf) you can see that compile_MET_all.sh has changed. The previous version of compile_MET_all.sh (v10.0.2) will not work with the new tar file package.
compile_MET_all_v10.0.2.sh.txt
compile_MET_all_v11.0.0.sh.txt

In the tar file package the version numbers are:

Suggested Fixes

• For compile_MET_all.sh there are two options to fix the issue:
  -- In the netcdf compile (Line 621) , export LIB_Z is not defined; changing it to ${LIB_DIR}/lib
  -- In the zlib compile (Line 425) add new line at the bottom of the if statement with export LIB_Z=${LIB_DIR}/lib

• For the tar file packages
  -- Create a separate tar file package for v10.0.2

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Organization level Project for support of the current coordinated release
• Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of main_<Version>.
  Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
• Fix the bug and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into main_<Version>.
  Pull request: bugfix <Issue Number> main_<Version> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Organization level software support Project for the current coordinated release
  Select: Milestone as the next bugfix version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
  Select: Reviewer(s) and Development issue
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Close this issue.