Amesos2: Tests building wrong target on Apple Silicon

[waveman68]

1. Build log: [attach here]
2. Input deck: [attach here]
3. Do this.

Bug Report

@srajama1

Building tests for all selected packages fails, because the tests are compiled for the wrong platform.

Description

Just trying to build a reduced set of Trilinos Packages. It fails from [1719/1772] to [1721/1772].

[1719/1772] Linking CXX executable packages/amesos2/test/solvers/Amesos2_KLU2_UnitTests.exe
FAILED: packages/amesos2/test/solvers/Amesos2_KLU2_UnitTests.exe

[1720/1772] Linking CXX executable packages/amesos2/test/solvers/Amesos2_SolverFactory_UnitTests.exe
FAILED: packages/amesos2/test/solvers/Amesos2_SolverFactory_UnitTests.exe

[1721/1772] Linking CXX executable packages/amesos2/test/solvers/Amesos2_Solver_Test.exe
FAILED: packages/amesos2/test/solvers/Amesos2_Solver_Test.exe

Steps to Reproduce

1. SHA1: GitHub fc55b9c (14.2.0)
2. Configure script: do-configure-trilinos.txt
3. CMakeCache.txt: CMakeCache.txt
4. ninja build: build.ninja.txt
5. CMake config output: CMake-config-log.txt
6. Build output: build.log](https://github.com/trilinos/Trilinos/files/15289305/build.log)

If I add -DAmesos2_ENABLE_TESTS=OFF, the build is successful.

[jhux2]

@trilinos/amesos2

[csiefer2]

@ndellingwood

[ndellingwood]

If Apple Silicon refers to Apple GPUs, that architecture is not yet supported by Kokkos. I don't have access to Apple Silicon for testing in any case

Looking over the build.log and the undefined references, it looks like there could be some missing info in the cmake configure line needed for the superlu_dist tpl, or some issues with the superlu_dist installation itself (assuming superlu_dist and tpl dependencies are supported on Apple Silicon and the dependency installs are ?)?

For reference, here is some configuration info I can share for how I have successfully configured+built Amesos2 with OpenMP backend using gcc/12.3 with superlu_dist/8.1.2, where I used openmpi/4.1.6, openblas/0.3.26 for my blas/lapack libs and parmetis/4.0.3 (32-bit install) TPLs in my Trilinos and superlu_dist builds, hopefully there is something helpful here to use:

BLAS_LIBRARIES="${BLAS_PATH}/lib/libopenblas.a;gfortran;gomp;m"
LAPACK_LIBRARIES="${LAPACK_PATH}/lib/libopenblas.a;gfortran;gomp;m"

PARMETIS_PATH=${TPL_PATH}/parmetis-4.0.3-32bit/install
METIS_PATH=${TPL_PATH}/metis-5.1.0-32bit/install

SUPERLU_DIST_PATH=${TPL_PATH}/superlu_dist-8.1.2-parmetis-32bit/install

cmake \
\
-G Ninja \
-D CMAKE_BUILD_TYPE:STRING=DEBUG \
-D BUILD_SHARED_LIBS:BOOL=OFF \
\
-D TPL_ENABLE_MPI:BOOL=ON \
  -D CMAKE_CXX_COMPILER:FILEPATH="`which mpicxx`" \
  -D CMAKE_C_COMPILER:FILEPATH="`which mpicc`" \
  -D CMAKE_Fortran_COMPILER:FILEPATH="`which mpifort`" \
\
-D TPL_ENABLE_METIS:BOOL=ON \
  -D METIS_INCLUDE_DIRS:PATH="${METIS_PATH}/include" \
  -D METIS_LIBRARY_DIRS:PATH="${METIS_PATH}/lib" \
  -D METIS_LIBRARY_NAMES:STRING="metis" \
-D TPL_ENABLE_ParMETIS:BOOL=ON \
  -D ParMETIS_INCLUDE_DIRS:PATH="${PARMETIS_PATH}/include;${METIS_PATH}/include" \
  -D ParMETIS_LIBRARY_DIRS:PATH="${PARMETIS_PATH}/lib" \
  -D ParMETIS_LIBRARY_NAMES:STRING="parmetis" \
-D TPL_ENABLE_BLAS:STRING=ON \
  -D BLAS_LIBRARY_DIRS:FILEPATH=${BLAS_PATH}/lib \
  -D TPL_BLAS_LIBRARIES:PATH="${BLAS_LIBRARIES}" \
-D TPL_ENABLE_LAPACK:STRING=ON \
  -D LAPACK_INCLUDE_DIRS:FILEPATH="${LAPACK_PATH}/include" \
  -D LAPACK_LIBRARY_DIRS:FILEPATH=${LAPACK_PATH}/lib \
  -D TPL_LAPACK_LIBRARIES:PATH="${LAPACK_LIBRARIES}" \
-D TPL_ENABLE_SuperLUDist:BOOL=ON \
  -D SuperLUDist_INCLUDE_DIRS:PATH="${SUPERLU_DIST_PATH}/include" \
  -D SuperLUDist_LIBRARY_DIRS:PATH="${SUPERLU_DIST_PATH}/lib;${METIS_PATH}/lib/libmetis.a;${PARMETIS_PATH}/lib/libparmetis.a;${BLAS_PATH}/lib/libopenblas.a;gfortran;m" \
\
-D Trilinos_ENABLE_EXAMPLES:BOOL=OFF \
-D Trilinos_ENABLE_TESTS:BOOL=OFF \
-D Trilinos_ENABLE_COMPLEX_DOUBLE:BOOL=ON \
-D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON \
\
-D Trilinos_ENABLE_OpenMP:BOOL=ON \
-D Trilinos_ENABLE_Kokkos:BOOL=ON \
  -D Kokkos_ENABLE_OPENMP:BOOL=ON \
  -D Kokkos_ENABLE_SERIAL:BOOL=ON \
  -D Kokkos_ENABLE_EXAMPLES:BOOL=OFF \
  -D Kokkos_ENABLE_TESTS:BOOL=OFF \
-D Trilinos_ENABLE_Epetra:BOOL=OFF \
-D Trilinos_ENABLE_Tpetra:BOOL=ON \
  -D Tpetra_ENABLE_EXAMPLES:BOOL=OFF \
  -D Tpetra_ENABLE_TESTS:BOOL=OFF \
  -D Tpetra_INST_SERIAL:BOOL=ON \
  -D Tpetra_INST_OPENMP:BOOL=ON \
  -D Tpetra_INST_INT_INT:BOOL=ON \
-D Trilinos_ENABLE_Amesos2:BOOL=ON \
  -D Amesos2_ENABLE_EXAMPLES:BOOL=ON \
  -D Amesos2_ENABLE_TESTS:BOOL=ON \
  -D Amesos2_ENABLE_KLU2:BOOL=ON \
  -D Amesos2_ENABLE_SuperLUDist:BOOL=ON \
${TRILINOS_DIR}

For the Amesos2 superlu_dist unit tests to run and pass, the -D Tpetra_INST_INT_INT:BOOL=ON option is necessary

Here is a sample of the cmake line I used when I configured superlu_dist (and installed, prior to use with Trilinos of course):

cmake \
    -DTPL_ENABLE_COMBBLASLIB=OFF \
    -DTPL_ENABLE_PARMETISLIB=ON \
    -DTPL_PARMETIS_INCLUDE_DIRS="${PARMETIS_PATH}/include;${METIS_PATH}/include" \
    -DTPL_PARMETIS_LIBRARIES="${PARMETIS_PATH}/lib/libparmetis.a;${METIS_PATH}/lib/libmetis.a" \
    -DTPL_ENABLE_BLASLIB=ON \
      -D TPL_BLAS_LIBRARIES="${BLAS_PATH}/lib/libopenblas.a" \
    -DTPL_ENABLE_LAPACKLIB=ON \
      -D TPL_LAPACK_INCLUDE_DIRS="${LAPACK_PATH}/include" \
      -D TPL_LAPACK_LIBRARIES="${LAPACK_PATH}/lib/libopenblas.a" \
    -DCMAKE_INSTALL_PREFIX="$SLUD_INSTALL_DIR" \
    -DCMAKE_C_FLAGS="-std=c99" \
    -DCMAKE_C_COMPILER=mpicc \
    -DCMAKE_CXX_COMPILER=mpicxx \
    -DCMAKE_CXX_FLAGS="-std=c++17" \
    -DBUILD_SHARED_LIBS=OFF \
$SUPERLUDIST_PATH

[waveman68]

Thanks for the input. I didn’t explicitly enable the Apple Silicon GPU. AFAIK (far from being an expert), OpenCL is the only path available for Apple Silicon GPU acceleration and Kokkos is far from alone.

[ndellingwood]

@waveman68 since you are not trying to explicitly run on the Apple Silicon, the arch should not be relevant then to the issues you are experiencing in this case (though I could be wrong). The build log posted showed some undefined references implying some linkage issues with your superlu_dist install; I can't say for certain where the breakdown occurs, but hopefully the configuration info I shared above can help you toward a working build.

Something worth noting, the configuration info I posted was for a build enabling the OpenMP backend; if your system and/or compilers do not support OpenMP (I believe Apple Clang does not), you may need to remove that setting (drop -D Trilinos_ENABLE_OpenMP:BOOL=ON -D Kokkos_ENABLE_OPENMP:BOOL=ON, or set each option to OFF), similarly if using OpenBLAS that is not built with OpenMP support drop the gomp lib from BLAS_LIBRARIES and LAPACK_LIBRARIES.
spack may provide a much easier path forward for handling all the library linkage paths etc., but I have not tried using it with superlu_dist

[waveman68]

@ndellingwood My multiple attempts using spack have all ended in grief on Apple Silicon. I am trying to run on Apple Silicon explicitly.