FTorch Intel Compiler Flags

[backfelix8]

Dear FTorch Community,

I am trying to set up FTorch on a rather large codebase on an HPC for a climate model (using the intel compilers).

Currently, when executing make, I receive the following error messages:
ld: warning: libcuda.so.1, needed by <path>/FTorch_install/lib64/libftorch.so, not found (try using -rpath or -rpath-link) lib/libicon.a(mo_art_linoz.o): In function mo_art_linoz_mp_art_calc_linoz_: <path>/icon-kit/externals/art/chemistry/mo_art_linoz.f90:66: undefined reference to torch_model_load_ <path>/icon-kit/externals/art/chemistry/mo_art_linoz.f90:90: undefined reference to torch_model_forward_ <path>/icon-kit/externals/art/chemistry/mo_art_linoz.f90:93: undefined reference to torch_tensor_to_array_ <path>/icon-kit/externals/art/chemistry/mo_art_linoz.f90:103: undefined reference to torch_delete_ <path>/icon-kit/externals/art/chemistry/mo_art_linoz.f90:104: undefined reference to torch_delete_ <path>/icon-kit/externals/art/chemistry/mo_art_linoz.f90:107: undefined reference to torch_delete_ <path>/FTorch_install/lib64/libftorch.so: undefined reference to std::__throw_bad_array_new_length()@GLIBCXX_3.4.29 make[1]: *** [icon.mk:486: bin/icon] Error 1

They indicate that the import of ftorch in my code is resolved, however, the actual functions are not being detected.

I am using the following compiler flags/directives:
FTORCH_LIBPATH='/<path>/FTorch_install/lib64'
FTORCH_FCFLAGS='-I/<path>/FTorch_install/include/ftorch'
FTORCH_LIBS='-L/<path>/FTorch_install/lib64 -lftorch'
FTORCH_LDFLAGS='-L/<path>/FTorch_install/lib64'

BUILD_ENV="export LD_LIBRARY_PATH=\"${INTEL_ROOT}/lib:${INTEL_ROOT}/compiler/lib/intel64:${MPI_ROOT}/lib64:${CUDA_ROOT}/lib64:${HDF5_ROOT}/lib:${NETCDF_ROOT}/lib:${NETCDFF_ROOT}/lib:${ECCODES_ROOT}/lib64:${FYAML_ROOT}/lib:${MKL_ROOT}/lib/intel64:/usr/lib64:/usr/lib:${FTORCH_LIBPATH}\"; export PATH=\"${HDF5_ROOT}/bin:${NETCDF_ROOT}/bin/:/usr/bin\";"
LDFLAGS="-L${INTEL_ROOT}/lib -L${INTEL_ROOT}/compiler/lib/intel64 -L${MPI_ROOT}/lib64 -L${CUDA_ROOT}/lib64 -L${HDF5_ROOT}/lib -L${NETCDF_ROOT}/lib -L${NETCDFF_ROOT}/lib -L${MKL_ROOT}/lib/intel64 -L${ECCODES_ROOT}/lib64 -L${FYAML_ROOT}/lib ${FTORCH_LDFLAGS}"
LIBS="-Wl,--disable-new-dtags -Wl,--as-needed ${MPI_LIBS} ${XML2_LIBS} ${ZLIB_LIBS} ${ECCODES_LIBS} ${FYAML_LIBS} ${MKL_LIBS} ${NETCDFF_LIBS} ${NETCDF_LIBS} ${HDF5_LIBS} ${FTORCH_LIBS}"
FCFLAGS="-I${INTEL_ROOT}/include -I${MPI_ROOT}/include -I${MPI_FORTRAN_MOD_DIR} -I${HDF5_ROOT}/include -I${NETCDFF_ROOT}/include ${FTORCH_FCFLAGS} -I${ECCODES_ROOT}/include ${ARCH} -g -gdwarf-4 -pc64 -fp-model source"

Do you have any suggestions on what I could try to resolve the error, so that all FTorch functions are imported correctly?

[jatkinson1000]

Hi @backfelix8

A few ideas:

• Check that the ftorch install contains lib64/ as you are using here - on some systems it is created as lib/.
• It also looks like there is a missing cuda library (though this is just a warning). If you need cuda can you import it, and if not you could try using the GPU-only version of libtorch.
• I would have perhaps expected an -lftorch at the end of the LDFLAGS, though I'm not super familiar with the way things are set up on your particular system.

Let me know how that goes, and if no luck we can discuss further or perhaps arrange a call.

I can see that you are trying to couple to ICON, and we have collaborators who have successfully done this in the past, so we might be able to reach out to them.

[jatkinson1000]

@helgehr are you able to advise how to set up a config file to use FTorch with ICON at all?
@backfelix8 is having issues linking during the build as described above.

[TomMelt]

@backfelix8 are you building this locally or on a HPC system?

Would it be possible for you to send a zip of the source so that I could try building it?

[backfelix8]

Hi @TomMelt

I am building ICON-ART on an HPC (HoreKa in Karlsruhe, Germany). Sure, I can try to give you the source code, but I am not completely confident that this will help, as ICON-ART has different configuration files for each venue and I do not know if the error can be reproduced on a different system.

If you still want the code, could you perhaps provide an upload link (or any other preferred form), as the code base is quite large (5.8gb)

[helgehr]

Hello all, its been a while since I built this library but my setup for icon includes these two lines in the Makefile:

FCFLAGS += -I/path_to_ftorch/build/usr/local/include/ftorch
LDFLAGS += -L/path_to_ftorch/build/usr/local/lib64 -lftorch

Otherwise I can confirm the point from @jatkinson1000 that it is important to compile ftorch with the compilers (for fortran, c, and cpp) used in icon.

[backfelix8]

@jatkinson1000 @helgehr @TomMelt

Thank you for your help, I was able to resolve the issue at least with the GNU compilers, so thank you again for mentioning the compilers to me.

The issue was a bit deeper than expected. I was using a torch version on my system that makes use of CUDA. So, while building FTorch with cmake, I was forced to use the GNU compilers in version 12, as version 13 is not compatible with CUDA (I am using module for software versioning, so it is not possible for me to use e.g. gfortran/13 and gcc/12 at the same time). However, the configuration script of ICON only allows GNU/13 to be used, which was causing the issue of non-detectable functions.

The solution was to install the CPU-only version of torch and re-building FTorch with GNU/13, which then allowed the project to compile.

[jatkinson1000]

Thanks @helgehr for your input, and thanks @backfelix8 for letting us know how you resolved it.

I hope you find FTorch useful, and please do get in touch if you need help, have feature requests, or have any scientific outputs you would like us to feature.