Fixes for clang16, gcc14.2, HIP/AMD #1006

valassi · 2024-09-18T08:01:02Z

This is a WIP PR with fixes for clang16, gcc14.2, HIP/AMD

work around for FPE on HIP ROCM6.0 clang Crash (FPE) in check_hip.exe on LUMI #1003 (and other minor issues on LUMI Test the latest master on AMD GPUs on LUMI with ROCm 6.0 #998)
work around for FPE on clang 16 FPEs in check.exe with clang16 and clang17 #1005
fix cxtype_ref on gcc14.2 cxtype_ref problem for some gcc versions #1004

I will run some tests like this, then if they are ok this can probably be merged, even if there are certainly other similar issues to be fixed

…th to libamdhip64 madgraph5#998 Also fix the LUMI setup to solve a second issue (move from 23.09 to 24.03) module load LUMI/24.03 partition/G module load cpeGNU/24.03 export CC="cc --cray-bypass-pkgconfig -craype-verbose" export CXX="CC --cray-bypass-pkgconfig -craype-verbose" export FC="ftn --cray-bypass-pkgconfig -craype-verbose -ffixed-line-length-132" (I checked that gg_tt.mad is regenerated as expected)

…e -hip is no longer available

…his identical to -nocuda for the moment (common random)

…runtime (1) Build tests on login node (~2h) ./tput/allTees.sh -makeonly STARTED AT Mon 16 Sep 2024 08:41:05 PM EEST ./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean -makeonly ENDED(1) AT Mon 16 Sep 2024 09:17:11 PM EEST [Status=1] ./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean -makeonly ENDED(2) AT Mon 16 Sep 2024 09:30:48 PM EEST [Status=0] ./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean -makeonly ENDED(3) AT Mon 16 Sep 2024 09:33:43 PM EEST [Status=1] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst -makeonly ENDED(4) AT Mon 16 Sep 2024 09:33:51 PM EEST [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -curhst -makeonly ENDED(5) AT Mon 16 Sep 2024 09:34:00 PM EEST [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -common -makeonly ENDED(6) AT Mon 16 Sep 2024 09:34:09 PM EEST [Status=0] ./tput/teeThroughputX.sh -mix -hrd -makej -susyggtt -susyggt1t1 -smeftggtttt -heftggbb -makeclean -makeonly ENDED(7) AT Mon 16 Sep 2024 09:59:55 PM EEST [Status=0] (2) Step 2 - run tests on worker nodes (~1h) ./tput/allTees.sh -hip STARTED AT Tue 17 Sep 2024 08:35:08 AM EEST ./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean -nocuda ENDED(1) AT Tue 17 Sep 2024 09:08:52 AM EEST [Status=2] ./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean -nocuda ENDED(2) AT Tue 17 Sep 2024 09:12:28 AM EEST [Status=2] ./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean -nocuda ENDED(3) AT Tue 17 Sep 2024 09:18:56 AM EEST [Status=2] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst -nocuda ENDED(4) AT Tue 17 Sep 2024 09:19:30 AM EEST [Status=2] SKIP './tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -common -nocuda' ENDED(5) AT Tue 17 Sep 2024 09:19:30 AM EEST [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -common -nocuda ENDED(6) AT Tue 17 Sep 2024 09:20:03 AM EEST [Status=2] ./tput/teeThroughputX.sh -mix -hrd -makej -susyggtt -susyggt1t1 -smeftggtttt -heftggbb -makeclean -nocuda ENDED(7) AT Tue 17 Sep 2024 09:26:15 AM EEST [Status=2]

Revert "[amd] rerun 96 tput tests on LUMI - many issues at build time and at runtime" This reverts commit d12d08a.

…nd improve error handling

…graph5#1003 by disabling SIMD in C++ objects for HIP builds - it does not help, will revert

Revert "[amd] in gg_tt.mad cudacpp.mk, try to work around the HIP crashes madgraph5#1003 by disabling SIMD in C++ objects for HIP builds - it does not help, will revert" This reverts commit 2fc102767ecc6ae2e95770f4cff18e5c08d31fc1.

…h5#1003 by disabling SIMD in C++ objects built with hipcc - it also does not help, will revert

Revert "[amd] in gg_tt.mad cudacpp.mk, try to work around HIP crashes madgraph5#1003 by disabling SIMD in C++ objects built with hipcc - it also does not help, will revert" This reverts commit 1e225fd7068eb0c67377f55c7e910af945a4d963.

…adgraph5#1003 by adding volatile - it does not work, will revert

Revert "[amd] in gg_tt.mad EventStatistics.h, try to work around HIP crashes madgraph5#1003 by adding volatile - it does not work, will revert" This reverts commit e2591da7b159b6d133a7cff7a4b583a8ad34d563.

…h5#1003 by printing out sum.nevtOK() - this avoids teh crash but is not practical, will revert

Revert "[amd] in gg_tt.mad EventStatistics.h, work around HIP crashes madgraph5#1003 by printing out sum.nevtOK() - this avoids teh crash but is not practical, will revert" This reverts commit 725dae88d89a61d005a0031c9462fe95f4ec6728.

madgraph5#1003 on hipcc by disabling optimizations for operator+=

…n const and non-const operator[] in cxtype_v (fix build error madgraph5#1004 on gcc14.2)

…e_ref is a const reference to two non-const fp variables

…rash madgraph5#1005 on clang16 by disabling optimizations for operator+= This extends to any clang the previous workaround for madgraph5#1003 which had been defined only for HIP clang

…#1005 and for gcc142 cxtype_ref madgraph5#1003

STARTED AT Wed Sep 18 10:03:30 AM CEST 2024 ./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean ENDED(1) AT Wed Sep 18 12:28:45 PM CEST 2024 [Status=0] ./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean ENDED(2) AT Wed Sep 18 12:49:20 PM CEST 2024 [Status=0] ./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean ENDED(3) AT Wed Sep 18 12:58:32 PM CEST 2024 [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst ENDED(4) AT Wed Sep 18 01:01:21 PM CEST 2024 [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -curhst ENDED(5) AT Wed Sep 18 01:04:08 PM CEST 2024 [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -common ENDED(6) AT Wed Sep 18 01:07:00 PM CEST 2024 [Status=0] ./tput/teeThroughputX.sh -mix -hrd -makej -susyggtt -susyggt1t1 -smeftggtttt -heftggbb -makeclean ENDED(7) AT Wed Sep 18 01:38:02 PM CEST 2024 [Status=0]

… expected STARTED AT Wed Sep 18 01:38:02 PM CEST 2024 (SM tests) ENDED(1) AT Wed Sep 18 05:31:59 PM CEST 2024 [Status=0] (BSM tests) ENDED(1) AT Wed Sep 18 05:42:22 PM CEST 2024 [Status=0] 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_heftggbb_mad/log_heftggbb_mad_d_inl0_hrd0.txt 1 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_heftggbb_mad/log_heftggbb_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_heftggbb_mad/log_heftggbb_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_smeftggtttt_mad/log_smeftggtttt_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_smeftggtttt_mad/log_smeftggtttt_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_smeftggtttt_mad/log_smeftggtttt_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_susyggt1t1_mad/log_susyggt1t1_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_susyggt1t1_mad/log_susyggt1t1_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_susyggt1t1_mad/log_susyggt1t1_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_susyggtt_mad/log_susyggtt_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_susyggtt_mad/log_susyggtt_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_susyggtt_mad/log_susyggtt_mad_m_inl0_hrd0.txt

valassi · 2024-09-18T15:58:46Z

Hi @oliviermattelaer this is ready for review, can you please have a look?

Summary

in EventStatistics.h, disable optimizations on clang in a function to work around some FPEs (crash only if optimized code... not a fix, but it avoids the problem)
in cudacpp.mk, fix some link issues on LUMI for HIP
in cxtype_ref headers, separate const and non-const operator to avoid build errors with the most recent gcc's
plus codegen and rerun some tests

Thanks

oliviermattelaer · 2024-09-18T19:50:04Z

Sounds reasonable to me. Go for it.

Thanks,

Olivier

…) - all as expected STARTED AT Wed 18 Sep 2024 03:07:46 PM EEST ./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean -nocuda ENDED(1) AT Wed 18 Sep 2024 05:21:56 PM EEST [Status=2] ./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean -nocuda ENDED(2) AT Wed 18 Sep 2024 06:00:57 PM EEST [Status=0] ./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean -nocuda ENDED(3) AT Wed 18 Sep 2024 06:09:10 PM EEST [Status=2] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst -nocuda ENDED(4) AT Wed 18 Sep 2024 06:11:01 PM EEST [Status=0] SKIP './tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -common -nocuda' ENDED(5) AT Wed 18 Sep 2024 06:11:01 PM EEST [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -common -nocuda ENDED(6) AT Wed 18 Sep 2024 06:12:50 PM EEST [Status=0] ./tput/teeThroughputX.sh -mix -hrd -makej -susyggtt -susyggt1t1 -smeftggtttt -heftggbb -makeclean -nocuda ENDED(7) AT Wed 18 Sep 2024 07:30:15 PM EEST [Status=0] ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0_bridge.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_d_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0_bridge.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_d_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0_bridge.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_f_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_f_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0_bridge.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_f_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0_bridge.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_f_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0_bridge.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_m_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_m_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_d_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_d_inl0_hrd0/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd1.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_m_inl0_hrd1/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd1.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_m_inl0_hrd1/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd1.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd1.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_d_inl0_hrd1/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd1.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_d_inl0_hrd1/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd1.txt:ERROR! C++ calculation (C++/GPU) failed ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd1.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_f_inl0_hrd1/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd1.txt:/users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/gq_ttq.mad/SubProcesses/P1_gux_ttxux/build.hip_f_inl0_hrd1/check_hip.exe: Segmentation fault ./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd1.txt:ERROR! C++ calculation (C++/GPU) failed

…or rocgdb on HIP (to debug the memory fault madgraph5#806)

…(heft fail madgraph5#833, skip ggttggg madgraph5#933, gqttq crash madgraph5#806) STARTED AT Wed 18 Sep 2024 09:02:01 PM EEST (SM tests) ENDED(1) AT Wed 18 Sep 2024 11:40:09 PM EEST [Status=0] (BSM tests) ENDED(1) AT Wed 18 Sep 2024 11:48:33 PM EEST [Status=0] 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_m_inl0_hrd0.txt 12 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_d_inl0_hrd0.txt 12 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_f_inl0_hrd0.txt 12 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_m_inl0_hrd0.txt 12 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt 12 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt 12 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_heftggbb_mad/log_heftggbb_mad_d_inl0_hrd0.txt 1 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_heftggbb_mad/log_heftggbb_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_heftggbb_mad/log_heftggbb_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_smeftggtttt_mad/log_smeftggtttt_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_smeftggtttt_mad/log_smeftggtttt_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_smeftggtttt_mad/log_smeftggtttt_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_susyggt1t1_mad/log_susyggt1t1_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_susyggt1t1_mad/log_susyggt1t1_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_susyggt1t1_mad/log_susyggt1t1_mad_m_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_susyggtt_mad/log_susyggtt_mad_d_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_susyggtt_mad/log_susyggtt_mad_f_inl0_hrd0.txt 16 /users/valassia/GPU2024/madgraph4gpu/epochX/cudacpp/tmad/logs_susyggtt_mad/log_susyggtt_mad_m_inl0_hrd0.txt

Revert "[amd] rerun 30 tmad tests on LUMI against AMD GPUs - all as expected (heft fail madgraph5#833, skip ggttggg madgraph5#933, gqttq crash madgraph5#806)" This reverts commit 458b834. Revert "[amd] rerun 96 tput builds and tests on LUMI worker node (small-g 72h) - all as expected" This reverts commit 0c947d1.

valassi · 2024-09-19T08:58:13Z

Sounds reasonable to me. Go for it.

Thanks to you Olivier. Will merge soon.

I just added also some commit logs for LUMI AMD GPUs #998 (and some commented out build options to debug #806 on LUMI)

I will merge when the CI completes.

valassi · 2024-09-19T09:06:56Z

The CI passed. Merging.

valassi added 22 commits September 17, 2024 06:47

[amd] regenerate all processes with fixes for libamdhip64 in cudacpp.mk

64249d2

[amd] in tput/allTees.sh clarify that -cpponly and -nocuda exist whil…

98dfbad

…e -hip is no longer available

[amd] in tput/allTees.sh, on second thought add back -hip, but make t…

c9a9ad9

…his identical to -nocuda for the moment (common random)

[amd] revert 96 tput logs on LUMI

734af0c

Revert "[amd] rerun 96 tput tests on LUMI - many issues at build time and at runtime" This reverts commit d12d08a.

[amd] in tput/throughputX.sh expose FPE crash madgraph5#1003 on HIP a…

32f1cb9

…nd improve error handling

[amd] in gg_tt.mad cudacpp.mk, try to work around the HIP crashes mad…

2f8e348

…graph5#1003 by disabling SIMD in C++ objects for HIP builds - it does not help, will revert

[amd] in gg_tt.mad cudacpp.mk, try to work around HIP crashes madgrap…

f6cca64

…h5#1003 by disabling SIMD in C++ objects built with hipcc - it also does not help, will revert

[amd] in gg_tt.mad EventStatistics.h, try to work around HIP crashes m…

3c8f9ca

…adgraph5#1003 by adding volatile - it does not work, will revert

[amd] in gg_tt.mad EventStatistics.h, revert the previous commit (1)

0a4e76d

Revert "[amd] in gg_tt.mad EventStatistics.h, try to work around HIP crashes madgraph5#1003 by adding volatile - it does not work, will revert" This reverts commit e2591da7b159b6d133a7cff7a4b583a8ad34d563.

[amd] in gg_tt.mad EventStatistics.h, work around HIP crashes madgrap…

f456668

…h5#1003 by printing out sum.nevtOK() - this avoids teh crash but is not practical, will revert

[amd] in gg_tt.mad and CODEGEN EventStatistics.h, work around FPE crash

055795d

madgraph5#1003 on hipcc by disabling optimizations for operator+=

[amd] in gg_tt.mad and CODEGEN EventStatistics.h, fix clang formatting

11bb959

[amd] regenerate all processes with the fix for madgraph5#1003

d07dcbf

[gcc14] in gg_tt.mad and CODEGEN mgOnGpuVectors.h, distinguish betwee…

79f45b6

…n const and non-const operator[] in cxtype_v (fix build error madgraph5#1004 on gcc14.2)

[gcc14] in gg_tt.mad and CODEGEN mgOnGpuCxtypes.h, clarify that cxtyp…

70f40cc

…e_ref is a const reference to two non-const fp variables

[clang] in gg_tt.mad and CODEGEN EventStatistics.h, work around FPE c…

a3b6ab4

…rash madgraph5#1005 on clang16 by disabling optimizations for operator+= This extends to any clang the previous workaround for madgraph5#1003 which had been defined only for HIP clang

[clang] regenerate all processes with fixes for clang16 FPE madgraph5…

da50539

…#1005 and for gcc142 cxtype_ref madgraph5#1003

valassi self-assigned this Sep 18, 2024

valassi marked this pull request as draft September 18, 2024 08:01

This was referenced Sep 18, 2024

cxtype_ref problem for some gcc versions #1004

Closed

Test builds with newer compilers (gcc13, clang16, icpx2023.2...) #734

Open

valassi added 2 commits September 18, 2024 17:50

valassi requested a review from oliviermattelaer September 18, 2024 15:54

valassi marked this pull request as ready for review September 18, 2024 15:54

This was linked to issues Sep 18, 2024

Crash (FPE) in check_hip.exe on LUMI #1003

Closed

cxtype_ref problem for some gcc versions #1004

Closed

FPEs in check.exe with clang16 and clang17 #1005

Closed

valassi added 5 commits September 19, 2024 10:49

[amd] in gq_ttq.mad and CODEGEN cudacpp.mk add optional debug flags f…

1b29900

…or rocgdb on HIP (to debug the memory fault madgraph5#806)

[amd] regenerate all processes (just with some comments in cudacpp.mk)

0ce5d06

valassi linked an issue Sep 19, 2024 that may be closed by this pull request

Test the latest master on AMD GPUs on LUMI with ROCm 6.0 #998

Closed

This was referenced Sep 19, 2024

Test the latest master on AMD GPUs on LUMI with ROCm 6.0 #998

Closed

Crash (FPE) in check_hip.exe on LUMI #1003

Closed

FPEs in check.exe with clang16 and clang17 #1005

Closed

valassi merged commit fe331ed into madgraph5:master Sep 19, 2024
169 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes for clang16, gcc14.2, HIP/AMD #1006

Fixes for clang16, gcc14.2, HIP/AMD #1006

valassi commented Sep 18, 2024

valassi commented Sep 18, 2024

oliviermattelaer commented Sep 18, 2024

valassi commented Sep 19, 2024

valassi commented Sep 19, 2024

Fixes for clang16, gcc14.2, HIP/AMD #1006

Fixes for clang16, gcc14.2, HIP/AMD #1006

Conversation

valassi commented Sep 18, 2024

valassi commented Sep 18, 2024

oliviermattelaer commented Sep 18, 2024

valassi commented Sep 19, 2024

valassi commented Sep 19, 2024