Manually run gc after chunk processing #265
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTargetBaselineTarget resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoBaseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoRuntime information
|
Pull Request Test Coverage Report for Build 5402717282
💛 - Coveralls |
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #265 +/- ##
==========================================
+ Coverage 75.34% 75.79% +0.44%
==========================================
Files 12 12
Lines 1858 1851 -7
==========================================
+ Hits 1400 1403 +3
+ Misses 458 448 -10
☔ View full report in Codecov by Sentry. |
|
@felixcremer you also mentioned having some issues with memory accumulating in one of your RQA runs. If you find some time, maybe you can try this branch? |
|
Just as a bit of a context, this is the error, that I get, when I do not manually call GC.gc() Failing mapcube call wtihout GCpath = "/eodc/private/pangeojulia/germany/E045N021T3_rqatrend_VV_D_thresh_3_year_2016"
tax = Time Axis with 1006 Elements from 2015-07-01T05:33:06 to 2017-07-01T05:40:11
Start of the processing: 2023-06-28T09:38:16.312
Progress: 19%|█████████████████████████████████████████████▍ | ETA: 0:42:58Worker 2 terminated.
Unhandled Task ERROR: EOFError: read end of file
Stacktrace:
[1] (::Base.var"#wait_locked#715")(s::Sockets.TCPSocket, buf::IOBuffer, nb::Int64)
@ Base ./stream.jl:947
[2] unsafe_read(s::Sockets.TCPSocket, p::Ptr{UInt8}, nb::UInt64)
@ Base ./stream.jl:955
[3] unsafe_read
@ ./io.jl:761 [inlined]
[4] unsafe_read(s::Sockets.TCPSocket, p::Base.RefValue{NTuple{4, Int64}}, n::Int64)
@ Base ./io.jl:760
[5] read!
@ ./io.jl:762 [inlined]
[6] deserialize_hdr_raw
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/messages.jl:167 [inlined]
[7] message_handler_loop(r_stream::Sockets.TCPSocket, w_stream::Sockets.TCPSocket, incoming::Bool)
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/process_messages.jl:172
[8] process_tcp_streams(r_stream::Sockets.TCPSocket, w_stream::Sockets.TCPSocket, incoming::Bool)
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/process_messages.jl:133
[9] (::Distributed.var"#103#104"{Sockets.TCPSocket, Sockets.TCPSocket, Bool})()
@ Distributed ./task.jl:514
Progress: 19%|██████████████████████████████████████████████▍ | ETA: 0:44:29ERROR: LoadError: ProcessExitedException(2)
Stacktrace:
[1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
@ Base ./task.jl:920
[2] wait()
@ Base ./task.jl:984
[3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)
@ Base ./condition.jl:130
[4] wait
@ ./condition.jl:125 [inlined]
[5] take_buffered(c::Channel{Any})
@ Base ./channels.jl:457
[6] take!(c::Channel{Any})
@ Base ./channels.jl:451
[7] take!(::Distributed.RemoteValue)
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/remotecall.jl:726
[8] remotecall_fetch(f::Function, w::Distributed.Worker, args::Tuple{UnitRange{Int64}, UnitRange{Int64}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/remotecall.jl:461
[9] remotecall_fetch(f::Function, w::Distributed.Worker, args::Tuple{UnitRange{Int64}, UnitRange{Int64}})
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/remotecall.jl:454
[10] #remotecall_fetch#162
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/remotecall.jl:492 [inlined]
[11] remotecall_fetch(f::Function, id::Int64, args::Tuple{UnitRange{Int64}, UnitRange{Int64}})
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/remotecall.jl:492
[12] remotecall_pool(rc_f::Function, f::Function, pool::WorkerPool, args::Tuple{UnitRange{Int64}, UnitRange{Int64}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/workerpool.jl:126
[13] remotecall_pool
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/workerpool.jl:123 [inlined]
[14] #remotecall_fetch#200
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/workerpool.jl:232 [inlined]
[15] remotecall_fetch
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/workerpool.jl:232 [inlined]
[16] #208#209
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/workerpool.jl:288 [inlined]
[17] #208
@ ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/workerpool.jl:288 [inlined]
[18] (::Base.var"#978#983"{Distributed.var"#208#210"{Distributed.var"#208#209#211"{WorkerPool, ProgressMeter.var"#56#59"{RemoteChannel{Channel{Bool}}, YAXArrays.DAT.var"#fnew#100"{YAXArrays.DAT.var"#108#112"{YAXArrays.DAT.DATConfig{1, 1}}, Future}}}}})(r::Base.RefValue{Any}, args::Tuple{Tuple{UnitRange{Int64}, UnitRange{Int64}}})
@ Base ./asyncmap.jl:100
[19] macro expansion
@ ./asyncmap.jl:234 [inlined]
[20] (::Base.var"#994#995"{Base.var"#978#983"{Distributed.var"#208#210"{Distributed.var"#208#209#211"{WorkerPool, ProgressMeter.var"#56#59"{RemoteChannel{Channel{Bool}}, YAXArrays.DAT.var"#fnew#100"{YAXArrays.DAT.var"#108#112"{YAXArrays.DAT.DATConfig{1, 1}}, Future}}}}}, Channel{Any}, Nothing})()
@ Base ./task.jl:514
Stacktrace:
[1] (::Base.var"#988#990")(x::Task)
@ Base ./asyncmap.jl:177
[2] foreach(f::Base.var"#988#990", itr::Vector{Any})
@ Base ./abstractarray.jl:3073
[3] maptwice(wrapped_f::Function, chnl::Channel{Any}, worker_tasks::Vector{Any}, c::DiskArrays.GridChunks{2})
@ Base ./asyncmap.jl:177
[4] wrap_n_exec_twice
@ ./asyncmap.jl:153 [inlined]
[5] #async_usemap#973
@ ./asyncmap.jl:103 [inlined]
[6] async_usemap
@ ./asyncmap.jl:84 [inlined]
[7] #asyncmap#972
@ ./asyncmap.jl:81 [inlined]
[8] asyncmap
@ ./asyncmap.jl:80 [inlined]
[9] pmap(f::Function, p::WorkerPool, c::DiskArrays.GridChunks{2}; distributed::Bool, batch_size::Int64, on_error::Nothing, retry_delays::Vector{Any}, retry_check::Nothing)
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/pmap.jl:126
[10] pmap(f::Function, p::WorkerPool, c::DiskArrays.GridChunks{2})
@ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/pmap.jl:99
[11] macro expansion
@ ~/.julia/packages/ProgressMeter/sN2xr/src/ProgressMeter.jl:1015 [inlined]
[12] macro expansion
@ ./task.jl:476 [inlined]
[13] macro expansion
@ ~/.julia/packages/ProgressMeter/sN2xr/src/ProgressMeter.jl:1014 [inlined]
[14] macro expansion
@ ./task.jl:476 [inlined]
[15] progress_map(::Function, ::Vararg{Any}; mapfun::Function, progress::ProgressMeter.Progress, channel_bufflen::Int64, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ ProgressMeter ~/.julia/packages/ProgressMeter/sN2xr/src/ProgressMeter.jl:1007
[16] #progress_pmap#60
@ ~/.julia/packages/ProgressMeter/sN2xr/src/ProgressMeter.jl:1032 [inlined]
[17] progress_pmap
@ ~/.julia/packages/ProgressMeter/sN2xr/src/ProgressMeter.jl:1032 [inlined]
[18] pmap_with_data(f::Function, p::WorkerPool, c::DiskArrays.GridChunks{2}; initfunc::Function, progress::ProgressMeter.Progress, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:669
[19] pmap_with_data(f::Function, c::DiskArrays.GridChunks{2}; initfunc::Function, kwargs::Base.Pairs{Symbol, ProgressMeter.Progress, Tuple{Symbol}, NamedTuple{(:progress,), Tuple{ProgressMeter.Progress}}})
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:674
[20] runLoop(dc::YAXArrays.DAT.DATConfig{1, 1}, showprog::Bool)
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:699
[21] mapCube(fu::typeof(RQADeforestation.rqatrend), cdata::Tuple{YAXArray{Union{Missing, Float16}, 3, DiskArrays.SubDiskArray{Union{Missing, Float16}, 3}, Vector{CubeAxis}}}, addargs::Int64; max_cache::Float64, indims::InDims, outdims::OutDims, inplace::Bool, ispar::Bool, debug::Bool, include_loopvars::Bool, showprog::Bool, irregular_loopranges::Bool, nthreads::Dict{Int64, Int64}, loopchunksize::Dict{Any, Any}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ YAXArrays.DAT ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:476
[22] #mapCube#37
@ ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:303 [inlined]
[23] mapCube
@ ~/.julia/dev/YAXArrays/src/DAT/DAT.jl:303 [inlined]
[24] #rqatrend#10
@ ~/RQADeforestation/src/analysis.jl:56 [inlined]
[25] rqatrend
@ ~/RQADeforestation/src/analysis.jl:55 [inlined]
[26] macro expansion
@ ./timing.jl:273 [inlined]
[27] (::var"#4#7"{Int64, YAXArray{Union{Missing, Float16}, 3, DiskArrays.SubDiskArray{Union{Missing, Float16}, 3}, Vector{CubeAxis}}, String})()
@ Main ~/RQADeforestation/scripts/germany_rqatrend.jl:128
[28] (::Base.RedirectStdStream)(thunk::var"#4#7"{Int64, YAXArray{Union{Missing, Float16}, 3, DiskArrays.SubDiskArray{Union{Missing, Float16}, 3}, Vector{CubeAxis}}, String}, stream::IOStream)
@ Base ./stream.jl:1429
[29] main()
@ Main ~/RQADeforestation/scripts/germany_rqatrend.jl:127
[30] top-level scope
@ ~/RQADeforestation/scripts/germany_rqatrend.jl:157
in expression starting at /home/ubuntu/RQADeforestation/scripts/germany_rqatrend.jl:157
┌ Warning: Forcibly interrupting busy workers
│ exception = rmprocs: pids [3, 4, 5, 6, 7, 8, 9] not terminated after 5.0 seconds.
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/cluster.jl:1253
┌ Warning: Failed to gracefully kill worker 3, sending SIGTERM
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/managers.jl:745
┌ Warning: Failed to gracefully kill worker 5, sending SIGTERM
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/managers.jl:745
┌ Warning: Failed to gracefully kill worker 6, sending SIGTERM
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/managers.jl:745
┌ Warning: Failed to gracefully kill worker 7, sending SIGTERM
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/managers.jl:745
┌ Warning: Failed to gracefully kill worker 8, sending SIGTERM
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/managers.jl:745
┌ Warning: Failed to gracefully kill worker 9, sending SIGTERM
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/managers.jl:745
From worker 3: pure virtual method called
┌ Warning: rmprocs: process 1 not removed
└ @ Distributed ~/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/share/julia/stdlib/v1.9/Distributed/src/cluster.jl:1049
From worker 3: terminate called without an active exceptionerror: <inline asm>:1:2: invalid character in input
��
[1300178] signal (11.128): Segmentation fault
in expression starting at none:0
unknown function (ip: 0x7f4d281a28a6)
exit at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_ZN4llvm11LLVMContext8diagnoseERKNS_14DiagnosticInfoE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
free(): corrupted unsorted chunks
[1300178] signal (6.-6): Aborted
in expression starting at none:0
_ZNSt17_Function_handlerIFvRKN4llvm12SMDiagnosticEbRKNS0_9SourceMgrERSt6vectorIPKNS0_6MDNodeESaISA_EEEZNS0_28MachineModuleInfoWrapperPass16doInitializationERNS0_6ModuleEEUlS3_bS6_SD_E_E9_M_invokeERKSt9_Any_dataS3_ObS6_SD_ at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm9MCContext8diagnoseERKNS_12SMDiagnosticE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN12_GLOBAL__N_19AsmParser11DiagHandlerERKN4llvm12SMDiagnosticEPv at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZNK4llvm9SourceMgr12PrintMessageERNS_11raw_ostreamENS_5SMLocENS0_8DiagKindERKNS_5TwineENS_8ArrayRefINS_7SMRangeEEENS8_INS_7SMFixItEEEb at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZNK4llvm9SourceMgr12PrintMessageENS_5SMLocENS0_8DiagKindERKNS_5TwineENS_8ArrayRefINS_7SMRangeEEENS6_INS_7SMFixItEEEb at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN12_GLOBAL__N_19AsmParser3RunEbb at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZNK4llvm10AsmPrinter13emitInlineAsmENS_9StringRefERKNS_15MCSubtargetInfoERKNS_15MCTargetOptionsEPKNS_6MDNodeENS_9InlineAsm10AsmDialectE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZNK4llvm10AsmPrinter13emitInlineAsmEPKNS_12MachineInstrE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm10AsmPrinter16emitFunctionBodyEv at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13X86AsmPrinter20runOnMachineFunctionERNS_15MachineFunctionE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc14SimpleCompilerclERNS_6ModuleE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
operator() at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1206
_ZN4llvm3orc14IRCompileLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16IRTransformLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
emit at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:631
_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc19MaterializationTask3runEv at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm6detail18UniqueFunctionBaseIvJSt10unique_ptrINS_3orc4TaskESt14default_deleteIS4_EEEE8CallImplIPFvS7_EEEvPvRS7_ at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession17OL_completeLookupESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EESt10shared_ptrINS0_23AsynchronousSymbolQueryEESt8functionIFvRKNS_8DenseMapIPNS0_8JITDylibENS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISF_vEEEENSG_ISD_vEENS_6detail12DenseMapPairISD_SI_EEEEEE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc25InProgressFullLookupState8completeESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession19OL_applyQueryPhase1ESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EENS_5ErrorE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS0_10LookupKindERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS8_EENS0_15SymbolLookupSetENS0_11SymbolStateENS_15unique_functionIFvNS_8ExpectedINS_8DenseMapINS0_15SymbolStringPtrENS_18JITEvaluatedSymbolENS_12DenseMapInfoISI_vEENS_6detail12DenseMapPairISI_SJ_EEEEEEEEESt8functionIFvRKNSH_IS6_NS_8DenseSetISI_SL_EENSK_IS6_vEENSN_IS6_SV_EEEEEE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EERKNS0_15SymbolLookupSetENS0_10LookupKindENS0_11SymbolStateESt8functionIFvRKNS_8DenseMapIS5_NS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISK_vEEEENSL_IS5_vEENS_6detail12DenseMapPairIS5_SN_EEEEEE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EENS0_15SymbolStringPtrENS0_11SymbolStateE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS0_15SymbolStringPtrENS0_11SymbolStateE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS_9StringRefENS0_11SymbolStateE at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/bin/../lib/julia/libLLVM-14jl.so (unknown line)
addModule at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1420
jl_add_to_ee at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1815
_jl_compile_codeinst at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:241
jl_generate_fptr_impl at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:460
jl_compile_method_internal at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2348 [inlined]
jl_compile_method_internal at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2237
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2750 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
show_exception_stack at ./errorshow.jl:895
display_error at ./client.jl:111
#647 at ./task.jl:577
jfptr_YY.647_42002.clone_1 at /home/ubuntu/.julia/juliaup/julia-1.9.1+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
start_task at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/task.c:1092
Allocations: 133148506 (Pool: 133105176; Big: 43330); GC: 142
Segmentation fault (core dumped) |
|
With this pull request the computation finishes in 47 minutes. |
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTargetBaselineTarget resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoBaseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoRuntime information
|
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTargetBaselineTarget resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoBaseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoRuntime information
|
Closed
meggart
changed the title
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTargetBaselineTarget resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoBaseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoRuntime information
|
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTargetBaselineTarget resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoBaseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoRuntime information
|
Benchmark resultJudge resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsA ratio greater than
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoTargetBaselineTarget resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoBaseline resultBenchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jlJob Properties
ResultsBelow is a table of this job's results, obtained by running the benchmarks.
Benchmark Group ListHere's a list of all the benchmark groups executed by this job:
Julia versioninfoRuntime information
|
|
Finally we have an integration test passing! |
lazarusA
pushed a commit
that referenced
this pull request
Jun 29, 2023
* Manually run gc after chunk processing * Add gc kwarg and CachingPool * Fix EDL integrarion Tests * remove pmap_with_data * Fix cube conversion * Allow to manually set prom_type
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This shoudl not be necessary, but it looks that in some situations for long-running jobs memory usage keeps increasing until running into an OOM error. We already had a use case where manually calling gc helped, it would be good if others could test as well if this would help for their use case @TabeaW @Qfl3x .