Fix mechanical simulation with MPI #301

masterleinad · 2024-07-17T18:00:36Z

No description provided.

Rombur · 2024-07-17T18:40:27Z

source/MechanicalPhysics.cc

+  for (auto const &cell : dealii::filter_iterators(
+           _dof_handler.active_cell_iterators(),
+           dealii::IteratorFilters::LocallyOwnedCell(),
+           dealii::IteratorFilters::ActiveFEIndexEqualTo(0)))


I see that you changed this for consistency but the reason why we don't always use pipe is because it wasn't part of deal.II when I started adamantine. I've tried to use pipes more consistently recently but I didn't change the old code.

While using LocallyOwnedCell() as a separate predicates makes the code easier to understand, we are dependent on the order in which the predicates are evaluated. With ActiveFEIndexEqualTo(0, true), we don't have that issue. I didn't go back and changed the code once I realize that but I think the current code is a better way to write the loop.

masterleinad · 2024-07-17T21:19:44Z

For the okuma demonstration case, this is my latest backtrace

0: nan nan -0.00195 nan nan 5e-05
1: nan nan -0.00195 nan nan 5e-05
2: nan nan -0.00195 nan nan 5e-05

--------------------------------------------------------
An error occurred in line <491> of file </opt/dealii/include/deal.II/base/bounding_box.h> in function
    dealii::BoundingBox<spacedim, Number>::BoundingBox(const std::pair<dealii::Point<dim, Number>, dealii::Point<dim, Number> >&) [with int spacedim = 3; Number = double]
The violated condition was: 
    boundary_points.first[i] <= boundary_points.second[i]
Additional information: 
    Bounding Box can't be created: the points' order should be bottom
    left, top right!

Stacktrace:
-----------
#0  /app/adamantine/build/bin/adamantine: dealii::BoundingBox<3, double>::BoundingBox(std::pair<dealii::Point<3, double>, dealii::Point<3, double> > const&)
#1  /app/adamantine/build/bin/adamantine: std::tuple<std::vector<dealii::BoundingBox<3, double>, std::allocator<dealii::BoundingBox<3, double> > >, std::vector<double, std::allocator<double> >, std::vector<double, std::allocator<double> >, std::vector<double, std::allocator<double> > > adamantine::deposition_along_scan_path<3>(boost::property_tree::basic_ptree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, adamantine::ScanPath const&)
#2  /app/adamantine/build/bin/adamantine: std::tuple<std::vector<dealii::BoundingBox<3, double>, std::allocator<dealii::BoundingBox<3, double> > >, std::vector<double, std::allocator<double> >, std::vector<double, std::allocator<double> >, std::vector<double, std::allocator<double> > > adamantine::create_material_deposition_boxes<3>(boost::property_tree::basic_ptree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, adamantine::HeatSources<3, dealii::MemorySpace::Host>&)
#3  /app/adamantine/build/bin/adamantine: std::pair<dealii::LinearAlgebra::distributed::Vector<double, dealii::MemorySpace::Host>, dealii::LinearAlgebra::distributed::Vector<double, dealii::MemorySpace::Host> > run<3, 0, adamantine::SolidLiquid, dealii::MemorySpace::Host>(ompi_communicator_t* const&, boost::property_tree::basic_ptree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<adamantine::Timer, std::allocator<adamantine::Timer> >&)
#4  /app/adamantine/build/bin/adamantine: main
--------------------------------------------------------

Calling MPI_Abort now.
To break execution in a GDB session, execute 'break MPI_Abort' before running. You can also put the following into your ~/.gdbinit:
  set breakpoint pending on
  break MPI_Abort
  set breakpoint pending auto

Thread 1 "adamantine" hit Breakpoint 1, 0x00007ffff7935cc0 in PMPI_Abort () from /opt/openmpi/lib/libmpi.so.40
[Thread 0x7fff8cffd000 (LWP 286956) exited]
#0  0x00007ffff7935cc0 in PMPI_Abort () from /opt/openmpi/lib/libmpi.so.40
[New Thread 0x7fff77ffe000 (LWP 286959)]
[Thread 0x7fff77ffe000 (LWP 286955) exited]
[New Thread 0x7fffc1cc0000 (LWP 286960)]
[New Thread 0x7fffc14bf000 (LWP 286961)]
[New Thread 0x7fffc0cbe000 (LWP 286962)]
[New Thread 0x7fffadfff000 (LWP 286963)]
[Thread 0x7fffc14bf000 (LWP 286961) exited]
[New Thread 0x7fffad7fe000 (LWP 286964)]
[New Thread 0x7fffacffd000 (LWP 286965)]
[Thread 0x7fffadfff000 (LWP 286963) exited]
[New Thread 0x7fff39faa000 (LWP 286966)]
[Thread 0x7fffacffd000 (LWP 286965) exited]
[Thread 0x7fffad7fe000 (LWP 286964) exited]
[Thread 0x7fff39faa000 (LWP 286966) exited]
[Thread 0x7fffc0cbe000 (LWP 286962) exited]
#1  0x00007fffef10b4c1 in dealii::deal_II_exceptions::internals::abort (exc=...) at /scratch/source/dealii/source/base/exceptions.cc:495
[Thread 0x7fff77ffe000 (LWP 286959) exited]
#2  0x000055555a6dc127 in dealii::deal_II_exceptions::internals::issue_error_noreturn<dealii::StandardExceptions::ExcMessage> (handling=dealii::deal_II_exceptions::internals::ExceptionHandling::abort_or_throw_on_exception, file=0x55555e5103f0 "/opt/dealii/include/deal.II/base/bounding_box.h", line=491, function=0x55555e5104f0 "dealii::BoundingBox<spacedim, Number>::BoundingBox(const std::pair<dealii::Point<dim, Number>, dealii::Point<dim, Number> >&) [with int spacedim = 3; Number = double]", cond=0x55555e510310 "boundary_points.first[i] <= boundary_points.second[i]", exc_name=0x55555e5102a8 "ExcMessage(\"Bounding Box can't be created: the points' \" \"order should be bottom left, top right!\")", e=...) at /opt/dealii/include/deal.II/base/exceptions.h:1443
#3  0x000055555c87d2a9 in dealii::BoundingBox<3, double>::BoundingBox (this=0x7ffffffee240, boundary_points={...}) at /opt/dealii/include/deal.II/base/bounding_box.h:491
#4  0x000055555c879c70 in adamantine::deposition_along_scan_path<3> (geometry_database=..., scan_path=...) at /app/adamantine/source/material_deposition.cc:229
#5  0x000055555c8736f4 in adamantine::create_material_deposition_boxes<3> (geometry_database=..., heat_sources=...) at /app/adamantine/source/material_deposition.cc:52
#6  0x000055555a80e1ce in run<3, 0, adamantine::SolidLiquid, dealii::MemorySpace::Host> (communicator=@0x7fffffff0558: 0x555561c63d60 <ompi_mpi_comm_world>, database=..., timers=std::vector of length 14, capacity 16 = {...}) at /app/adamantine/application/adamantine.hh:956
#7  0x000055555a6c9447 in main (argc=2, argv=0x7fffffffd048) at /app/adamantine/application/adamantine.cc:990

Rombur · 2024-07-17T21:38:22Z

That's an issue with the scan path. There was an issue where Slicer would create scan path that we cannot handler. I don't know if @AshGannon pushed the lasted version of the scan path on the repo.

AshGannon · 2024-07-17T22:13:20Z

That's an issue with the scan path. There was an issue where Slicer would create scan path that we cannot handler. I don't know if @AshGannon pushed the lasted version of the scan path on the repo.

Is this an event series? I think these were generated using a code from Yousub and the laser power is off by an index. If that's the case and you have the g-code, I can run it through the modified code to get an event series that will work in Adamantine. @stvdwtt have the event series been updated here?

I'm on my phone at the moment, so I am not sure which print the okuma test case is.

stvdwtt · 2024-07-18T03:15:12Z

It doesn't look like the event series has been updated for the Okuma curved wall demonstration case. I'll look into this tomorrow (Thursday) and push updated files. Sorry for letting this get out of date.

AshGannon · 2024-07-18T11:19:54Z

@stvdwtt sorry, I think I was supposed to do this and it fell off my to-do list. 🤦‍♀️

stvdwtt · 2024-07-18T20:17:54Z

@masterleinad, these input files should work, but I haven't had a chance to test them: https://github.com/adamantine-sim/adaptive-control-driver/tree/main/demo_input_files/control3/reference_simulation

masterleinad · 2024-07-18T21:45:17Z

@masterleinad, these input files should work, but I haven't had a chance to test them: adamantine-sim/adaptive-control-driver@main/demo_input_files/control3/reference_simulation

That one has

  mechanical false

Use ghosted vector with locally relevant dofs for compute_stress

ad58000

Rombur reviewed Jul 17, 2024

View reviewed changes

Be explicit about the second argument of ActiveFEIndexEqualTo

2c9e269

masterleinad force-pushed the fix_mechanical_mpi branch from e2e671d to 2c9e269 Compare July 17, 2024 19:35

masterleinad added 19 commits July 19, 2024 18:01

Fix warning

a0e3812

Fix

38cec19

Intermediate

0a4721d

Split mechanical_physics->setup_dofs

efb1bb8

Intermediate

82a92b0

Use SolutionTransfer

98ceb7f

Fix

ec86ce9

Intermediate

fd6ec45

Almost working

6d9510c

Cleanup

d48c4ab

Indentation

a204ddf

Compiling again

2924e30

Almost working

95f309c

Intermediate

48051d3

Almost working

caecb44

Intermediate

7caa9fd

Set active fe indices

d061914

Remove debug output

3656e3c

Merge branch 'master' into fix_mechanical_mpi

14a5548

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix mechanical simulation with MPI #301

Fix mechanical simulation with MPI #301

masterleinad commented Jul 17, 2024

Rombur Jul 17, 2024

masterleinad commented Jul 17, 2024

Rombur commented Jul 17, 2024

AshGannon commented Jul 17, 2024

stvdwtt commented Jul 18, 2024

AshGannon commented Jul 18, 2024

stvdwtt commented Jul 18, 2024

masterleinad commented Jul 18, 2024

Fix mechanical simulation with MPI #301

Are you sure you want to change the base?

Fix mechanical simulation with MPI #301

Conversation

masterleinad commented Jul 17, 2024

Rombur Jul 17, 2024

Choose a reason for hiding this comment

masterleinad commented Jul 17, 2024

Rombur commented Jul 17, 2024

AshGannon commented Jul 17, 2024

stvdwtt commented Jul 18, 2024

AshGannon commented Jul 18, 2024

stvdwtt commented Jul 18, 2024

masterleinad commented Jul 18, 2024