Skip to content

Commit

Permalink
Synchronize after copying in 2D reduced diagnostics (#5726)
Browse files Browse the repository at this point in the history
Add `Gpu::streamSynchronize();` after copying `TableData` from either
host-to-device and device-to-host in the `ParticleHistogram2D` and
`DifferentialLuminosity2D` reduced diagnostics.

We have observed unstable behaviors with the `DifferentialLuminosity2D`
diagnostic and @RemiLehe suggested this could be a reason.

Marked as bug, unless otherwise recommended.

---------

Co-authored-by: Remi Lehe <remi.lehe@normalesup.org>
  • Loading branch information
aeriforme and RemiLehe authored Mar 4, 2025
1 parent 304a38e commit 529723f
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,7 @@ void DifferentialLuminosity2D::ComputeDiags (int step)

// Copy data from GPU memory
m_h_data_2D.copy(m_d_data_2D);
Gpu::streamSynchronize();

// reduced sum over mpi ranks
const int size = static_cast<int> (m_d_data_2D.size());
Expand Down
3 changes: 3 additions & 0 deletions Source/Diagnostics/ReducedDiags/ParticleHistogram2D.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@ void ParticleHistogram2D::ComputeDiags (int step)
}

d_data_2D.copy(m_h_data_2D);
// Gpu::streamSynchronize() is not needed, because there is a sync in WarpXParIter constructor later

auto d_table = d_data_2D.table();

// get a reference to WarpX instance
Expand Down Expand Up @@ -244,6 +246,7 @@ void ParticleHistogram2D::ComputeDiags (int step)

// Copy data from GPU memory
m_h_data_2D.copy(d_data_2D);
Gpu::streamSynchronize();

// reduced sum over mpi ranks
const int size = static_cast<int> (d_data_2D.size());
Expand Down

0 comments on commit 529723f

Please sign in to comment.