Skip to content

Commit 45ef533

Browse files
prkkumarax3lPrabhat KumarRemiLehe
authored
1D3V Cartesian Support (#2307)
* Build System: Add 1D Geometry * test PR * test PR * 1D cartesian yee algorithm * fixed typo * Fixes for PML * 1D support related multiple changes * Fix compilation * change 1D to 1D_Z * 1D Field Gather and typo fix * 1D Charge Deposition * Particle Pusher * multiple changes related to 1D * 1D diagnostics and initialization * PlasmaInjector and PEC fixes for 1D * clean-up delete diags file * mobility 1D laser particle and bilinear filter * deleted diags files * update laser particle weight formula * delete diags files * Azure: Add 1D Cartesian Runner * 1D fixes for FieldProbe * Update Docs/source/developers/dimensionality.rst Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> * 1d laser injection and langmuir test input files * 1d tests * clean up : delete print statements * analyse simulation result for laser injection and Langmuir tests * EOL * delete input files for which there are no automated tests * delete input files for which there are no automated tests * add ignore_unused to remove warnings * remove space * Fix compilation issues * fix error : macro name must be an identifier * Small bug fix * cleanup Python script for analysis * bug fix * bug fix * Update ParticleProbe: Check 1D in-domain * Update Source/Make.WarpX * Update .azure-pipelines.yml * Add USE_OPENPMD=FALSE to .azure-pipeline.yml * resolve conflict * resolve conflict * fix typo * Correct out-of-bound access * Fix Particle BC in WarpXParticleContainer and correct path to checksumAPI in python analysis scripts * EOL * Fix bug : accessing out of bound index of cell in 1D * remove 1d test for cartesian3d * Fix CI check * Slight style change * Address review comments * Fix GPU compilation Filter.cpp * Fix CI * Fix Indentation * Address review comments * More consistent ifdef for dimension bigger than 1 * Update Examples/Tests/Langmuir/analysis_langmuir_multi_1d.py Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update GNUmakefile Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Regression/prepare_file_ci.py Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/FieldSolver/FiniteDifferenceSolver/FiniteDifferenceAlgorithms/CartesianCKCAlgorithm.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/FieldSolver/FiniteDifferenceSolver/FiniteDifferenceAlgorithms/CartesianNodalAlgorithm.H Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Filter/Filter.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Initialization/PlasmaInjector.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * Update Source/Initialization/PlasmaInjector.cpp Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * add comment inline to explain twice push_back * Add amrex::Abort for NCIGodfreyFilter Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Prabhat Kumar <prabhatkumar@kraken.dhcp.lbl.gov> Co-authored-by: Remi Lehe <remi.lehe@normalesup.org> Co-authored-by: Remi Lehe <rlehe@lbl.gov>
1 parent ff8b73f commit 45ef533

File tree

64 files changed

+1646
-177
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+1646
-177
lines changed

.azure-pipelines.yml

+4
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,10 @@ jobs:
2020

2121
strategy:
2222
matrix:
23+
cartesian1d:
24+
WARPX_CI_REGULAR_CARTESIAN_1D: 'TRUE'
25+
WARPX_CI_PSATD: 'FALSE'
26+
WARPX_CI_OPENPMD: 'FALSE'
2327
cartesian2d:
2428
WARPX_CI_REGULAR_CARTESIAN_2D: 'TRUE'
2529
cartesian3d:

.github/workflows/source/test_ci_matrix.sh

+3-1
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,10 @@ grep "\[" ci-tests.ini > ci_all_tests.txt
1313
export WARPX_CI_PSATD=TRUE
1414

1515
# Concatenate the names of all elements in CI matrix into another test file
16-
WARPX_CI_REGULAR_CARTESIAN_2D=TRUE python prepare_file_ci.py
16+
WARPX_CI_REGULAR_CARTESIAN_1D=TRUE python prepare_file_ci.py
1717
grep "\[" ci-tests.ini > ci_matrix_elements.txt
18+
WARPX_CI_REGULAR_CARTESIAN_2D=TRUE python prepare_file_ci.py
19+
grep "\[" ci-tests.ini >> ci_matrix_elements.txt
1820
WARPX_CI_REGULAR_CARTESIAN_3D=TRUE python prepare_file_ci.py
1921
grep "\[" ci-tests.ini >> ci_matrix_elements.txt
2022
WARPX_CI_PYTHON_MAIN=TRUE python prepare_file_ci.py

CMakeLists.txt

+7-7
Original file line numberDiff line numberDiff line change
@@ -60,8 +60,8 @@ option(WarpX_SENSEI "SENSEI in situ diagnostics" OFF)
6060
option(WarpX_QED "QED support (requires PICSAR)" ON)
6161
option(WarpX_QED_TABLE_GEN "QED table generation (requires PICSAR and Boost)" OFF)
6262

63-
set(WarpX_DIMS_VALUES 2 3 RZ)
64-
set(WarpX_DIMS 3 CACHE STRING "Simulation dimensionality (2/3/RZ)")
63+
set(WarpX_DIMS_VALUES 1 2 3 RZ)
64+
set(WarpX_DIMS 3 CACHE STRING "Simulation dimensionality (1/2/3/RZ)")
6565
set_property(CACHE WarpX_DIMS PROPERTY STRINGS ${WarpX_DIMS_VALUES})
6666
if(NOT WarpX_DIMS IN_LIST WarpX_DIMS_VALUES)
6767
message(FATAL_ERROR "WarpX_DIMS (${WarpX_DIMS}) must be one of ${WarpX_DIMS_VALUES}")
@@ -263,6 +263,8 @@ if(WarpX_DIMS STREQUAL 3)
263263
target_compile_definitions(WarpX PUBLIC WARPX_DIM_3D)
264264
elseif(WarpX_DIMS STREQUAL 2)
265265
target_compile_definitions(WarpX PUBLIC WARPX_DIM_XZ)
266+
elseif(WarpX_DIMS STREQUAL 1)
267+
target_compile_definitions(WarpX PUBLIC WARPX_DIM_1D_Z)
266268
elseif(WarpX_DIMS STREQUAL RZ)
267269
target_compile_definitions(WarpX PUBLIC WARPX_DIM_RZ)
268270
endif()
@@ -331,12 +333,10 @@ install(TARGETS ${WarpX_INSTALL_TARGET_NAMES}
331333
# simplified library alias
332334
# this is currently expected by Python bindings
333335
if(WarpX_LIB)
334-
if(WarpX_DIMS STREQUAL 3)
335-
set(lib_suffix "3d")
336-
elseif(WarpX_DIMS STREQUAL 2)
337-
set(lib_suffix "2d")
338-
elseif(WarpX_DIMS STREQUAL RZ)
336+
if(WarpX_DIMS STREQUAL RZ)
339337
set(lib_suffix "rz")
338+
else()
339+
set(lib_suffix "${WarpX_DIMS}d")
340340
endif()
341341
if(WIN32)
342342
set(mod_ext "dll")

Docs/source/developers/dimensionality.rst

+23-20
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ Dimensions CMake Option
1313
========== ===================
1414
**3D3V** ``WarpX_DIMS=3``
1515
**2D3V** ``WarpX_DIMS=2``
16+
**1D3V** ``WarpX_DIMS=1``
1617
**RZ** ``WarpX_DIMS=RZ``
1718
========== ===================
1819

@@ -23,35 +24,37 @@ Defines
2324

2425
Depending on the build variant of WarpX, the following preprocessor macros will be set:
2526

26-
================== =========== =========== ===========
27-
Macro 3D3V 2D3V RZ
28-
================== =========== =========== ===========
29-
``AMREX_SPACEDIM`` ``3`` ``2`` ``2``
30-
``WARPX_DIM_3D`` **defined** *undefined* *undefined*
31-
``WARPX_DIM_XZ`` *undefined* **defined** *undefined*
32-
``WARPX_DIM_RZ`` *undefined* *undefined* **defined**
33-
================== =========== =========== ===========
27+
================== =========== =========== =========== ===========
28+
Macro 3D3V 2D3V 1D3V RZ
29+
================== =========== =========== =========== ===========
30+
``AMREX_SPACEDIM`` ``3`` ``2`` ``1`` ``2``
31+
``WARPX_DIM_3D`` **defined** *undefined* *undefined* *undefined*
32+
``WARPX_DIM_1D_Z`` *undefined* *undefined* **defined** *undefined*
33+
``WARPX_DIM_XZ`` *undefined* **defined** *undefined* *undefined*
34+
``WARPX_DIM_RZ`` *undefined* *undefined* *undefined* **defined**
35+
================== =========== =========== =========== ===========
3436

3537
At the same time, the following conventions will apply:
3638

37-
==================== =========== =========== ===========
38-
**Convention** **3D3V** **2D3V** **RZ**
39-
-------------------- ----------- ----------- -----------
39+
==================== =========== =========== =========== ===========
40+
**Convention** **3D3V** **2D3V** **1D3V** **RZ**
41+
-------------------- ----------- ----------- ----------- -----------
4042
*Fields*
41-
-----------------------------------------------------------
42-
AMReX Box dimensions ``3`` ``2`` ``2``
43-
WarpX axis labels ``x, y, z`` ``x, z`` ``x, z``
44-
-------------------- ----------- ----------- -----------
43+
------------------------------------------------------------------------
44+
AMReX Box dimensions ``3`` ``2`` ``1`` ``2``
45+
WarpX axis labels ``x, y, z`` ``x, z`` ``z`` ``x, z``
46+
-------------------- ----------- ----------- ----------- -----------
4547
*Particles*
46-
-----------------------------------------------------------
47-
AMReX AoS ``.pos()`` ``0, 1, 2`` ``0, 1`` ``0, 1``
48-
WarpX position names ``x, y, z`` ``x, z`` ``r, z``
49-
extra SoA attribute ``theta``
50-
==================== =========== =========== ===========
48+
------------------------------------------------------------------------
49+
AMReX AoS ``.pos()`` ``0, 1, 2`` ``0, 1`` ``0`` ``0, 1``
50+
WarpX position names ``x, y, z`` ``x, z`` ``z`` ``r, z``
51+
extra SoA attribute ``theta``
52+
==================== =========== =========== =========== ===========
5153

5254
Please see the following sections for particle AoS and SoA details.
5355

5456
Conventions
5557
-----------
5658

5759
In 2D3V, we assume that the position of a particle in ``y`` is equal to ``0``.
60+
In 1D3V, we assume that the position of a particle in ``x`` and ``y`` is equal to ``0``.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
#! /usr/bin/env python
2+
3+
# Copyright 2021 Prabhat Kumar, Remi Lehe
4+
#
5+
# This file is part of WarpX.
6+
#
7+
# License: BSD-3-Clause-LBNL
8+
9+
# This file is part of the WarpX automated test suite. Its purpose is to test the
10+
# injection of a Gaussian laser pulse from an antenna in a 1D simulation.
11+
# The test calculates the envelope of each component of the laser pulse at the end of
12+
# the simulation and it compares it with theory. It also checks that the
13+
# central frequency of the Fourier transform is the expected one.
14+
15+
import yt
16+
import sys
17+
import matplotlib
18+
matplotlib.use('Agg')
19+
import matplotlib.pyplot as plt
20+
import numpy as np
21+
from scipy.signal import hilbert
22+
sys.path.insert(1, '../../../../warpx/Regression/Checksum/')
23+
import checksumAPI
24+
25+
# Maximum acceptable error for this test
26+
relative_error_threshold = 0.05
27+
28+
# A small number
29+
small_num = 1.0e-8
30+
31+
# Physical parameters
32+
um = 1.e-6
33+
fs = 1.e-15
34+
c = 299792458
35+
36+
# Parameters of the gaussian beam
37+
wavelength = 1.*um
38+
w0 = 5.*um
39+
tt = 10.*fs
40+
t_c = 24.*fs
41+
E_max = 4e12
42+
43+
# laser direction
44+
dir_vector = np.array([0,0,1.0])
45+
dir_vector /= np.linalg.norm(dir_vector)
46+
47+
48+
# polarization vector
49+
pol_vector = np.array([1.0,1.0,0.0])
50+
pol_vector /= np.linalg.norm(pol_vector)
51+
52+
# Calculates the envelope of a Gaussian beam
53+
def gauss_env(T,Z):
54+
'''Function to compute the theory for the envelope
55+
'''
56+
inv_tau2 = 1./tt/tt
57+
exp_arg = - inv_tau2 / c/c * (Z-T*c)*(Z-T*c)
58+
return E_max * np.real(np.exp(exp_arg))
59+
60+
# Checks envelope and central frequency for a given laser component
61+
def check_component(data, component, t_env_theory, coeff,Z,dz):
62+
print("*** Checking " + component + " ***")
63+
field = data['boxlib', component].v.squeeze()
64+
env = abs(hilbert(field))
65+
66+
env_theory = t_env_theory*np.abs(coeff)
67+
68+
# Plot results
69+
fig = plt.figure(figsize=(12,6))
70+
71+
ax1 = fig.add_subplot(221)
72+
ax1.set_title('PIC field')
73+
ax1.plot(Z,field)
74+
75+
ax2 = fig.add_subplot(222)
76+
ax2.set_title('PIC envelope')
77+
ax2.plot(Z,env)
78+
79+
ax3 = fig.add_subplot(223)
80+
ax3.set_title('Theory envelope')
81+
ax3.plot(Z,env_theory, label="theory")
82+
ax3.plot(Z,env, label="simulation")
83+
ax3.legend(loc="upper right")
84+
85+
ax4 = fig.add_subplot(224)
86+
ax4.set_title('Difference')
87+
ax4.plot(Z,env-env_theory)
88+
89+
plt.tight_layout()
90+
plt.savefig("plt_" + component + ".png", bbox_inches='tight')
91+
92+
if(np.abs(coeff) < small_num):
93+
is_field_zero = np.sum(np.abs(env)) < small_num
94+
if is_field_zero :
95+
print("[OK] Field component expected to be 0 is ~ 0")
96+
else :
97+
print("[FAIL] Field component expected to be 0 is NOT ~ 0")
98+
assert(is_field_zero)
99+
print("******\n")
100+
return
101+
102+
fft_field = np.fft.fft(field)
103+
104+
freq_cols = np.fft.fftfreq(fft_field.shape[0],dz/c)
105+
106+
pos_max = np.unravel_index(np.abs(fft_field).argmax(), fft_field.shape)
107+
108+
freq = np.abs(freq_cols[pos_max[0]])
109+
exp_freq = c/wavelength
110+
111+
relative_error_freq = np.abs(freq-exp_freq)/exp_freq
112+
is_freq_ok = relative_error_freq < relative_error_threshold
113+
if is_freq_ok :
114+
print("[OK] Relative error frequency: {:6.3f} %".format(relative_error_freq*100))
115+
else :
116+
print("[FAIL] Relative error frequency: {:6.3f} %".format(relative_error_freq*100))
117+
assert(is_freq_ok)
118+
119+
print("******\n")
120+
121+
relative_error_env = np.sum(np.abs(env-env_theory)) / np.sum(np.abs(env_theory))
122+
is_env_ok = relative_error_env < relative_error_threshold
123+
if is_env_ok :
124+
print("[OK] Relative error envelope: {:6.3f} %".format(relative_error_env*100))
125+
else :
126+
print("[FAIL] Relative error envelope: {:6.3f} %".format(relative_error_env*100))
127+
assert(is_env_ok)
128+
129+
def check_laser(filename):
130+
ds = yt.load(filename)
131+
132+
# yt 4.0+ has rounding issues with our domain data:
133+
# RuntimeError: yt attempted to read outside the boundaries
134+
# of a non-periodic domain along dimension 0.
135+
if 'force_periodicity' in dir(ds): ds.force_periodicity()
136+
137+
z = np.linspace(
138+
ds.domain_left_edge[0].v,
139+
ds.domain_right_edge[0].v,
140+
ds.domain_dimensions[0])
141+
142+
dz = (ds.domain_right_edge[0].v-ds.domain_left_edge[0].v)/(ds.domain_dimensions[0]-1)
143+
144+
# Compute the theory for envelope
145+
env_theory = gauss_env(+t_c-ds.current_time.to_value(),z)+gauss_env(-t_c+ds.current_time.to_value(),z)
146+
147+
# Read laser field in PIC simulation, and compute envelope
148+
all_data_level_0 = ds.covering_grid(level=0, left_edge=ds.domain_left_edge, dims=ds.domain_dimensions)
149+
150+
b_vector = np.cross(dir_vector, pol_vector)
151+
152+
components = ["Ex", "Ey", "Ez", "Bx", "By", "Bz"]
153+
coeffs = [
154+
pol_vector[0],
155+
pol_vector[1],
156+
pol_vector[2],
157+
b_vector[0],
158+
b_vector[1],
159+
b_vector[2]]
160+
161+
field_facts = [1, 1, 1, 1/c, 1/c, 1/c]
162+
163+
for comp, coeff, field_fact in zip(components, coeffs, field_facts):
164+
check_component(all_data_level_0, comp, field_fact*env_theory, coeff, z, dz)
165+
166+
def main():
167+
filename_end = sys.argv[1]
168+
169+
check_laser(filename_end)
170+
171+
test_name = filename_end[:-9] # Could also be os.path.split(os.getcwd())[1]
172+
checksumAPI.evaluate_checksum(test_name, filename_end)
173+
174+
if __name__ == "__main__":
175+
main()
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Maximum number of time steps
2+
max_step = 240
3+
4+
# number of grid points
5+
amr.n_cell = 352
6+
7+
# Maximum allowable size of each subdomain in the problem domain;
8+
# this is used to decompose the domain for parallel calculations.
9+
amr.max_grid_size = 32
10+
11+
# Maximum level in hierarchy (for now must be 0, i.e., one level in total)
12+
amr.max_level = 0
13+
14+
# Geometry
15+
geometry.coord_sys = 0 # 0: Cartesian
16+
geometry.prob_lo = -15.e-6 # physical domain
17+
geometry.prob_hi = 15.e-6
18+
19+
boundary.field_lo = pec
20+
boundary.field_hi = pec
21+
22+
warpx.serialize_ics = 1
23+
24+
# Verbosity
25+
warpx.verbose = 1
26+
27+
# Algorithms
28+
algo.current_deposition = esirkepov
29+
warpx.use_filter = 0
30+
31+
# CFL
32+
warpx.cfl = 0.9
33+
34+
# Order of particle shape factors
35+
algo.particle_shape = 1
36+
37+
# Laser
38+
lasers.names = laser1
39+
laser1.profile = Gaussian
40+
laser1.position = 0.e-6 0.e-6 0.e-6 # This point is on the laser plane
41+
laser1.direction = 0. 0. 1. # The plane normal direction
42+
laser1.polarization = 1. 1. 0. # The main polarization vector
43+
laser1.e_max = 4.e12 # Maximum amplitude of the laser field (in V/m)
44+
laser1.wavelength = 1.0e-6 # The wavelength of the laser (in meters)
45+
laser1.profile_waist = 5.e-6 # The waist of the laser (in meters)
46+
laser1.profile_duration = 10.e-15 # The duration of the laser (in seconds)
47+
laser1.profile_t_peak = 24.e-15 # The time at which the laser reaches its peak (in seconds)
48+
laser1.profile_focal_distance = 13.109e-6 # Focal distance from the antenna (in meters)
49+
# With this focal distance the laser is at focus
50+
# at the end of the simulation.
51+
52+
# Diagnostics
53+
diagnostics.diags_names = diag1
54+
diag1.intervals = 20
55+
diag1.diag_type = Full
56+
57+
# Moving window
58+
warpx.do_moving_window = 1
59+
warpx.moving_window_dir = z
60+
warpx.moving_window_v = 1.0 # in units of the speed of light
61+
warpx.start_moving_window_step = 20
62+
warpx.end_moving_window_step = 200

0 commit comments

Comments
 (0)