try (hard) to support WSL2 (Windows) as an environment

[wwlwpd]

I think this is feasible within the current accepted definition of what we call a "platform" and would give us a native Windows build. If this is successful, maybe look at WSL (Windows Subsystem for Linux) again to see if we can treat it as a "platform", too.

[jasonfleming]

Sounds challenging, looking forward to discussing details. :-)

[wwlwpd]

required dependencies seem prohibitive for now, I think it might still be worth trying in Microsoft Windows Subsystem for Linux (WSL2).

[wwlwpd]

Update title to reflect WSL2 rather than MSYS2

[jasonfleming]

Just tried installing ASGS in WSL2 (Ubuntu). It ran until it failed at the python build/install stage with the following message:

Collecting netCDF4
  Downloading netCDF4-1.6.5.tar.gz (764 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 765.0/765.0 kB 4.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Package hdf5 was not found in the pkg-config search path.
      Perhaps you should add the directory containing `hdf5.pc'
      to the PKG_CONFIG_PATH environment variable
      No package 'hdf5' found
      reading from setup.cfg...
      using /home/jason/asgs/opt/bin/nc-config...
      checking /home/jason/asgs/opt/include ...
      HDF5 library version: 1.12.2 headers found in /home/jason/asgs/opt/include
      Traceback (most recent call last):
        File "/home/jason/asgs/opt/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/jason/asgs/opt/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/jason/asgs/opt/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-jnwqtmb9/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-jnwqtmb9/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-jnwqtmb9/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 547, in <module>
        File "<string>", line 166, in getnetcdfvers
        File "/home/jason/asgs/opt/lib/python3.7/ctypes/__init__.py", line 7, in <module>
          from _ctypes import Union, Structure, Array
      ModuleNotFoundError: No module named '_ctypes'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Traceback (most recent call last):
  File "cloud/general/t/netcdf4-bench.py", line 7, in <module>
    from numpy.random.mtrand import uniform
ModuleNotFoundError: No module named 'numpy'
post condition for "step for installing python 3 and required modules" FAILED, stopping. Please fix and rerun.

[jasonfleming]

Looking back up through the preceding python build messages, I see that the building of the python _ctypes module failed with the following messages:

gcc -pthread -shared -L/home/jason/asgs/opt/lib -L/home/jason/asgs/opt/lib -L/home/jason/asgs/opt/lib -I/home/jason/asgs/opt/include build/temp.linux-x86_64-3.7/home/jason/asgs/tmp/Python-3.7.9/Modules/xxlimited.o -L/home/jason/asgs/opt/lib -L/usr/lib/x86_64-linux-gnu -L/usr/local/lib -o build/lib.linux-x86_64-3.7/xxlimited.cpython-37m-x86_64-linux-gnu.so
INFO: Could not locate ffi libs and/or headers

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2                  _curses               _curses_panel
_dbm                  _gdbm                 _sqlite3
_tkinter              readline
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd
time


Failed to build these modules:
_ctypes

I don't have enough build history in my window to go back to see why this module failed to build.

[jasonfleming]

I seem to have fixed the python build errors with apt-get install libffi-dev

[jasonfleming]

I just observed this same build failure on mike due to my testing process for #1224

[jasonfleming]

I tried ./asgsh and got the following error message

./asgsh: line 125: syntax error near unexpected token `('

Looking at line 125 I see in my PATH the following string:

:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common

I manually enclosed that PATH part in double quotes to become

:"/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common"

and found that ./asgsh started successfully.

[jasonfleming]

Once inside the shell, build adcirc initially failed because it could not clone the adcirc repo. Strangely, I was able to do this manually, successfully. I fixed this by adding my id_rsa.pub ssh key to my GitHub account.

[jasonfleming]

btw my full (broken) PATH was set as follows:

export PATH=/home/jason/asgs/opt/perl5/bin:/home/jason/asgs/opt/perl5/perls/perl-5.38.0/bin:/home/jason/asgs/opt/gfortran/bin:/home/jason/asgs/opt/bin:/home/jason/asgs/cloud/general:/home/jason/asgs/bin:/home/jason/asgs/my-bin:/home/jason/asgs/cloud/general:/home/jason/asgs/config:/home/jason/asgs/config/tests:/home/jason/asgs/input:/home/jason/asgs/input/data_assimilation:/home/jason/asgs/monitoring:/home/jason/asgs/output:/home/jason/asgs/output/cera_contour:/home/jason/asgs/output/cpra_postproc:/home/jason/asgs/output/cpra_postproc/MEX:/home/jason/asgs/output/cpra_postproc/old/tools:/home/jason/asgs/output/Cuba_post:/home/jason/asgs/output/FG49:/home/jason/asgs/output/NGOM_post:/home/jason/asgs/output/PartTrack:/home/jason/asgs/output/postProcessFlux:/home/jason/asgs/output/POSTPROC_KMZGIS:/home/jason/asgs/output/POSTPROC_KMZGIS/FigGen:/home/jason/asgs/output/POSTPROC_KMZGIS/RenciGETools-10/src:/home/jason/asgs/output/test:/home/jason/asgs/output/TRACKING_FILES:/home/jason/asgs/output/validation:/home/jason/asgs/tides:/home/jason/asgs/util:/home/jason/asgs/util/admin:/home/jason/asgs/util/input:/home/jason/asgs/util/input/mesh:/home/jason/asgs/util/input/nodalattr:/home/jason/asgs/util/output:/home/jason/asgs/util/troubleshooting:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/mnt/c/Program Files/Microsoft MPI/Bin/:/mnt/c/Windows/system32:/mnt/c/Windows:/mnt/c/Windows/System32/Wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0/:/mnt/c/Windows/System32/OpenSSH/:/mnt/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/mnt/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/mnt/c/Program Files/dotnet/:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Program Files/Git/cmd:/Docker/host/bin:/mnt/c/Users/jason/AppData/Local/Programs/Python/Python310/Scripts/:/mnt/c/Users/jason/AppData/Local/Programs/Python/Python310/:/mnt/c/Users/jason/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/jason/AppData/Local/Programs/Microsoft VS Code/bin:/snap/bin

I wonder if this could be fixed in an automated way by surrounding every PATH part with double quotes.

[jasonfleming]

I was able to build and run ADCIRC (v53release) successfully and then run a GFS spinup/nowcast with SWAN coupling on the Shinnecock Inlet mesh under WSL2 on my Windows 11 laptop. :-)

[jasonfleming]

Everything works. Making this into a Discussion to capture the lessons learned and then making a separate issue for dealing with the PATH on a WSL2 platform.