Facing troubles with Matplotlib: not compiled against NumPy v2

[agriyakhetarpal]

Hi, we are trying out a Xeus-based environment for Matplotlib in the browser as a part of matplotlib/matplotlib#29506, where import matplotlib doesn't seem to work with NumPy 2.2.2, as it hasn't been compiled against NumPy v2.

What would be the way to do so? Extension modules compiled against NumPy 2.x work with both 1.x and 2.x, so a re-compilation and subsequent upload should work.

[agriyakhetarpal]

I guess changing this line for NumPy as a host dependency is the way to go:

recipes/recipes/recipes_emscripten/matplotlib/recipe.yaml

Line 38 in 06fba5c

- numpy

unless I'm missing something?

[ianthomas23]

Yes, this seems to be pinned in matplotlib-base https://beta.mamba.pm/channels/emscripten-forge/packages/matplotlib-base. I don't really know why the pin is there.

The good news is that the latest builds using emscripten 3.1.73 and python 3.13 are just becoming available at https://prefix.dev/channels/emscripten-forge-dev. They are probably not fully reliable yet as they are so new and there are probably some useful packages not yet built, but if you want to try it out now I think all you need to is change the channel in your environment yaml file from
https://repo.mamba.pm/emscripten-forge
to
https://repo.prefix.dev/emscripten-forge-dev

[agriyakhetarpal]

Thanks for looking into it!

I don't really know why the pin is there.

I think it might be coming from the global pin, as conda_build_config.yml pins NumPy to 1.25.2. Does this mean many other packages might also have been built against NumPy <2? (If yes, I don't know how many of them are working with NumPy >2 which is already present). I would be happy to help out by bumping the pin to, say, 2.2.2, now that most common packages are supported (numpy/numpy#26191).

the latest builds using emscripten 3.1.73 and python 3.13 are just becoming available at prefix.dev/channels/emscripten-forge-dev.

I'll switch to that and see if that works; Matplotlib would likely have been built by now. I think this server sounds useful to document under https://emscripten-forge.org/usage/package_server/ as an experimental choice for those who need it.

[agriyakhetarpal]

@ianthomas23, no luck. I switched to the dev channel, but I think there's a check somewhere that looks for the Emscripten 3.1.45 ABI as an underlying metapackage of sorts:

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

To activate this environment, use

     $ mamba activate /var/folders/b3/2bq1m1_50bs4c7305j8vxcqr0000gn/T/tmp4hslral7/env/envs/mpl-wasm-docs

To deactivate an active environment, use

     $ mamba deactivate


Looking for: ['numpy', 'matplotlib', 'ipympl', 'xeus-python', 'emscripten-abi=3.1.45']

https://repo.prefix.dev/emscripten-forge-dev/emscripten-wasm32          Using cache
https://repo.prefix.dev/emscripten-forge-dev/noarch          Using cache
conda-forge/emscripten-wasm32                               Using cache
conda-forge/noarch                                          Using cache
Could not solve for environment specs
The following package could not be installed
└─ emscripten-abi 3.1.45**  does not exist (perhaps a typo or a missing channel).
[LiteBuildApp] ERROR | [lite] [post_build] [jupyterlite-xeus] [ERR] Command '['/opt/homebrew/bin/mamba', 'install', '--yes', '--prefix', PosixPath('/var/folders/b3/2bq1m1_50bs4c7305j8vxcqr0000gn/T/tmp4hslral7/env/envs/mpl-wasm-docs'), '-c', 'https://repo.prefix.dev/emscripten-forge-dev', '-c', 'conda-forge', 'numpy', 'matplotlib', 'ipympl', 'xeus-python', 'emscripten-abi=3.1.45']' returned non-zero exit status 1.

[ianthomas23]

@ianthomas23, no luck. I switched to the dev channel, but I think there's a check somewhere that looks for the Emscripten 3.1.45 ABI as an underlying metapackage of sorts

Here's a simple repo for a Lite deployment including just pyodide and xeus python 3.13 kernels: https://github.com/ianthomas23/jlite-kernels. And it is deployed to github pages at https://ianthomas23.github.io/jlite-kernels. It works for numpy but not matplotlib at this stage, so we'd have to conclude that it is not ready for real use quite yet.

[agriyakhetarpal]

Thanks! It's good that Matplotlib doesn't force NumPy 2 at runtime at the moment (and probably won't until the coming two years perhaps?), and is otherwise up to date on the stable channel, so there's no hurry about it.

[ianthomas23]

@agriyakhetarpal JupyterLite deployment https://github.com/ianthomas23/jlite-kernels (deployed at https://ianthomas23.github.io/jlite-kernels) is now working with xeus python 3.13, matplotlib 3.9.4 and numpy 2.2.2. It requires jupyterlite-core 0.5.1 and jupyterlite-xeus 3.1.2 that were released today. We're still working on updating matplotlib to 3.10.0, hopefully it won't be long now.

[agriyakhetarpal]

Thanks, @ianthomas23! Is there a timeline for this kernel's change from canary to stable, and for a general rollout of 3.1.73 ABI packages? Is there an issue/PR I can subscribe to to know more about the transition?

[ianthomas23]

There is no commitment to a particular timeline but it is being actively worked on. And there isn't really a particular issue or PR as the work is distributed across emscripten-forge, jupyterlite and xeus. Probably the transition to the 3.1.73 packages being recommended will be when jupyterlite-xeus is altered to use them in the demo site, via the channel in this file: https://github.com/jupyterlite/xeus/blob/main/docs/environment-python.yml. It is already mentioned in the docs in https://github.com/jupyterlite/xeus/blob/main/docs/environment.md with a warning.

By the way, matplotlib 3.10.0 is now available for the xeus python 3.13 kernel.

[DerThorsten]

fyi I created an issue with the missing packages for the 3.1.73 branch #1822