Fix Inspect Request Failure in xeus-cpp-lite

[kr-2003]

Description

This PR resolves the issue where ?std::vector and similar inspect requests were failing in xeus-cpp-lite when built with WebAssembly (WASM) options. The inspect request was unable to locate the tag files correctly.

Fixes #229

Changes

1. Ensured that --preload-file correctly includes the required tag files by adding

PUBLIC "SHELL: --preload-file ${XEUS_CPP_DATA_DIR}@/share/xeus-cpp"
PUBLIC "SHELL: --preload-file ${XEUS_CPP_CONF_DIR}@/etc/xeus-cpp"

in target-link-options while building for EMSCRIPTEN.

2. Added demo of inspect request for getting documentation in /notebooks/xeus-cpp-lite-demo.ipynb.

Screenshot

Type of change

Please tick all options which are relevant.

• Bug fix

[anutosh491]

I think that's perfect and exactly what was required.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.72%. Comparing base (f054951) to head (6eb8cdd).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #258   +/-   ##
=======================================
  Coverage   80.72%   80.72%           
=======================================
  Files          19       19           
  Lines         970      970           
  Branches       93       93           
=======================================
  Hits          783      783           
  Misses        187      187           

[vgvassilev]

We need a way to test this change. Can you figure out how to write a test that runs as part of our ci?

[anutosh491]

We need a way to test this change. Can you figure out how to write a test that runs as part of our ci?

Hi,
Yes technically we need to test all of this https://github.com/compiler-research/xeus-cpp/blob/main/notebooks/xeus-cpp-lite-demo.ipynb

So let me open an issue and start discussing more with @kr-2003 there. (Won't expect something through this PR just yet considering @kr-2003 is a xeus-cpp newbie 😬 )

I think I have couple approaches in mind (one directly inspired by Tharun's work running UI tests for the native notebook).

Coming to this PR. The --preload-file flag only make changes to the xcpp.data file and not the xcpp.js/wasm file, so this can't affect what already works in any possible way. Also preloading for emscripten is kinda straightforward (blindly copies the files in the *.data file)

So yeah nothing apart from the size of the xcpp.data file would be changing here.

[anutosh491]

Here's the issue #259

[vgvassilev]

Is there any chance to run this notebook as part of the CI the way we do this for the non-wasm version?

[anutosh491]

I think this is the closest/fastest approach to get that #259 (comment)

Let's move this in and we can try it out ?

Tharun's work might have potential too.(I shall be meeting him soon to discuss more)

[vgvassilev]

I think this is the closest/fastest approach to get that #259 (comment)

Let's move this in and we can try it out ?

Tharun's work might have potential too.(I shall be meeting him soon to discuss more)

Let's spend a day in investigating if what Tharun has done applies here. If we can't solve it in a day or two we can move forward by squashing both commits into one.

[anutosh491]

Perfect. Shall meet him tomorrow and discuss more ( I am just thinking if this might involve compiling papermill and nbformat against emscripten as this is what he made use of )

[kr-2003]

https://github.com/kr-2003/xeus-cpp/actions/runs/13543087790/

These are the runs for testing xeus-cpp-lite in ubuntu 24.04 and macos 15. They are passing without having any issue.

[kr-2003]

For now, its testing for simple std::cout << "Hello World" << std::endl;.

[vgvassilev]

For now, its testing for simple std::cout << "Hello World" << std::endl;.

Does it break if there is an error?

[anutosh491]

Hey @kr-2003 thanks for this.

I shall review your branch by tomorrow and then we can possibly take this in.

Technically we shouldn't be having all the ui-test code here and it should be placed somewhere mainstream and we just import the framework. I think a jupyterlite repo should be made (maybe jupyterlite/ui-tests or something). I shall talk to a jupyterlite maintainer about this.

That being said, untill something like that is in place, we don't really have any quick way to just check things on a lite notebook yet ..... so possibly we'll just go ahead with the approach above (if Tharun and I can't find a better one)

For starters, we just need like 2-3 simple tests i would assumer (1 execute request test ... hello world should do I suppose, 1 inspect request). Kernel info request might be a bonus just to verify that only the cpp20 kernel is supported through xeus-cpp-lite for now !

[kr-2003]

For now, its testing for simple std::cout << "Hello World" << std::endl;.

Does it break if there is an error?

Yes, if there is an error, then the screenshot will be different than the expected screenshots. Therefore, it will mark those test cases as failed. Even the slightest difference of pixels are not tolerable by playwright.

[kr-2003]

1 execute request test ... hello world should do I suppose, 1 inspect request)

@anutosh491
Regarding this, doing ?std::vector fetches the cppreference website. This website might have different ads when ran at different times, which will cause the difference between the expected and resultant screenshot. This would lead to unnecessary failures of test cases.

For example,

[anutosh491]

arghhh that sucks. I was interested in introducing something for the main requests that work :
Not sure if we can get past it !

[vgvassilev]

For now, its testing for simple std::cout << "Hello World" << std::endl;.

Does it break if there is an error?

Yes, if there is an error, then the screenshot will be different than the expected screenshots. Therefore, it will mark those test cases as failed. Even the slightest difference of pixels are not tolerable by playwright.

We need something that compares the expected value to the cell's current execution value and if they differ the test fails. We already have similar infrastructure for the non-wasm builds.

[kr-2003]

We need something that compares the expected value to the cell's current execution value and if they differ the test fails. We already have similar infrastructure for the non-wasm builds.

Even if we have something like this, it won't compare the inspect request properly. Let's say we compare the cell outputs in this case. Here, the cell output will be in the form of html. 2 html contents with different ads will again differ and the test fails.

[kr-2003]

arghhh that sucks. I was interested in introducing something for the main requests that work : Not sure if we can get past it !

The temporary solution here can be: We can set maxDiffPixels as a property while comparing screenshots.
Reference : https://playwright.dev/docs/test-snapshots#options
We have to decide how much difference we can tolerate.

[anutosh491]

Even if we have something like this, it won't compare the inspect request properly. Let's say we compare the cell outputs in this case. Here, the cell output will be in the form of html. 2 html contents with different ads will again differ and the test fails.

Hmmm, I think we can. We can extract out the json and just compare the links being directed too if we really want to. Something like this

xeus-cpp/test/test_interpreter.cpp

Line 107 in f054951

std::string inspect_result = "https://en.cppreference.com/w/cpp/container/vector";

But that's not what the discussion is here, I think. Taking a step back and trying to look at this from a hawk-eye view (just because I haven't had the time to dive deeper), this is what we have

1. Approach 1 that I suggested (which you are trying out is) :

execute a cell -> capture a screenshot through some framework -> compare with stored screenshot

Nothing really wrong (and we would go with this if we don't find better options) but yeah a bit cumbersome to maintain
Ui test folder lies in our source code. P.S I spoke to a jupyterlite maintainer on moving this all at a mainstream location just like jupyter_kernel_test ... which might take some time.

2. Approach 2 is what we tried for testing a notebook for the native case.

I) Store the expected json otputs somewhere
2) Throw a notebook against a testing framework
3) Compare the resulting output from the stored output

For example a cell would look like this

    {
      "id": "9bd7f767-6c22-4a1b-a1e2-cd4184fd0367",
      "cell_type": "code",
      "source": "#include <iostream>\n\nstd::cout << \"some output\" << std::endl;",
      "metadata": {
        "vscode": {
          "languageId": "c++"
        },
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": "some output\n"
        }
      ],
      "execution_count": 1
    }
    

Just keeping track of source and outputs should be good to compare stuff isn't it. I need to discuss the feasibility of this approach for the lite use case with Tharun which I'll do soon. If not feasible, probably approach 1 is the easiest approach

[anutosh491]

But yeah that being said, I think getting the inspect request to work with lite is great. I was just telling Sylvain about this yesterday and he was really happy about it (cause no other Jupyterlite kernel like python-pydodie, xeus-python-lite, xeus-r-lite etc can execute an inspect_request)

So yeah I would not stall this PR for too long. Shouldn't we keep track of all ideation for the UI tests here #259 (which is why I had opened the issue)

[vgvassilev]

@anutosh491, convinced me offline to move forward with this pull request given it has no tests this one time. We should open an issue to remind us we to add tests at our earlier convenience.

@anutosh491 convinced me offline to move forward with this pull request given it has no tests this one time. We should open an issue to remind us we to add tests at our earlier convenience.