Julia kernel is not found anymore from 1.3 to 1.4 update when available through Conda.jl

[cderv]

This was reported in quarto-dev/quarto-actions#98

Simple document:

---
title: test
jupyter: julia-1.10
---

```{julia}
1 + 1
```

Only Julia is available no python, and it is configured using https://quarto.org/docs/computations/julia.html#installation

using Pkg
Pkg.add("IJulia")
using IJulia
notebook()

Using quarto render test.qmd will throw an error

quarto render test.qmd
Check file:///quarto-cli/src/quarto.ts
ERROR: Jupyter kernel 'julia-1.10' not found.

Stack trace:
    at jupyterKernelspecFromMarkdown (file:///quarto-cli/src/core/jupyter/jupyter.ts:514:9)
    at async quartoMdToJupyter (file:///quarto-cli/src/core/jupyter/jupyter.ts:281:34)
    at async createNotebookforTarget (file:///quarto-cli/src/execute/jupyter/jupyter.ts:551:14)
    at async Object.target (file:///quarto-cli/src/execute/jupyter/jupyter.ts:183:12)
    at async fileExecutionEngineAndTarget (file:///quarto-cli/src/execute/engine.ts:213:18)
    at async renderFileInternal (file:///quarto-cli/src/command/render/render-files.ts:429:32)
    at async renderFiles (file:///quarto-cli/src/command/render/render-files.ts:310:9)
    at async render (file:///quarto-cli/src/command/render/render-shared.ts:87:18)
    at async Command.fn (file:///quarto-cli/src/command/render/cmd.ts:212:26)
    at async Command.execute (file:///quarto-cli/src/vendor/deno.land/x/cliffy@v0.25.4/command/command.ts:1790:7)

it is working in 1.3.450 but fails this way with new 1.4 version

[cderv]

Digging into this here is what happens :

Using Built conda was adding in this PR

• Use IJulia's built-in conda environment / jupyter install for julia notebooks / qmds #1931

It worked in "default" way

quarto-cli/src/core/jupyter/capabilities.ts

Lines 26 to 33 in 83aa4af

	if (!jupyterCapsCache.has(language)) {
	// if we are targeting julia then prefer the julia installed miniconda
	const juliaCaps = await getVerifiedJuliaCondaJupyterCapabilities();
	if (language === "julia" && juliaCaps) {
	jupyterCapsCache.set(language, juliaCaps);
	return juliaCaps;
	}

juliacaps was always computed, and was supposed to be set for language Julia in kernel spec

At first detection, when jupyterCapsCache is filled in our process, I don't think language is ever set (but unsure about that). So what happened is that - in case no other python install is found, the Julia Conda installation was supposed to be used a the default one for language (none)

quarto-cli/src/core/jupyter/capabilities.ts

Lines 65 to 69 in 83aa4af

	// if the version we discovered doesn't have jupyter and we have a julia provided
	// jupyter then go ahead and use that
	if (!jupyterCapsCache.get(language)?.jupyter_core && juliaCaps) {
	jupyterCapsCache.set(language, juliaCaps);
	}

During 1.4, this was change in two commits ab5722f...48795e9 so that it "Only search for Julia conda installation when the engine language is Julia"

My understanding is juliacaps is no more check at first detection as in 1.3 and this is no more set

quarto-cli/src/core/jupyter/capabilities.ts

Lines 67 to 71 in e80b903

	// if the version we discovered doesn't have jupyter and we have a julia provided
	// jupyter then go ahead and use that
	if (!jupyterCapsCache.get(language)?.jupyter_core && juliaCaps) {
	jupyterCapsCache.set(language, juliaCaps);
	}

So when kernelspec from Markdown is checked and see julia-1.10, it will check the capabilities - there will be no language information, and so there will be none found

quarto-cli/src/core/jupyter/jupyter.ts

Lines 507 to 516 in 48795e9

	} else if (typeof (yamlJupyter) === "string") {
	const kernel = yamlJupyter;
	const kernelspec = await jupyterKernelspec(kernel);
	if (kernelspec) {
	return [kernelspec, {}];
	} else {
	return Promise.reject(
	new Error("Jupyter kernel '" + kernel + "' not found."),
	);
	}

I don't know the context of ab5722f...48795e9 changes so I need to understand further for a correct fix, but the logic for Julia detection needs to be adapted for this specific case.

@jjallaire do you remember about those commits and why "Only search for Julia conda installation when the engine language is Julia" ? Was there an issue with previous logic ?

Also I am not sure when language should be set to Julia and so Julia conda found.

[runjaj]

Have you updated Julia to 1.10 recently? If you have, you need to run this command:

Pkg.build("IJulia")

See: Updating Julia and IJulia

I’m not sure if this is the problem, but I know that it has solved me these kinds of problems (kernel not found) before.

[cderv]

Thanks @runjaj ! That is good advice indeed ! In most case, IJulia needs to be rebuild so that it finds Jupyter and configure kernel.

Though I do believe here that the issue is happening because of how quarto tries to determine kernel. I reproduced and searched through our code as documented above.

This is exclusively happening when no python installation on the system and Conda.jl only is used.

[Lightup1]

Is there a quick solution at the current status? I just followed the document to install IJulia and find that Quarto cannot find julia kernel but jupyter kernelspec list says julia-1.10 is available.

[Lightup1]

I just removed the python that quarto was using, then the quarto can find the julia kernel.
However, it still keeps warning me that "Specified QUARTO_PYTHON 'C:\Python312\python.exe' does not exist".
Actually, I checked my system environment there is no parameter named QUARTO_PYTHON.

Warning disappears after reboot the machine.