[DOC] dynamic session loading in non jupyter environments (marimo, python scripts, streamlit) #4497
Conversation
Signed-off-by: lucharo <luis@merqato.eu>
lucharo
changed the title
Signed-off-by: lucharo <luis@merqato.eu>
There was a problem hiding this comment.
Thanks a lot for this contribution @lucharo !
A thought: would it be useful to modify https://docs.kedro.org/en/stable/kedro_project_setup/session.html too? Asking because yesterday I copy-pasted that block on a notebooks/main.ipynb and failed because of similar reasons. So perhaps this is not just a marimo problem.
It does make use of private API though, so I brought this PR to the attention of the team in #4473
Having said all this, I don't want to block this PR on these discussions, which might take long 🙏🏼 So I'll leave some comments just on the changes you made.
docs/source/integrations/marimo.md
Outdated
| # How to use Kedro from Marimo and non-Jupyter environments | ||
|
|
||
| This guide explains how to set up Kedro programmatically without relying on IPython magics, making it compatible with non-Jupyter environments (python scripts or streamlit apps) and modern notebook interfaces like Marimo. | ||
|
|
||
| ## Overview | ||
|
|
||
| Kedro Jupyter notebook users rely on the magic command `%load_ext kedro.ipython` to source their catalog, context, pipelines and session objects, see [Kedro and Notebooks](../notebooks_and_ipython/kedro_and_notebooks.md) for more. This magic command automatically finds the location of the root kedro project, avoiding having users hardcode paths to their configuration files. | ||
|
|
||
| Such magic commands are not available in modern notebook interfaces such as marimo or in simple python scripts. Yet, using Kedro for sharing configuration, credentials or the data catalog across a project is still desired when developing data science projects. |
There was a problem hiding this comment.
While factually correct, I think I'd focus this page on describing how to run Kedro on marimo, so trim down this introduction a bit. Some other possible additions:
- Do not assume that the reader knows marimo, hence include a sentence linking to its official website and explaining briefly what it is
- Add the necessary
marimo editormarimo runcommands - Add a screenshot that demonstrates how this looks like
There was a problem hiding this comment.
Would you want to focus solely on marimo? I think it could be relevant to talk about non jupyter environments as well: streamlit, python scripts or even cli scripts
There was a problem hiding this comment.
and also, what is the screenshot meant to show? I think what I might do is create a start that uses marimo, I'd need a screenshot of a starter project that's not my work projects anyway
|
@astrojuanlu I've changed the block in https://docs.kedro.org/en/stable/kedro_project_setup/session.html I will have a look at kedro starters and hopefully get more content for a doc page, feel free to re-write anything of what I've written. I thought it made sense to treat it as a non jupyter doc page rather than just marimo but it's you guys call the shots! |
Description
As discussed in #4440 currently kedro users that decide to not use jupyter for exploration cannot dynamically load their kedro config (catalog, credentials, etc) as those non jupyter environments lack the
%magic commands that kedro currently relies on for its ipython/jupyter integrationThis PR simply documents how to go ahead with such integration with today's codebase, making use of the private
kedro.utils._find_kedro_projectDevelopment notes
nothing changed in development, just a doc PR
Developer Certificate of Origin
We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a
Signed-off-byline in the commit message. See our wiki for guidance.If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.
Checklist
RELEASE.mdfile