How to include external files (qmd, md) when rendering and deplying via Github actions?

[wiesehahn]

Description

We have a website which we render and deploy via Github actions to Github pages.

# file: .github/workflows/publish.yml
name: Publish Website

# Allow one concurrent deployment
concurrency:
  group: "pages"
  cancel-in-progress: true

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
  contents: read
  pages: write
  id-token: write

on:
  push:
    branches: ['main']

jobs:
  quarto:
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Install Quarto
        uses: quarto-dev/quarto-actions/setup@v2
      - uses: r-lib/actions/setup-r@v2
      - uses: r-lib/actions/setup-r-dependencies@v2
        with:
          packages:
            any::rmarkdown
            any::knitr
      - name: Setup Pages
        uses: actions/configure-pages@v5
      - name: Render Website
        run: quarto render
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v3
        with:
          path: '_site'
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v4

Now we would like to add content to a post from an external repository/file. We could of course copy paste the content but to keep just one code base we would like to include the external content.

---
title: Some Title

description: |
  Some description

categories:
  - Some category

date: today
---

<!--------------- post content ----------------->

{{< include https://raw.githubusercontent.com/quarto-dev/quarto-cli/main/CONTRIBUTING.md >}}

Like this it does not work and we receive

ERROR: Include directive failed.
  in file /home/runner/work/username.github.io/username.github.io/posts/post_folder/index.qmd, 
  could not find file /home/runner/work/username.github.io/username.github.io/posts/post_folder/https://raw.githubusercontent.com/quarto-dev/quarto-cli/main/CONTRIBUTING.md.

What is the correct way of including and rendering external markdown content?

[cscheid]

Quarto currently doesn't support that usage and we don't intend it to. Quarto resolves images in URLs because that's required for embed-resources and formats like PDF to work correctly, but this usage makes it impossible to determine things like the engine under which the content will need to execute without going to the web.

Instead, you can create a pre-render script that fetches the readme and puts it somewhere that's visible to the rest of your project.

[jimjam-slam]

If it's only Markdown (no computation) and you really need it to be up to date based on when the user loads the page (rather than on page render), you might be able to write an OJS block to fetch the Markdown file and pop it straight into the doc with HTL, like:

```{ojs}
fileContent = FileAttachment("https://example.com/some/readme.md").text()

md`${fileContent}`
```

Important

I feel like you'd want to think really carefully about any possible security implications of that, though — you'd want to be pretty confident that you trust the source of the Markdown file and that there's no way anything can end up in it that'll have adverse effects for users (for example, I'd want to know how OJS's Markdown renderer handles iframes or other HTML tags). I definitely wouldn't do it with an HTML tag, especially if the remote file is hosted elsewhere.

[wiesehahn]

Yes you are right, it's only Markdown, I simply want to fetch it and render it as part of the html site with the same style (css/scss). It would definitely be an asset that the included markdown is fetched and rendered directly from external source on page loading.
@jimjam-slam Simply inserting your OJS Block with a link to some markdown does not work for me.

But for now I could also live with loading and rendereing the content on page rendering (in the example above whenever something is pushed to the main branch). This would not mean the most up to date version is automatically rendered, but at least I dont have to maintain a copied version of the external markdown.

Instead, you can create a pre-render script that fetches the readme and puts it somewhere that's visible to the rest of your project.

Is there some resource how to do this?

[mcanouil]

This would not mean the most up to date version is automatically rendered, but at least I dont have to maintain a copied version of the external markdown.

Using CI/CD workflows it is definitely something you can do.

Is there some resource how to do this?

Pre-render scripts are regular scripts executed before Quarto renders the files. The scripts have access to some environment variables set by Quarto.
See the documentation: https://quarto.org/docs/projects/scripts.html#pre-and-post-render