docs: ADR introducing mobile offline content support

[GlugovGrGlib]

Description

This PR introduces an ADR outlining the approach to enable offline content support in the Open edX mobile application. The ADR documents the proposed solution, including content generation, API extensions, and synchronization mechanisms.

Key Decisions:

Offline Content Generation:

ZIP archives of xBlock content (HTML, assets, and assessments) will be pre-generated during course publishing and stored for mobile download.

Retries failed generation tasks with progressive delays.

Supports both local and S3 storage.

Mobile API:

Extends the Course Home API to include offline content availability (URL, size, last modified timestamp).

JavaScript Bridge:

Enables CAPA problem submissions via JSON messages through native mobile bridges.
Partial support for hints/feedback in problem types.

Supported xBlocks:

Includes common problem types (checkboxes, dropdown, text/numerical input), text/video blocks, and partial support for blocks with hints/feedback.

Consequences:

✅ Pros: Improved accessibility for learners with unreliable connectivity, increased engagement.
⚠️ Cons: Added complexity in content sync/storage, potential app size increase.

Rejected Solutions:

• Storing shared JS/CSS files separately (tracking changes impractical).
• On-the-fly content generation (performance concerns).

Supporting information

FC-0047

[openedx-webhooks]

Thanks for the pull request, @GlugovGrGlib!

This repository is currently maintained by @openedx/wg-maintenance-edx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[angonz]

I suggest adding a comprehensive list of content types that will support offline viewing (maybe not all will be implemented at a time).
I think that video content deserves special attention. Many courses rely on video content. Open edX usually relies on external streaming services for video (YouTube, Vimeo, etc.) Video files can be huge. How will it handle offload video? Maybe uploading a downloadable version of the video in a separate storage?
Another thing to consider is milestones. If grading cannot be done offline, then probably unlocking content based on subsection grade will not be possible.

[salman2013]

@GlugovGrGlib Just for curiosity how this ADR is different from this PR?

[NiedielnitsevIvan]

I suggest adding a comprehensive list of content types that will support offline viewing (maybe not all will be implemented at a time).
I think that video content deserves special attention. Many courses rely on video content. Open edX usually relies on external streaming services for video (YouTube, Vimeo, etc.) Video files can be huge. How will it handle offload video? Maybe uploading a downloadable version of the video in a separate storage?
Another thing to consider is milestones. If grading cannot be done offline, then probably unlocking content based on subsection grade will not be possible.

Hello @angonz! I have updated the ADR and now there is a list of blocks that are planned to be supported. I also added a diagram to make it clearer how the content generation process will work. Please check it out.

As for videos, video uploading is already implemented in mobile applications.
As for unlocking content based on subsection grade, yes, such functionality is not planned within this work scope and, accordingly, it is not reflected in the ADR.

[NiedielnitsevIvan]

@GlugovGrGlib Just for curiosity how this ADR is different from this PR?

Hi @salman2013, the functionality added in the PR you mentioned corresponds to what is described in this ADR.
This ADR describes the overall vision, but focuses on how it is planned to be done on the backend.

[kdmccormick]

This is very exciting. Here's a first pass. I'll need to think through the implications over the next few days, so I should have more review comments next week--feel free to bug me if I don't get back to you with those.

[kdmccormick]

Links?

[NiedielnitsevIvan]

Added.

[e0d]

@kdmccormick was this your only blocking feedback?

[kdmccormick]

No, there is #35011 (comment), and I also have concerns about the feature's coupling to modulestore, which I will elaborate on shortly.

[kdmccormick]

I'm glad you enumerated these. I agree that it makes sense to start with a subset of XBlocks.

I think it would be good to keep the door open for future XBlocks to be supported. Could you build this in a way that would allow that? For example, can we make sure that it is possible to add support for an external XBlock like xblock-drag-and-drop-v2, without adding knowledge of xblock-drag-and-drop-v2 to edx-platform?

Relatedly, please keep in mind that we are extracting all built-in edx-platform XBlocks into a separate repo. It will take some time, but the CAPA ProblemBlock will eventually be extracted as part of that project. So, when you implement this, it is important that we are not hard-coding more CAPA knowledge into the publishing process-- the CAPA-specific archiving code should stay within the ProblemBlock definition.

[NiedielnitsevIvan]

Thank you for your question!
For offline content generation, we will use an html renderer very similar to the one used in the LMS, so there shouldn't be any significant issues with adding support for new blocks.

As for your question about xblock-drag-and-drop-v2 or other external XBlocks, yes, we won't need to add changes to the edx platform for each of them. I think that some refactoring/improvements will be needed in the future to expand the types of supported blocks, but it definitely shouldn't be for each new block separately.

As for CAPA, yes, the CAPA block will not be used.

[kdmccormick]

As for CAPA, yes, the CAPA block will not be used.

@NiedielnitsevIvan Can you clarify this statement? "CAPA" is just another name for the ProblemBlock, which implements most of the core problem types. To implement offline support for most problems, the app will need to download archives of CAPA problem data and render CAPA html/javascript.

[NiedielnitsevIvan]

Sorry, I probably misunderstood the question.
I meant that there should be no direct imports from xmodule.capa...
You can see the draft implementation in this PR.

[kdmccormick]

Thanks. Taking a look through the draft...

[kdmccormick]

Please provide the exact URL

[NiedielnitsevIvan]

Added

[kdmccormick]

What Python function will call in order to render the HTML?

Does that function require a user_id? If so, how will you handle that?

[NiedielnitsevIvan]

The new function will collect the context needed to render the block and call render_to_string of the courseware/courseware-chromeless.html template.
Some existing functions require a user, so a special service user must be created and used for this.

[kdmccormick]

Have you investigated rendering the content on the CMS side instead?

The major difference between CMS and LMS is that LMS supports user state, whereas CMS does not. So, if we just need to render content for a generic stateless enrolled user, then I think it should be possible for CMS to do this (consider that CMS is already able to render the author_view of problems for authors, which are essentially identical to what learners see in the student_view).

This would eliminate the need for cross-communication between LMS and CMS, which I sense would be challenging to implement.

[NiedielnitsevIvan]

Yes, we've been looking into this, and theoretically, after some modifications, it would indeed be possible to do rendering on the CMS side.
But besides the problem you described with the lack of user state in the case of rendering on the CMS side, even standard xblocks look a little different from what the student sees in the LMS, which can confuse the student, and the appearance of custom/non-standard blocks can be even more different from the appearance in the LMS. Therefore, in the future, this may become a blocker to expanding the types of supported blocks.

[kdmccormick]

even standard xblocks look a little different from what the student sees in the LMS, which can confuse the student, and the appearance of custom/non-standard blocks can be even more different from the appearance in the LMS.

@NiedielnitsevIvan can you elaborate or show an example? Is this because author_view and student_view can differ, or area you saying that even rendering student_view in CMS will yield a different-looking result than student_view in LMS?

Having to handle an event across LMS/CMS boundaries would make this implementation significantly less robust and more complicated. So, unless there is some severe blocker, I would prefer that we render the blocks on the CMS side, even if that requires some technical investment in CMS's rendering capabilities.

[kdmccormick]

Note: If we do end up going with LMS-side rendering, I think should use the anonymous public_views rather than creating a service user specifically for this task. This would involve some investment in CAPA's public_view, as it currently contains a comment explaining that it will only render anonymously in the Learning Core runtime, not in the ModuleStore runtime.

[kdmccormick]

I can see that this would work, but I am curious if there is a more elegant approach. I have asked in #architecture.

[NiedielnitsevIvan]

So far, we've tried the implementation described in ADR and found a problem that in this case, content generation can occur before the course structure is updated. Therefore, I propose a new attribute for SignalHandler.course_cache_updated, which will be called in update_course_in_cache_v2 and the signal listening to this event, which will start generation task after the course structure is updated.
I will add this to the ADR.

[NiedielnitsevIvan]

Updated

[kdmccormick]

Please say more about:

• How the offline app will respond when a user hits Submit
• What local state the app will store and in what format it will store it
• How the app will synchronize this state with the server

[NiedielnitsevIvan]

The JS bridge will intercept ajax requests in the mobile application and store the responses. The user will see the message "Your response is accepted" and the Submit button should be disabled.
Then, when the user gets the Internet, the mobile application will send the saved responses to the backend one by one, to the standard xblock handler.
So far, this is the behavior planned within FC-0047, which should be improved in the future within FC-7879, where it is planned to add the ability for the application to send all responses to the backend at once.

[kdmccormick]

I am concerned that without any de-duping of JS, we will be saving and serving thousands of exactly-identical copies of the ProblemBlock's javascript and css.

[kdmccormick]

Thanks for your patience. This is an ambitious and exciting project; it took a lot of research and thinking on my part develop feedback for it. Generally, pre-rendering content for offline consumption is a great idea. I have some blocking concerns with the proposed approach as it stands now:

1. The new XBlockRenderer. XBlock rendering is already complicated. Creating a new renderer specifically for offline content will make it even harder to understand and maintain, and it will open us up to behavior drift between online and offline blocks. Because the renderer would be used exclusively for offline content, the only way to test it would be to enable offline content by-default and have Build-Test-Release WG specifically test the offline content system.
2. Duplication of static resources. Duplicating CAPA JS & CSS into every single ProblemBlock (potentially 100+ per course) would be quite a waste of space, bandwidth, and server computing resources.
3. The offline service user. This raises red flags for me. To quote Dave O: Having non-real users in the database would have difficult-to-predict knock-on effects in different places that deal with users like reporting, enrollments, analytics, gradebook, etc.
4. External assumptions about block internals. The proposed system would externally determine which blocks are and aren't offline-ready. That determination would be based on assumptions about each block's internals, and we'd just have to hope and trust that the assumptions don't break in practice, or change over time as the blocks get new features, bugfixes, or refactorings.

With those in mind, I would propose to investigate an alternative approach: introduce a new standard XBlock view, offline_view. This view would render an user-agnostic fragment. In other words, student state would not be available in this view--only content and settings fields would be. Because the rendered HTML would be user-agnostic, the JSS and CSS bundled into offline_view would have to call JSON APIs defined on the XBlock in order to retrieve student state and synchronize user events.

By default, an XBlock class would not support offline_view, but it could opt-in by defining offline_view like any other view method. XBlock classes could also partially implement the view: that is, they could try to render themselves statically, or raise an exception if it's not possible based on its content+settings. For example, the ProblemBlock would need to implement something like this:

class ProblemBlock(...):
    ...
    def offline_view(self, context):
        ...
        if self.randomize != RANDOMIZATION.NEVER:
            raise OfflineViewUnavailable  # Randomization requires student state (the seed)!
        if 'loncapa/python' in response_types:
            raise OfflineViewUnavailable  # Python problems require student state!
        if ...:
            raise OfflineViewUnavailable  # ... any other thing that would require student state
        fragment = ...  # Render fragment based on content and settings
        return fragment

How would offline_view be used?

• Mobile offline mode, obviously. Upon course publish, CMS would fire off a task to walk through the course hierarchy and pre-render any blocks with offline_view available. If a block raises OfflineViewUnavailable, it is skipped. All pre-rendered blocks could be saved into a single archive and pushed to media storage. JS, CSS, and other shared block resources should be de-duped by using a hash of the file contents.
• Anonymous access. Out of scope for this project, but we can (in the future) re-implement public_view on top of offline_view. In other words, if it is possible to pre-render a block without knowing user state, then it is possible to serve that pre-renderable view as the public experience for logged-out users.
• Non-mobile offline mode. Again, out of scope, but realize that there is nothing mobile-specific about this proposal. With enough investment in our Web frontend, it could leverage offline_mode in order to make the Web experience resilient under poor connections.
• Regular student access. Even further out of scope, but this system leaves open the possibility of eventually implementing student_view on top of offline_view wherever it's supported. In other words, for compatible XBlocks, we would always serve offline-ready content, and if a user has a good connection, that's a special case.

Regarding CMS vs LMS rendering: If implemented to this design, I believe it should be possible to call offline_view from LMS or CMS and get identical results, with the exception of asset paths, which will need to be special-handled either way. Because the result is identical either way, I believe the simplicity of calling a CMS task from CMS makes it more attractive for pre-rendering than LMS.

[e0d]

@kdmccormick and I had a chance to talk through some of the details in his review. I agree that adding a view to XBlocks to support this new case is a superior approach to introducing a new facility to render XBlocks that is outside of the design we currently have.

I'm not as convinced on the duplicate asset question as I think that the transfer and storage cost of problem blocks is probably a rounding error compared to video. We should do some analysis of a highly complex course to see what the impact would be. We are trading off exported blocks being self-sufficient against optimizing transfer and storage. All to say, I think more discovery is required here.

We also discussed that even with XBlocks being able to advertise whether they support offline mode that we probably need a site-wide configuration option that allows operators to allow/deny list which blocks they want available on mobile. This would be new scope here.

It's also worth noting that because the problem block wraps different things, some of which will be offline ready and some of which will not, that there will be some messiness in the internals of the block. This would also complicate implementing the site-wide config because an operator might want to allow some types of problem renderings and not others.

As finalizing this is blocking closing our a project, I recommend that we close the project, leave this ADR open, and come back to it when offline mode is back on the frontlog.