Media Overlay: "On demand audio" mode, interactive control.

[sschule]

Hi everyone. I'm currently working on a demo of an interactive Comic / BD ePub with audio included.

My first idea is to use absolute positioning of translucid div over each dialog/text zone. It works fine inside Thorium Reader, but continuous reading is not the way to use it. People read comics at a very personal speed.

I could not find a way to use Thorium so that it plays audio "on click", and does not continue after current element. Did I miss it? Either you start the reading, and it doesn't stop automatically, or else it doesn't play anything on click.

This would be a great feature for people using textual Media Overlay books as well. People learning the language or having difficulties reading might prefer to choose when the next sentence is starting.

Both "Next sentence" on click anywhere on the screen and "Play on click" on a specific zone would be very useful, if not necessary features for both these use cases.

Simon Schulé
Bibliothèque Sonore Romande, Lausanne, Switzerland

[danielweck]

Thank you, I posted the feature request: #2797

[danielweck]

Hello Simon, please let us know if the new feature works for you. You will be able to try it once the automated build is finished (all green):
https://github.com/edrlab/thorium-reader/actions/runs/13338809838

Scroll down to download a build for your operating system / architecture:
https://github.com/edrlab/thorium-reader/releases

To try the new feature with TTS readaloud or EPUB3 Media Overlays, tick the checkbox labeled "Disable continuous playback" in the playback options (you must start playback first, then the configuration icon will appear and you will be able to reveal the options panel).

[sschule]

Hi Daniel,

That was amazingly fast, thank you ! Those "Playback options" are not easy to notice, (the configuration icon is not expression "settings"), but it is great.

If it was possible, it would be great to keep the last sentence/zone highlighted after the stop, for people reading slowly. For DYS readers, it is often tricky to find back the location of the current position. Is it? Not urgent for the demos we are making, but would be even better.

Great work with this player.

Cheers,
Simon

[danielweck]

Hello Simon, as you know (but just stating here for other readers) content creators / authors can include CSS styling for the standard Media Overlays "active" classes that highlight (i.e. put visual emphasis on) the currently-playing audio+text fragment, as well as the top-level document root element (combining the two can achieve interesting effects, like masking inactive content to provide even more contrast for the highlighted fragment). However there is no class for the "paused" state, which in Thorium under the hood would be the "current reading location" that gets outlined temporarily during navigation (e.g. TOC hyperlink). In other words, each reading system would choose an arbitrary highlighting style for the "paused" state ... I am not sure that would be the right approach. Another option would be for Thorium to introduce support for a non-standard "active / paused" CSS class during EPUB3 Media Overlays playback, but again this kind of ad-hoc proprietary extension seems wrong.

[danielweck]

Simon, I completely agree about the options for TTS readaloud and EPUB3 Media Overlays playback, which only reveal themselves once the user starts playing the content. Ideally the "settings" affordance would be accessible without requiring active play. The configurable boolean options toggles are: enable simple/caption view,
enable skippability, disable continuous, enable sentence detection / fragmentation (this last one is TTS only, doesn't apply to EPUB3 MO).

[sschule]

Hi Daniel, thank you for the feedback. I think I understand how the classes dynamically change during playback, and how CSS is used to react.

If there is an "active" class on the current segment while playing, why does it have to be removed when pausing? Cannot a div stay "active", without adding extra information about the state change, until another element gets focus, by interaction or by activating "play"?

[sschule]

It is very clear indeed, thank you. Back to default colors when playback stopped.

I am note sure about the flexibility of the terms, but it could be argued that playback is not stopped when "on hold", even if it is not a formal new state. Active status is not formally linked to the audio aspect of the playback, or is it?

Could it not be considered as a long silence, from the "duration of that element's playback"? Probably a bit far stretched.

Thank you for your time. I can work with that, but I am pretty convinced of the usefulness of the feature for users with reading and focus disabilities.

[danielweck]

2013 :)
w3c/epub-specs#428

[danielweck]

In response to this 2013 issue, back in 2016 I wrote: "The proposal is to add corresponding media:playback-paused-class and media:paused-class, respectively. This would allow authors to define visual styles specifically for the "paused" state (e.g. CSS highlighting of the location from which playback will resume)"
:)

[sschule]

12 years ago, wow. Well, the need is more explicit and obvious now that we make audio-text syncronized books for the reading impaired.

The process of selecting new features for next revisions is very impressive, I had not seen it before. Soo, is it being considered for the 3.3 revision? Is see the label "The issue affects the core EPUB 3.3 Recommendation".

Is there a way for us to express our support for this feature? I have no idea how this works.

[danielweck]

Feel free to chime in, it is an open GitHub issue. There's a chicken-and-egg problem though, because content authored with the new hypothetical "paused" CSS class (assuming this passes the standardization process) would only work with a very small number of reading systems (probably just Thorium) for a very long time.

[sschule]

Well, from what I read, there are dozen of billions of chickens on earth, that problem has been solved ;)

We are having good conversations with Dolphin EasyReader developer team as well, as we are using their app ecosystem. I think they would be interested in implementing it if partner libraries and users ask for it.

Thanks again.