TTS readaloud highlighting styles

[danielweck]

I am attaching screenshots in this GitHub "discussion" to illustrate a new feature we are testing in Thorium (the GUI is not final yet). The static screen captures are pretty self-explanatory, I wanted to demonstrate various combinations of presentation settings (including light and dark modes), in different languages / scripts.

As you can see, users can now pick highlight colours from a palette (same as for annotations, with the additional yellow-background + orange-underline which has been the TTS default from day one). Users can also select different highlighting styles for the TTS utterance and for the word tracking, including solid background, underline, outline, mask, or a block mask which spans the width (or height for vertical Japanese text) of the document.

I am pinging the following GitHub users who I think will be interested (based on previous interactions in the issue tracker): @ways2read @sjabsr @gautierchomel @HadrienGardeur @civodulab @CircularKen @LaZay @tedvandertogt @gregoriopellegrino @CLnordcompo @mlouie-prcvi @brenojac @oscarlcarlsson @dazrand @togregg @MyDK @Moonbase59 @hajonnes @dolevu @7cio @zstanecic @gexgd0419 @mayuhamada

As usual, Thorium test builds are available in GitHub releases: https://github.com/edrlab/thorium-reader/releases

[ways2read]

These look great, kudos on the amazing work! I was able to test these in the alpha build.

Unfortunately, I will not have a chance to do any end user demos in the next couple of weeks, so these brief comments are based only on my past experiences.

Allow highlights to be turned off/hidden

Whilst most users find visual highlighting in read aloud extremely useful, over the years I met some who complained that it is distracting or worse. In this case I have directed them to solutions where the highlighting can be turned off, or the colour set to the same as the background rendering it invisible. In the alpha release I do not see an option for "No highlighting" and the range of colours do not allow for the second approach.

Perhaps the simplest would be to offer a TTS highlight style of "None". However, I feel that the "mask" and "block mask" experiences may be appreciated without the word highlighting. My concern then is that setting the colours to the same as the background color is quite a lot for someone to work out.

I am afraid my recommendation is to do both! Offer a TTS highlight style of "None" and also extend the range of possible colours to include white, black and sepia.

Allow the position of mask and block mask to be fixed in position

It would be wonderful if there was an option to lock the position of the text being read, for example in the centre of the display. I believe there are some users who would appreciate this, including but not limited to those with limited field of vision or who use eccentric viewing or steady eye strategies.

I think this is a big ask though.

[danielweck]

Hi Richard, thank you for your comments. I will address each point separately.

[danielweck]

Allow the position of mask and block mask to be fixed in position. It would be wonderful if there was an option to lock the position of the text being read, for example in the centre of the display. I believe there are some users who would appreciate this, including but not limited to those with limited field of vision or who use eccentric viewing or steady eye strategies.

Good suggestion. This "stay centered" feature is currently implemented in the "captions / clean view" mode, but the concept could be generalised to any document presented in scroll mode (column pagination rigidly snaps on the "page" boundaries so the active text cannot be "moved" to an arbitrary location in the content viewport).

Normal document scrolling view, in dark mode with a user-selected serif typeface and increased line spacing, playing TTS using the "mask / block" highlighting style:

Caption view (picks-up that the user selected a "dark" mode, but applies its own non-configurable presentation styles), displays the raw utterances outside of the context of the originating document (completely ignores publisher styles) by rendering one utterance per line / block of lines:

[danielweck]

I am afraid my recommendation is to do both! Offer a TTS highlight style of "None" and also extend the range of possible colours to include white, black and sepia.

Slight topic deviation before I steer back into Text To Speech:

In the spirit of "keeping things simple" (for end users, and to some extent for application developers / maintainers too), Thorium introduced its annotation feature with a limited colour palette. Another practical reason is that exported / imported annotations that rely on "named colours" (i.e. a preset of symbolic, translatable names instead of arbitrary colour codes, such as RGB triplets / hex, or other encodings) will hopefully facilitate interchange with other reading systems.

As a technically-minded end-user, I personally would not mind a feature-rich colour picker (i.e. a greater selection of recommended colour swatches, or even a full "colour wheel"), but I understand the tradeofs and I appreciate the benefits of UI / UX simplicity, and baseline interoperability. Better aim for a realistic lowest common denominator that stands a chance to be adopted by other actors in the ecosystem. EDRLab is set to participate actively in W3C-DPUB standardisation efforts regarding a pragmatic format for annotations echange, essentially a continuation / revisitation of the existing specification.

Thorium ships with several predefined non-customisable colours themes. At the moment the highlighting styles for "TTS" (as discussed here), "annotations" (currently a 6-colour palette), and "search results" (currently hard-coded to green and pink depending on inactive / active state) work reasonably well with all the themes, in the sense that the contrast ratio is acceptable. In the ReadiumCSS project we have started discussing how to make these disparate sub-systems more coherent. Ideally, Thorium would offer a unified user experience when selecting a colour theme, including a best-match highlighting preset (ping @JayPanoz ).

So, I agree about a wholesale "no highlight" option ("turn everything off" !). One might argue that a "no word tracking" option would be nice too (while allowing utterance highlighting at the sentence / paragraph level), but this can easily be achieved with the same-colour trick. As for extending the choice of TTS colours, I propose that we don't rush into adding specific combinations unless this ties neatly into Thorium's existing themes. I suggest that we think this through as part of the ReadiumCSS effort.

PS: EPUB3 Media Overlays synchronised text-audio highlights are configured by CSS styles / classes authored by content creators. Thorium does not currently allow users to customise / override the publisher's choice of highlighting style and colour. Typically, this is a solid background / foreground combination, but it's "just CSS" so it could be anything (which makes it difficult for reading systems to take over colours only ... essentially the RS must override the entire style of the active playback element).