Enable Callouts in AsciiDoc Extension for Highlight

[tajmone]

Update the highlight-treeprocessor.rb extension to add support for callouts in Highlight extension, so that Highlight can be used instead of Highlight.js for syntax highlighting HTML-based backends.

This is prefarable to using Hihglight.js for it allows static syntax highlighting, easier usage of custom syntaxes, and because Highlight offers more options to customize results.

For more info, see #99:

• Help With Callouts: Any Examples? asciidoctor/asciidoctor-extensions-lab#99

[tajmone]

Still Struggling...

Since the creation of this issue, I've updated various times my custom version of highlight-treeprocessor_mod.rb — now it fully supports substitutions, correctly allows using line-numbering, and even allows enforcing custom syntax definitions over built-in syntaxes via env vars.

Basically, it's now possible to use Highlight for the HTML backend of every document that doesn't use call-outs.

To Switch, or Not to Swtich? That Is The Dilemma

On the one hand, I'm tempted to switch to Highlight all those docs that don't use call-outs, but on the other hand I think that call-outs are such a cool feature that it might be best to wait that Highlight supports them. The problem here is that, in the meantime, Asciidoctor switched to a new syntax highlighter interface, so my extension is already obsolete (although it works).

Help! Any Rubyists Listening?

I wish I knew Ruby better, for I looked into the new Asciidoctor API for highlighters, but I just can't hack my way through it (I've lost count of the hours spent in attempting to solve this).

Unfortunately, it seems that Highlight integration has a very low priority in the Asciidoctor agenda (even though Highlight is one of the oldest highlighters on the scene, and the one offering more built-in syntaxes than any other, more energy has been invested in other highlighters).

I could really do with some help from someone fluent in Ruby!!! Together we could quickly create an extension using the new Asciidoctor API 2.0 and settle the matter of Highlight integration once and for all.

[tajmone]

Switching to Rouge?

EDIT — moved to #107!

This problem of adding callouts support to Highlight has been pending a solution for so many years that I think it might be worth considering switching to Rouge instead, and create an ALAN definition for it.

Rouge has the following pros and cons over Highlight:

• PROS:
  • It's natively supported by Asciidoctor Ruby, for both the HTML and Asciidoctor PDF backends; see: §46.2. Available Source Highlighters.
    • When the Asciidoctor v2.0.0 update will be released, syntax highlight via Rouge won't break (whereas our Highlight extension will probably stop working).
  • It's written in Ruby, so it integrates well with Asciidoctor Ruby.
  • Its Lexer is state-driven, so it can handle powerful syntax constructs (e.g. nested states), pretty much like Pygments.
  • Doesn't require end users to install another language dependency (like Python for Pygments), since Asciidoctor users will have installed Ruby already.
  • Its themes are 100% compatible with Pygments stylesheets.
• CONS:
  • Only supports UTF-8! so we might have problems with ISO-8859-1 ALAN sources (but this is already a problem with Asciidoctor anyhow).
  • The documentation on how to write a Lexer was lacking (or poor) in the past; but now it seems it's been documented, and that some tutorials are available.
  • Using a custom syntax for the repository might be a bit trickier, compared to Highlight.

[tajmone]

Updated Asciidoctor Manual

See:

• https://docs.asciidoctor.org/asciidoctor/latest/syntax-highlighting/custom/

[tajmone]

New Highlight 4 Feature Request at Asciidoctor

Follow also this new Issue I've opened:

asciidoctor/asciidoctor#4069

[tajmone]

Downgraded From "Important" to "Useful"

Now that we can also use Rouge for highlighting, Highlight is only used for those documents which need to add spans or highlight/marked to code blocks (which is currently unsupported in Rouge), so fixing the HL ADoc Extensions is no longer a high priority (still, it would be nice to see it done).