Intro

For several years now, I’ve been using Markdown for pretty much all my formatted text writing needs. It’s easy to learn. It’s portable and the resulting documents can be easily post-processed to output a variety of versions of your Markdown document, such as HTML, PDF, etc. In fact, way back in 2012, I wrote a learned treatise about Markdown, in this very boutique.

Any roads up, whilst Markdown is pretty nifty, it does have its limitations. There’s not much support for styling text [outside of basics like bold and italic], or for aligning or justifying text, and image positioning requires a bit of hackery-pokery which is not part of the official spec…​ or rather, wouldn’t be, if there actually was an official spec.

One of the other problems with Markdown is precisely that there isn’t an official spec [unless you count John Gruber’s announcement post with I linked to above]. This means that, in attempt to address these shortcomings, the world and his wife has come up with their own "flavours" of Markdown, most of which are not completely compatible with each other.

T’is all turning into a bit of a mess.

Anyway, a couple of weeks back, after being irritated by something I was trying to achieve in Markdown, I thought I’d look around for an alternative. After briefly considering Restructured Text, I decided to plump for AsciiDoc, which uses a lot of similar syntax to Markdown, but offers a lot more typographical [and image] control. And also benefits from having a standard, which means you don’t have to worry about what particular "flavour" you’re going to opt for.

AsciiDoc Meets Hugo

At the moment, this 'ere site is built using Hugo, a static site generator which chews up various templates & Markdown content pages and converts them to this astounding HTML website, that my vast readership craves.

Any roads up; one of the 'supposedly' cool features about Hugo [in comparison to other SSGs out there], is that, as well as Markdown, Hugo supports your content being written in various other markup languages, including AsciiDoc.

In order to process AsciiDoc, Hugo needs you to install the asciidoc and asciidoctor Ruby Gems as external helpers. So I did this…​

$ gem install asciidoc
$ gem install asciidoctor

…​and then set about converting my 300 odd posts from Markdown to AsciiDoc.

Given the generally fairly basic Markdown used in most of the posts here, I thought that this might be a reasonably easy task, involving not much more than a hefty Find/Replace session in my text editor. But, I decided to see if there was "an app for that" first. And, luckily, there was.

kramdown-asciidoc is another Ruby goody that can convert documents from Markdown to AsciiDoc. So I installed that [see the link for instructions] and then set about converting all my Markdown documents [which live in the content folder inside my Hugo site project] into AsciiDoc ones.

I couldn’t find a way of getting kramdown-asciidoctor [or kramdoc in its command-line parlance] to do this as a batch. I tried pointing it at content/* but it didn’t like that use of a wildcard. So I ended up knocking together a quick script in V-lang to 'walk' the content directory and pass each .md file to kramdoc for processing.

import os

fn main()
{ //start main
	// some vars
	workingdir := os.getenv('HOME')
	contentpath := '$workingdir/Sites/stiobhart.net/content/'
    extension := "md"

        file2process := os.walk_ext(contentpath, extension)
        mut i := 0
        for i=0; i < file2process.len; i++
        {
        	println(file2process[i])
            kramdoccmd := ("kramdoc "+file2process[i])
            println("Going to execute --> $kramdoccmd")
            os.system(kramdoccmd)
        }



}//end main

That worked a treat and I ended up with an AsciiDoc version of each Markdown file in my content directory.

I extracted all the .md files from my content folder and backed them up elsewhere, just in case everything went horribly wrong and I needed to roll back. Then I fired up hugo server, to build the new AsciiDoc version of my site.

And that’s when I ran into into my first small problem which, as it turned out presaged Hugo’s more serious shortcomings, when it comes to processing content authored in AsciiDoc.

What’s the matter with my Shortcodes?

One of the features of Hugo is that it allows you to create 'Shortcodes'. These are reusable chunks of Golang template code, which you can store in separate files and include whenever needed, to save you, when composing your articles, from having to write loads of boilerplate HTML, to cover things it’s not possible to do natively in Markdown.

I make heavy use of a {{ figure }} shortcode I’ve written which allows me to place an image in my Markdown file with a simple one-liner:

which, when Hugo builds the site is converted into:

<figure>
<img src="https://stiobhart.net/someimage.jpg" alt="somecaption">
<figcaption>somecaption</figcaption>
</figure>

I have created other Shortcodes for such things as embedding Vimeo or YouTube video, creating image slideshows, etc.

Anyway, when I built my new AsciiDoc-flavoured site and previewed it, I found that all my shortcodes were being printed verbatim to the page, instead of being rendered into the appropriate HTML.

I opened a thread about this on the Hugo Discussion forum while I tried to find an answer.

Fortunately, as it turned out, the answer was pretty simple, although I had to suss it out for myself, as I got no response on the forum and nor could I find anything online. The secret lay in RTFM ie. the AsciiDoc syntax guide and particular the bits relating to Passthrough Macros.

It turned out that all I had to do was insert a line containing +++ at the beginning of the shortcode and a similar line at the end, to make AsciiDoctor "pass through" the content unaltered to Hugo er…​ or vice-versa. I’m not sure in exactly what order things are processed internally, when building the site. But, whatever the nitty-gritty is, escaping the AsciiDoc content in this way causes the {{ figure }} shortcode to be correctly processed and the image in question to display.

Actually, that’s not entirely true. Again for reasons beyond me, I found that the triple plus caused the resulting <figure> tag to be further wrapped in <p> tags, which caused unwanted extra spacing around my images. However, using a quadruple plus ++++ around the shortcode causes it to render as desired.

++++
{{ "<!-- begin image shortcode //-->" | safeHTML }}
<figure {{ with .Get "class" }}class="{{.}}"{{ end }}>
{{ with .Get "link"}}<a href="{{.}}">{{ end }}
<img src="{{ .Get "src" }}" {{ if or (.Get "alt") (.Get "caption") }}alt="{{ with .Get "alt"}}{{.}}{{else}}{{ .Get "caption" }}{{ end }}"{{ end }} />
{{ if .Get "link"}}</a>{{ end }}
{{ if or (or (.Get "title") (.Get "caption")) (.Get "attr")}}<figcaption>{{ if isset .Params "title" }}{{ .Get "title" }}{{ end }}{{ if or (.Get "caption") (.Get "attr")}}{{ .Get "caption" }}{{ with .Get "attrlink"}}<a href="{{.}}"> {{ end }}{{ .Get "attr" }}{{ if .Get "attrlink"}}</a> {{ end }}{{ end }}
</figcaption>{{ end }}
</figure>
{{ "<!-- end image shortcode //-->" | safeHTML }}
++++

More problems

Triumph was short-lived however, for it wasn’t long after solving the Shortcode problem that I ran into another one.

As hugo server ran, I noticed my terminal window filling up with hundreds of ERRORs/WARNINGs from asciidoctor via Hugo:

<snip>
...
ERROR 2020/03/30 11:23:02 2013-03-15-ios-prototyping-prerequisites.adoc: asciidoctor: WARNING: <stdin>: line 52: section title out of sequence: expected level 1, got level 3
ERROR 2020/03/30 11:23:02 2013-03-15-ios-prototyping-prerequisites.adoc: asciidoctor: WARNING: <stdin>: line 77: section title out of sequence: expected level 1, got level 3
ERROR 2020/03/30 11:23:03 2015-02-27-samsung-android-memory-error.adoc: asciidoctor: WARNING: <stdin>: line 48: section title out of sequence: expected level 1, got level 2
ERROR 2020/03/30 11:23:06 2012-05-12-getting-stuff-into-and-out-of-your-ios-devices.adoc: asciidoctor: WARNING: <stdin>: line 7: section title out of sequence: expected level 1, got level 3
ERROR 2020/03/30 11:23:06 2012-05-12-getting-stuff-into-and-out-of-your-ios-devices.adoc: asciidoctor: WARNING: <stdin>: line 17: section title out of sequence: expected level 1, got level 3
ERROR 2020/03/30 11:23:06 2012-05-12-getting-stuff-into-and-out-of-your-ios-devices.adoc: asciidoctor: WARNING: <stdin>: line 27: section title out of sequence: expected level 1, got level 3
ERROR 2020/03/30 11:23:06 2012-05-12-getting-stuff-into-and-out-of-your-ios-devices.adoc: asciidoctor: WARNING: <stdin>: line 35: section title out of sequence: expected level 1, got level 3
...
<snip>

And, after what seemed to be an interminably long time—​Hugo builds the markdown version of this site in a few seconds—​the hugo server build command bailed out with…​

Error: Error building site: logged XXX error(s)

…​with XXX being some ridiculously huge number.

There were a few odd things about this:

Clunky Initial workaround

Since there’s no way of making hugo server run 'standalone' without building the site first, I had to resort to firing up a one-liner Python webserver on the same port [1313] that hugo server uses, just so I could preview my Hugo site. Which added yet more hassle and waiting to an already bloated build time!

python3 -m http.server 1313 --directory /Users/madra/Sites/stiobhart.net/public

Anyway, as I said, the HTML files had all built correctly and, using the Python server, I could preview my site and make sure everything was working as it should. So, from that side of things, all was good. But, what to do about the fact that Hugo refused to build the site [and took an age to decide not to do so]?

The cause of the WARNINGs

A bit of research on the WARNINGs that asciidoctor was throwing up revealed that they were caused by the fact that AsciiDoc likes your documents to be logically structured in that they should start with a 'level two' header, which is rendered as an <h1> and then subsequent sub-headers and sub-sub-headers should use <h2>…​ <h3>…​ and so on.

I’d actually fallen into a bad habit of using <h3> headers as section titles throughout most of my Markdown documents. So, when these were converted to AsciiDoc syntax and then processed, the asciidoctor processor complained: section title out of sequence: expected level 1, got level 3 --ie. my documents had lots of <h3> headings in them but these weren’t following on from <h1> and <h2> ones.

OK. I’ll admit that this is entirely my fault. I should have been using <hn> headers semantically, instead of as a convenient shortcut to get section titles of a size I wanted. But that’s beside the point. asccidoctor was quite happy to process the pages anyway and just complain about the headers being 'out of order', but Hugo was receiving these harmless WARNINGs, inwardly reclassifying them as fatal ERRORs and consequently reporting that the site build had failed and therefore refusing to boot up its internal server.

Your humble author again vainly asks for help

I mentioned this problem on the Hugo Discussion Forum but, like my previous post re Shortcodes, it didn’t get any response.

I also posted it as an Issue on Hugo’s Github. Again, no response. So, as with the Shortcodes issue, it looked like I was going to have to solve the problem by myself.

First attempt at a solution

Trying to research a solution to this problem, I came to the unfortunate conclusion that either;

because I only found about 2 or 3 seemingly relevant articles across the entire intarwebs. Which means that this post has vastly increased the sum total of human knowledge on that particular subject—​if not on anything else!

[Although, again, in the interest of fairness, I’m converting an existing and fairly large Markdown Hugo site to AsciiDoc. Anyone writing in AsciiDoc from the outset is likely to have stamped out these issues at the very early stages of creating their site and probably not thought them worth documenting]

I did find out that the command line options which Hugo passes to asciidoctor are actually hard-wired in and it’s not possible to override them or pass any other options in [Tsk! Tsk!---not very helpful design].

This was unfortunate as I’d been hoping to pass the --quiet option to asciidoctor so it would shut these WARNINGs up and let hugo server do its stuff without thinking anything was amiss.

Fortunately though, I came across this post by 'Ratfactor' wherein s/he outlines a sneaky way of passing commandline options to asciidoctor when called from Hugo. You can read the details in the post linked to but, in essence, it involves mv-ing your asciidoctor binary to another name [asciidoctor-real in the example] and then replacing it with a fake asciidoctor binary which is really a shell script which calls the genuine and newly named asciidoctor-real with whatever options you want to pass to it.

It’s a bit "hacky" but it works:

#!/usr/local/bin/zsh
#this file is not the real asciidotor binary but points to it and passes some
#command line options. Needed coz Hugo has crappy options hardwired in
/Users/madra/.gem/bin/asciidoctor-real --quiet "$@"

Using that hack, to suppress any asciidoctor warnings, I was able to get Hugo to build the site and run its preview server, without bailing out.

Unfortunately it made no difference to the build times at all. The AsciiDoc version of the site was still taking around 60 or 70 seconds to build, compared to only a few seconds, when it was written in Markdown. I had hoped that it had been the asciidoctor WARNINGs that had been slowing Hugo down and that, without them, the build would be faster. But, unfortunately not.

Incidentally, just to eliminate the possibility that asciidoctor itself that was the culprit in causing the slowdown, I ran asciidoctor standalone on my content directory and it converted the 300+ .adoc files in there to HTML in barely a couple of seconds. So, wherever the problem lay, it wasn’t with asciidoctor itself, but somehow related to Hugo’s use of asciidoctor to process .adoc files.

Cleaning up the WARNINGs

For my next nibble at the problem, I decided to eliminate the WARNINGs from asciidoctor so that, even though it now seemed unlikely that these were having any effect on the build times, I could rule that out as well.

Out with the venerable but still the best GUI text editor by far, for OSX and a hefty Find/Replace session and I managed to kick all my .adoc documents into proper semantic shape [basically by replacing all the ==== and === headings with == ones]. Whilst doing so, I removed the --quiet flag from my 'pretendy-asciidoctor' shell script and kept fixing and rebuilding the site [reverting to Python for the testing server again, whenever hugo server bailed out] until, finally, the site built with no Hugo ERRORs nor ascidoctor WARNINGs.

Sadly, this didn’t make any difference either. The site still takes 60-70 seconds to build using hugo server which is really fecking annoying. But at least the live rebuild, after a file has been edited, happens almost instantly, like it used to in the good old days.

Conclusion

So, as the dust settles, the obvious question is; 'Do I regret moving from markdown to asciidoc?'

If you’d asked me that a few days ago, I’d probably have said 'Yes'. But now that I’ve got it sort of working [albeit with an irritating first startup delay] I’m pretty chuffed that it’s done now.

I still think Markdown is a great markup tool to use. But, after several years, I found I was running up against its limitations more and more often. So I’m chuffed to have been able to convert my site to AsciiDoc relatively painlessly and am looking forward to digging deeper into just what AsciiDoc can do, both for site generation and for any of the other drivel I need to write.