tur/markdown
CommonMark Markdown: parse a document, render it as HTML.
markdown/parse reads a String into an MdDoc and markdown/render-html writes it out; markdown/doc-free releases it. The parser follows the CommonMark spec, 0.31.2, and passes all 652 of its examples. With markdown/default-opts it adds GitHub's four extensions -- tables, strikethrough, extended autolinks and task list items -- as cmark-gfm 0.29.0.gfm.13 implements them, and passes the 24 GFM extension examples. MdOpts's math field adds TeX math -- $...$, $$...$$ and $$ display blocks -- written out for KaTeX's auto-render to find. The renderer can give headings ids by python-markdown's rule or GitHub's (MdSlugMode). tests/run-markdown-spec.sh runs the spec's examples.
The tree is walked by id: markdown/children and markdown/block for blocks, then markdown/inlines, markdown/inline-children and markdown/inline for a paragraph's, heading's or table cell's inline content. markdown/headings and markdown/code-blocks list the blocks of those kinds, wherever they are.
Pure Turmeric apart from the Unicode, entity and heading-id tables, which the interpreter carries as natives, so it runs compiled, under the JIT and under --interpret.
Since: markdown-library-plan M1
markdown/default-opts
(markdown/default-opts :MdOpts)
the GFM extensions on, math off.
An MdOpts with tables, strikethrough, autolinks and task lists on.
(.tables (markdown/default-opts)) ; => true
Since: markdown-library-plan M1
markdown/commonmark-opts
(markdown/commonmark-opts :MdOpts)
plain CommonMark: every extension off.
An MdOpts with every field false.
(.tables (markdown/commonmark-opts)) ; => false
Since: markdown-library-plan M1
markdown/default-html-opts
(markdown/default-html-opts :MdHtmlOpts)
the renderer's defaults: raw HTML through GitHub's tag filter, and no heading ids.
The output is the CommonMark spec's and cmark-gfm's. For heading ids, build an MdHtmlOpts with (MdSlugPython) or (MdSlugGitHub) instead.
An MdHtmlOpts whose raw-html is MdRawTagFilter and heading-ids MdNoIds.
(markdown/render-html d (markdown/default-html-opts))
Since: markdown-library-plan M1
markdown/parse
(markdown/parse [src :String opts :MdOpts] :MdDoc)
parse Markdown source into a document.
Any String parses: Markdown has no syntax errors. Lines may end in LF, CR or CR LF, and a NUL byte reads as U+FFFD. The document keeps its own copy of what it needs, so src can be released straight away.
| src | the Markdown text (borrowed) | |
| opts | the extensions to recognise |
A fresh MdDoc; release it with markdown/doc-free.
(markdown/parse (string/from-cstr "# Hi") (markdown/default-opts))
Since: markdown-library-plan M1
markdown/render-html
(markdown/render-html [d :MdDoc opts :MdHtmlOpts] :String)
the HTML for a document.
Block structure as the CommonMark spec writes it: one line per block tag, a tight list's paragraphs without <p>, a code block's info string's first word as a language- class.
| d | the document (borrowed) | |
| opts | how to treat raw HTML |
A fresh owned String.
(markdown/render-html d (markdown/default-html-opts)) ; => "<h1>Hi</h1>\n"
Since: markdown-library-plan M1
markdown/doc-free
(markdown/doc-free [d :MdDoc] :void)
release a document and everything it holds.
| d | the document; do not use it afterwards |
(markdown/doc-free d)
Since: markdown-library-plan M1
markdown/escape-html
(markdown/escape-html [s :String] :String)
s with & < > and " replaced by their HTML entities.
| s | the text to escape (borrowed) |
A fresh owned String, safe to place in HTML text or a quoted attribute.
(string/to-cstr (markdown/escape-html (string/from-cstr "a<b"))) ; => "a<b"
Since: markdown-library-plan M1
MdBlockId
(defstruct MdBlockId :copy [ix :int])
a block of a parsed document. Valid only with the MdDoc it came from.
MdAlign
(defdata MdAlign :copy (MdAlignNone) (MdAlignLeft) (MdAlignCenter) (MdAlignRight))
a table column's alignment, from the colons of its delimiter cell.
MdTask
(defdata MdTask :copy (MdNotTask) (MdUnchecked) (MdChecked))
a list item's GFM task box: none, [ ], or [x].
MdBlock
(defdata MdBlock :copy (MdDocument) (MdBlockQuote) (MdList bool int bool) (MdItem MdTask) (MdParagraph) (MdHeading int) (MdThematicBreak) (MdCodeBlock bool) (MdHtmlBlock) (MdTable int) (MdTableRow bool) (MdTableCell MdAlign) (MdMathBlock))
what a block is, with the facts that go with its kind.
(MdList ordered start tight): start is the first item's number (1 for a bullet list). (MdItem task): whether the item opens with a GFM task box. (MdHeading level): 1 to 6. (MdCodeBlock fenced). (MdTable columns), (MdTableRow header?) and (MdTableCell alignment) are a GFM table: its rows are its children, and a row's cells are its children. (MdMathBlock) is a $$ display formula, with math on; markdown/block-text has its TeX.
md/-node-of
(md/-node-of [d :MdDoc id :MdBlockId] :MdNode)
internal: the arena node behind id.
md/-new-ids
(md/-new-ids :(Vec MdBlockId))
internal: an empty MdBlockId Vec.
md/-pool-copy
(md/-pool-copy [d :MdDoc i :int] :(Option String))
internal: a fresh copy of pool String i, or (none) when i is -1.
md/-align
(md/-align [a :int] :MdAlign)
internal: the MdAlign for alignment code a (1 left, 2 center, 3 right).
markdown/root
(markdown/root [d :MdDoc] :MdBlockId)
the document block, the root of the tree.
| d | the document |
The MdBlockId of the MdDocument block.
(markdown/block d (markdown/root d)) ; => (MdDocument)
Since: markdown-library-plan M1
markdown/children
(markdown/children [d :MdDoc id :MdBlockId] :(Vec MdBlockId))
the blocks directly inside a block, in order.
A Vec rather than first-child / next-sibling Options: an Option of a struct is a heap box that a recursive walk would leak (see docs/reported/carrier-sum-option-boxes-have-no-owner.md).
| d | the document | |
| id | a block of d |
A fresh (Vec MdBlockId), empty for a leaf block; free it with vec-free.
(vec-len (markdown/children d (markdown/root d))) ; => the number of top-level blocks
Since: markdown-library-plan M1
markdown/block
(markdown/block [d :MdDoc id :MdBlockId] :MdBlock)
what a block is.
| d | the document | |
| id | a block of d |
Its MdBlock: kind, and the level, list facts or fencing that go with it.
(markdown/block d id) ; => (MdHeading 2) for "## Title"
Since: markdown-library-plan M1
markdown/block-text
(markdown/block-text [d :MdDoc id :MdBlockId] :(Option String))
a leaf block's text.
A paragraph's or heading's source text, trimmed, with its inner line endings and its inline syntax kept (markdown/inlines has it parsed); a table cell's, the same, with each \| written as |; a code block's contents, ending in a line feed; an HTML block's HTML; or a display formula's TeX, ending in a line feed.
| d | the document | |
| id | a block of d |
(some s), a fresh owned String, or (none) for a block that holds blocks rather than text.
(markdown/block-text d id) ; => (some "code\n") for " code"
Since: markdown-library-plan M1
markdown/code-info
(markdown/code-info [d :MdDoc id :MdBlockId] :(Option String))
a fenced code block's info string.
| d | the document | |
| id | a block of d |
(some s), a fresh owned String with escapes and references resolved (possibly empty), or (none) for any other block.
(markdown/code-info d id) ; => (some "python") for a fence whose info string is python
Since: markdown-library-plan M1
markdown/list-marker
(markdown/list-marker [d :MdDoc id :MdBlockId] :(Option Char))
a list's marker: a bullet list's bullet, or the delimiter after an ordered list's numbers.
| d | the document | |
| id | a block of d |
(some c): #\-, #\+ or #\* for a bullet list, #\. or #\) for an ordered one; (none) for any other block.
(markdown/list-marker d id) ; => (some #\)) for "1) one\n2) two"
Since: markdown-library-plan M5
markdown/block-lines
(markdown/block-lines [d :MdDoc id :MdBlockId] :(Pair int int))
the source lines a block spans.
| d | the document | |
| id | a block of d |
(pair first last): 1-based line numbers, inclusive.
(markdown/block-lines d id) ; => (pair 3 4) for a paragraph on lines 3-4
Since: markdown-library-plan M1
markdown/render-block-html
(markdown/render-block-html [d :MdDoc id :MdBlockId opts :MdHtmlOpts] :String)
the HTML for one block and everything in it, as markdown/render-html writes it in place.
A heading keeps the id it has in the whole document's HTML, and a paragraph of a tight list is still written without <p>, so rendering each of the root's children in turn gives the document's HTML.
| d | the document (borrowed) | |
| id | a block of d | |
| opts | how to treat raw HTML and give headings ids |
A fresh owned String.
(markdown/render-block-html d id (markdown/default-html-opts)) ; => "<p>Hi</p>\n"
Since: markdown-library-plan M5
md/-blocks-of
(md/-blocks-of [d :MdDoc keep? :(fn [MdKind] bool)] :(Vec MdBlockId))
internal: the blocks for which keep? holds, in document order, as MdBlockIds.
markdown/headings
(markdown/headings [d :MdDoc] :(Vec MdBlockId))
every heading of a document, in order.
Headings inside block quotes and list items are in it too, where they come in the text. markdown/heading-ids lists their ids in the same order.
| d | the document |
A fresh (Vec MdBlockId); free it with vec-free.
(vec-len (markdown/headings d)) ; => 2 for "# A\n\n> ## B\n"
Since: markdown-library-plan M3
markdown/heading-text
(markdown/heading-text [d :MdDoc id :MdBlockId] :(Option String))
a heading's content as plain text, for a table of contents.
The text of its inlines: code spans' contents included, raw HTML and images' alt text left out, character references decoded, and each run of whitespace one space. It is the text the heading-id rules start from.
| d | the document | |
| id | a block of d |
(some s), a fresh owned String, or (none) when the block is not a heading.
(markdown/heading-text d id) ; => (some "Use make here") for "## Use `make` *here*"
Since: markdown-library-plan M3
markdown/heading-html
(markdown/heading-html [d :MdDoc id :MdBlockId opts :MdHtmlOpts] :(Option String))
a heading's content as markdown/render-html writes it, for a table of contents.
The HTML between the heading's tags: its inlines rendered with opts, so the raw-HTML rule is the page's. No id is in it; markdown/heading-ids has those.
| d | the document | |
| id | a block of d | |
| opts | the options the document is (or would be) rendered with |
(some s), a fresh owned String, or (none) when the block is not a heading.
(markdown/heading-html d id (markdown/default-html-opts)) ; => (some "Use <code>make</code>") for "## Use `make`"
Since: markdown-library-plan M4
markdown/heading-ids
(markdown/heading-ids [d :MdDoc opts :MdHtmlOpts] :(Vec String))
the id markdown/render-html gives each heading.
One per heading, in the order markdown/headings lists them, by the rule opts's heading-ids names, made unique in the document as that rule does. Under (MdNoIds) the renderer writes none, and the Vec is empty.
| d | the document | |
| opts | the options the document is (or would be) rendered with |
A fresh (Vec String); release it with string/release-all.
(markdown/heading-ids d (MdHtmlOpts (MdRawTagFilter) (MdSlugPython))) ; => ["intro" "intro_1"] for "# Intro\n# Intro\n"
Since: markdown-library-plan M3
markdown/code-blocks
(markdown/code-blocks [d :MdDoc] :(Vec MdBlockId))
every code block of a document, fenced or indented, in order.
markdown/code-info, markdown/block-text and markdown/block-lines read each one's info string, contents and source lines.
| d | the document |
A fresh (Vec MdBlockId); free it with vec-free.
(vec-len (markdown/code-blocks d)) ; => 1 for "```c\nx;\n```\n"
Since: markdown-library-plan M3
MdInlineId
(defstruct MdInlineId :copy [ix :int])
an inline of a parsed document. Valid only with the MdDoc it came from.
MdInline
(defdata MdInline :copy (MdText) (MdCode) (MdEmph) (MdStrong) (MdLink) (MdImage) (MdHtmlInline) (MdSoftBreak) (MdHardBreak) (MdStrikethrough) (MdMath bool))
what an inline is.
MdText, MdCode, MdHtmlInline and MdMath carry text (markdown/inline-text). MdEmph, MdStrong, MdStrikethrough, MdLink and MdImage hold inlines (markdown/inline-children). A link's or image's destination and title come from markdown/link-dest and markdown/link-title, and an image's children are its alt text. An autolink, extended ones included (www., scheme:// or an email address), is an MdLink. (MdMath display) is a formula, with math on: $...$, or $$...$$ when display is true.
md/-inl-of
(md/-inl-of [d :MdDoc id :MdInlineId] :MdInl)
internal: the arena inline behind id.
md/-inl-ids
(md/-inl-ids [d :MdDoc i :int] :(Vec MdInlineId))
internal: inline i's children as MdInlineIds.
markdown/inlines
(markdown/inlines [d :MdDoc id :MdBlockId] :(Vec MdInlineId))
a paragraph's, heading's or table cell's inlines, in order.
| d | the document | |
| id | a block of d |
A fresh (Vec MdInlineId), empty for any other block; free it with vec-free.
(vec-len (markdown/inlines d id)) ; => 2 for the paragraph "a *b*": text, then emphasis
Since: markdown-library-plan M2
markdown/inline-children
(markdown/inline-children [d :MdDoc id :MdInlineId] :(Vec MdInlineId))
the inlines inside emphasis, strong emphasis, strikethrough, a link or an image.
| d | the document | |
| id | an inline of d |
A fresh (Vec MdInlineId), empty for an inline that holds none; free it with vec-free.
(vec-len (markdown/inline-children d id)) ; => 1 for *b*: the text b
Since: markdown-library-plan M2
markdown/inline
(markdown/inline [d :MdDoc id :MdInlineId] :MdInline)
what an inline is.
| d | the document | |
| id | an inline of d |
Its MdInline.
(markdown/inline d id) ; => (MdStrong) for **b**
Since: markdown-library-plan M2
markdown/inline-text
(markdown/inline-text [d :MdDoc id :MdInlineId] :(Option String))
the text of a text, code, raw-HTML or math inline.
A text inline's escapes and character references are resolved; a code span's contents have their line endings as spaces; raw HTML is as written, and so is a formula's TeX, without its $ delimiters.
| d | the document | |
| id | an inline of d |
(some s), a fresh owned String, or (none) for an inline that holds inlines or is a break.
(markdown/inline-text d id) ; => (some "x") for `x`
Since: markdown-library-plan M2
markdown/link-dest
(markdown/link-dest [d :MdDoc id :MdInlineId] :(Option String))
a link's or image's destination.
With escapes and references resolved and the URL percent-encoded, as the HTML renderer writes it; mailto: comes first for an email autolink.
| d | the document | |
| id | an inline of d |
(some s), a fresh owned String (empty for [x]()), or (none) for any other inline.
(markdown/link-dest d id) ; => (some "/a%20b") for [x](</a b>)
Since: markdown-library-plan M2
markdown/link-title
(markdown/link-title [d :MdDoc id :MdInlineId] :(Option String))
a link's or image's title.
| d | the document | |
| id | an inline of d |
(some s), a fresh owned String with escapes and references resolved, or (none) when the link has no title or id is not a link or image.
(markdown/link-title d id) ; => (some "t") for [x](/u "t")
Since: markdown-library-plan M2