tur/markdown/block

(defdata MdKind :copy (MdKDocument) (MdKBlockQuote) (MdKList) (MdKItem) (MdKParagraph) (MdKHeading) (MdKThematicBreak) (MdKCodeBlock) (MdKHtmlBlock) (MdKTable) (MdKTableRow) (MdKTableCell) (MdKMathBlock))

what a block is.

(defstruct MdListData [])

a list's or list item's marker: kind, bullet or delimiter byte, start number, tightness, and the item's content column.

(defstruct MdFence :copy [fenced :bool char :int len :int off :int])

a code block's fence: fenced?, the fence byte, its length and its indentation.

(defstruct MdNode [])

one block in the arena.

parent, first-child, last-child, prev and next are arena indices, -1 for none. buf is the block's StringBuilder index while it takes lines, text and info are String-pool indices (-1 for none), and start-line and end-line are 1-based source lines. inl is the root of a paragraph's, heading's or table cell's inline tree in the document's inline arena, -1 for none.

level is a heading's level, a table's column count, 1 on a table's header row, a table cell's alignment (0 none, 1 left, 2 center, 3 right), a list item's task box (0 none, 1 unchecked, 2 checked), 1 on a paragraph once it has failed to become a table's header (cmark-gfm's TABLE_VISITED), and 2 on the paragraph of the lines above a table's header, which cmark-gfm never reads link reference definitions from, and 1 on a display formula that its closing $$ ended.

(defstruct MdRef :copy [label :int dest :int title :int])

a link reference definition: String-pool indices of its normalised label, destination and title (-1 for no title).

(defdata MdIKind :copy (MdIRoot) (MdIText) (MdISoftBreak) (MdIHardBreak) (MdICode) (MdIEmph) (MdIStrong) (MdILink) (MdIImage) (MdIHtml) (MdIStrike) (MdIMath) (MdIMathDisplay))

what an inline is. MdIRoot holds a paragraph's or heading's inlines.

(defstruct MdInl [])

one inline in the arena.

parent, first-child, last-child, prev and next are inline-arena indices, -1 for none. A text, code or raw-HTML inline's literal is the bytes [start, end) of pool String str. dest and title are a link's or image's pool Strings (title -1 for none).

(defstruct MdOpts [])

what markdown/parse recognises beyond CommonMark.

The GitHub Flavored Markdown extensions -- tables, strikethrough, extended autolinks and task list items, as cmark-gfm 0.29 implements them -- and TeX math: $...$ and $$...$$ in text, and a $$ display block.

(defstruct MdDoc [])

a parsed document: the block arena, its String pool, its link reference definitions, the inline arena and the options it was parsed with.

(defstruct MdSt [])

the parser: the document being built, the open leaf's builders, and the cursor in the current line.

(defdata MdCont :copy (MdMatched) (MdUnmatched) (MdLineDone))

a container's answer to a line: it continues, it does not, or it consumed the line.

(defdata MdStart :copy (MdNoStart) (MdContainerStart) (MdLeafStart))

a block start's answer: none, a container (keep looking), or a leaf (stop).

(md/-new-nodes :(Vec MdNode))

internal: an empty node Vec.

(md/-new-strs :(Vec String))

internal: an empty String Vec.

(md/-new-refs :(Vec MdRef))

internal: an empty reference Vec.

(md/-new-bufs :(Vec StringBuilder))

internal: an empty builder Vec.

(md/-new-ints :(Vec int))

internal: an empty int Vec.

(md/-new-inls :(Vec MdInl))

internal: an empty inline Vec.

(md/-no-list :MdListData)

internal: the list data of a block that is not a list or item.

(md/-new-node [k :MdKind parent :int line :int] :MdNode)

internal: an open block of kind k under parent, starting at line.

(md/-para? [k :MdKind] :bool)

internal: is k a paragraph?

(md/-list? [k :MdKind] :bool)

internal: is k a list?

(md/-accepts-lines? [k :MdKind] :bool)

internal: does a block of kind k take text lines?

(md/-can-contain? [parent :MdKind child :MdKind] :bool)

internal: can a block of kind parent hold a child of kind child?

(md/-nodes [^borrow st :rc<MdSt>] :(Vec MdNode))

internal: the parser's node Vec.

(md/-strs [^borrow st :rc<MdSt>] :(Vec String))

internal: the parser's String pool.

(md/-node [^borrow st :rc<MdSt> i :int] :MdNode)

internal: node i.

(md/-put! [^borrow st :rc<MdSt> i :int n :MdNode] :void)

internal: replace node i.

(md/-kind [^borrow st :rc<MdSt> i :int] :MdKind)

internal: node i's kind.

(md/-add-str! [^borrow st :rc<MdSt> s :String] :int)

internal: put s in the pool; its index.

(md/-str [^borrow st :rc<MdSt> i :int] :String)

internal: pool String i (borrowed).

(md/-append-child! [^borrow st :rc<MdSt> p :int c :int] :void)

internal: make node c the last child of node p.

(md/-unlink! [^borrow st :rc<MdSt> i :int] :void)

internal: take node i out of its parent's children.

(md/-insert-after! [^borrow st :rc<MdSt> a :int b :int] :void)

internal: put node b just after node a, under a's parent.

(md/-insert-before! [^borrow st :rc<MdSt> a :int b :int] :void)

internal: put node b just before node a, under a's parent.

(md/-src [^borrow st :rc<MdSt>] :String)

internal: the parser's copy of the source (borrowed).

(md/-len [^borrow st :rc<MdSt>] :int)

internal: the current line's length in bytes, without its line ending.

(md/-at [^borrow st :rc<MdSt> i :int] :int)

internal: the byte at offset i of the current line, or -1 past its end.

(md/-indented? [^borrow st :rc<MdSt>] :bool)

internal: is the next non-space character four or more columns in?

(md/-find-next-nonspace! [^borrow st :rc<MdSt>] :void)

internal: find the next character that is not a space or tab, and the indentation before it.

(md/-advance-tab! [^borrow st :rc<MdSt> n :int columns :bool] :int)

internal: advance over the tab at the offset; the columns it used up.

(md/-advance! [^borrow st :rc<MdSt> n0 :int columns :bool] :void)

internal: advance the offset by n characters, or by n columns when columns is true (a tab may then be consumed part way).

(md/-advance-next-nonspace! [^borrow st :rc<MdSt>] :void)

internal: move the offset to the next non-space character.

(md/-advance-to-end! [^borrow st :rc<MdSt>] :void)

internal: move the offset to the end of the line.

(md/-add-line! [^borrow st :rc<MdSt>] :void)

internal: append the rest of the line, and a line feed, to the tip's text.

(md/-count-byte [s :String c :int i0 :int end :int] :int)

internal: how many times byte c occurs in [i, end) of s, plus one.

(md/-find-ref [^borrow st :rc<MdSt> label :String] :int)

internal: the reference whose normalised label is label, or -1.

(md/-dest-inner-start [s :String p :int] :int)

internal: where the destination [p, e) starts once its <> are dropped.

(md/-dest-inner-end [s :String p :int e :int] :int)

internal: where the destination [p, e) ends once its <> are dropped.

(md/-parse-ref-title! [^borrow st :rc<MdSt> s :String i :int le :int p :int de :int end :int] :int)

internal: the title and the line end of a definition whose label and destination are read.

(md/-parse-ref! [^borrow st :rc<MdSt> s :String i :int] :int)

internal: read the link reference definition at i of s, recording it; the index past it, or -1.

(md/-strip-ref-defs! [^borrow st :rc<MdSt> s :String] :int)

internal: read the definitions at the start of s; the index of the first byte that is not one.

(md/-blank-from? [s :String i :int] :bool)

internal: is [i, end) of s all spaces, tabs and line endings?

(md/-paragraphs [^borrow st :rc<MdSt> i :int out :(Vec int)] :void)

internal: append the paragraphs under node i, in document order, to out.

(md/-strip-paragraph! [^borrow st :rc<MdSt> i :int] :void)

internal: remove the definitions at the start of paragraph i, unless it holds the lines above a table's header.

(md/-remove-ref-defs! [^borrow st :rc<MdSt>] :void)

internal: take the reference definitions off the front of every paragraph, dropping paragraphs left blank.

(md/-new-buf! [^borrow st :rc<MdSt>] :int)

internal: a fresh builder; its index in bufs.

(md/-take-text! [^borrow st :rc<MdSt> i :int] :int)

internal: finish node i's builder into a pool String; its index.

(md/-ends-with-blank? [^borrow st :rc<MdSt> i :int] :bool)

internal: is node i followed, after a blank line, by a sibling?

(md/-loose-children? [^borrow st :rc<MdSt> i :int] :bool)

internal: does a child of node i, before the last, end with a blank line?

(md/-finalize-list! [^borrow st :rc<MdSt> i :int] :void)

internal: a list is loose when an item, or a block in an item, is followed by a blank line before more content.

(md/-finalize-item! [^borrow st :rc<MdSt> i :int] :void)

internal: an item ends where its last block does, or on its own line when empty.

(md/-trim-index [s :String i :int end :int] :int)

internal: the first index of [i, end) of s that is not ASCII whitespace, or end.

(md/-trim-end-index [s :String i :int end0 :int] :int)

internal: one past the last index of [i, end) of s that is not ASCII whitespace, or i.

(md/-code-keep-end [s :String len :int] :int)

internal: the end of an indented code block's text once trailing blank lines are dropped.

(md/-with-final-lf [s :String end :int] :String)

internal: [0, end) of s and a line feed, as a fresh String.

(md/-finalize-code! [^borrow st :rc<MdSt> i :int] :void)

internal: a fenced block's first line is its info string; an indented block drops its trailing blank lines.

(md/-finalize-html! [^borrow st :rc<MdSt> i :int] :void)

internal: an HTML block's text loses its final line feed.

(md/-finalize-math! [^borrow st :rc<MdSt> i :int] :void)

internal: a display formula's text loses its first line, which is the opening $$ line's (empty) rest.

(md/-finalize-kind! [^borrow st :rc<MdSt> i :int] :void)

internal: the per-kind work when node i closes.

(md/-finalize! [^borrow st :rc<MdSt> i :int line :int] :void)

internal: close node i, ending at line, and make its parent the tip.

(md/-add-child! [^borrow st :rc<MdSt> k :MdKind] :int)

internal: open a block of kind k under the tip, closing blocks that cannot hold it; its index.

(md/-close-unmatched! [^borrow st :rc<MdSt>] :void)

internal: close the blocks this line did not continue.

(md/-skip-quote-marker! [^borrow st :rc<MdSt>] :void)

internal: step over a > and the space after it.

(md/-continue-quote [^borrow st :rc<MdSt>] :MdCont)

internal: a block quote continues on a line starting with >.

(md/-continue-item [^borrow st :rc<MdSt> i :int] :MdCont)

internal: a list item continues on a blank line or one indented to its content.

(md/-fence-run [^borrow st :rc<MdSt> i :int c :int] :int)

internal: how many of byte c repeat from offset i of the line.

(md/-rest-blank? [^borrow st :rc<MdSt> i :int] :bool)

internal: is the line from offset i only spaces and tabs?

(md/-continue-code [^borrow st :rc<MdSt> i :int] :MdCont)

internal: a fenced block runs to its closing fence; an indented one takes indented and blank lines.

(md/-continue-html [^borrow st :rc<MdSt> i :int] :MdCont)

internal: an HTML block of type 6 or 7 ends at a blank line.

(md/-tables? [^borrow st :rc<MdSt>] :bool)

internal: are GFM tables on?

(md/-line-cells [^borrow st :rc<MdSt>] :(Option (Vec String)))

internal: the cells of the current line from its first non-space, or (none).

(md/-continue-table [^borrow st :rc<MdSt>] :MdCont)

internal: a table continues on any non-blank line that is a row.

(md/-continue [^borrow st :rc<MdSt> i :int] :MdCont)

internal: does the current line continue open block i?

(md/-start-quote [^borrow st :rc<MdSt>] :MdStart)

internal: > opens a block quote.

(md/-rest-blank-range? [^borrow st :rc<MdSt> i :int end :int] :bool)

internal: is [i, end) of the line only spaces and tabs?

(md/-atx-text-end [^borrow st :rc<MdSt> from :int] :int)

internal: where an ATX heading's text ends once an optional closing run of # is dropped.

(md/-start-atx [^borrow st :rc<MdSt>] :MdStart)

internal: one to six # and a space (or the line end) open an ATX heading.

(md/-backtick-after? [^borrow st :rc<MdSt> i :int] :bool)

internal: is there a ` anywhere on the line from offset i?

(md/-start-fence [^borrow st :rc<MdSt>] :MdStart)

internal: three or more or ~ open a fenced code block (a fence's line holds no other `).

(md/-math? [^borrow st :rc<MdSt>] :bool)

internal: is $ math on?

(md/-math-block? [k :MdKind] :bool)

internal: is k a display formula?

(md/-int-in? [v :(Vec int) x :int] :bool)

internal: does v hold x?

(md/-backslash-pair? [^borrow st :rc<MdSt> j :int] :bool)

internal: is offset j of the line a backslash with a character after it?

(md/-closing-dollars? [^borrow st :rc<MdSt> j :int] :bool)

internal: is offset j of the line a $$ with only spaces and tabs after it?

(md/-math-close-at [^borrow st :rc<MdSt> i0 :int] :int)

internal: where a $$ that ends the current line (but for spaces and tabs) starts, searching from offset i past any backslash pair; -1 when there is none.

(md/-start-math [^borrow st :rc<MdSt>] :MdStart)

internal: with math on, $$ alone on its line opens a display formula, and $$, a formula and $$ on one line are one.

(md/-has-at? [s :String needle :cstr i :int] :bool)

internal: do the bytes of needle occur in s at index i?

(md/-html-bang-type [s :String] :int)

internal: <!-- is kind 2, <![CDATA[ kind 5, <! and a letter kind 4.

(md/-html7? [s :String] :bool)

internal: is s a complete open or close tag and nothing else but whitespace?

(md/-html-type-of [s :String can7 :bool] :int)

internal: the kind of HTML block s (which starts with <) opens, or 0.

(md/-line-string [^borrow st :rc<MdSt> i :int] :String)

internal: the current line from offset i, as a fresh String.

(md/-html-start-type [^borrow st :rc<MdSt> i :int can7 :bool] :int)

internal: the kind (1 to 7) of HTML block the line from offset i starts, or 0.

(md/-start-html [^borrow st :rc<MdSt> container :int] :MdStart)

internal: < and one of the seven HTML block openings.

(md/-setext-level [^borrow st :rc<MdSt>] :int)

internal: 1 for a line of =, 2 for a line of -, else 0 (trailing spaces allowed).

(md/-start-setext [^borrow st :rc<MdSt> container :int] :MdStart)

internal: a line of = or - under a paragraph turns it into a heading.

(md/-thematic-break? [^borrow st :rc<MdSt> i :int] :bool)

internal: are three or more of one of * - _ (and spaces and tabs) all the line holds from offset i?

(md/-start-break [^borrow st :rc<MdSt>] :MdStart)

internal: a thematic break.

(md/-ordered-run [^borrow st :rc<MdSt> i :int] :int)

internal: the number of digits (at most 9) at offset i.

(md/-line-int [^borrow st :rc<MdSt> i :int n :int] :int)

internal: the value of the digits [i, i + n) of the line.

(md/-consume-marker! [^borrow st :rc<MdSt> data :MdListData mlen :int] :MdListData)

internal: step over a list marker of length mlen and the spaces after it; the data with its padding.

(md/-parse-marker [^borrow st :rc<MdSt> container :int] :(Option MdListData))

internal: the list marker at the next non-space character, consuming it and the spaces after it, or none.

(md/-lists-match? [a :MdListData b :MdListData] :bool)

internal: do a list and a new item agree in kind, delimiter and bullet?

(md/-start-item [^borrow st :rc<MdSt> container :int] :MdStart)

internal: a list marker opens a list item, and a list when the tip is not a matching one.

(md/-start-indented [^borrow st :rc<MdSt>] :MdStart)

internal: four columns of indentation open an indented code block, outside a paragraph.

(md/-no-start? [r :MdStart] :bool)

internal: is r MdNoStart?

(md/-para-text! [^borrow st :rc<MdSt> i :int] :String)

internal: paragraph i's lines so far, as a fresh String; its builder is spent.

(md/-para-refill! [^borrow st :rc<MdSt> i :int s :String] :void)

internal: give paragraph i a fresh builder holding s.

(md/-last-line-start [s :String] :int)

internal: where the last line of s (which ends in a line feed) starts.

(md/-align-of [c :String] :int)

internal: a delimiter cell's alignment: 1 left, 2 center, 3 right, 0 none.

(md/-add-row! [^borrow st :rc<MdSt> table :int header :bool line :int] :int)

internal: a closed table row under table, on line; its index.

(md/-add-cell! [^borrow st :rc<MdSt> row :int s :String align :int line :int] :void)

internal: a closed cell under row holding s (which the pool takes over), with alignment align.

(md/-col-align [^borrow st :rc<MdSt> table :int i :int] :int)

internal: the alignment of column i of table, from its header row.

(md/-make-table! [^borrow st :rc<MdSt> para :int text :String ls :int hdr :(Vec String) delim :(Vec String)] :void)

internal: turn paragraph para into a table with header cells hdr and delimiter cells delim; its lines before ls (the header's start in text) stay a paragraph.

(md/-table-header! [^borrow st :rc<MdSt> para :int] :MdStart)

internal: the current line, a delimiter row under paragraph para whose last line is a header row of as many cells, starts a table.

(md/-table-row! [^borrow st :rc<MdSt> table :int] :MdStart)

internal: the current line, under table, is a row: cells up to the table's width, the missing ones empty.

(md/-start-table [^borrow st :rc<MdSt> container :int] :MdStart)

internal: a table's header or one of its rows, with tables on and the line not indented.

(md/-table-candidate? [^borrow st :rc<MdSt> container :int] :bool)

internal: could the current line start a table header or row under container (so the quick start filter must not skip it)?

(md/-task-any-char [s :String i :int end :int] :int)

internal: the index past the one character at i before end, which is not a line feed and is well-formed UTF-8, or -1.

(md/-task-ordinal-end [s :String i0 :int z :int] :int)

internal: the index past an ordered list marker as the task scanner reads it -- digits, then any one character, which is the byte after them or, when spaces follow, their last digit -- or -1.

(md/-task-box-at [^borrow st :rc<MdSt>] :bool)

internal: whether the current line holds a task list item's box, scanned from the line's start as cmark-gfm's generated scanner does: spaces, then -, + or * or digits and any one character, then spaces, [ ], [x] or [X], and a space.

(md/-task-box! [^borrow st :rc<MdSt> container :int] :void)

internal: with task lists on, mark item container as a task when its line holds a box, checked when the line holds [x] or [X] anywhere, and step past the box.

(md/-try-starts [^borrow st :rc<MdSt> container :int] :MdStart)

internal: the block starts in the spec's order of precedence.

(md/-maybe-special? [c :int] :bool)

internal: can byte c begin a block start (a quick filter)?

(md/-match-containers [^borrow st :rc<MdSt>] :int)

internal: walk the open blocks the line continues; the last that matched, or -1 when the line is used up.

(md/-open-new-blocks [^borrow st :rc<MdSt> container0 :int] :int)

internal: try block starts until a leaf starts or none does; the container the text goes to.

(md/-has? [s :String needle :cstr] :bool)

internal: does s contain the bytes of needle?

(md/-lower-all [s :String] :String)

internal: s in ASCII lower case, as a fresh String.

(md/-html-closes? [^borrow st :rc<MdSt> i :int t :int] :bool)

internal: does the line from offset i hold the end condition of HTML block kind t (1 to 5)?

(md/-add-text! [^borrow st :rc<MdSt> container :int] :void)

internal: what is left of the line is text: a lazy paragraph line, a line of the open leaf, or a new paragraph.

(md/-incorporate-line! [^borrow st :rc<MdSt> start :int end :int] :void)

internal: process the line [start, end) of the source.

(md/-copy-source [src :String] :String)

internal: a copy of src with each NUL byte replaced by U+FFFD, as CommonMark requires.

(md/-line-end [s :String i0 :int n :int] :int)

internal: the index of the first CR or LF at or after i, or n.

(md/-parse-lines! [^borrow st :rc<MdSt>] :int)

internal: feed every line of the source to the parser; how many there were.

(md/-free-doc [d :MdDoc] :void)

internal: release a document and everything it holds.

(md/-open-math-lines [d :MdDoc] :(Vec int))

internal: the opening lines of the display formulas no closing $$ ended.

(md/-parse-blocks [src :String opts :MdOpts] :MdDoc)

internal: the block structure of src.

Parsed again, with their $$ lines as text, while a display formula is left open; see "Display formulas" above.

(md/-parse-once [src :String opts :MdOpts no-math :(Vec int)] :MdDoc)

internal: the block structure of src, with no display formula opening on the lines in no-math.