tur/markdown/scan
CommonMark scanners: escapes, references, link pieces, tags and URLs.
The lexical pieces both parsers share: backslash escapes and character references (md/-push-unescaped!), the link label, destination and title scanners, label normalisation, the HTML tag scanners (an HTML block of type 7, and raw inline HTML), the autolink scanners, and the URL normalisation a link destination goes through. The block parser uses them for link reference definitions; the inline parser (stdlib/markdown/inline.tur) for links, autolinks and raw HTML. Every scanner reads a String between two byte indices and answers with the index just past what it matched, or -1.
Since: markdown-library-plan M1
md/-byte
(md/-byte [s :String i :int] :int)
internal: the byte at i of s, or -1 outside it.
md/-space-tab?
(md/-space-tab? [b :int] :bool)
internal: is b a space or a tab?
md/-ws?
(md/-ws? [b :int] :bool)
internal: is b ASCII whitespace: space, tab, LF, VT, FF or CR?
md/-punct?
(md/-punct? [b :int] :bool)
internal: is b ASCII punctuation, which a backslash escapes?
md/-digit?
(md/-digit? [b :int] :bool)
internal: is b an ASCII digit?
md/-alpha?
(md/-alpha? [b :int] :bool)
internal: is b an ASCII letter?
md/-alnum?
(md/-alnum? [b :int] :bool)
internal: is b an ASCII letter or digit?
md/-hex?
(md/-hex? [b :int] :bool)
internal: is b an ASCII hex digit?
md/-hex-val
(md/-hex-val [b :int] :int)
internal: the value of hex digit b.
md/-lower
(md/-lower [b :int] :int)
internal: b in ASCII lower case.
md/-skip-ws
(md/-skip-ws [s :String i0 :int end :int] :int)
internal: the first index at or after i, before end, that is not ASCII whitespace.
md/-skip-spaces
(md/-skip-spaces [s :String i0 :int end :int] :int)
internal: the first index at or after i, before end, that is not a space.
md/-run-end
(md/-run-end [s :String i0 :int end :int max :int cls :int] :int)
internal: the end of the run of bytes from i (at most max) that satisfy the class: 0 digits, 1 hex digits, 2 letters and digits.
md/-num-val
(md/-num-val [s :String i0 :int end :int base :int] :int)
internal: the value of the digits [i, end) of s in base 10 or 16.
md/-push-cp!
(md/-push-cp! [b :StringBuilder cp :int] :void)
internal: append the code point a numeric reference names, U+FFFD for 0.
md/-push-numeric!
(md/-push-numeric! [b :StringBuilder s :String i :int end :int] :int)
internal: append the numeric reference whose digits start at i; the index past its ; or -1.
md/-push-named!
(md/-push-named! [b :StringBuilder s :String i :int end :int] :int)
internal: append the named reference whose name starts at i; the index past its ; or -1.
md/-push-entity!
(md/-push-entity! [b :StringBuilder s :String i :int end :int] :int)
internal: append the character reference at i (an &); the index past it, or -1 when there is none.
md/-entity-into!
(md/-entity-into! [b :StringBuilder s :String run :int i :int end :int] :int)
internal: at an & at i, flush [run, i) and append the reference; the index past it, or -1 (nothing appended).
md/-escape-at?
(md/-escape-at? [s :String j :int end :int] :bool)
internal: is the byte at j of s a backslash escaping an ASCII punctuation character?
md/-push-unescaped!
(md/-push-unescaped! [b :StringBuilder s :String i0 :int end :int] :void)
internal: append [i, end) of s with backslash escapes and character references replaced.
md/-unescape
(md/-unescape [s :String i :int end :int] :String)
internal: [i, end) of s with escapes and references replaced, as a fresh String.
md/-scan-label
(md/-scan-label [s :String i :int end :int] :int)
internal: the end of the link label whose [ is at i: [ ... ] with at most 999 bytes inside and no unescaped bracket.
md/-spnl
(md/-spnl [s :String i :int end :int] :int)
internal: skip spaces, at most one line ending, and spaces.
md/-space-eol
(md/-space-eol [s :String i :int end :int] :int)
internal: past spaces and the line ending (or the end) after i, or -1 when something else comes first.
md/-scan-dest-angle
(md/-scan-dest-angle [s :String i :int end :int] :int)
internal: the end of an <...> destination whose content starts at i.
md/-scan-dest-plain
(md/-scan-dest-plain [s :String i :int end :int] :int)
internal: the end of a destination without <>: balanced parentheses, no whitespace.
md/-scan-dest
(md/-scan-dest [s :String i :int end :int] :int)
internal: the end of the link destination at i, or -1.
md/-scan-title
(md/-scan-title [s :String i :int end :int] :int)
internal: the end of the link title at i, in "", '' or ().
md/-normalize-label
(md/-normalize-label [s :String i :int end :int] :String)
internal: the label [i, end) of s (brackets included) as references match it: trimmed, inner whitespace collapsed to one space, case-folded.
md/-name-start?
(md/-name-start? [b :int] :bool)
internal: can b start an attribute name?
md/-name-byte?
(md/-name-byte? [b :int] :bool)
internal: can b continue an attribute name?
md/-tag-name-end
(md/-tag-name-end [s :String i0 :int end :int] :int)
internal: the end of the tag name at i (a letter, then letters, digits and -), or -1.
md/-attr-name-end
(md/-attr-name-end [s :String i0 :int end :int] :int)
internal: the end of the attribute name at i.
md/-unquoted-byte?
(md/-unquoted-byte? [b :int] :bool)
internal: can b be in an unquoted attribute value?
md/-attr-value-end
(md/-attr-value-end [s :String i :int end :int] :int)
internal: the end of the attribute value at i, or -1.
md/-attr-end
(md/-attr-end [s :String i :int end :int] :int)
internal: the end of one attribute (whitespace, name, optional value) at i, or -1.
md/-open-tag-end
(md/-open-tag-end [s :String i :int end :int] :int)
internal: the end of the open tag whose < is at i, or -1.
md/-close-tag-end
(md/-close-tag-end [s :String i :int end :int] :int)
internal: the end of the close tag whose < is at i, or -1.
md/-lower-run
(md/-lower-run [s :String i0 :int end :int] :String)
internal: the letters and digits from i of s, lower-cased, as a fresh String.
md/-word-in?
(md/-word-in? [list :cstr name :String] :bool)
internal: is name one of the |-separated words of list?
md/-find-from
(md/-find-from [s :String needle :cstr i :int end :int] :int)
internal: the index of the bytes of needle in [i, end) of s, or -1.
md/-has-cstr-at?
(md/-has-cstr-at? [s :String lit :cstr i :int] :bool)
internal: do the bytes of lit occur in s at i?
md/-utf8-next
(md/-utf8-next [s :String i :int end :int] :int)
internal: the index past the well-formed UTF-8 sequence at i, when it ends by end; else -1.
md/-ci-cstr-at?
(md/-ci-cstr-at? [s :String lit :cstr i :int] :bool)
internal: do the bytes of lit, which is lower case, occur in s at i, ignoring ASCII case?
md/-comment-end
(md/-comment-end [s :String i :int end :int] :int)
internal: the end of the HTML comment whose < is at i: <!-->, <!---> or <!-- ... -->.
md/-pi-end
(md/-pi-end [s :String i :int end :int] :int)
internal: the end of the processing instruction <? ... ?> at i.
md/-cdata-end
(md/-cdata-end [s :String i :int end :int] :int)
internal: the end of the CDATA section <![CDATA[ ... ]]> at i.
md/-decl-end
(md/-decl-end [s :String i :int end :int] :int)
internal: the end of the declaration <! and letters and anything but > ... > at i.
md/-html-tag-end
(md/-html-tag-end [s :String i :int end :int] :int)
internal: the end of the raw HTML at i (an open or close tag, comment, processing instruction, declaration or CDATA section), or -1.
md/-uri-autolink-end
(md/-uri-autolink-end [s :String i :int end :int] :int)
internal: the end of <scheme:...> at i: a scheme of 2 to 32 letters, digits, + . -, a colon, and no space, control, < or >.
md/-word-in-bytes?
(md/-word-in-bytes? [c :int] :bool)
internal: is c one of the punctuation an email local part allows: . ! # $ % & ' * + / = ? ^ _ ` { | } ~ -
md/-email-local?
(md/-email-local? [c :int] :bool)
internal: can byte c be in the local part of an email autolink?
md/-email-label-end
(md/-email-label-end [s :String j0 :int end :int] :int)
internal: the end of one domain label at j (letters, digits and inner hyphens, 1 to 63 bytes), or -1.
md/-email-autolink-end
(md/-email-autolink-end [s :String i :int end :int] :int)
internal: the end of <local@domain> at i, or -1.
md/-url-safe?
(md/-url-safe? [c :int] :bool)
internal: does byte c stay as it is in a normalised URL?
md/-hex-digit
(md/-hex-digit [n :int] :int)
internal: the upper-case hex digit for n (0 to 15).
md/-pct-escape-at?
(md/-pct-escape-at? [s :String i :int n :int] :bool)
internal: does a %XX escape start at byte i of s?
md/-normalize-uri
(md/-normalize-uri [s :String] :String)
internal: s percent-encoded as a link destination, as a fresh String.
md/-dest-string
(md/-dest-string [s :String i :int end :int] :String)
internal: the destination [i, end) of s, unescaped and normalised, as a fresh String.
md/-byte-in
(md/-byte-in [s :String j :int end :int] :int)
internal: byte j of s when it is before end, else -1.
md/-table-space?
(md/-table-space? [b :int] :bool)
internal: a table's space character: space, tab, VT or FF.
md/-skip-table-space
(md/-skip-table-space [s :String i0 :int end :int] :int)
internal: past the table spaces from i.
md/-table-marker-end
(md/-table-marker-end [s :String i :int end :int] :int)
internal: past a delimiter-row cell at i (spaces, an optional :, one or more -, an optional :, spaces), or -1.
md/-table-start?
(md/-table-start? [s :String i :int end :int] :bool)
internal: is [i, end) of s a delimiter row: cells of dashes with optional colons, separated by pipes?
md/-cell-end
(md/-cell-end [s :String i :int end :int] :int)
internal: the end of a table cell's text at i: before the first pipe that no backslash precedes.
md/-pipe-len
(md/-pipe-len [s :String i :int end :int] :int)
internal: the length of the pipe and spaces at i, or 0 when there is no pipe.
md/-cell-text
(md/-cell-text [s :String a :int b :int] :String)
internal: a table cell's text [a, b) of s: each backslash-pipe as a pipe, trimmed, as a fresh String.
md/-new-cells
(md/-new-cells :(Vec String))
internal: an empty Vec of cell texts.
md/-row-cells
(md/-row-cells [s :String i :int end :int] :(Option (Vec String)))
internal: the cells of the table row [i, end) of s, each a fresh String, or (none) when it is not a row.