tur/forms/generators
resumable bodies that yield values one at a time.
(gen [] body ...) makes a generator: a body that runs until a yield, hands that value out, and picks up where it left off the next time it is asked. The compiler turns the body into a C state machine, so a yield allocates nothing. The body does not start until the first gen-next.
gen-next returns the yielded value as a pointer-sized word, or a null pointer once the body has finished; gen-unwrap and gen-some? in tur/gen read that word, and gen-for-each, gen-collect and gen-nth there consume a whole generator. The lazy Seq combinators (tur/seq) are built on these. See generators-guide.
A generator's type is (Generator T), T being what it yields (its first yield's type). Write it on a parameter, a return, a let or a struct field to pass a generator around; gen-unwrap reads the element back at T, a struct or ADT included, and a generic (Generator T) parameter binds T.
Limits today (the post-1.0 CPS pass lifts them): the capture vector must be empty, a generator cannot contain another generator, and yield cannot appear in a match arm (TUR-E0702) or in a recursive generator (TUR-E0703).
gen
(gen [] body ...)
create a generator from a body.
The capture vector is always []: the body sees the variables around it, and the compiler works out which ones from the body itself ((gen [x] ...) is an error). The body runs lazily, a yield at a time. A scalar ^mut variable (a number, bool, string or symbol) that the body names is shared with the generator, on both engines: a set! outside is seen inside, and the body's own set! reaches the variable. Every other captured local is copied into the generator when it is created, and assigning a captured ^mut of another type inside the body is an error (it would change only the copy). Keep a generator's own state in let bindings made inside the body.
| [] | the capture vector, which must be empty | |
| body | the forms to run; yield hands a value out |
A generator. Drive it with gen-next.
(let [g (gen [] (yield 1) (yield 2))] (gen-next g) (gen-unwrap (gen-next g))) ; => 2
yield
(yield value)
hand a value out of a generator body and pause.
Valid only inside a gen body (anywhere else is yield is only valid inside a gen body). The first yield fixes the generator's element type -- the T of its (Generator T): ints, floats, strings, structs and ADTs work, and a later yield of a different type is an error (integer widths may mix). A yield inside a fn literal in the body is an error too: the fn is its own function, not part of the generator. A yield form has type void, so (let [r (yield 1)] ...) is TUR-E0023.
| value | the value to yield |
(let [g (gen [] (yield 7.25))] (gen-unwrap (gen-next g))) ; => 7.25
gen-next
(gen-next [g :(Generator T)] :ptr<void>)
run a generator to its next yield.
Starts or resumes the body and returns what it yields, as a pointer-sized word. When the body runs off its end the result is a null pointer (gen-some? is false for it) and every later call returns null again. Passing something that is not a generator is a compile error. Reading a null result with gen-unwrap is a panic ("the generator is exhausted") on both engines, so test with gen-some? first.
| g | the generator to advance |
The next yielded value as a ptr<void>, or null when finished.
(let [g (gen [] (yield 1))] (gen-unwrap (gen-next g))) ; => 1
gen-done?
(gen-done? [g :(Generator T)] :bool)
has a generator run to its end?
False for a fresh generator, and still false after the last yield has been taken: it turns true only after a gen-next drives the body past its end and gets null back. A loop that tests gen-done? before each gen-next therefore makes one more gen-next call than there are values and must check gen-some? on each result.
| g | the generator to ask |
true once the body has finished, else false.
(let [g (gen [] (yield 1))] (gen-next g) (gen-done? g)) ; => false (let [g (gen [] (yield 1))] (gen-next g) (gen-next g) (gen-done? g)) ; => true