Metaprogramming

[TimWhiting]

While I understand if this isn't a priority or even something that Daan would like for Koka. I do want to explore the idea.

Metaprogramming solutions span a broad spectrum. You can have metaprograms that transform tokens, ones that transform ASTs, function decorators that wrap functions, or even more introspective metaprograms (such as the macros introduced to Dart recently).

I think the following are particularly worthy of mention:
Rackets new metaprogramming solution:
Rhombus: A New Spin on Macros without All the Parentheses

• A token based metaprogramming solution that is very flexible due to working on a sparse AST + token representation
• After a bit of preprocessing of raw lexical tokens into a bit more structure (such as after Koka's format rules adding semicolons and braces).
• So it's tokens plus a bit of structure, and interleaves it with an AST.
• Also supports different lexical macro contexts or scopes for extension (such as binding position, type position, match positions, or user specified positions)

Leans metaprogramming solution:
Beyond notations: Hygienic macro expansion for theorem proving languages

• Similar to the above, but more restricted.

And Darts solution: (there is a proposal and a few examples there)
https://github.com/dart-lang/language/tree/8194951697a090bc23f6a84e92044d368a00bf12/working/macros

• Happens after the AST is known and runs in 3 phases, a types phase, declaration phase, and definition phase.
• Types phase allows you to define or augment new types declarations or definitions, and don't allow much introspection
• Declaration phase allows you to reflect on types they are annotating
• Definition phase allows you to reflect on declarations and give definitions for ones that have empty bodies still, or augment (wrap) a definition
• There are more details, but the basics are that Dart tries to balance inference and reflection, because it allows inference of fields in types, you can't really know those types until the types are defined that it needs to reflect on.

All of these solutions are worth looking at, though I think Dart made it more complicated to make it really powerful, I don't necessarily think that is the approach that Koka should take.

Of course you need to think about hygiene for macros. I think ideally that macros should be run an environment where syntactic information get's added using algebraic effects, allowing for creating unique symbols based off a name etc. Pattern based macros are also something that would be very nice.
In general I think this should wait until the compiler is written in Koka, so we can experiment and use algebraic effect handlers in a macro system, which is something that I haven't heard of being done before.

[kuchta]

Nim also has a quite nice macro/template system:

• Templates
• Macros

[TimWhiting]

Thanks for mentioning that. Nim's is one that I've also thought was nicely designed.

[complyue]

until the compiler is written in Koka

I quite anticipate that day, that Koka starts to bootstrap itself, with some ergonomic parser/combinator libs digesting innovative new DSL grammars into some IR form, that Koka understands, type-checks, and compiles.

I feel especially comfortable with Julia, for its single runtime accommodating both compile-time and run-time programs, where you have the full ecosystem in doing compile-time programming, though its AST/macro system seems a LISP under disguise, which I don't like that much. Other meta-programming mechanisms I've ever worked with so far are all way more restrictive, i.e. C++ templates, type-level Haskell, Template Haskell. Though Python as a dynamic PL is much easier to work with, wrt meta-programming, and I also baked my own dynamic/scripting PL for some parts of my target system, there still needs static compilers in critical parts for performance.

I see Koka very promising to be a great infrastructure for performant DSL making, meta-programming with full-fledged ecosystem on algebraic effects shall be wonderful experiences I'd expect.