The long journey towards Dok AST
Now I can parse Dok (a first version of it) and generate its Abstract Syntax Tree (Dok AST). commit/a43417024115fa586dd7b4be2a27f92b87a91c0b
The followed approach is far from pragmatic, but I'm experimenting some paradigms that I would use also in the final project:
- Attribute Grammars using JastAdd JastAdd as temporary tool (AG)
- a staged bootstrapping approach where every created tool, library and Domain Specific Language (DSL) can be used for generating new piece of the source code
I have the Dok grammar, written using the tree-sitter formalism. Tree-Sitter tools are used for generating a JSON description of the Dok grammar.
According the Tombstone Diagram notation, I have an AG definition of JSON (input language), and JastAdd (the implementation language) is used for deriving Java classes (the target language).
A tree-sitter parser (the implementation language) reads the Dok grammar in JSON format (the input language) and emit the Java objects associated to the JSON AG.
JastAdd AG rules (the implementation language) convert the JSON AG objects (the input language) into Java Objects representing a tree-sitter grammar specification in JastAdd AG format that I previously defined (the target language).
JastAdd AG rules (the implementation language) convert the tree-sitter grammar represented as AG nodes (the input language) into a JastAdd AG representing the Concrete Syntax (CST) of the language to parse.
JastAdd AG rules (the implementation language) convert the tree-sitter grammar represented as AG nodes (the input language) into Java code that query the tree-sitter parser of the original grammar and emit the AG with the CST.
This process converts the Dok tree-sitter grammar into Java code parsing it and emitting Dok CST in JastAdd AST nodes. But, this is generic code that can be used for every tree-sitter grammar.
The final pass was asking to an AI model to generate glue code, mapping the Dok CST into Dok AST. This is specific code for the Dok case.
I also created some view of the JastAdd AG nodes using the Java ELK graph layout library and JavaFX as UI library. This helped me to figure out the structure of the data to process and generate, in an AG friendly format.
Obviously a more direct approach would be more pragmatic, but I tested a way to transform step by step a tree-sitter grammar into a CST and an AST, using AG nodes and rules.
