Hi all,

I need to use macros similar to how scalameta does under the hood, in order to generate AST information. Given a specific case class, I also want to generate instances of Eq, Show, Functor etc. into companions.

I need this sooner rather then later, so I would like to help out by implementing this myself if the rest of the scalameta team is happy with this (I'm going to write this anyway for personal use, so I might aswell expose it to the rest of the community)

Below is a table of what is possible with code gen, not necessarily what will be implemented.

Note: The above is largely incomplete

• • It would be possible to provide a semantic database of sources not modifiable by code gen (eg. out of the current SBT project, so it's plausible to be able to add library semantics, and possibly other projects)
• ** Syntactic only, this would involve populating the parent field of the tree, initially, the answer to this would be no, but this is not a hard constraint.
• *** It it possible, by converting statics into methods.
  ??? = cannot remember whether it is valid

Features explained

Syntactic

By leveraging scalameta, we can ensure all tree's are syntactically valid. This does mean the code will compile. There is no semantic support such as fully qualified names, nor can we resolve any types.

Example:

If we wanted to generate a very trivial equals method.

 @GenEquals
 class Point(val x: Int, val y: Int)

could become

class Point(val x: Int, val y: Int) {
  def equals(a: Any): Boolean = {
    a match {
       case Point(ox, oy) => 
           ox == x && oy == y
       case _ => false
    }
  }
}

Enclosing

Due to being a post-order traversal, we know anything that has already been traversed has already been fully expanded. We know that extension generators do not tamper with existing information, thus are safe to introspect. Thus the only issues we have, are with not-yet-traversed tree's that have Manipulation based generators. I propose we only allow introspection of the source code of enclosing scopes, any generated code is off limits, this dramatically simplifies how it can implemented and will improve performance.

Same return

These generators take an input and output of the same type.

eg. Defn.Def => Defn.Def

Different return

These generators can optionally alter the return type.

eg Defn.Def => Defn.Val

Composition

The ability to compose multiple generators into a single one.

This will initially only be available to extension generators.

Extensions

Extend existing code.

Have the type A => Seq[Stat]

They can only generate new methods etc. for existing trees.

In theory, these would cover most use cases.

Note: If the entire file only uses extension generator's, the entire process can potentially happen in parallel.

Manipulators

Modify existing code, these are the "outlaws" of code generation, and what the macro annotations are based off

Have the type A => A, or A => B

Nested

@GeneratorA
 object Foo {
    @GeneratorB
     def bar = ???
  }

Multiple

@GeneratorA @GeneratorB
 object Foo

Recursive

Basically, allowing a generator, to generate another generator annotation.

Source Maps

Since we have the original and expanded tree's with correct line numbers. We can generate source maps to put errors in the right place. Any position inside synthetic code becomes an error at the position of the annotation. Given that the output... Technical jump to definition to could even end up at the resulting file, instead of the source.

Diffs

With code gen, we can actually show diffs, like scalafmt. We could also have dry-runs etc.

Seeing outputs

With code gen, the output is a literal file, not part of the compilation pipeline. Thus it is easy for the user just to open the resulting file, and look for problems.

Value parameters

@SomeGenerator(1)
class Point(x: Int, y: Int)

Type Parameters

@Deriving[Eq, Show]
class Point(x: Int, y: Int)

Plan

1. Add a new module to scalameta, called gen which is tool agnostic.
2. Add a cli interface
3. Write plugins for popular build systems (Primarily SBT)

I believe it would make sense for these projects to live and evolve with scalameta.
If necessary however, they can be put in a separate repo. I do not mind either way.

Gen

Interface

Almost identical to scalameta paradise macro annotations, the benefit of this is it makes migration from scalameta paradise almost trivial.

1. Define an annotation
2. Annotate the definition for which code generation must happen
3. Write a method in for the generator, using the following api: https://github.com/scalacenter/macros/issues/6
4. Register generators

Specifics on how generation happens.

Macro generation would be a post-order traversal of the original tree.
We scan the resulting tree for generators and recursively generate code until no more generators are available. Then we continue the traversal. So effectively we recursively run post-order traversals, until no more generation is necessary.

SBT Plugin

• Compiles the generator sources
• Builds up a set of available generators
• Traverses input files and runs generators.
• Outputs to generated sources
• Excludes the appropriate sources from compilation.

I am happy to go into more detail on how this would work, but I think code examples would be explain my logic better.

PS: If work has already been done in order to do something similar let me know.