使用 scala 宏操作变量声明

Question

我想使用 scala (v2.12.8) 宏来操作给定块的所有变量声明。在此示例中添加值 23.

例如：

val myblock = mymanipulator {
    var x = 1
    x = 4
    var y = 1
    x + y
  }
print( myblock )

变成

{
  var x = (1).+(23);
  x = 4;
  var y = (1).+(23);
  x.+(y)
}

为此，我这样实现了 mymanipulator：

import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
import scala.language.implicitConversions

object mymanipulator {
  def apply[T](x: => T): T = macro impl

  def impl(c: Context)(x: c.Tree) = { import c.universe._

    val q"..$stats" = x
    val loggedStats = stats.flatMap { stat =>

      stat match {
        case ValDef(mods, sym, tpt, rhs) => {
          List( q"var $sym : $tpt = $rhs + 23" )
        }

        case _ => {
          List( stat )
        }
      }

    }

    val combined = q"..$loggedStats"

    c.info(c.enclosingPosition, "combined: " + showRaw(combined), true)

    combined
  }
}

我在编译宏的过程中得到了这些信息：

Information:(21, 31) combined: {
  var x = (1).+(23);
  x = 4;
  var y = (1).+(23);
  x.+(y)
}
  val myblock = mymanipulator {

但是当我使用上面给定的块执行 mymanipulator 时，我收到此错误消息：

Error:scalac: Error while emitting Test.scala
variable y

当我将实现更改为不执行任何操作时也会出现此错误：

 stat match {
        case ValDef(mods, sym, tpt, rhs) => {
          List( q"var $sym : $tpt = $rhs" )
        }

        case _ => {
          List( stat )
        }
      }

只有当我 return stat 时，错误才会消失

 stat match {
    case ValDef(mods, sym, tpt, rhs) => {
       List( stat )
     }

     case _ => {
        List( stat ) 
      }
  }

有人可以告诉我我做错了吗？谢谢

Answer 1

你应该在转换之前取消对树的类型检查

  object mymanipulator {
    def apply[T](x: => T): T = macro impl

    def impl(c: blackbox.Context)(x: c.Tree): c.Tree = {
      import c.universe._

      val q"..$stats" = c.untypecheck(x) // here
      val loggedStats = stats.flatMap { stat =>

        stat match {
          case ValDef(mods, sym, tpt, rhs) /*q"$mods var $sym : $tpt = $rhs"*/ => {
            List( q"$mods var $sym : $tpt = $rhs + 23" )
          }

          case _ => {
            List( stat )
          }
        }

      }

      val combined = q"..$loggedStats"

      c.info(c.enclosingPosition, "combined: " + showRaw(combined), true)

      combined
    }
  }

What is wrong with this def macro?

Answer 2

取消对源代码树的类型检查会删除子树的类型。这使得无法根据表达式类型进行树操作。那么如何使用类型检查树并替换宏源代码中的术语定义？

如果我们替换术语的定义（或者甚至简单地重新组合相同的术语定义，实际上有新的树）那么编译器会在该术语的标识上失败，并出现如下错误：

Could not find proxy for val <val-name>

REPL 错误类似于

Error while emitting <console>
variable j

对生成的树进行简单的取消类型检查或什至进一步的类型检查都无济于事。我在不同的答案中发现了几个原因：

旧的（从源代码树中重用）Ident 仍然指的是结果树中已经不存在的旧符号定义（我们替换了它）
重用的未更改符号定义（其树）已更改其所有者（代码被包装或重新包装）

对我有帮助的解决方案是重新创建所有引用本地（源代码）定义的标识。引用外部符号定义的标识应保持不变（如 'scala'、内部引用的外部类型等），否则编译失败。

以下示例（可在 REPL 模式下在 IDEA Worksheet 中运行，在 2.12 中尝试过）显示了使用 Transformer 重新创建仅引用本地定义的 IDents。新替换的 Ident 现在不会引用旧定义。

它使用的语法树涵盖了实现目标所需的唯一 Scala 语法。这种语法所不知道的一切都变成了 OtherTree，它包含原始源代码的子树。

import scala.reflect.macros.blackbox
import scala.language.experimental.macros

trait SyntaxTree {
  val c: blackbox.Context
  import c.universe._

  sealed trait Expression

  case class SimpleVal(termName: TermName, expression: Expression) extends Expression
  case class StatementsBlock(tpe: Type, statements: Seq[Expression], expression: Expression) extends Expression
  case class OtherTree(tpe: Type, tree: Tree) extends Expression

  object Expression {
    implicit val expressionUnliftable: Unliftable[Expression] = Unliftable[Expression] (({
      case q"val ${termName: TermName} = ${expression: Expression}" =>
        SimpleVal(termName, expression)
      case tree@Block(_, _) => // matching on block quosiquotes directly produces Whosebug in this Syntax: on the single OtherTree node
        val q"{ ..${statements: Seq[Expression]}; ${expression: Expression} }" = tree
        StatementsBlock(tree.tpe, statements, expression)
      case otherTree =>
        OtherTree(otherTree.tpe, otherTree)
    }: PartialFunction[Tree, Expression]).andThen(e => {println("Unlifted", e); e}))

    implicit val expressionLiftable: Liftable[Expression] = Liftable[Expression] {
      case SimpleVal(termName, expression) =>
        q"val $termName = $expression + 23"
      case StatementsBlock(_, statements, expression) =>
        q"{ ..${statements: Seq[Expression]}; ${expression: Expression} }"
      case OtherTree(_, otherTree) =>
        c.untypecheck(otherTree) // untypecheck here or before final emitting of the resulting Tree: fun, but in my complex syntax tree this dilemma has 0,01% tests impact (in both cases different complex tests fails in ToolBox)
    }
  }
}

class ValMacro(val c: blackbox.Context) extends SyntaxTree {
  import c.universe._

  def valMacroImpl(doTransform: c.Expr[Boolean], doInitialUntypecheck: c.Expr[Boolean])(inputCode: c.Expr[Any]): c.Tree = {
    val shouldDoTransform = doTransform.tree.toString == "true"
    val shouldUntypecheckInput = doInitialUntypecheck.tree.toString == "true"

    val inputTree = if (shouldUntypecheckInput)
      c.untypecheck(inputCode.tree) // initial untypecheck helps but we loose parsed expression types for analyses
    else
      inputCode.tree

    val outputTree: Tree = inputTree match {
      case q"${inputExpression: Expression}" =>
        val liftedTree = q"$inputExpression"
        if (shouldDoTransform) {
          val transformer = new LocalIdentsTransformer(inputTree)
          transformer.transform(liftedTree)
        } else
          liftedTree
      case _ =>
        q"{ ${"unexpected input tree"} }"
    }
    println(s"Output tree: $outputTree")
    /*c.typecheck(c.untypecheck(*/outputTree/*))*/ // nothing commented helps without transforming (recreating) Idents
  }

  class LocalIdentsTransformer(initialTree: Tree) extends Transformer {
    // transform is mandatory in any case to relink (here: reset) Ident's owners when working with typechecked trees
    private val localDefSymbols: Set[Symbol] = initialTree.collect {
      case t if t != null && t.isDef && t.symbol.isTerm =>
        t.symbol
    }.toSet

    println("localDefSymbols", localDefSymbols)

    override def transform(tree: Tree): Tree = tree match {
      case tree@Ident(termName: TermName) if localDefSymbols.contains(tree.symbol) =>
        println("replacing local Ident", termName, tree.symbol)
        Ident(termName)
      case _ =>
        super.transform(tree)
    }
  }
}

def valMacro(doTransform: Boolean, doInitialUntypecheck: Boolean)(inputCode: Any): Any = macro ValMacro.valMacroImpl

val outerVal = 5
// 1) works with pre untypechecking, but we loose types info
valMacro(false, true) {
  val i = 1
  i + outerVal
}
// 2) works with Transformer
valMacro(true, false) {
  val i = 1
  i + outerVal
}
// 3) does not work
valMacro(false, false) {
  val i = 1
  i + outerVal
}
// 4) cases when we reuse old tree without changes: fails
valMacro(false, false) {
  var j = 1
  j
}
// 5) cases when we reuse old tree without changes: works
valMacro(true, false) {
  var j = 1
  j
}

输出：

// 1) works with pre untypechecking, but we loose types info
(Unlifted,OtherTree(null,1))
(Unlifted,SimpleVal(i,OtherTree(null,1)))
(Unlifted,OtherTree(null,i.+($line8.$read.INSTANCE.$iw.$iw.outerVal)))
(Unlifted,StatementsBlock(null,List(SimpleVal(i,OtherTree(null,1))),OtherTree(null,i.+($line8.$read.INSTANCE.$iw.$iw.outerVal))))
Output tree: {
  val i = 1.$plus(23);
  i.+($line8.$read.INSTANCE.$iw.$iw.outerVal)
}
res0: Any = 29

// 2) works with Transformer
Unlifted,OtherTree(Int(1),1))
(Unlifted,SimpleVal(i,OtherTree(Int(1),1)))
(Unlifted,OtherTree(Int,i.+($line8.$read.INSTANCE.$iw.$iw.outerVal)))
(Unlifted,StatementsBlock(Int,List(SimpleVal(i,OtherTree(Int(1),1))),OtherTree(Int,i.+($line8.$read.INSTANCE.$iw.$iw.outerVal))))
(localDefSymbols,Set(value i))
(replacing local Ident,i,value i)
Output tree: {
  val i = 1.$plus(23);
  i.+($line8.$read.INSTANCE.$iw.$iw.outerVal)
}
res1: Any = 29

// 3) does not work
(Unlifted,OtherTree(Int(1),1))
(Unlifted,SimpleVal(i,OtherTree(Int(1),1)))
(Unlifted,OtherTree(Int,i.+($line8.$read.INSTANCE.$iw.$iw.outerVal)))
(Unlifted,StatementsBlock(Int,List(SimpleVal(i,OtherTree(Int(1),1))),OtherTree(Int,i.+($line8.$read.INSTANCE.$iw.$iw.outerVal))))
Output tree: {
  val i = 1.$plus(23);
  i.+($line8.$read.INSTANCE.$iw.$iw.outerVal)
}

Error while emitting <console>
value i

// 4) case when we reuse old tree without changes: fails
(Unlifted,OtherTree(<notype>,var j: Int = 1))
(Unlifted,OtherTree(Int,j))
(Unlifted,StatementsBlock(Int,List(OtherTree(<notype>,var j: Int = 1)),OtherTree(Int,j)))
Output tree: {
  var j = 1;
  j
}

Error while emitting <console>
variable j

// 5) case when we reuse old tree without changes: works with Transformer
(Unlifted,OtherTree(<notype>,var j: Int = 1))
(Unlifted,OtherTree(Int,j))
(Unlifted,StatementsBlock(Int,List(OtherTree(<notype>,var j: Int = 1)),OtherTree(Int,j)))
(localDefSymbols,Set(variable j))
(replacing local Ident,j,variable j)
Output tree: {
  var j = 1;
  j
}

如果跳过 OtherTree 的取消类型检查（同时提升它）或不对结果树进行取消类型检查，那么我们将在 2) 示例宏调用中得到错误：

java.lang.AssertionError: assertion failed: 
  transformCaseApply: name = i tree = i / class scala.reflect.internal.Trees$Ident
     while compiling: <console>
        during phase: refchecks
     library version: version 2.12.12
    compiler version: version 2.12.12

实际上这个示例展示了 2 种不同的方法来处理类型检查的输入树：

如何使用只覆盖Scala语法树部分所需类型集合的简单AST（实际上可能不会使用）。如果不使用这样的 AST 并且生成的树是使用源类型检查树的一部分（生成的部分类型检查树）“动态”构建的，那么生成的树应该在发出之前未进行类型检查（转换后）
如何使用 LocalIdentsTransformer 修复部分类型检查结果树中的 IDents

使用 scala 宏操作变量声明

Using scala macro to manipulate variable declaration

scala

scala-macros