Runtime exception when running Dotty's native image about graal HOT 5 OPEN

Note: same output when adding the --no-fallback option.

from graal.

For the record I also tried to compile it through Scala CLI's --native-image options, yielding the same result:

➜  ~/dotty-fresh git:(main) scala-cli --power package -o dotty-native --native-image --main-class dotty.tools.dotc.Main --classpath 'dist/target/pack/lib/*' --graalvm-args '-cp dist/target/pack/lib/*' --graalvm-args "--no-fallback"

from graal.

By the way, not directly related to the issue, but there actually is a much nicer possible setup using Scala CLI and using Dotty as a dependency, which avoids fiddling with JARs manually:

Let dotty.scala contain:

//> using scala "3.3.1"
//> using dep "org.scala-lang::scala3-compiler:3.3.1"
@main def main(args: String*) = dotty.tools.dotc.Main.process(args.toArray)

Then we can run:

$ scala-cli --power package dotty.scala -o dotty-native --native-image --graalvm-args "--no-fallback"

This yields the same executable—which throws the same error—as the instructions in the issue description:

$ ./dotty-native -cp $SCALA_LIB hello.scala
Exception while compiling hello.scala
Exception in thread "main" java.lang.AssertionError: assertion failed: asTerm called on not-a-Term val <none>
        at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
        at dotty.tools.dotc.core.Symbols$Symbol.asTerm(Symbols.scala:169)
        at dotty.tools.dotc.core.Definitions.ObjectClass(Definitions.scala:321)
...

from graal.

We are one step further thanks to @sjrd!

A problem was apparently due to Dotty trying to read Java standard library modules, which did not work in native image.

We instead generated a JAR with Java standard library classes using a script in tasty-query (https://github.com/scalacenter/tasty-query/blob/main/build.sbt#L176-L206). This script can be run from SBT as follows:

$ git clone [email protected]:scalacenter/tasty-query.git
$ cd tasty-query
$ sbt "tastyQueryJS / Test / javalibEntry"

Using this JAR, we can now successfully compile hello.scala using the Dotty native image! 🥳

$ mkdir classes
$ time ./dotty-native -bootclasspath extracted-rt.jar -cp $SCALA_LIB -d classes hello.scala 
./dotty-native -bootclasspath extracted-rt.jar -cp $SCALA_LIB -d classes   0.13s user 0.05s system 79% cpu 0.227 total

However, the generated class file seems to be corrupted:

$ java -cp $SCALA_LIB:$(pwd)/classes main                                                       
Error: Unable to initialize main class main
Caused by: java.lang.VerifyError: Constructor must call super() or this() before return
Exception Details:
  Location:
    main.<init>()V @0: return
  Reason:
    Error exists in the bytecode
  Bytecode:
    0000000: b1

Which is not the case if we generate it with the JVM version:

scala-cli dotty.scala -- -bootclasspath extracted-rt.jar -cp $SCALA_LIB -d classes-jvm hello.scala

My decompiler shows the same text for both:

// Source code is decompiled from a .class file using FernFlower decompiler.
import hello.package.;
import scala.util.CommandLineParser;

public final class main {
   public main() {
   }

   public static void main(final String[] args) {
      try {
         .MODULE$.main();
      } catch (CommandLineParser.ParseError var2) {
         scala.util.CommandLineParser..MODULE$.showError(var2);
      }

   }
}

But the binaries of main.class are indeed different:

(File truncated, end not shown)

from graal.

Hey @mbovel, thanks for digging into this! I'm not really familiar with Dotty, maybe you can help me understand the actual error a bit more. I assume asTerm() is supposed to be called in your app, only the assertion is not supposed to fail, right?

Exception in thread "main" java.lang.AssertionError: assertion failed: asTerm called on not-a-Term val <none>
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
	at dotty.tools.dotc.core.Symbols$Symbol.asTerm(Symbols.scala:171)

If Dotty or some other part are marked for build-time initialization, it's possible that "non-a-Term" object is created a build-time and frozen into the image. Maybe you could run with the -H:+PrintClassInitialization option for Native Image, which should provide you with an overview of all initialization policies.

from graal.