Code Monkey home page Code Monkey logo

Comments (5)

thesamet avatar thesamet commented on May 24, 2024 1

When running the Dockerfile you provided I noticed that the sbt protocGenScalaNativeImage/nativeImage step ends up downloading an older version of native-image (20.2.0), while make_reflect_config is using a more recent version. Since we install native-image manually using gu install, we need to disable the auto-download feature of sbt's native image plugin. The following dockerfile worked for me on x86 (the fix is the in ENV line):

FROM sbtscala/scala-sbt:graalvm-ce-21.3.0-java11_1.8.0_2.12.17
RUN mkdir -p /scala-protobuf
RUN curl -sSL https://api.github.com/repos/scalapb/ScalaPB/tarball/v0.11.12 | tar xz --strip 1 -C /scala-protobuf
WORKDIR /scala-protobuf
RUN gu install native-image
RUN ./make_reflect_config.sh
ENV NATIVE_IMAGE_INSTALLED=true
RUN sbt protocGenScalaNativeImage/nativeImage
RUN install -D /scala-protobuf/target/protoc-gen-scala /out/usr/bin/protoc-gen-scala

from scalapb.

thesamet avatar thesamet commented on May 24, 2024

Hi @strophy

ScalaPB works on all JVMs>=8, so using a JDK that supports Java 11 should be fine. Can you confirm that the make_reflect.config.sh step generates a new reflect-config.json ? Does it have a queriedMethods key in it? If so, why graalvm doesn't recognize it? This sounds more like a question for the GraalVM team.

from scalapb.

strophy avatar strophy commented on May 24, 2024

Yes, running make_reflect.config.sh generates protoc-gen-scala-nativ-image/native-image-config/reflect-config.json, is this the file you are expecting to see? The contents of the relevant section appear as follows:

  {
    "name": "com.google.protobuf.DescriptorProtos$DescriptorProto",
    "methods": [
      {
        "name": "newBuilder",
        "parameterTypes": []
      }
    ],
    "queriedMethods": [
      {
        "name": "getEnumType",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getEnumTypeCount",
        "parameterTypes": []
      },
      {
        "name": "getEnumTypeList",
        "parameterTypes": []
      },
      {
        "name": "getExtension",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getExtensionCount",
        "parameterTypes": []
      },
      {
        "name": "getExtensionList",
        "parameterTypes": []
      },
      {
        "name": "getExtensionRange",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getExtensionRangeCount",
        "parameterTypes": []
      },
      {
        "name": "getExtensionRangeList",
        "parameterTypes": []
      },
      {
        "name": "getField",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getFieldCount",
        "parameterTypes": []
      },
      {
        "name": "getFieldList",
        "parameterTypes": []
      },
      {
        "name": "getName",
        "parameterTypes": []
      },
      {
        "name": "getNameBytes",
        "parameterTypes": []
      },
      {
        "name": "getNestedType",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getNestedTypeCount",
        "parameterTypes": []
      },
      {
        "name": "getNestedTypeList",
        "parameterTypes": []
      },
      {
        "name": "getOneofDecl",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getOneofDeclCount",
        "parameterTypes": []
      },
      {
        "name": "getOneofDeclList",
        "parameterTypes": []
      },
      {
        "name": "getOptions",
        "parameterTypes": []
      },
      {
        "name": "getReservedName",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getReservedNameCount",
        "parameterTypes": []
      },
      {
        "name": "getReservedNameList",
        "parameterTypes": []
      },
      {
        "name": "getReservedRange",
        "parameterTypes": [
          "int"
        ]
      },
      {
        "name": "getReservedRangeCount",
        "parameterTypes": []
      },
      {
        "name": "getReservedRangeList",
        "parameterTypes": []
      },
      {
        "name": "hasName",
        "parameterTypes": []
      },
      {
        "name": "hasOptions",
        "parameterTypes": []
      }
    ]
  },

from scalapb.

strophy avatar strophy commented on May 24, 2024

This can be reproduced quite easily using the following short Dockerfile:

FROM sbtscala/scala-sbt:${SCALA_SBT_VERSION} as protoc_gen_scala
RUN mkdir -p /scala-protobuf
RUN curl -sSL https://api.github.com/repos/scalapb/ScalaPB/tarball/v0.11.12 | tar xz --strip 1 -C /scala-protobuf
WORKDIR /scala-protobuf
RUN gu install native-image
RUN ./make_reflect_config.sh
RUN sbt protocGenScalaNativeImage/nativeImage
RUN install -D /scala-protobuf/target/protoc-gen-scala /out/usr/bin/protoc-gen-scala

Running this file with different vars for SCALA_SBT_VERSION produces the following results, just under x86_64 for now to try and get native build working under Java >8:

SCALA_SBT_VERSION=graalvm-ce-21.2.0-java8_1.8.0_2.12.17  //works
SCALA_SBT_VERSION=graalvm-ce-21.3.0-java11_1.8.0_2.12.17 //fails
SCALA_SBT_VERSION=graalvm-ce-21.3.0-java17_1.8.0_2.12.17 //fails
SCALA_SBT_VERSION=graalvm-ce-21.3.0-java17_1.8.0_3.2.1   //fails

Can you reproduce this on x86? Or should I ask GraalVM for help?

from scalapb.

strophy avatar strophy commented on May 24, 2024

Thanks, this works perfectly now, including under ARM. You can track inclusion of the Scala plugin in rvolosatovs/protoc here: rvolosatovs/docker-protobuf#121

from scalapb.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.