Kotlin 1.3 support about rules_kotlin HOT 34 CLOSED

Hi everyone,

The timelines for the new Kotlin rules have slipped. Due to these delays in releasing we’ve decided to accept @cgruber’s fork as a stop-gap measure to support the community.

We’re actively working with him to get this merged, and will be open to receiving pull requests for features and bug fixes until the Google rules are released.

Thank you for your patience through this process, and your support for an awesome Kotlin + Bazel experience!

from rules_kotlin.

I note with interest that the @Globegitter rules are failing for me when using bazel 0.25.2 with:

ERROR: /private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/internal/js/js.bzl:57:17: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/internal/js/js.bzl", line 50
		rule(attrs = {"srcs": attr.label_list...")}, <4 more arguments>)
	File "/private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/internal/js/js.bzl", line 57, in rule
		attr.label_list(allow_files = True, default = [], cf...")
cfg must be either 'host' or 'target'.
ERROR: error loading package '': in /private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/kotlin.bzl: Extension file 'kotlin/internal/js/js.bzl' has errors
ERROR: error loading package '': in /private/var/tmp/_bazel_shs/9d46fd02a00b44f62e259110d18e4303/external/io_bazel_rules_kotlin/kotlin/kotlin.bzl: Extension file 'kotlin/internal/js/js.bzl' has errors

and the "official" bazel rules don't yet support 1.3. Any chance of the merged ruleset being released in the near future?

from rules_kotlin.

Fixed by #205

from rules_kotlin.

from rules_kotlin.

Thanks to @cgruber there's a way forward for people, but is there an ETA on the google version of these rules being updated? Or are there plans to merge @cgruber's rules_kotlin into this tree?

from rules_kotlin.

@Globegitter - sadly no. You'd have to open an issue on square's moshi project and explore it there.

from rules_kotlin.

KT-27895 (the annotations bug) as been fixed in Kotlin 1.3.10.

from rules_kotlin.

I will at least update my fork. I'm hoping thomas and co can come on-stream soon and start taking maintenance pulls here, even while they look at re-doing the whole ruleset.

from rules_kotlin.

Sorry, I hvaen't updated my fork, as there are quite a few other issues going on, not hte least of which is that the 1.3 kotlinc package doesn't inclue kotlin-runtime (which is now in kotlin-stdlib) and so the rule needs some special handling in order to handle that case. I have prototyped it a bit, but in our square internal fork. Once I settle on an actual working version, I'll throw it into my fork.

It may not matter, insofar as the #174 suggests the google-internal rules may be open-sourced soon, but not knowing the timelines yet on availability, I'll probably still do it as a stop-gap, so folks can use the legacy rules with 1.3 until the harmonized super-awesome rules are in place.

from rules_kotlin.

from rules_kotlin.

Back on my machine now. The error is:

bazel run --verbose_failures //examples:hello
INFO: Invocation ID: ea40c621-eb51-424c-a75d-12169d03538e
INFO: Analysed target //examples:hello (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: missing input file '@com_github_jetbrains_kotlin//:lib/kotlin-runtime.jar'
ERROR: /path/examples/BUILD.bazel:10:1: //examples:hello: missing input file '@com_github_jetbrains_kotlin//:lib/kotlin-runtime.jar'
Target //examples:hello failed to build
ERROR: /path/examples/BUILD.bazel:10:1 1 input file(s) do not exist
INFO: Elapsed time: 0.871s, Critical Path: 0.06s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

The BUILD file is:

kt_jvm_binary(
    name = "hello",
    main_class = "examples.Hello",
    srcs = [
        "Hello.kt",
    ]
)

And yeah if I switch back to the default kotlin compiler the application runs as expected.

Maybe also relevant I am running on the latest bazel rc: release 0.20.0rc1

Edit:
The url I am providing is: https://github.com/JetBrains/kotlin/releases/download/v1.3.0/kotlin-compiler-1.3.0.zip

from rules_kotlin.

Can you post (or exerpt) your WORKSPACE file as well? Bazel itself shouldn't be an issue, as this is nearly certainly a matter of how the toolchain is specified, and what ends up imported where.

from rules_kotlin.

@cgruber at its simplest it is

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

rules_kotlin_version = "d6711b288f4066954aedb3c800131aa8e970b443"
rules_kotlin_compiler_release = {
    "urls": [
        "https://github.com/JetBrains/kotlin/releases/download/v1.3.0/kotlin-compiler-1.3.0.zip",
    ],
    "sha256": "ff851cb84dd12df6078ae1f4a5424de9be6dcb4ac578b35455eeb7106dc52592",
}

http_archive(
    name = "io_bazel_rules_kotlin",
    urls = ["https://github.com/globegitter/rules_kotlin/archive/%s.zip" % rules_kotlin_version],
    type = "zip",
    strip_prefix = "rules_kotlin-%s" % rules_kotlin_version,
    sha256 = "2c4f2ecc8184243a893be21ac684ea0d0bdf93877d6163659a17c00092b0d99e",
)

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kotlin_repositories", "kt_register_toolchains")
kotlin_repositories(compiler_release=rules_kotlin_compiler_release)
kt_register_toolchains()

from rules_kotlin.

I'm also stuck at the same point using Kotlin 1.3, with the same compiler release (basically same workspace as above), but with a different error:

ERROR: /private/var/tmp/_bazel_pmatri/c08565c5c6c1bed750978f81a404fb89/external/io_bazel_rules_kotlin/src/main/kotlin/io/bazel/kotlin/builder/BUILD:34:1: Building external/io_bazel_rules_kotlin/src/main/kotlin/io/bazel/kotlin/builder/libbuilder.jar (2 source files) and running annotation processors (ComponentProcessor) failed (Exit 1).
error: cannot access NotNull
  class file for org.jetbrains.annotations.NotNull not found
  Consult the following stack trace for details.
  com.sun.tools.javac.code.Symbol$CompletionFailure: class file for org.jetbrains.annotations.NotNull not found
Target //services/HelloWorldService/src/kotlin/com/hurow/services/HelloWorld:HelloWorld failed to build
Use --verbose_failures to see the command lines of failed build steps.

Seems that there is a mismatch between the third-party dependencies of the rules and those of the project sources.

from rules_kotlin.

After commenting out:

• rules_kotlin/kotlin/internal/repositories/BUILD.com_github_jetbrains_kotlin

  Line 64 in a4e902d

  "runtime",

• rules_kotlin/kotlin/internal/jvm/jvm.bzl

  Line 137 in 46b867e

  default = Label("@" + _KT_COMPILER_REPO + "//:kotlin-runtime"),

• rules_kotlin/kotlin/internal/toolchains.bzl

  Line 115 in a4e902d

  default = Label("@" + _KT_COMPILER_REPO + "//:kotlin-runtime"),

I get the same error as @pierreis above. There seem to be a few other people who have run into this issue: https://discuss.kotlinlang.org/t/org-jetbrains-annotations-notnull-problem-with-android-build/629 & JakeWharton/timber#295 but have not been able to fix it with rules_maven. I tried to put it on the omit list but no luck.

from rules_kotlin.

Ah the error even happens for a hello world example. And I am seeing that org.jetbrains:annotations is a transitive dependency specified by rules_kotlin.

So adding "//third_party/jvm/org/jetbrains:annotations", to:
https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/builder/BUILD#L46

I seem ot be getting to the next error:

INFO: Invocation ID: 524a8a96-f56b-4f0a-84ee-798868b6d6e2
INFO: Analysed target //:tempjar (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Compiling Kotlin to JVM //:tempjar { kt: 1, java: 0, srcjars: 0 }:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/home/user/.cache/bazel/_bazel_user/98954e6e9031b033f581d48ad4da2ceb/external/io_bazel_rules_kotlin_com_google_protobuf_protobuf_java/jar/io_bazel_rules_kotlin_com_google_protobuf_protobuf_java.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Target //:tempjar up-to-date:
  bazel-bin/tempjar.jar
INFO: Elapsed time: 3.377s, Critical Path: 3.26s
INFO: 1 process: 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
INFO: Build completed successfully, 2 total actions
Exception in thread "main" java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics
	at digester.ServerKt.main(Server.kt)
Caused by: java.lang.ClassNotFoundException: kotlin.jvm.internal.Intrinsics
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)

So the build actually works and I am getting a runtime error.

from rules_kotlin.

@cgruber Getting a bit off point here (happy to open up a separate issue) but just trying to use https://github.com/square/moshi with the kotlin-codegen plugin. But for some reason the generated does not seem to be picked up by the resulting jar. Do you know by any chance how to get this to work? Is there something missing in bazel for it to include the generated code?

from rules_kotlin.

@Globegitter, before I try it myself, did you try updating the dependencies in third_party/jvm/workspace.bzl based on the dependency YAML file?

from rules_kotlin.

@pierreis not all of them, in fact I only tried to update kotlinx-coroutines and it did seem like it got me a bit further but I wonder if any of the source code needs updating like https://github.com/bazelbuild/rules_kotlin/blob/master/src/main/kotlin/io/bazel/kotlin/compiler/BazelK2JVMCompiler.kt or anything within https://github.com/bazelbuild/rules_kotlin/tree/master/src/main/kotlin/io/bazel/kotlin/builder

from rules_kotlin.

@pierreis you can actually see my progress here: https://github.com/Globegitter/rules_kotlin/tree/exports

from rules_kotlin.

@pierreis @cgruber using the patched repo from https://github.com/Globegitter/rules_kotlin/tree/exports with 1.3.10 as well as manually adding "@com_github_jetbrains_kotlin//:kotlin-stdlib", as a dep to a kt_jvm_binary, or runtime_dep to a java_binary let's me compile and run an app with 1.3

Not sure why that is necessary but at least it is running.

from rules_kotlin.

@Globegitter awesome, thanks.
Any fix planned from a repo maintainer?

from rules_kotlin.

Yeah - thanks @Globegitter. No love from maintainers yet. I believe there are plans for more focus/ownership on the kotlin side, but it's all informal discussions I'm having at this point, with no firm commitments yet. To be fair, some people were out for personal reasons, and are only getting back recently. Hopefully we'll see more action soon.

from rules_kotlin.

@Globegitter thanks for your effort!

To be able to compile sourcecode with 1.3 features, I had to add the version in toolchains.bzl.
Not sure how to do this without setting the default version.

Index: kotlin/internal/toolchains.bzl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- kotlin/internal/toolchains.bzl	(revision 170:95be9131aa2d04c6206a46c3b5077b507a1547f3)
+++ kotlin/internal/toolchains.bzl	(revision 170+:95be9131aa2d+)
@@ -80,18 +80,20 @@
         ),
         "language_version": attr.string(
             doc = "this is the -languag_version flag [see](https://kotlinlang.org/docs/reference/compatibility.html)",
-            default = "1.2",
+            default = "1.3",
             values = [
                 "1.1",
                 "1.2",
+                "1.3",
             ],
         ),
         "api_version": attr.string(
             doc = "this is the -api_version flag [see](https://kotlinlang.org/docs/reference/compatibility.html).",
-            default = "1.2",
+            default = "1.3",
             values = [
                 "1.1",
                 "1.2",
+                "1.3",
             ],
         ),
         "debug": attr.string_list(

Without this, I got an error:

error: the feature "inline classes" is only available since language version 1.3

from rules_kotlin.

Hi @cgruber we're hoping to upgrade to Kotlin 1.3 -- are you still planning on updating your fork with a working version? If so would be greatly appreciated.
Thanks!
Trevor

from rules_kotlin.

Hey - sorry, I'm just working on our internal fork now, for 1.3 support. I can back-port what I do there to my repo, in the near-term. But also, Globegitter has a 1.3-based fork here: https://github.com/Globegitter/rules_kotlin/tree/exports

I haven't tried it, but it should work. Regardless, I'm deep in the structure of the present rules, making them 1.3 workable, and so once I have that fixed internally, I"ll externalize those changes. Hopefully I'll have something external by next week. Hoping for something working internally in the next couple of days.

from rules_kotlin.

@cgruber thanks so much I appreciate your update and work. Looking forward to seeing what you create.

from rules_kotlin.

We've been using this fork, with this patch applied https://github.com/loreto/rules_kotlin/pull/1

from rules_kotlin.

Seems org.jetbrains.kotlin:kotlin-runtime was moved/renamed to org.jetbrains.kotlin:std-lib. I'm guessing a bunch of other things have broken in the process too otherwise it would've been an easy fix.

I'm able to download the artifact with the following change:

_COMPILER_RELEASE = {
    "urls": [
        "https://github.com/JetBrains/kotlin/releases/download/v1.3.31/kotlin-compiler-1.3.31.zip",
    ],
    "sha256": "107325d56315af4f59ff28db6837d03c2660088e3efeb7d4e41f3e01bb848d6a",
}

kotlin_repositories(compiler_release=_COMPILER_RELEASE)

It is an incomplete build graph though yielding the error @Globegitter mentioned. Looking at the maven repo it looks like the std-lib has been back filled for all versions of Kotlin... in theory maybe that means it could be used as a maven_jar dep instead of kotlin-runtime? Slight waste in bandwidth perhaps but 🤷‍♂.

from rules_kotlin.

Again, internally I have this fixed, and am under pretty intense deadline pressure. I'll try to push something out, but it's very likely that it won't support both 1.2 and 1.3 kotlinc. (That is, you can use kotlinc 1.3 in 1.2 language mode, but it is frustrating to make the change agnostic to the kotlinc version.

from rules_kotlin.

I'm actively (like, at this literal moment) working on making my fork work. I'm happy to port any of the changes back to the legacy branch, though I'll need a sponsor on the google side for reviews and merging.

from rules_kotlin.

Pressure's on, but I'll be looking at starting this process next week.

from rules_kotlin.

So... this week apparently. :) Sorry for the slip.

from rules_kotlin.

Is there an ETA for a new release which will includes this fix?

from rules_kotlin.