Different behavior from Google protoc generated message for accessing proto extensions about scalapb HOT 15 CLOSED

Trying to understand the issue.

1. You seem to be generating the descriptor in two ways: "by the protocol buffer compiler", and through "Scala plugin". Can you specify exactly how the descriptors are being generated in each of the two options?

2. currently I'm using AnnotationsProto.http.get(methDescriptor) from ScalaBP generated class for com.google.api.annotations.proto, which has a lens that access extension from unknownFields, and doesn't work for javadsl(bcs it's using pbc generated java class).

Like you indiciate, the ScalaPB extensions API works on ScalaPB descriptors/extensions. Is the expectation that it would work with Java descriptors?

Can you somehow make the problem statement more clear?

from scalapb.

Trying to understand the issue.

1. You seem to be generating the descriptor in two ways: "by the protocol buffer compiler", and through "Scala plugin". Can you specify exactly how the descriptors are being generated in each of the two options?

2. currently I'm using AnnotationsProto.http.get(methDescriptor) from ScalaBP generated class for com.google.api.annotations.proto, which has a lens that access extension from unknownFields, and doesn't work for javadsl(bcs it's using pbc generated java class).

Like you indiciate, the ScalaPB extensions API works on ScalaPB descriptors/extensions. Is the expectation that it would work with Java descriptors?

Can you somehow make the problem statement more clear?

sry for being confusing.

for 1, to be precisely, "by the protocol buffer compiler" in which I mean sbtprotoc.ProtocPlugin.gens.java that from sbt plugin protocbridge, and "Scala plugin" in which I mean scalapb.gen.SandboxedGenerator that using ScalaPbCodeGenerator.

for 2, yes, I was expect ScalaPB generated descriptors' lens work with Java descriptors, it feels weird to me that ScalaPB generated descriptors' implementation for proto extensions differ from the Java one's.

from scalapb.

I am sorry I still don't follow. Can you provide a minimal project that demonstrate the problem and the expectation that is not met by ScalaPB? In any case, the implementation details do not have to match between ScalaPB and Java - the respective APIs for each should give the same results.

from scalapb.

I guess I can, but would you like to elaborate about which point confused you? I mean if it is like why setup a project like this anyway or what specific code structure I was made to make this situation?

Also you actually answered my problem, and I would like to argue a little bit more about it.

To be fair, I'm far from protobuf expert by any means, but from a user perspective, I see Descriptor as a data that represent the programmable structure of a proto, also the API javaDescriptor returns a class that lays in com.google.protobuf which makes it looks like the "common part" of all different protobuf implementations), it just feels wrong to have the same data class/type but works actually different.

And from implementation perspective, it looks reasonable to put extension into extensions field instead of unknownFields, also relatively easy to do it I guess?

from scalapb.

sry I was not make my point very clear, I've update the Problem section.

from scalapb.

I am still confused. The two screenshots you share show the Java protobufs, none of them is a ScalaPB object, but then you mention you try to access them through the ScalaPB extensions API which isn't supposed to work with the Java objects.

from scalapb.

I am still confused. The two screenshots you share show the Java protobufs, none of them is a ScalaPB object, but then you mention you try to access them through the ScalaPB extensions API which isn't supposed to work with the Java objects.

uh, I knew what I was missed now, I mean I did manually transform Java protobufs(com.google.protobuf.MethodOptions) to SPB ones(com.google.protobuf.descriptor.MethodOptions) by copying each fields, and then access the extension through generated lens, sry for cause confusion.

so just for sure, which I was misunderstand and you pointed is, ScalaPB extensions API(generated lens) isn't supposed to work with neither pbc generated Java classes or ScalaPB's javaDescriptor. thanks for clearing that.

if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.

from scalapb.

And for clarify what I want to achieve, akka-grpc support both javadsl and scaladsl, and for some reason, javadsl and scaladsl use different code generator, when you try to implement functionality like HTTP transcoding which requires you to parse HttpRule from Descriptors, naturally you would to choose the common part of both generated classes and use Scala API for accessing, and it's javaDescriptor from ScalaPB and descriptor from Java ones.

Even they're the same type, but the implementation is actually different as I pointed, so after the same transformation procedure(to ScalaPB class), one works and one doesn't. that's why I think it is an issue.

from scalapb.

Also to be fair, I do heard that unknownSets is the widely used approach, it maybe protobuf-java suddenly changes it's implementation at some point.

from scalapb.

if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.

The Java API's getExtension should work with javaDescriptor.

...one works and one doesn't. that's why I think it is an issue.

Can you provide a minimal project that demonstrates what doesn't work?

from scalapb.

if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.

The Java API's getExtension should work with javaDescriptor.

Bcs some weird issues of akka-grpc I couldn't get the Java API works(google common proto library's Java classes missing from the target), so I haven't test it yet, still working on that.

And if that's the case, sry for misunderstanding the API, I will try it.

from scalapb.

...one works and one doesn't. that's why I think it is an issue.

Can you provide a minimal project that demonstrates what doesn't work?

minimal reproduce

Also I just realize that since I'm only copy unknownSets and there is no place for extensions in ScalaPB's class, so use ScalaPB's lens is not gonna work anyway.

from scalapb.

if it's true, may I ask is that expected to have Java protobuf generated API(Annotations.http.getExtension) to work with ScalaPB's javaDescriptor, if it is, as I pointed, I don't think it works bcs one just simply differ from other.

The Java API's getExtension should work with javaDescriptor.

...one works and one doesn't. that's why I think it is an issue.

Can you provide a minimal project that demonstrates what doesn't work?

I just update the reproduce project to show it seems getExtension doesn't work with javaDescriptor

from scalapb.

Thanks for providing the reproduction. It helps understanding the issue you are seeing better.

When ScalaPB generates the code for javaDescriptor it has a limitation: it can't assume that there are Java classes for anything that gets imported by the proto file. In this case, the generator has no way of knowing that AnnotatationsProto exists, so it can't reliably emit code like registry.add(com.google.api.AnnotationsProto.http);

There are two solutions you can have:

1. Enable javaConversions=true generator option. This makes javaDescriptor to be a reference to the actual java descriptors.
2. You could use javaDescriptor.toProto.toByteString and re-parse it using an extension registry you manually build in your code.

The code in Options.scala seems unnecessarily complex. What is it actually trying to do? Why doesn't it use the ScalaPB extensions API directly?

from scalapb.

Thanks for your patience and comprehensive answer.

I can see the limitation here now, since akka-grpc doesn't include java classes for scaladsl and vice verse, I guess reconstruct descriptor is the only option.

The code in Options.scala seems unnecessarily complex. What is it actually trying to do? Why doesn't it use the ScalaPB extensions API directly?

I was thinking that use javaDescriptor would work, also wanna using Scala API for convenience, and parse from bytes seems expensive, so i've decide why not just manually copy every fields, after all it's just field mapping...
yeah, I guess it is a bad idea.

Anyway, I have no further question, pls feel free to close this issue.

from scalapb.