Can I use the C++ proto message object in Python? about pybind11_protobuf HOT 25 CLOSED

@davidtwomey I have a working build with Bazel and an older version of pybind11_protobuf here:
https://github.com/beancount/beancount
I'm not using it in the codebase yet, I've been using the lower level API.

from pybind11_protobuf.

I'm using CMake as my build system at the moment...any proof of concept demo would be much appreciated!

I'm not aware of anything that could help directly.
I'd start by looking at the commands emitted by the bazel build, then emulate those with the cmake system.
@scal444, @michaelreneer: do you have a bazel output handy that could help here?

from pybind11_protobuf.

from pybind11_protobuf.

However, I am getting a Segmentation fault (core dumped) when attempting to run.

There is a small chance that you may need the latest version of pybind11 (github master branch), for loader_life_support fixes, depending on what you're trying, but that's only a wild guess.

If that's not it, I'm hoping @scal444 can help. Details about the segfault may be helpful.

Also is pybind11::google::ImportProtoModule(); now deprecated?

I think so, but I saw something flying by recently that left me confused about this. Tagging @laramiel for advice.

from pybind11_protobuf.

Thanks, I've merge it. I'll make a trivial change internally to see if the automatic exporting is still happy.

from pybind11_protobuf.

@davidtwomey Looks like it "just works". I think we're good to add your example under contrib/.

from pybind11_protobuf.

is this repo still under construction?

from pybind11_protobuf.

Yes, very much so. Help is welcome.
The most recent activity is #2.
(I'll add some auto-assign-issue tool so I'll see issues faster in the future.)

from pybind11_protobuf.

Hi - What is the current state of the repo?

A lot of things seem to be pointing to google-internal resources/dependencies, at least from a quick scan.

Can the latest master branch be linked against the latest public protobuf / pybind11?

Kind regards,

David

from pybind11_protobuf.

Hi - What is the current state of the repo?

A lot of things seem to be pointing to google-internal resources/dependencies, at least from a quick scan.

Can the latest master branch be linked against the latest public protobuf / pybind11?

Yes I think so, although I don't have 1st hand experience using the repo externally, but I know at least one project is doing that. Tagging @scal444 for that.

The master branch is gone, it's now main.

Sorry for the messy state of the documentation, it's something we need to clean up. This repo is exported from a Google-internal source tree. We have a fully automated process that tests all Google-internal changes on GitHub before they can be merged internally.

I think the main challenge is integrating this into your build system. We're using bazel in the automated testing.

from pybind11_protobuf.

Thanks for the quick response @rwgk!

Sounds good - I will check out anything done by @scal444

I'm using CMake as my build system at the moment...any proof of concept demo would be much appreciated!

from pybind11_protobuf.

@scal444 If you do have time to put together simple bazel example that would be fantastic!

(and don't worry about cmake if you suspect it will require a lot of effort...i can work from bazel for now)

Either way i'll keep my eye out for updates re the upcoming open source project

from pybind11_protobuf.

Quick update @rwgk - I have managed to create a simple example which builds in bazel (mostly taken from @scal444's bug repo):

See: https://github.com/davidtwomey/pybind11_protobuf_example.git

However, I am getting a Segmentation fault (core dumped) when attempting to run.

Am i missing something obvious in my implementation here?
Also is pybind11::google::ImportProtoModule(); now deprecated?

Thanks,

David

from pybind11_protobuf.

Your guess was correct! I switched to latest version of pybind11 and so far seems to work.

I have updated https://github.com/davidtwomey/pybind11_protobuf_example.git accordingly

from pybind11_protobuf.

from pybind11_protobuf.

We're trying to retire proto_casters.h and the related ImportProtoModule(). I can understand some confusion around this, as we're in the middle of migrating, and we should probably remove them from the open source release.

But what, specifically, left you confused?

from pybind11_protobuf.

from pybind11_protobuf.

Hi @rwgk @blais @scal444 @laramiel

I was able to put together a small example of pybind11_protobuf here: https://github.com/davidtwomey/pybind11-protobuf-bazel-example (I have stripped everything non-essential out to make it as simple as possible)

My goal is to extend this example to support package installation using a setup.py.

Are there any plans for this soon? Or would one of you be able to help me in accomplishing this?

The only open-source attempts I can find atm are:

• https://github.com/OpenMined/PSI/blob/master/setup.py
• https://github.com/deepmind/envlogger/blob/main/envlogger/setup.py

from pybind11_protobuf.

Hi @davidtwomey, sorry for the super long delay in responding. I'm interested in helping, although I don't have the free bandwidth to do substantial work, but I usually respond very quickly once things start rolling, and I do have full ownership of this repo internally and externally.

Currently we're not set up to accept external PRs changing files that live on the Google side ("internal"), but I wonder if we could create a subdirectory here under which we can work like in any other GitHub repo. @michaelreneer, do you know if that's possible? Maybe a contrib/ subdirectory here that is ignored by the current automated workflow exporting from Google to GitHub? If I could import contrib/ manually from GitHub to Google that would be even better, and fully sufficient for a good while I believe.

If that idea works, we could host contrib/bazel-example here.

from pybind11_protobuf.

Hi @rwgk,

Thanks for getting back to me - sounds good. I will just wait for updates

from pybind11_protobuf.

It's going to be unpredictable because pybind11_protobuf is "best effort" (in other words not staffed).
We could do a quick experiment if you're interested:

• Could you send a PR adding contrib/README.md with "Hello, world."?
• I'll merge it, then we see if that breaks anything when I make a small change internally that is exported here.
• Assuming that goes well, we could add your code under contrib.
  If we need changes outside contrib, I could handle them manually (assuming it's mainly small tweaks) via our internal workflow.

from pybind11_protobuf.

@rwgk - great! Have made that PR now - hope it works

from pybind11_protobuf.

Fantastic! I'll try and put a PR together this week

from pybind11_protobuf.

Fantastic! I'll try and put a PR together this week

Big disappointment and apologies: I only just now (coincidentally) noticed that your contrib/README.md file was deleted when my "Trivial change" was exported by the copybara-service bot.

I need to ask around what we can do.

from pybind11_protobuf.

@rwgk ah that is a shame!.

I had been holding off until fast_cpp_proto was able to work against non-hermetic builds.
Thankfully, #44 fixed the issue, so I am in a position to submit a PR (if, of course, you do manage to find a solution!)

Just keep me posted if you find out

Thanks,

David

from pybind11_protobuf.