humans-of-julia / typedbclient.jl Goto Github PK
View Code? Open in Web Editor NEWA client interface to read from and write to the TypeDB Knowledge Graph
License: MIT License
A client interface to read from and write to the TypeDB Knowledge Graph
License: MIT License
GOAL
In order to be able to eventually have a grpc library available in Julia, either via writing a new one or revamping gRPC.jl or GRPC.jl, there would need to be a way to make use of the HTTP/2 protocol.
Current situation and possible solutions
There are several solutions that have been started, but are either not completed or not maintained anymore.
Solution 1:
HTTP2.jl
This library is not maintained anymore.
It would require the HPack.jl 0.2.0 (Julia v1.0 compatibility), which itself is not maintained anymore. It looks like it already has been done maybe here in this tree, but has never been released.
The author @samoconnor is not active on GitHub anymore, his last contribution is from March 2019.
Solution: HTTP2.jl could be forked and setup to be working again.
Solution 2:
There is HTTP.jl
@samoconnor started to integrate the HPack.jl into HTTP.jl here
And there is an open PR here: #JuliaWeb/HTTP.jl#331
I asked @quinnj on Slack about the state and he wants to look into it and estimate the required workload and if it would need sponsoring or not.
Solution: We could wait for his response and decide, wether it makes sense to finish that PR or not or look for sponsors or smth.
Solution 3:
Calling into C
Somebody mentioned this MIT licensed pack to be a solution in this issue: #JuliaWeb/HTTP.jl#448
It is about this technology: litespeed-quic-http3-open-source
This is the pack, not sure if HTTP/3 or QUIC actually offer the HTTP/2 parts
https://github.com/litespeedtech/lsquic
But there is the C compression library here
Found on their official site here
Solution: This could be called within Julia.
Solution 4:
Calling grpcio from Python via PyCall and don't care for a Julia implementation for now.
Grakn themselves use the grpcio for their Python Client.
Implement graql steps
Work on the implementation is needed.
The Grakn core server communicates via gRPC with it's clients.
Currently there is no working gRPC library in Julia. There are gRPC.jl or GRPC.jl, but these need to be checked and updated to a working state.
See issue #19
The long term goal would require a native Julia implementation of gRPC.
There has been several attempts to find people doing it here: #grpc/grpc#24786
These let to an offer to get help facilitate the work, clarify requirements, do API reviews, set up interop tests as mentioned in the issue.
And here: https://discourse.julialang.org/t/grpc-api-community-request/51029
which led to no avail yet.
I also wrote a mail to @tanmaykm, he replied and might give it a shot to get existing stuff working back again (no promise), but may not be able to spend a lot of time on it. He would be happy if anyone else is willing to contribute.
gRPC is supposed to be a non trivial protocol, so I think it doesn't seem feasible to tackle this first and wait for completion until we can start with the GraknClient.
Instead we should find an interim way by either calling into the Python gRPC library with Python or using Julias native C calling functionality to call into the shared gRPC core library, which is used by all client implementations. I guess this would require additional work, as the C lib seems to be more low level code.
Quickest solution to get started with developing the GraknClient.jl seems to be to call into Python and then see if somebody wants to take a leap on implementing/revamping a Julia native gRPC lib in the long run to eventualy avoid the Python call.
Implement common parsing functions
Graknlabs follows the Behaviour driven development (BDD) principle. It would be great to also have this imlemented at some point.
There is the Grakn Behaviour Specification and there is a native Julia package called ExecutableSpecifications.jl and some talk in this issue #erikedin/Behavior.jl#38
Some things that the Grakn features and steps require, would need some work on this ExecutableSpecifications.jl to be able to use that in our client.
This could maybe end up in a small stipple.app to show the BDD results of the tests like in the picture below:
Or we might even be able to use Grabl at some point.
In this Grakn 2.0 announcement 13). post it has been mentioned, that it would be available to 3rd partys as well some day.
It would be a great help for me to have a few reference examples to work with.
The feature file I reference is here: https://github.com/Humans-of-Julia/GraknClient.jl/blob/BDD--steps/test/behaviour/features/concept/thing/attribute.feature
Feature: Concept Attribute
Background:
Given connection does not have any database
@step("connection does not have any database")
def step_impl(context: Context):
assert len(context.client.databases().all()) == 0
@Given("connection does not have any database")
public void connection_does_not_have_any_database() {
super.connection_does_not_have_any_database();
}
@given("connection does not have any database") do context
@expect length(get_all(databases(context[:client]))) == 0
end
Background:
# Write schema for the test scenarios
Given put attribute type: is-alive, with value type: boolean
@When("{var} = attribute\\( ?{type_label} ?) as\\( ?boolean ?) put: {bool}")
public void attribute_type_as_boolean_put(String var, String typeLabel, boolean value) {
put(var, tx().concepts().getAttributeType(typeLabel).asBoolean().asRemote(tx()).put(value));
}
@when("{var:Var} = attribute({type_label}) as(boolean) put: {value:Bool}") do context
context[Symbol(var)] = [:tx(), :concepts(), get_attribute_type(:type_label::String), :as_remote(context[:tx()]), :as_boolean(), :put(value::bool)]
end
especially this is pretty surely wrong:
[:tx(), :concepts(), get_attribute_type(:type_label::String), :as_remote(context[:tx()]), :as_boolean(), :put(value::bool)]
Feature: Concept Attribute
Background:
Given put entity type: person
Given entity(person) set owns attribute type: is-alive
@When("{root_label}\\( ?{type_label} ?) set owns attribute type: {type_label}")
public void thing_type_set_has_attribute_type(RootLabel rootLabel, String typeLabel, String attributeLabel) {
AttributeType attributeType = tx().concepts().getAttributeType(attributeLabel);
get_thing_type(rootLabel, typeLabel).asRemote(tx()).setOwns(attributeType);
}
@given("entity(person) set owns attribute type: is-alive") do context
@fail "Implement me"
end
@given("{root_label}({type_label}) set owns attribute type: {type_label}") do context
@expect "Implement me"
end
Hi there!
My name is Tran Nguyen Thuong Truong - @thuongtruong1009
Currently, I learning front-end.
I want to learn more and more about open-source code.
Hope to cooperate with you in the future.
Thank you!
Those can be found here: https://github.com/Humans-of-Julia/GraknClient.jl/blob/dev/client-python-reference/tests/integration/test_concept.py
That way we understand the concept classes and can test our translations accordingly.
By opening transaction according the behavior scenario
Scenario: one database, many sessions, many transactions in parallel to read and write in transaction.feature
The order of the upcoming transactions should be the same as they are fired up.
Implement context code for initializing context object
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
Implement background code with hooks before/after the run and each feature test
Hi :)
I am currently working on the Wikipedia page of graph based databases.
TypeDB has/had currently only Java declared as compatible language.
I attend to change this, and I am currently uncertain, if this project here is alive and well maintained.
Can you clearify?
As can be seen in the picture below and can be read up in the docs here, we would need special .proto files to make the client be able to communicate with the grakn-core server.
@jakewilliami has started to create these files here by using the Protobuf.jl but the automatic generation failed like this:
┌─[usr@vs-214-01-l: generated] [main ?] └──╼ $ protoc -I=proto --julia_out=. proto/session.proto protobuf/options.proto: File not found. session.proto:23:1: Import "protobuf/options.proto" was not found or had errors. session.proto:45:13: "Options" is not defined.
But options.proto should be found:
`┌─[usr@vs-214-01-l: generated] [main ?]
└──╼ $ tree
.
├── concept_pb.jl
├── database_pb.jl
├── grakn.jl
├── options_pb.jl
└── proto
├── Build
├── answer.proto
├── concept.proto
├── database.proto
├── options.proto
├── query.proto
├── session.proto
└── transaction.proto
1 directory, 12 files`
In the proto directory are all of the original protoc files here
These would also need to be updated to the latest versions now, in case we want to generate the files in pure Julia.
However, there is a grakn-protocol repo here
that is also on PiPy and compatible with the 2.0 alpha here.
Decision has to be made, on how we go on with this as well.
The attached script runs not without preparing the right paths individually for your machine. Today, the command of preparing the generated file from proto-files didn't work. This should be done by hand. Above and below this every line can be executed after preparing the right paths. Because the issues don't allow attaching Julia files I will place the script file in the translate_java directory in the project GraknClient.jl.
This issue can be closed after the final translation is done.
To build a Attribute the building block for the constructor is a Label. Until now the constructor requires a string that isn't given by the function calling the constructor. Excuse the ending txt in the pinned file. But GitHub didn't support .jl files.
AttributeType.txt
This is a summary issue that keeps track of the work involved for implementing the BDD test suite for GraknClient.jl (for background, see issue #22). Note that ExecutableSpecifications.jl needs to be enhanced to support this project, and that's being tracked separately.
We should be able to largely model the work from Grakn's python-client. The links below point to that repo for reference.
Anything goes that helps.
Check docs for what is available already in contributing #Useful sources
In Julia versions > 1.7.2 the every 4 seconds needed ping to the database server isn't stable after a few minutes of running a session. Above the mentioned Julia version something was changed in the Thread mechanism where the ping lives in the lifespan of the session. As it seems to be the grpc client isn't thread safe in conjunction with the thread mechanism of Julia above 1.7.2
Implement connection steps
To connect to the grakn server, we need the .proto files translated.
https://github.com/Humans-of-Julia/GraknClient.jl/tree/protoc-gen/src/generated
Configure CI to run BDD process (pull feature files, install Grakn, and execute the tests)
Implement concept steps
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.