debasishdebs / janusgraph-py Goto Github PK

The Index builder classes needs to be designed as follows so that the following syntax can be followed to create indexes.

For Composite index:

graph = JanusGraph().connect()
mgmt = graph.openManagement()

index_builder = mgmt.buildCompositeIndex("indexName", "Vertex")
index_builder.addKey("prop1").addKey("prop2").indexOnly("label1").unique().make()

For Mixed Index:

graph = JanusGraph().connect()
mgmt = graph.openManagement()

index_builder = mgmt.buildMixedIndex("indexName", "Vertex")
index_builder.addKey("prop1", maping("TEXT"/"STRING"))..addKey("prop2", maping("TEXT"/"STRING")).indexOnly("label1").make()

For Vertex Centrix Index:

graph = JanusGraph().connect()
mgmt = graph.openManagement()

index_builder = mgmt.buildVertexCentricIndex("indexName")
index_builder.addEdge("label").direction("BOTH").order("ASC").on(["prop1", "prop2"]).make()

The Schema helpers are part of janusgraph_python.core.schema package.
Build a CompositeIndexHelper, MixedIndexHelper and VertexCentricHelper which builds stringified query which submits against server when make() is created.

Currently the import of modules happen from absolute paths, i.e. from where the libs are installed on system. We will need to move that to relative path corresponding to current module.

Currently, in IndexBuilders, we have a base query, and based on method invoked like addKey, indexOnly etc, the corresponding query gets appended to base query.

Such structure needs to be followed as uniform across all schema element builders, like LabelBuilder, PropertyBuilder etc.

The GeoShape as well as RelationIdentifier implements equality method. The same currently works only if the other object in of same type, and equal.

Ex: Circle(x,y,z) == Circle(x,y,z) but Circle(x,y,z) != CIRCLE(lat: x, lon: y, r: z) (String representation)

Extend it to check for equality even if String object is passed.

Update build so that is used "pyb package publish" instead of just "pyb"

Currently, all development is being carried out with TinkerPop 3.3.3 and GraphSON 3.0.

Once we publish candidates with various TinkerPop versions, we will then need to add GraphSON 2.0 and 1.0 support so that older versions of JanusGraph can also use the client drivers.

Currently, for the following:

• Edge Label
• Vertex Label
• Property Key

We use a if else loop as gremlin query to first query for the element, if present retrive it or else create it as follows:

q = "if (mgmt.getEdgeLabel('{}')) {{" 
 "   {} = mgmt.getEdgeLabel('{}'); }}" 
"else {{" 
 "   {} = mgmt.makeEdgeLabel('{}').multiplicity({}).make(); }}\n".format(
self.label, self.label, self.label, self.label, self.label, self.multiplicity) 

This level of complex string formatting can be avoided by using JanusGraph's inbuilt APIs which does the same check.

Instead of we makeEdgeLabel in above example we will need to move to getOrCreateEdgeLabel and similarly for all other elements too.

JanusGraph creates relationIdentifier object for Edge IDs. We will need to create an object for the same, Serializer and De-serializer for the same too.

The Schema helpers are part of janusgraph_python.core.schema package.
Build a VertexwhicLabelHelper and EdgeLabelHelper which builds stringified query which submits against server when make() is created.

This involves cleaning up the code base for 0.1beta release. Issue for adding comments will be added in separate one.

Remove unused file for release. Includes:
GeoPredicates for GeoContains & GeoWithin.
GeoShape for Point and Circle.
JanusGraph Client.
Serializer.

Currently we assert the equality with Class object once the object is deserialized. We need to update that, so that the string representation of class is asserted against. This will help us in representing the String representation of class instead of the class itself while we do valueMap thus increasing readability.

Add methods to JanusGraph client for closing connection and dropping the remote Graph.

The method signature will be close() and drop()

While close() : Closes both types of connection (Traversal and Management) to Remote server, the drop() will issue a query which drops remote Graph.

For creating Indexes, it is very necessary that any transactions aren't open. If a Transaction is open, and we create an Index, then there is good chance that the Index will remain in INSTALLED state and might not move to REGISTERED state.

graph.tx().rollback() and Closed Transactions are needed to be in sync to create Indexes properly.

Currently, there is no way from JanusGraph Python to visualize the open Transactions, and a way to close them. We need to create Management APIs for the same.

The will follow following syntax:

graph = JanusGraph().connect()
mgmt = graph.openManagement()

mgmt.getOpenInstances()
# Returns list of IDs
mgmt.forceCloseInstance(ID)
# Returns null, closses the Instance referenced by ID

# Further Steps to create Index

Based on comments from @pluradj it looks like the order of Longitude & Latitude is reversed in GeoShapes.

The reversed order was based on conversation. An issue #10 is created on official repo to track the same.

Docstrings needs to be added to every class and method, so that the same can be used by Spinx if needed. And, method definations needs to be defined also.

JanusGraph stores its IDs as RelationIdentifies objects. We will need to serialize it into strings to that correcponding Python scripts can access the object.

Create index page with reference to existing pages, except contributions.html

JanusGraphClientBuilder currently accepts URL, PORT, Graph for creating remote connection and returns traversal. The same needs to be changed so that it is aligned with Gremlin.NET declaration, JanusGraph.NET declaration, as well as the way TinkerPop suggests us to create connection.

The class should take GremlinServer instance, and return updated GremlinServer instance with JanusGraph writer/reader classes updated.

The Schema helpers are part of janusgraph_python.core.schema package.
Build a PropertyKeyBuilder which builds stringified query which submits against server when make() is created.

Looks like the file core/attribute/GeoPredicate/geoWithin.py should be renamed GeoWithin. Doing so allows me to use: from janusgraph_python.structure.io.GraphsonReader import JanusGraphSONReader. Otherwise, I get the error: ModuleNotFoundError: No module named 'janusgraph_python.core.attribute.GeoPredicate.GeoWithin'

The default_task is split and standardizing with mode granular task control.

Changed from ['clean', 'install_dependencies', 'publish', 'pycharm_generate'] to ['clean', 'install_dependencies', 'prepare', 'compile_sources', 'pycharm_generate']

Now, pyb just compiles package, package to build tar and publish takes care of publishing.

TinkerPop version needs to be updated corresponding to each version of library being pushed to PyPi. For ease of use, we will keep the name of Python libraries same version no. as JanusGraph version it is supporting.

Accordingly 4 version will be released corresponding to separate TinkerPop version with active support for 2 versions (0.2.1 and 0.3.0)

Now, when a Index query is build to create an Index, the set of queries being submitted to Server contains queries for

• Creating Index.

• Await till Graph Index is in Installed state.

• Update Index into existing data.

We need to move the last 2 points, i.e. awaitGraphIndex and updateIndex out of Index builder so that user can invoke it based on their needs. Like UpdateIndex isn't required if the schema loading is first thing being done on graph.

The syntax to be followed would be:

graph = JanusGraph().connect()
mgmt = graph.openManagement()

index_builder = mgmt.buildCompositeIndex("indexName", "Vertex")
index_builder.addKey("prop1").addKey("prop2").indexOnly("label1").unique().make()

mgmt.awaitGraphIndex("indexName").make()

mgmt.updateIndex("indexName").make()

Testing on GraphOfGods graph.

We will need to implement something like JanusGraph P class along with proper serialization/de-serialization so that the same functionalities can be used.

Currently, we have snapshot of Master from TinkerPop copied manually from TinkerPop repo. We would want to pull that information dynamically so that we can be in sync anytime changes to TinkerPop happens.

A new interface to connecting to JanusGraph in Gremlin server mode needs to be defined.

The interface will have connect method which connects to JanusGraph running in Gremlin server mode.
The interface provides 2 APIs, like JanusGraph
namely traversal() to create traversal object to query data.
and openManagement() to create management object which is used to create schema elements.

It also needs to have close method to close connection.

Push to Artifactory, in this case PyPi when we merge into Master. Also note that the push to artifactory will be done when the library is tagged with a major version number change.

We need to make use of Travis CI to push it to PyPi.

While PR is being approved for 1.0.0 release, the repo needs to be updated with CodeBase for 1.0.1.

1.0.1 version plans to add following features, but a few can be added or removed later on depending on feasibility.

1: Schema Management
-> Make Property keys.
-> Make Elements, vertex or edge label.
-> Make Composite Index.
-> Make Mixed Index.
-> Make Vertex Centric Index.

2: Single Graph object of type JanusGraph which contains 2 APIs
-> Traversal API which returns GraphTraversalSource and has JanusGraph specific requirements (1.0.0 release) serializers and deserializers registered.
-> Management API which will be used to create Schema elements.

3: Update docs whereever necessary