prabushitha / gremlin-visualizer Goto Github PK
View Code? Open in Web Editor NEWvisualize the graph network corresponding to a gremlin query
License: MIT License
visualize the graph network corresponding to a gremlin query
License: MIT License
This is a great visualizer. We are trying to connect to Neptune over secure websocket. we changed the proxy-server.js to connect to wss:// instead of ws://. However, its still unable to connect and does not give any errors as well. Has anyone been able to connect this library to Neptune securely over SSH tunnel recently (after they mandated HTTPS/SSL/WSS)? Please help.
It seems like this UI is not supporting Janusgraph 5. For any query with size limit more than 1 in the settings panel, it throws this error. I have not debugged properly but seems like there is a change of response in the new gremlin version (3.4.6) used by Janusgraph 5.
ResponseError: Server error: java.lang.NullPointerException (597)
[0] at Connection._handleMessage (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/gremlin/lib/driver/connection.js:269:9)
[0] at WebSocket.<anonymous> (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/gremlin/lib/driver/connection.js:129:43)
[0] at WebSocket.emit (events.js:321:20)
[0] at Receiver._receiver.onmessage (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/ws/lib/WebSocket.js:141:47)
[0] at Receiver.dataMessage (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/ws/lib/Receiver.js:380:14)
[0] at Receiver.getData (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/ws/lib/Receiver.js:330:12)
[0] at Receiver.startLoop (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/ws/lib/Receiver.js:165:16)
[0] at Receiver.add (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/ws/lib/Receiver.js:139:10)
[0] at Socket.<anonymous> (/Users/abhijeet/eclipse-workspace/opensource/gremlin-visualizer/node_modules/ws/lib/WebSocket.js:138:22)
[0] at Socket.emit (events.js:321:20)
Hello all! I try install gremlin-visualizer on kubernetes, i start the container in pod, but i can not connect to my janusgraph with use web gui. I got error - Invalid query. Please execute a query to get a set of vertices. All requests go to Request URL: http://localhost:3001/query
Not sure if you've seen this use case before. I have a new Neptune cluster in my test environment which by default forces connections over https not http. I'm port forwarding via a bastion, so for manual queries from the command line I simply tell curl not to worry about the ssl hostname mismatch (localhost Vs the cert AWS is displaying). Just wondering if anyone has gotten around this with the visualiser? It's a fabulous tool.
Instead of "Type" at the top, I recommend following the TinkerPop convention and calling it "Label". Also, it would be helpful if the table of properties were sorted by key so that it's easier to find something when there are many properties.
when running this docker file I cannot query Vertex:
version: '3.4'
services:
workflowenginedb:
image: docker.io/janusgraph/janusgraph:latest
container_name: workflow-engine-db
restart: always
ports:
- "8182:8182"
environment:
- gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
- gremlin.tinkergraph.vertexIdManager=ANY
- gremlin.tinkergraph.edgeIdManager=ANY
- gremlin.tinkergraph.graphLocation=/opt/gremlin-server/data/graph.json
- gremlin.tinkergraph.graphFormat=graphson
- graph.set-vertex-id=true
# The mounted volume only makes sense if JanusGraph is being run with the BerekeleyDB storage.
volumes:
- "janusgraph-default-data:/var/lib/janusgraph"
gremlinvisualizer:
image: docker.io/prabushitha/gremlin-visualizer:latest
container_name: gremlin-visualizer
restart: always
ports:
- "3000:3000"
- "3001:3001"
network_mode: host
volumes:
janusgraph-default-data:
however if i then stop the gremlinvisualizer container I and execute the src using NPM it connects as expected. I'm assuming that
Have been trying to simply display a small graph as I have always done before. Have updated software environment to run
and the latest clone of gremlin-visualizer. Created a couple of graphs through the Console at "a" and "g", with 455 and 11 nodes respectively. Running gremlin-visualizer client on port 3000 and server on 8182 as have always done. I've tried to run it using
issuing "g.V()" and either getting nothing (npm start) or "Invalid query. Please execute a query to get a set of vertices." (npm run client). I've tried to set the proxy server port in package.json and proxy-server.js to 8182 from 3001 to see if that could help but no change in outcomes. Really like your app but am just at a loss.
Hello Mr. Prabushitha, my message does not refer to any issue. I just wanted to thank you for your work on this magnificent visualization tool. I am a student from Germany and actually working on my own visualization-tool for a project that uses Gremlin to visualize the programming language C# in a Graph. I am supposed to visualize the graph in a Maui application myself. I was wondering if you would be able to help me out. Maybe with some informations about the most crucial parts of your visualization tool. For example the way you serialize your graph. I am having issues to understand how you bind the graph to the UI. Did you use an external library to display the graph or did you build the control yourself. Maybe I can get the oppurtunity to ask some questions about your code? Please let me know. You can contact me at [email protected]. Thank you very much. I am very impressed by this tool. :)
It looks like vertexes that you can search for will check if the values in your gremlin query are there but, it won't display it on the GUI.
This project is fantastic! Might I ask for the following capabilities to be added?
@prabushitha, I wonder if it could be feasible to call your code and run it directly in Jupyter like it can be done with d3 https://blog.thedataincubator.com/2015/08/embedding-d3-in-an-ipython-notebook/, any ideas ? In order to query the server with the right query and draw using your code.
I ran npm install
and npm start
, went to http://localhost:3000
and used query g.V()
, and got no results or even an error message.
I know my SSH tunnel to my Neptune cluster works, becuase I use GraphExp without issue. I also tried your Docker version.
In Chrome developer tools, I see a pending xhr request and then a HTTP 204 No Content reply to the http://localhost:3001/query
request:
I'd like to use your frontend to access a database but I get an error when accessing the server remotely :
Invalid Host header
Is there anything besides adding disableHostCheck: true
to the proxy configuration ?
Prabushitha,
Thank you for your project, it's very rare to find a visualization tool for Gremlin.
I did all the process but I can't see a sample graph, may be I'm doing something wrong. Please see below.
I've started a Gremlin Server followed by a Gremlin console. I've connected into the Server from the console:
gremlin> :remote connect tinkerpop.server conf/remote.yaml session
==>Configured localhost/127.0.0.1:8182-[d3a7a1cb-2ba4-4c68-b93d-fd15ae3d8ade]
After that I've created a sample graph as below:
gremlin> graph = TinkerFactory.createModern()
==>tinkergraph[vertices:6 edges:6]
gremlin> g = traversal().withEmbedded(graph)
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
I can query the graph as below:
gremlin> g.V()
==>v[1]
==>v[2]
==>v[3]
==>v[4]
==>v[5]
==>v[6]
I'm able to connect into the Gremlin Server from the Gremlin-Visualizer and run "g.V()" but nothing appears on the screen. I can see the command in the history too.
Best !
Julio
Thanks for sharing this graph visualization tool!
Could you please add an explicit LICENSE
file to the repo so that it's clear under what terms the content is provided, and under what terms user contributions are licensed?
[...] without a license, the default copyright laws apply, meaning that you retain all rights to your source code and no one may reproduce, distribute, or create derivative works from your work. If you're creating an open source project, we strongly encourage you to include an open source license.
It might also make sense to mention the license type in the package.json
file as well.
Thanks!
Hi,
In this reducer, you are mutating the state directly. As you can see https://redux.js.org/recipes/troubleshooting#never-mutate-reducer-arguments, this should not be done.
So, this should either be fixed, or if there is very specific reason for you doing this, it should be mentioned in a comment.
Thank you.
It seems some functions are not supported. For instance using has('email', containing("test"))
does not work
I have the grateful dead json loaded into a graph, using this graph database: https://github.com/pietermartin/sqlg
I am running g.V()
on your visualizer tool, and getting an error.
The logs from this tool are:
[0] ResponseError: Server error: Error during serialization: Conflicting getter definitions for property "id": org.umlg.sqlg.structure.RecordId#getId(0 params) vs org.umlg.sqlg.structure.RecordId#getID(0 params) (599)
[0] at Connection._handleMessage (/gremlin-visualizer-master/node_modules/gremlin/lib/driver/connection.js:269:9)
[0] at WebSocket._ws.on (/gremlin-visualizer-master/node_modules/gremlin/lib/driver/connection.js:129:43)
[0] at WebSocket.emit (events.js:198:13)
[0] at Receiver._receiver.onmessage (/gremlin-visualizer-master/node_modules/ws/lib/WebSocket.js:141:47)
[0] at Receiver.dataMessage (/gremlin-visualizer-master/node_modules/ws/lib/Receiver.js:380:14)
[0] at perMessageDeflate.decompress (/gremlin-visualizer-master/node_modules/ws/lib/Receiver.js:349:40)
[0] at _decompress (/gremlin-visualizer-master/node_modules/ws/lib/PerMessageDeflate.js:306:9)
[0] at _inflate.flush (/gremlin-visualizer-master/node_modules/ws/lib/PerMessageDeflate.js:386:7)
[0] at afterWrite (_stream_writable.js:485:3)
[0] at onwrite (_stream_writable.js:476:7)
The following logs from the gremlin-server:
could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Conflicting getter definitions for property "id": org.umlg.sqlg.structure.RecordId#getId(0 params) vs org.umlg.sqlg.structure.RecordId#getID(0 params)
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.from(JsonMappingException.java:295)
at org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider.reportMappingProblem(SerializerProvider.java:1309)
at org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1427)
at org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:521)
at org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:799)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:308)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3058)
at org.apache.tinkerpop.shaded.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
at org.apache.tinkerpop.gremlin.structure.io.graphson.JavaUtilSerializersV3d0$MapJacksonSerializer.serialize(JavaUtilSerializersV3d0.java:59)
at org.apache.tinkerpop.gremlin.structure.io.graphson.JavaUtilSerializersV3d0$MapJacksonSerializer.serializeWithType(JavaUtilSerializersV3d0.java:67)
at org.apache.tinkerpop.gremlin.structure.io.graphson.JavaUtilSerializersV3d0$MapJacksonSerializer.serializeWithType(JavaUtilSerializersV3d0.java:49)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrappedSerializer.java:32)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3058)
at org.apache.tinkerpop.shaded.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
at org.apache.tinkerpop.gremlin.structure.io.graphson.JavaUtilSerializersV3d0$ListJacksonSerializer.serialize(JavaUtilSerializersV3d0.java:127)
at org.apache.tinkerpop.gremlin.structure.io.graphson.JavaUtilSerializersV3d0$ListJacksonSerializer.serializeWithType(JavaUtilSerializersV3d0.java:135)
at org.apache.tinkerpop.gremlin.structure.io.graphson.JavaUtilSerializersV3d0$ListJacksonSerializer.serializeWithType(JavaUtilSerializersV3d0.java:118)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrappedSerializer.java:32)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV2d0.java:281)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0$ResponseMessageSerializer.serializeWithType(AbstractGraphSONMessageSerializerV2d0.java:255)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0$ResponseMessageSerializer.serializeWithType(AbstractGraphSONMessageSerializerV2d0.java:240)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrappedSerializer.java:32)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4409)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsBytes(ObjectMapper.java:3687)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.serializeResponseAsBinary(AbstractGraphSONMessageSerializerV2d0.java:85)
at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.makeFrame(AbstractOpProcessor.java:291)
at org.apache.tinkerpop.gremlin.server.op.AbstractOpProcessor.handleIterator(AbstractOpProcessor.java:167)
at org.apache.tinkerpop.gremlin.server.op.AbstractEvalOpProcessor.lambda$evalOpInternal$5(AbstractEvalOpProcessor.java:263)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:283)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: Conflicting getter definitions for property "id": org.umlg.sqlg.structure.RecordId#getId(0 params) vs org.umlg.sqlg.structure.RecordId#getID(0 params)
at org.apache.tinkerpop.shaded.jackson.databind.introspect.POJOPropertyBuilder.getGetter(POJOPropertyBuilder.java:440)
at org.apache.tinkerpop.shaded.jackson.databind.introspect.BeanPropertyDefinition.getAccessor(BeanPropertyDefinition.java:188)
at org.apache.tinkerpop.shaded.jackson.databind.ser.BeanSerializerFactory.removeIgnorableTypes(BeanSerializerFactory.java:698)
at org.apache.tinkerpop.shaded.jackson.databind.ser.BeanSerializerFactory.findBeanProperties(BeanSerializerFactory.java:573)
at org.apache.tinkerpop.shaded.jackson.databind.ser.BeanSerializerFactory.constructBeanOrAddOnSerializer(BeanSerializerFactory.java:384)
at org.apache.tinkerpop.shaded.jackson.databind.ser.BeanSerializerFactory.findBeanOrAddOnSerializer(BeanSerializerFactory.java:286)
at org.apache.tinkerpop.shaded.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:231)
at org.apache.tinkerpop.shaded.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:165)
at org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1474)
at org.apache.tinkerpop.shaded.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1422)
... 36 more
This points to: https://github.com/pietermartin/sqlg/blob/master/sqlg-core/src/main/java/org/umlg/sqlg/structure/RecordId.java#L187-L198
Is there anything that can be done to avoid this?
Can it use v3d0?
I've got an implementation of Google OAuth2 signin (using the new Google Signin framework) available at master...obaqueiro-primetrust:gremlin-visualizer:google_open_id . Ready to be merged once the Neptune change is made (the oauth2 changes are done on top of the others).
Additionally I have additional changes that will be adding in a separate PR with minor UI improvements (double click node to "expand" both in and out edges) and perform query search when pressing [Enter] in the query input text.
I plan on doing additional improvements to my fork:
npm audit
issues and deprecation notices.Long Term / Maybe
Instead of displaying all kinds of edges, I'd like to be able to filter and only show certain edge types (by their label in gremlin).
Is this possible?
If not, could it be requested as a feature?
janusgraph/janusgraph:latest
is running with port 8182
Ran:
> [email protected] start /gremlin-visualizer-master
> concurrently "npm run server" "npm run client"
[1]
[1] > [email protected] client /gremlin-visualizer-master
[1] > react-scripts start
[1]
[0]
[0] > [email protected] server /gremlin-visualizer-master
[0] > node proxy-server.js
[0]
[0] Simple gremlin-proxy server listening on port 3001!
[1] Starting the development server...
[1]
[1] Browserslist: caniuse-lite is outdated. Please run next command `npm update`
[1] Compiled successfully!
[1]
[1] You can now view gremlin-visualizer in the browser.
[1]
[1] Local: http://localhost:3000/
[1] On Your Network: http://172.17.0.2:3000/
[1]
[1] Note that the development build is not optimized.
[1] To create a production build, use npm run build.
[1]
UI gives:
xhr.js:166 POST http://localhost:3001/query net::ERR_EMPTY_RESPONSE
dispatchXhrRequest @ xhr.js:166
xhrAdapter @ xhr.js:16
dispatchRequest @ dispatchRequest.js:49
Promise.then (async)
request @ Axios.js:56
Axios.<computed> @ Axios.js:80
wrap @ bind.js:11
sendQuery @ HeaderComponent.js:16
callCallback @ react-dom.development.js:337
invokeGuardedCallbackDev @ react-dom.development.js:386
invokeGuardedCallback @ react-dom.development.js:439
invokeGuardedCallbackAndCatchFirstError @ react-dom.development.js:454
executeDispatch @ react-dom.development.js:585
executeDispatchesInOrder @ react-dom.development.js:610
executeDispatchesAndRelease @ react-dom.development.js:713
executeDispatchesAndReleaseTopLevel @ react-dom.development.js:722
forEachAccumulated @ react-dom.development.js:694
runEventsInBatch @ react-dom.development.js:739
runExtractedPluginEventsInBatch @ react-dom.development.js:881
handleTopLevel @ react-dom.development.js:5831
batchedEventUpdates$1 @ react-dom.development.js:24378
batchedEventUpdates @ react-dom.development.js:1414
dispatchEventForPluginEventSystem @ react-dom.development.js:5927
attemptToDispatchEvent @ react-dom.development.js:6044
dispatchEvent @ react-dom.development.js:5947
unstable_runWithPriority @ scheduler.development.js:701
runWithPriority$2 @ react-dom.development.js:12231
discreteUpdates$1 @ react-dom.development.js:24395
discreteUpdates @ react-dom.development.js:1439
dispatchDiscreteEvent @ react-dom.development.js:5914
redux-logger.js:377 action SET_ERROR @ 23:26:03.666
redux-logger.js:388 prev state {gremlin: {โฆ}, graph: {โฆ}, options: {โฆ}}
redux-logger.js:392 action {type: "SET_ERROR", payload: "Invalid query. Please execute a query to get a set of vertices"}
redux-logger.js:401 next state {gremlin: {โฆ}, graph: {โฆ}, options: {โฆ}}
xhr.js:166 POST http://localhost:3001/query net::ERR_EMPTY_RESPONSE
dispatchXhrRequest @ xhr.js:166
xhrAdapter @ xhr.js:16
dispatchRequest @ dispatchRequest.js:49
Promise.then (async)
request @ Axios.js:56
Axios.<computed> @ Axios.js:80
wrap @ bind.js:11
sendQuery @ HeaderComponent.js:16
callCallback @ react-dom.development.js:337
invokeGuardedCallbackDev @ react-dom.development.js:386
invokeGuardedCallback @ react-dom.development.js:439
invokeGuardedCallbackAndCatchFirstError @ react-dom.development.js:454
executeDispatch @ react-dom.development.js:585
executeDispatchesInOrder @ react-dom.development.js:610
executeDispatchesAndRelease @ react-dom.development.js:713
executeDispatchesAndReleaseTopLevel @ react-dom.development.js:722
forEachAccumulated @ react-dom.development.js:694
runEventsInBatch @ react-dom.development.js:739
runExtractedPluginEventsInBatch @ react-dom.development.js:881
handleTopLevel @ react-dom.development.js:5831
batchedEventUpdates$1 @ react-dom.development.js:24378
batchedEventUpdates @ react-dom.development.js:1414
dispatchEventForPluginEventSystem @ react-dom.development.js:5927
attemptToDispatchEvent @ react-dom.development.js:6044
dispatchEvent @ react-dom.development.js:5947
unstable_runWithPriority @ scheduler.development.js:701
runWithPriority$2 @ react-dom.development.js:12231
discreteUpdates$1 @ react-dom.development.js:24395
discreteUpdates @ react-dom.development.js:1439
dispatchDiscreteEvent @ react-dom.development.js:5914
And yes I can manually run the same query on the janusgraph gremlin and get results.
It's not possible to connect to a Neptune DB that has IAM Authentication enabled
Hi - I'm trying to install the visualizer on Windows. What is the first step I take to "clone git" in Windows? In your instructions, the first command seems to be a Unix command.
I tried to query on local gremlin server but its not working, please refer attached snippet for more details.
Any pointers if I am missing anything
Running in dockerrized mode
docker run --rm -d -p 3000:3000 -p 3001:3001 --name=gremlin-visualizer prabushitha/gremlin-visualizer:latest
Hi @prabushitha, thanks for the great work on the visualizer. I had some issues with Microsoft Cosmos DB, because of the executed query ${query}${nodeLimitQuery}.dedup().as('node').project('id', 'label', 'properties', 'edges').by(__.id()).by(__.label()).by(__.valueMap().by(__.unfold())).by(__.outE().project('id', 'from', 'to', 'label', 'properties').by(__.id()).by(__.select('node').id()).by(__.inV().id()).by(__.label()).by(__.valueMap().by(__.unfold())).fold())
.
It contains Gremlin constructs not supported by Cthe Cosmos DB Graph implementation of Gremlin: by(__.valueMap().by(__.unfold())
.
I created a version that works correctly for Cosmos DB Graph where I did split the query into three different queries to get it working correctly. I also added an additional endpoint in the service to request the incoming and outgoing edges. Because Cosmos B Graph contains an authenticated endpoint I configure the proxy server with an included config.js
file:
config.js:
var config = {}
config.endpoint = "wss://<yourservername>.gremlin.cosmosdb.azure.com:443/gremlin";
config.primaryKey = "<your primary key as can be found under keys in Azure>"
config.database = "<name of the database>";
config.collection = "<name of the collection>";
config.port = 3001;
module.exports = config;
proxy-server.js:
const express = require('express');
const bodyParser = require('body-parser');
const Gremlin = require('gremlin');
const cors = require('cors');
const app = express();
const fs = require('fs');
const path = require('path');
const config = require('./config');
app.use(cors({
credentials: true,
}));
// parse application/json
app.use(bodyParser.json());
// Each property has as value an array with length 1 - take first element
function mapVertexPropertiesToObj(propInObj) {
let propOutObj = {};
Object.keys(propInObj).forEach(k => propOutObj[k] = propInObj[k][0]);
return propOutObj;
}
function edgesToVisualizationStructure(edges) {
if (!!edges) {
return edges.map(
edge => ({
id: typeof edge.id !== "string" ? JSON.stringify(edge.id) : edge.id,
from: edge.from,
to: edge.to,
label: edge.label,
properties: edge.properties,
})
);
} else {
return [];
}
}
function nodesToVisualizationStructure(nodeList) {
return nodeList.map(
node => ({
id: node.id,
label: node.label,
properties: mapVertexPropertiesToObj(node.properties),
edges: edgesToVisualizationStructure(node.edges)
})
);
}
function makeSelfQuery(query) {
const theQuery = `${query}.
as('node').
project('id', 'label', 'properties').
by(__.id()).
by(__.label()).
by(__.valueMap())
`;
return theQuery;
}
function makeInQuery(query, nodeLimit) {
// original query: `${query}${nodeLimitQuery}.dedup().as('node').project('id', 'label', 'properties', 'edges').by(__.id()).by(__.label()).by(__.valueMap().by(__.unfold())).by(__.outE().project('id', 'from', 'to', 'label', 'properties').by(__.id()).by(__.select('node').id()).by(__.inV().id()).by(__.label()).by(__.valueMap().by(__.unfold())).fold())`;
const nodeLimitQuery = !isNaN(nodeLimit) && Number(nodeLimit) > 0 ? `.limit(${nodeLimit})` : '';
const theQuery = `${query}${nodeLimitQuery}.
dedup().
as('node').
project('id', 'label', 'properties', 'edges').
by(__.id()).
by(__.label()).
by(__.valueMap()).
by(__.outE().as('outEdge').
project('id', 'from', 'to', 'label', 'properties').
by(__.id()).
by(select('node').id()).
by(__.inV().id()).
by(__.label()).
by(__.valueMap()).
fold()
)
`;
// coalesce(select('outEdge').inV().count().is(gt(0)).select('outEdge').inV().id(), constant("NO_TO_VERTEX"))
return theQuery;
}
function makeOutQuery(query, nodeLimit) {
// original query: `${query}${nodeLimitQuery}.dedup().as('node').project('id', 'label', 'properties', 'edges').by(__.id()).by(__.label()).by(__.valueMap().by(__.unfold())).by(__.outE().project('id', 'from', 'to', 'label', 'properties').by(__.id()).by(__.select('node').id()).by(__.inV().id()).by(__.label()).by(__.valueMap().by(__.unfold())).fold())`;
const nodeLimitQuery = !isNaN(nodeLimit) && Number(nodeLimit) > 0 ? `.limit(${nodeLimit})` : '';
const theQuery = `${query}${nodeLimitQuery}.
dedup().
as('node').
project('id', 'label', 'properties', 'edges').
by(__.id()).
by(__.label()).
by(__.valueMap()).
by(__.inE().
project('id', 'from', 'to', 'label', 'properties').
by(__.id()).
by(__.outV().id()).
by(select('node').id()).
by(__.label()).
by(__.valueMap()).
fold()
)
`;
return theQuery;
}
async function executeQuery(query) {
const authenticator = new Gremlin.driver.auth.PlainTextSaslAuthenticator(`/dbs/${config.database}/colls/${config.collection}`, config.primaryKey)
const client = new Gremlin.driver.Client(
config.endpoint,
{
authenticator,
traversalsource : "g",
rejectUnauthorized : true,
mimeType : "application/vnd.gremlin-v2.0+json"
}
);
console.log(query);
try {
const result = await client.submit(query, {})
console.log(JSON.stringify(result, null, 2));
return result;
}
catch(err) {
console.error(err);
return null;
}
}
app.post('/query', async (req, res, next) => {
const nodeLimit = req.body.nodeLimit;
let query = "" + req.body.query;
let visualizationNodesAndEdges = [];
// Support for sample files to show possible
if (query.startsWith("sample:")) {
try {
const sample = query.split(":")[1];
visualizationNodesAndEdges = JSON.parse(fs.readFileSync(path.join(__dirname, "samples", `${sample}.json`), 'utf8'));
}
catch(err) {
console.error(err);
}
} else {
let theQuery;
if(query.endsWith(".out()")) {
theQuery = makeOutQuery(query, nodeLimit);
} else if (query.endsWith(".in()")) {
theQuery = makeInQuery(query, nodeLimit);
} else {
theQuery = makeSelfQuery(query);
}
const result = await executeQuery(theQuery);
if (result !== null) {
visualizationNodesAndEdges = nodesToVisualizationStructure(result._items);
}
}
const visualizationNodesAndEdgesPrettyfiedJSon = JSON.stringify(visualizationNodesAndEdges, null, 2);
console.log(visualizationNodesAndEdgesPrettyfiedJSon);
res.send(visualizationNodesAndEdgesPrettyfiedJSon);
});
app.get('/edgecount/:nodeId', async (req, res, next) => {
const nodeId = req.params.nodeId;
let query = `g.V("${nodeId}").project("inEdgesCount", "outEdgesCount").by(__.inE().count()).by(__.outE().count())`;
const result = await executeQuery(query); // result._items in format: [ { "inEdgesCount": 2, "outEdgesCount": 0 } ]
let countInfo;
if (result === null || result._items.length === 0) {
countInfo = { 'inEdgesCount': -1, 'outEdgesCount': -1}; // error - node does not exist?
} else {
countInfo = result._items[0];
}
res.send(JSON.stringify(countInfo, null, 2));
});
app.listen(config.port, () => console.log(`Simple Gremlin proxy-server listening on port ${config.port}!`));
I also added support for sample graph visualization files without using a Gremlin server, and no support for drilling through the graph. Create next to the proxy-server.js
file a folder samples
, and add for example the following file:
modern.json: (patterned after the graph created with TinkerFactory.createModern()
as described in the Tinkerpop Reference Documentation
[
{
"id": "1",
"label": "person",
"properties": {
"name": "marko",
"age": 29
},
"edges": [
{
"id": "7",
"from": "1",
"to": "2",
"label": "knows",
"properties": {
"weight": 0.5
}
},
{
"id": "8",
"from": "1",
"to": "4",
"label": "knows",
"properties": {
"weight": 1.0
}
},
{
"id": "9",
"from": "1",
"to": "3",
"label": "created",
"properties": {
"weight": 0.4
}
}
]
},
{
"id": "2",
"label": "person",
"properties": {
"name": "vadas",
"age": 27
}
},
{
"id": "3",
"label": "software",
"properties": {
"name": "lop",
"lang": "java"
}
},
{
"id": "4",
"label": "person",
"properties": {
"name": "josh",
"age": 32
},
"edges": [
{
"id": "11",
"from": "4",
"to": "3",
"label": "created",
"properties": {
"weight": 0.4
}
},
{
"id": "10",
"from": "4",
"to": "5",
"label": "created",
"properties": {
"weight": 1.0
}
}
]
},
{
"id": "5",
"label": "software",
"properties": {
"name": "ripple",
"lang": "java"
}
},
{
"id": "6",
"label": "person",
"properties": {
"name": "peter",
"age": 35
},
"edges": [
{
"id": "12",
"from": "6",
"to": "3",
"label": "created",
"properties": {
"weight": 0.2
}
}
]
}
]
If instead of a query the following string is specified sample:modern
the sample JSON is read and returned.
@prabushitha could you provide a way to specify those when starting the server and remove them from interface ? something like GREMLIN_SERVER and GREMLIN_PORT (defaulting to 8182).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.