kraison / vivace-graph-v3 Goto Github PK

This looks very interesting. How close is it to "production ready?"

This is only true of edges that have no slots, as their BYTES slot will always have value :INIT

The value :INIT is not of type SEQUENCE.
0: (LENGTH :INIT)
1: (GRAPH-DB::TRANSACTION-NODE-VECTOR-SIZE #<HAS-MEMBER 0b79edc96f175427931d7095d16a69ff REV 0 ..)

Something about how automatically generated prolog functor names are used is keeping them from being garbage collected. Look through all calls to prolog-gensym in prologc.lisp and trace the leak.

I have tested on 2.2.3, 2.2.4, 2.2.6, and 2.2.9, and they all give the same error message:

Recursive lock attempt #<SB-THREAD:MUTEX taken owner=worker {1016D70D03}>.

when running

(with-transaction ()
  (let ((c1 (make-customer :first-name "Joe" :last-name "Blow" :email "[email protected]"))
        (c2 (make-customer :first-name "Jill" :last-name "Blow" :email "[email protected]"))
        (m1 (make-merchant :name "Snake Oil, Inc."))
        (p1 (make-product :name "Oil of Longevity" :upc "1234567890"))
        (p2 (make-product :name "Oil of Slipperiness" :upc "abcdefghijk")))
    (make-sells :from m1 :to p1)
    ;; The above is equivalent to
    ;; (make-edge 'sells m1 p1 1 nil)
    (make-sells :from m1 :to p2)
    (make-likes :from c1 :to p1 :weight 100.0)
    (make-likes :from c1 :to p2 :weight 20.0)
    (make-likes :from c2 :to p2 :weight 50.0)))

2.2.1 and 2.2.2 are fine.

This issue deals with

I noticed that after making the graph, its schema is stored without any of its class locks or locks. Then when it's restored, the class locks nor locks return, resulting in (schema-class-locks (schema graph)) being nil. This prevents some functions that use instantiate-node-type, e.g. def-edge.

To recreate, make an empty graph, close it, open it again, just to notice schema-class-locks being nil. Then you can't def-edge.

Temporarily I have made a workaround to this issue in my fork.

What's the status on the upcoming documentation? This looks like a very interesting project, a manual would really lower the barrier to start experimenting with it.

Storing nodes (vertices or edges) in the slots of other nodes doesn't make sense, and causes an error downstream. Throw an error immediately when this is attempted.

The right thing to do is connect a node to another node with an edge.

Right now, class definitions are not stored with the graph itself; rather, it is required that the programmer load them when loading the rest of her code. It would be nice to find a way to get around this. However, class definitions must come before method definitions in CLOS, so it is not immediately clear how to make this work. A compromise might be to store the schema with the graph simply as a reference.

On mmap.lisp(151) there is a call to a function osicat-posix:mremap. This seems to be unavailable on Darwin. Is there a way to support MacOS in this regard?

When mapping a graph view for a class and attempting to modify a node of that class in the lambda expression passed to MAP-VIEW, a deadlock condition occurs. MAP-VIEW holds a read lock on the view-group, and attempting to save a node of the same class will try to grab a write-lock for the same view group. The simple rw-lock system that vivace uses isn't smart enough to detect that the writer is already a reader. This perhaps should be fixed. However, it is also debatable whether or not modifying a node of the type you are mapping is really a good practice to begin with.

Any chance to publish it in QL dist?

In seriousness, really interesting. Linear hash is very cool. Schema at first glance seems so much better than the static, minimalist, unabstracted approach Ive used I'm very impressed and a little bit embarrassed.

I havent played with v-g-3 yet but it certainly seems to be a significantly more mature design than v-g-2. (I actually was a fan of v-g-1, which I thought was remarkably elegant in its simple brevity, although I wasn't clear about where the abrupt change in direction that the Javascript server, etc were leading.

I am looking forward to studying this, but if you are indeed working on some docs, would you mind saying a few words abt the overall architecture and how the various techjnologies fit together? Ie, I assume the skiplist stuff is for text indexing, the linear hashing for node indexes (with an eye toward ability for future of distributed index capability?) etc. Im interested in the design, but also any insights that motivated it.

Finally, what made you move to node-property model? im kind of shocked about that, actually.

Well, it is looking quite promising... As they say, third time's the charm!

I hate to be the one that needs to ask this question, but do you happen to have any additional example materials sitting around? Especially wrt to prolog query/rules, but also useful general data/schema models? Its a really nice library I wish I were better at applying it

Not sure if having multiple graphs is the intended use case, but on my machine with SBCL, after calling make-graph on four graphs, calling make-graph halts the interpreter.

SBCL 1.4.16 on MacOS

; caught ERROR:
;   READ error during COMPILE-FILE:
;   
;     Symbol "MREMAP" not found in the OSICAT-POSIX package.
;   
;       Line: 151, Column: 33, File-Position: 5176
;   
;       Stream: #<SB-INT:FORM-TRACKING-STREAM for "file /Users/ar/quicklisp/local-projects/vivace-graph-v3/mmap.lisp" {100C1D3FB3}>

When a view is defined in terms of a parent class, its child classes don't update the view on change. However, if you regenerate the view from scratch, the child classes are incorporated. Fix this for consistent behavior when updating child class objects.

Unique slot values would be very useful. Enforcing uniqueness now is cumbersome.

If a transaction relies on a view for which it has added an item, that item will not appear in the view until the transaction is complete, making for inconsistent results. You can program your way around this, but it would be more natural to have a version of the view that is private to the transaction which could be referenced before referencing the global view.

I have taken a keen interest in this project; I have a small project and have both a vivace-graph version and a neo4j version, with significant speedup with vivace-graph version, so much so that I 'm willing to work with a largely unmaintained project. What are the authors' plan with this project? I am curious.

VG was intended to mimic couchdb's behavior of keeping old revisions of nodes around until manually garbage collected, or perhaps until some condition is triggered. It would be a simple matter to add an PREVIOUS-REVISION-POINTER field to the NODE base class and write the address of the old node to the heap with the new revision.

Currently, views on edges only update when the edge itself is changed. In practice, most views on edges use data from the vertices of the edge. Thus, the view should update when the vertices change.