Code Monkey home page Code Monkey logo

falkordb / falkordb Goto Github PK

View Code? Open in Web Editor NEW
397.0 13.0 16.0 73.42 MB

A super fast Graph Database uses GraphBLAS under the hood for its sparse adjacency matrix graph representation. Our goal is to provide the best Knowledge Graph for LLM (KG-RAG).

Home Page: https://www.falkordb.com/

License: Other

Shell 0.58% C++ 2.28% Python 19.93% C 56.48% Makefile 0.28% CMake 0.03% Dockerfile 0.03% Gherkin 20.40%
graph-database knowledge-graph database-as-a-service cloud-database database developer-tools devtools realtime-database

falkordb's Introduction

Dockerhub Discord Workflow

FalkorDB

Try Free

Our objective is to create an outstanding Knowledge Graph specifically for Large Language Models (LLM) that boasts exceptionally low latency, ensuring swift delivery of information through our Graph Database, known as KG-RAG.

FalkorDB is the first queryable Property Graph database to use sparse matrices to represent the adjacency matrix in graphs and linear algebra to query the graph.

Primary features:

  • Adopting the Property Graph Model
    • Nodes (vertices) and Relationships (edges) that may have attributes
    • Nodes can have multiple labels
    • Relationships have a relationship type
  • Graphs represented as sparse adjacency matrices
  • OpenCypher with proprietary extensions as a query language
    • Queries are translated into linear algebra expressions

To see FalkorDB in action, visit Demos.

Quickstart

  1. Trying FalkorDB
  2. Build
  3. Start
  4. Use from any client
  5. Documentation

Give it a try

To quickly try out FalkorDB, launch an instance using docker:

docker run -p 6379:6379 -it --rm falkordb/falkordb:edge

Or, to use the built-in browser-based interface, run:

docker run -p 6379:6379 -p 3000:3000 -it --rm falkordb/falkordb:edge

Then, open your browser and navigate to http://localhost:3000.

You can also interact with FalkorDB using any of the supported client libraries

Here we'll use FalkorDB Python client to create a small graph representing a subset of motorcycle riders and teams taking part in the MotoGP league, once created we'll start querying our data.

from falkordb import FalkorDB

# Connect to FalkorDB
db = FalkorDB(host='localhost', port=6379)

# Create the 'MotoGP' graph
g = db.select_graph('MotoGP')
g.query("""CREATE (:Rider {name:'Valentino Rossi'})-[:rides]->(:Team {name:'Yamaha'}),
                  (:Rider {name:'Dani Pedrosa'})-[:rides]->(:Team {name:'Honda'}),
                  (:Rider {name:'Andrea Dovizioso'})-[:rides]->(:Team {name:'Ducati'})""")

# Query which riders represents Yamaha?
res = g.query("""MATCH (r:Rider)-[:rides]->(t:Team)
                 WHERE t.name = 'Yamaha'
                 RETURN r.name""")

for row in res.result_set:
	print(row[0])

# Prints: "Valentino Rossi"

# Query how many riders represent team Ducati ?
res = g.query("""MATCH (r:Rider)-[:rides]->(t:Team {name:'Ducati'})
                 RETURN count(r)""")

print(row[0])
# Prints: 1

Building

Compiling

Requirements:

  • The FalkorDB repository: git clone --recurse-submodules -j8 https://github.com/FalkorDB/FalkorDB.git

  • On Ubuntu Linux, run: apt-get install build-essential cmake m4 automake peg libtool autoconf python3 python3-pip

  • On OS X, verify that homebrew is installed and run: brew install cmake m4 automake peg libtool autoconf.

    • The version of Clang that ships with the OS X toolchain does not support OpenMP, which is a requirement for FalkorDB. One way to resolve this is to run brew install gcc g++ and follow the on-screen instructions to update the symbolic links. Note that this is a system-wide change - setting the environment variables for CC and CXX will work if that is not an option.

To build, run make in the project's directory.

Congratulations! You can find the compiled binary at bin/<arch>/src/falkordb.so.

Running tests

First, install required Python packages by running pip install -r requirements.txt from the tests directory.

If you've got redis-server in PATH, just invoke make test.

Otherwise, invoke REDIS_SERVER=<redis-server-location> make test.

For more verbose output, run make test V=1.

Building in a docker

The FalkorDB build system runs within docker. For detailed instructions on building, please see here.

Loading FalkorDB into Redis

FalkorDB is hosted by Redis, so you'll first have to load it as a Module to a Redis server. Redis 6.2 is required for FalkorDB 2.12.

We recommend having Redis load FalkorDB during startup by adding the following to your redis.conf file:

loadmodule /path/to/module/src/falkordb.so

In the line above, replace /path/to/module/src/falkordb.so with the actual path to FalkorDB's library. If Redis is running as a service, you must ensure that the redis user (default) has the necessary file/folder permissions to access falkordb.so.

Alternatively, you can have Redis load FalkorDB using the following command line argument syntax:

~/$ redis-server --loadmodule /path/to/module/src/falkordb.so

Lastly, you can also use the MODULE LOAD command. Note, however, that MODULE LOAD is a dangerous command and may be blocked/deprecated in the future due to security considerations.

Once you've successfully loaded FalkorDB your Redis log should see lines similar to:

...
30707:M 20 Jun 02:08:12.314 * Module 'graph' loaded from <redacted>/src/falkordb.so
...

If the server fails to launch with output similar to:

# Module /usr/lib/redis/modules/falkordb.so failed to load: libgomp.so.1: cannot open shared object file: No such file or directory
# Can't load module from /usr/lib/redis/modules/falkordb.so: server aborting

The system is missing the run-time dependency OpenMP. This can be installed on Ubuntu with apt-get install libgomp1, on RHEL/CentOS with yum install libgomp, and on OSX with brew install libomp.

Using FalkorDB

You can call FalkorDB's commands from any Redis client.

With redis-cli

$ redis-cli
127.0.0.1:6379> GRAPH.QUERY social "CREATE (:person {name: 'roi', age: 33, gender: 'male', status: 'married'})"

With any other client

You can interact with FalkorDB using your client's ability to send raw Redis commands.

Depending on your client of choice, the exact method for doing that may vary.

Python example

This code snippet shows how to use FalkorDB with from Python using falkordb-py:

from falkordb import FalkorDB

# Connect to FalkorDB
db = FalkorDB(host='localhost', port=6379)

# Select the social graph
g = db.select_graph('social')

reply = g.query("CREATE (:person {name:'roi', age:33, gender:'male', status:'married'})")

Client libraries

Some languages have client libraries that provide support for FalkorDB's commands:

Project Language License Author Stars Package Comment
jfalkordb Java BSD FalkorDB Stars Maven
falkordb-py Python MIT FalkorDB Stars pypi
falkordb-ts Node.JS MIT FalkorDB Stars npm
nredisstack .NET MIT Redis Stars nuget
redisgraph-rb Ruby BSD Redis redisgraph-rb-stars GitHub
redgraph Ruby MIT pzac redgraph-stars GitHub
redisgraph-go Go BSD Redis redisgraph-go-stars GitHub
rueidis Go Apache 2.0 Rueian rueidis-stars GitHub
ioredisgraph JavaScript ISC Jonah ioredisgraph-stars GitHub
@hydre/rgraph JavaScript MIT Sceat rgraph-stars GitHub
php-redis-graph PHP MIT KJDev php-redis-graph-stars GitHub
redisgraph_php PHP MIT jpbourbon redisgraph_php-stars GitHub
redisgraph-ex Elixir MIT crflynn redisgraph-ex-stars GitHub
redisgraph-rs Rust MIT malte-v redisgraph-rs-stars GitHub
redis_graph Rust BSD tompro redis_graph-stars GitHub
rustis Rust MIT Dahomey Technologies rustis-stars Crate Documentation
NRedisGraph C# BSD tombatron NRedisGraph-stars GitHub
RedisGraph.jl Julia MIT xyxel RedisGraph.jl-stars GitHub

License

Licensed under the Server Side Public License v1 (SSPLv1). See LICENSE.

falkordb's People

Contributors

swilly22 avatar jeffreylovitz avatar aviavni avatar gkorland avatar dvirdukhan avatar rafie avatar chayim avatar liorkogan avatar ofirmos avatar raz-mon avatar nafraf avatar filipecosta90 avatar k-jo avatar itamarhaber avatar dudizimber avatar alonre24 avatar dependabot[bot] avatar iddm avatar tomerhekmati avatar leibale avatar nermiller avatar meirshpilraien avatar odanoburu avatar sel-fish avatar slorello89 avatar rueian avatar omrib1 avatar sazzad16 avatar danitseitlin avatar chrisatredislabs avatar

Stargazers

Daniel Fahey avatar Caleb Del Begio avatar  avatar KimSia Sim avatar Flaire avatar Nikolaus Schlemm avatar Wonhong Jang avatar  avatar  avatar  avatar Jaromir Hamala avatar Daniel Dalsgaard avatar St. Kek O'Paris avatar Cameron Kelley avatar Seva Baskin avatar Eugene Klimov avatar Nur Arifin Akbar avatar Robin avatar  avatar  avatar  avatar Dhruv Anand avatar Jeroen Steggink avatar Ryan Orban avatar  avatar Tane Piper avatar Allan Berger avatar Harsh Pundhir avatar  avatar Andrew McElroy avatar Arthur Lee avatar Dao_er avatar  avatar Gunes Korkmaz avatar Can Evgin avatar Hans Speijer avatar Takao Mizuno avatar  avatar Bruno Capelão avatar zxf945 avatar  avatar Tobias avatar FFengIll avatar Nevo David avatar Felipe Menegazzi avatar Rinor Hoxha avatar SamLast avatar  avatar Jeffrey Morgan avatar Michael avatar Jeff Blake avatar Allan Kobelansky avatar Juan Pablo Manson avatar Scott Joyner avatar Moore123 avatar Nissim Bitan avatar Kevin Elliott avatar Sumit Mittal avatar Ariel Shtul avatar Alok avatar silaskenneth avatar HeTao avatar David Mauger avatar Murray Hopkins avatar Tim avatar Chris Evers avatar  avatar Chih-Yu Yeh avatar  avatar 宁培培 avatar  avatar Nikita Koksharov avatar pberton avatar  avatar Andreas Motl avatar Daniel Popsuevich avatar nizq avatar Dixon Dick avatar  avatar Samuel Galanakis avatar Igor Gilmutdinov avatar xQ avatar Richard Dijkstra avatar  avatar  avatar Igal Rozen avatar Andreas Thoelke avatar  avatar Denys Lazarenko avatar BA7LYA avatar Dylan Yang avatar CIH avatar ‮rekcäH nitraM‮ avatar  avatar  avatar  avatar Dan Frommer avatar Andrey ``Bass'' Shcheglov avatar Muharrem Okutan avatar  avatar

Watchers

 avatar Peter Styk avatar James Cloos avatar Arthur Hirel avatar  avatar Kasper Piskorski avatar p0xiao avatar Zim avatar  avatar  avatar Jérémy Reynard avatar  avatar Sean Macdonald avatar

falkordb's Issues

Unexpected result about number of nodes in graph

This issue was imported from the original issue: RedisGraph/RedisGraph#3192
Created by: @ljhhuxiaoba

I execute a query on my empty RedisGraph and get an unexpected result.

Redis version:6.2.10
RedisGraph version: v2.10.9
operating system: windows 10
API:Cypher version 9
query:create (n) delete n with 1 as a match (n0) return count(n0)

The query creates a node and then deletes it, at which point the number of nodes in the graph is 0, but the return result is 1.

[Feature] Ability for loading/unloading graphs to/from disk

I understand that this feature might not be possible to implement if Falkor remains tied to Redis as a module, nevertheless, one of the problems I had with RedisGraph was that at any given time I would have a set of graphs that I would be actively querying, while the graphs outside of that set would be wasting the limited amount of RAM on my VPS. It would be nice if Falkor had a command or the ability to automatically load/unload graphs to/from disk in order to remediate this issue.

Redis Graph doesn't allow single quote to be escaped when creating a vertex/edge

Created by: @tperry
Source: RedisGraph/RedisGraph#188
Here's an example of the problem we've encountered when trying to create a vertex that has a string attribute whose value contains a single quote.

    RedisGraphAPI api = new RedisGraphAPI("videoDB");
    api.query( "CREATE (:person{name:'roi\'s', age:32})" );

Exception in thread "main" redis.clients.jedis.exceptions.JedisDataException: Syntax error at offset 28 near 's'
at redis.clients.jedis.Protocol.processError(Protocol.java:131)
at redis.clients.jedis.Protocol.process(Protocol.java:165)
at redis.clients.jedis.Protocol.read(Protocol.java:219)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:309)
at redis.clients.jedis.Connection.getRawObjectMultiBulkReply(Connection.java:276)
at redis.clients.jedis.Connection.getObjectMultiBulkReply(Connection.java:280)
at com.redislabs.redisgraph.RedisGraphAPI.query(RedisGraphAPI.java:58)
at com.oncam.saas.redisgraph.OrientDbImport.main(OrientDbImport.java:239)
Oct 30, 2018 7:30:26 AM com.orientechnologies.common.log.OLogManager log
INFO: Orient Engine is shutting down...

We also occasionally get a syntax error when trying to create a vertex that contains a String property. I believe the error is caused by the occurrence of a special character in the value of a String property. It would be nice if the list of special characters that need to be escaped was documented and also if the syntax error provided more information about the specific cause of the error.

Thomas Perry
Oncam

Make error on Debian for v.2.12.2

Created by: @alperensayar
Source: RedisGraph/RedisGraph#3182
after downloand the graph and extarct and insatll cmake make python3 and other dependis in folder write make command to get .so but i got this output

Building into /src

Using CC=cc

Makefile:216: /rules: No such file or directory
make: *** No rule to make target '/rules'. Stop.

How are strings compared in redisgraph?

Created by: @zhangliang-zl
Source: RedisGraph/RedisGraph#3189

My test case is as follows. When using entity attributes for comparison, it does not work properly

GRAPH.QUERY "asset" "MATCH  (n2 :NetworkSegment)  WHERE n2.ipRangeStart <= '000000000000000000000000000001684277521'  RETURN  count(n)" 
1) 1) "count(n2)"
2) 1) 1) "0"

GRAPH.QUERY "asset" "MATCH  (n2 :NetworkSegment)  WHERE '000000000000000000000000000003658716353' <= '000000000000000000000000000001684277521'  RETURN  n2.ipRangeStart" 
 "n2.ipRangeStart"

2) 1) 1) "000000000000000000000000000001684277770"
   2) 1) "000000000000000000000000000001684277761"
   3) 1) "000000000000000000000000000000016843027"
   4) 1) "000000000000000000000000000003232235777"
   5) 1) "000000000000000000000000000001684276800"
   6) 1) "000000000000000000000000000001684277258"
   7) 1) "000000000000000000000000000001684277523"
   8) 1) "000000000000000000000000000001684277760"
   9) 1) "000000000000000000000000000000151126795"
   10) 1) "000000000000000000000000000000151126805"
   11) 1) "000000000000000000000000000000923010159"
   12) 1) "000000000000000000000000000000016843009"
   13) 1) "000000000000000000000000000001684276068"
   14) 1) "000000000000000000000000000001684276481"
   15) 1) "000000000000000000000000000001684276800"
   16) 1) "000000000000000000000000000001684276993"
   17) 1) "000000000000000000000000000001684277256"
   18) 1) "000000000000000000000000000001684277258"
   19) 1) "000000000000000000000000000001684277313"
   20) 1) "000000000000000000000000000001684277312"
   21) 1) "000000000000000000000000000001684277523"
   22) 1) "000000000000000000000000000001684277768"
   23) 1) "000000000000000000000000000001684277761"
   24) 1) "000000000000000000000000000001684277760"
   25) 1) "000000000000000000000000000001684277761"
   26) 1) "000000000000000000000000000001684277770"
   27) 1) "000000000000000000000000000000016843009"
   28) 1) "000000000000000000000000000000791621423"
   29) 1) "000000000000000000000000000000175077917"
   30) 1) "000000000000000000000000000000173812096"
   31) 1) "000000000000000000000000000001902708481"
   32) 1) "000000000000000000000000000002000564224"
   33) 1) "000000000000000000000000000003074423393"
   34) 1) "000000000000000000000000000003683091104"
   35) 1) "000000000000000000000000000003735870976"
   36) 1) "000000000000000000000000000000992558608"
   37) 1) "000000000000000000000000000000175073934"
   38) 1) "000000000000000000000000000000167772160"
   39) 1) "000000000000000000000000000000992653562"
   40) 1) "000000000000000000000000000000989546753"
   41) 1) "000000000000000000000000000000989397729"
   42) 1) "000000000000000000000000000000989338112"
   43) 1) "000000000000000000000000000000989336648"
   44) 1) "047896470883454335353216816918185377792"
   45) 1) "000000000000000000000000000003708277760"
   46) 1) "000000000000000000000000000003721753184"
   47) 1) "000000000000000000000000000003701696256"
   48) 1) "000000000000000000000000000003683231496"
   49) 1) "000000000000000000000000000003683191616"
   50) 1) "000000000000000000000000000003683091106"
   51) 1) "000000000000000000000000000003658716353"
   52) 1) "000000000000000000000000000003524613825"
   53) 1) "000000000000000000000000000003395849985"
   54) 1) "000000000000000000000000000003395609956"
   55) 1) "000000000000000000000000000003232235520"
   56) 1) "000000000000000000000000000003070657490"
   57) 1) "000000000000000000000000000002566952001"
   58) 1) "000000000000000000000000000002096352513"
   59) 1) "000000000000000000000000000002030610688"
   60) 1) "000000000000000000000000000002032147296"
   61) 1) "000000000000000000000000000002031059426"
   62) 1) "000000000000000000000000000002028782017"
   63) 1) "000000000000000000000000000002028780164"
   64) 1) "000000000000000000000000000002028770361"
   65) 1) "000000000000000000000000000002028770331"
   66) 1) "000000000000000000000000000002000564225"
   67) 1) "000000000000000000000000000001962681984"
   68) 1) "000000000000000000000000000001902952178"
   69) 1) "000000000000000000000000000001902710112"
   70) 1) "000000000000000000000000000001902710178"
   71) 1) "000000000000000000000000000001866146752"
   72) 1) "000000000000000000000000000000173817600"
   73) 1) "000000000000000000000000000000173816832"
   74) 1) "000000000000000000000000000000175342976"
   75) 1) "000000000000000000000000000000175068892"
   76) 1) "000000000000000000000000000000175078347"
   77) 1) "000000000000000000000000000000175077898"
   78) 1) "000000000000000000000000000003524613824"
   79) 1) "000000000000000000000000000001032860000"
   80) 1) "000000000000000000000000000002566951937"
   81) 1) "000000000000000000000000000001780964865"
   82) 1) "000000000000000000000000000000175074020"
   83) 1) "000000000000000000000000000000175069185"
   84) 1) "000000000000000000000000000000173817344"
   85) 1) "000000000000000000000000000000175052241"
   86) 1) "000000000000000000000000000001866146752"
   87) 1) "000000000000000000000000000001962681984"
   88) 1) "000000000000000000000000000002030610688"
   89) 1) "000000000000000000000000000002028647905"
   90) 1) "000000000000000000000000000002028780164"
   91) 1) "000000000000000000000000000002030987009"
   92) 1) "000000000000000000000000000002032147296"
   93) 1) "000000000000000000000000000002886729728"
   94) 1) "000000000000000000000000000003395609602"
   95) 1) "000000000000000000000000000003395636481"
   96) 1) "000000000000000000000000000000974184449"
   97) 1) "000000000000000000000000000003524614272"
   98) 1) "000000000000000000000000000003683191552"
   99) 1) "000000000000000000000000000003683238646"
   100) 1) "000000000000000000000000000003701696256"
   101) 1) "000000000000000000000000000003708277760"
   102) 1) "000000000000000000000000000000989397729"
   103) 1) "000000000000000000000000000000989404385"
   104) 1) "000000000000000000000000000002028770387"
   105) 1) "000000000000000000000000000001902710202"
   106) 1) "000000000000000000000000000002028770360"
   107) 1) "000000000000000000000000000003683191618"
   108) 1) "000000000000000000000000000000989337968"
   109) 1) "000000000000000000000000000002095316224"

RG can store binary, but can't pretty_print() it !

Created by: @vsraptor
Source: RedisGraph/RedisGraph#445
In [12]: from bitarray import *

In [13]: b = bitarray('110101100001011010001')

In [27]: b.tobytes()
Out[27]: '\xd6\x16\x88'

In [28]: str(b.tobytes())
Out[28]: '\xd6\x16\x88'

In [29]: g = Graph('graph2', r)

In [30]: node1 = Node(label='node', properties={'abc': b.tobytes() })

In [31]: g.add_node(node1)

In [32]: g.commit()
Out[32]: <redisgraph.query_result.QueryResult at 0x7fb5401bcc50>

In [33]: q = g.query('match (n:node) return n')
In [35]: q.result_set
Out[35]: [['n.abc'], ['\xd6\x16\x88']]

In [34]: q.pretty_print()

UnicodeDecodeError Traceback (most recent call last)
in ()
----> 1 q.pretty_print()

/usr/local/lib/python2.7/dist-packages/redisgraph/query_result.pyc in pretty_print(self)
33 tbl.add_row(['No data returned.'])
34
---> 35 print(str(tbl) + '\n')
36
37 for stat in self.statistics:

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in str(self)
213 else:
214 def str(self):
--> 215 return self.unicode().encode(self.encoding)
216
217 def unicode(self):

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in unicode(self)
216
217 def unicode(self):
--> 218 return self.get_string()
219
220 ##############################

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in get_string(self, **kwargs)
1141
1142 # Turn all data in all rows into Unicode, formatted as desired
-> 1143 formatted_rows = self._format_rows(rows, options)
1144
1145 # Compute column widths

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in _format_rows(self, rows, options)
1095
1096 def _format_rows(self, rows, options):
-> 1097 return [self._format_row(row, options) for row in rows]
1098
1099 ##############################

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in _format_row(self, row, options)
1092
1093 def _format_row(self, row, options):
-> 1094 return [self._format_value(field, value) for (field, value) in zip(self._field_names, row)]
1095
1096 def _format_rows(self, rows, options):

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in _format_value(self, field, value)
998 elif isinstance(value, float) and field in self._float_format:
999 value = self._unicode(("%%%sf" % self._float_format[field]) % value)
-> 1000 return self._unicode(value)
1001
1002 def _compute_table_width(self, options):

/usr/local/lib/python2.7/dist-packages/prettytable/prettytable.pyc in unicode(self, value)
154 value = str(value)
155 if not isinstance(value, unicode
):
--> 156 value = unicode_(value, self.encoding, "strict")
157 return value
158

/usr/lib/python2.7/encodings/utf_8.pyc in decode(input, errors)
14
15 def decode(input, errors='strict'):
---> 16 return codecs.utf_8_decode(input, errors, True)
17
18 class IncrementalEncoder(codecs.IncrementalEncoder):

UnicodeDecodeError: 'utf8' codec can't decode byte 0xd6 in position 0: invalid continuation byte

Used memory for source and target databases are not the same

Created by: @K-Jo
Source: RedisGraph/RedisGraph#170
Steps:

  1. Use bulk insert to insert 200k nodes
    I ran:
    sudo -iEn python /root/modules/redisgraph/demo/bulk_insert/bulk_insert.py G -n edges.csv -h -p 11921
    The memory usage will be around 260mb
    If I ran it twice, the memory is around 600mb.
  2. Create target database.
    The memory consumption isn't the same.
    For source database - 600mb
    For target database - 500mb

Crash on cypher_astnode_instanceof

Title: Running a query causes the Docker container to exit abnormally

I execute a query on my RedisGraph and get an error.

Redis version:6.2.10
RedisGraph version: v2.10.9
operating system: windows 10
API:Cypher version 9
query:UNWIND NULL AS a0 WITH a0 ORDER BY NULL DESC WHERE false REMOVE (NULL).k RETURN *
error message:"Reached the max retries per request limit (which is 1). Refer to "maxRetriesPerRequest" option for details."

Actual behavior:
The query failed and the redisgraph container exited abnormally with the error message.

ref RedisGraph/RedisGraph#3191
@ljhhuxiaoba

7:M 22 Sep 2023 11:38:02.475 * Ready to accept connections tcp
redis-server: /FalkorDB/deps/libcypher-parser/lib/src/ast_identifier.c:78: cypher_ast_identifier_get_name: Assertion `cypher_astnode_instanceof(astnode, CYPHER_AST_IDENTIFIER)' failed.


=== REDIS BUG REPORT START: Cut & paste starting from here ===
7:M 22 Sep 2023 11:38:44.834 # Redis 7.2.0 crashed by signal: 6, si_code: -6
7:M 22 Sep 2023 11:38:44.834 # Crashed running the instruction at: 0x7fc8b4c99a7c

------ STACK TRACE ------
EIP:
/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7fc8b4c99a7c]

Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fc8b4c45520]
/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7fc8b4c99a7c]
/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7fc8b4c45476]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7fc8b4c2b7f3]
/lib/x86_64-linux-gnu/libc.so.6(+0x2871b)[0x7fc8b4c2b71b]
/lib/x86_64-linux-gnu/libc.so.6(+0x39e96)[0x7fc8b4c3ce96]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(+0x184d950)[0x7fc8b3a06950]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(+0x2378bb)[0x7fc8b23f08bb]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(AST_BuildReferenceMap+0x109)[0x7fc8b23f0a69]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(AST_NewSegment+0x18b)[0x7fc8b23ed33b]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(ExecutionPlan_FromTLS_AST+0xff)[0x7fc8b240b4ef]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(ExecutionCtx_FromQuery+0x13e)[0x7fc8b240008e]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(_query+0x67)[0x7fc8b23ff927]
/FalkorDB/bin/linux-x64-release/src/falkordb.so(+0x2a4b7f)[0x7fc8b245db7f]
/lib/x86_64-linux-gnu/libc.so.6(+0x94b43)[0x7fc8b4c97b43]
/lib/x86_64-linux-gnu/libc.so.6(+0x126a00)[0x7fc8b4d29a00]

Store date type as node property in redisgraph and use >= or <= operator on that property using WHERE clause

Created by: @mayankbharadwaj
Source: RedisGraph/RedisGraph#597
Hi ,

I am doing RnD on redisgraph to use it in my project. In that i need to store date as one attribute of the node and then run some query using WHERE clause on it.
For example i need to store something like below which i could do in neo4j using below cypher query.

Cypher query :
CREATE (b:Book {author:'o reilly', id:1, writtenDate: date('2017-08-02')})

MATCH (b:Book)
WHERE b.writtenDate = date("2017-08-02")
RETURN COUNT(b)

Currently redisgraph doesn't support temporal values like date . So Is there any way to store date as node property in redisgraph and use >= or <= operator on that property using WHERE clause.

Slow performance with large but simple graph

Created by: @jonrscott
Source: RedisGraph/RedisGraph#524
Hi again,

I have a graph with 14M nodes, 21M edges and now just 28 edge types. Running on an instance with 32GB RAM it occupies just over 50% of that. Every node is of the same type (n) has a single id property, which is indexed. Edges have no properties.

I am puzzled by the speed of some basic queries, so would love to understand if I'm doing something wrong, or how I can speed things up. I'd be very happy to be told I've made some simple fundamental misunderstanding :)

I've also tried the same queries on a differently-structured graph with only a single edge type, with similar results.

I'll list some examples below, along with the query plan from GRAPH.EXPLAIN:

Count total edges

127.0.0.1:6379> GRAPH.QUERY test "MATCH ()-[r]->() RETURN COUNT(r)"
1) 1) "COUNT(r)"
2) 1) 1) (integer) 21189569
3) 1) "Query internal execution time: 90482.647483 milliseconds"
(90.48s)
Query plan:
127.0.0.1:6379> GRAPH.EXPLAIN test "MATCH ()-[r]->() RETURN COUNT(r)"
1) "Results"
2) "    Aggregate"
3) "        Conditional Traverse"
4) "            All Node Scan"

Find number of nodes connecting 2 specified nodes (single hop each side)

127.0.0.1:6379> GRAPH.QUERY test "MATCH (:n{id:'test0'})-[]->(m)<-[]-(:n{id:'test1'}) RETURN COUNT(m)"
1) 1) "COUNT(m)"
2) 1) 1) (integer) 10
3) 1) "Query internal execution time: 2175.715580 milliseconds"
(2.18s)
Query plan:
127.0.0.1:6379> GRAPH.EXPLAIN test "MATCH (:n{id:'test0'})-[]->(m)<-[]-(:n{id:'test1'}) RETURN COUNT(m)"
1) "Results"
2) "    Aggregate"
3) "        Filter"
4) "            Conditional Traverse"
5) "                Conditional Traverse"
6) "                    Index Scan"

Find number of nodes connecting 2 specified nodes (up to 2 hops each side)

127.0.0.1:6379> GRAPH.QUERY test "MATCH (:n{id:'test0'})-[*1..2]->(m)<-[*1..2]-(:n{id:'test1'}) RETURN COUNT(m)"
1) 1) "COUNT(m)"
2) 1) 1) (integer) 138
3) 1) "Query internal execution time: 219439.508123 milliseconds"
(219.44s)
Query plan:
127.0.0.1:6379> GRAPH.EXPLAIN test "MATCH (:n{id:'test0'})-[*1..2]->(m)<-[*1..2]-(:n{id:'test1'}) RETURN COUNT(m)"
1) "Results"
2) "    Aggregate"
3) "        Conditional Traverse"
4) "            Filter"
5) "                Conditional Variable Length Traverse"
6) "                    Conditional Traverse"
7) "                        Conditional Variable Length Traverse"
8) "                            Index Scan"

Find number of nodes connecting 3 specified nodes (up to 3 hops each side)

127.0.0.1:6379> GRAPH.QUERY test "MATCH (:n{id:'test0'})-[*1..3]->(r)<-[*1..2]-(:n{id:'test1'}) RETURN COUNT(r)"
1) 1) "COUNT(r)"
2) 1) 1) (integer) 363
3) 1) "Query internal execution time: 895543.309874 milliseconds"
(895.54s)

(same query plan as previous)

High query latency (also for execution plan)

Created by: @jonrscott
Source: RedisGraph/RedisGraph#530
Hi again, sorry for the barrage of questions :)

I've been running a large bunch of the same type of query, along the lines of

MATCH (:n{id:'test0'})-[:R1|:R2|:R3*0..5]->(:n)-[:R1]->(m:n) WHERE (m.id = 'test1' OR m.id = 'test2') RETURN COUNT(m)

Every query of this form that I run (varying the numbers of conditions both in the relationship match and the WHERE clause) takes almost exactly the same time (around 0.84s) to execute. Interestingly, running GRAPH.EXPLAIN on the query takes the same execution time. The graph is the same one from my other recent issues (14M nodes, 21M edges over 28 types), running on a machine with 64GB RAM and plenty of CPU, with no other load on the box.

The query plan is as follows:

1) "Results"
2) "    Aggregate"
3) "        Filter"
4) "            Conditional Traverse"
5) "                Conditional Variable Length Traverse"
6) "                    Index Scan"
(0.84s)

I noticed something similar came up in #306 but there wasn't enough info for you to resolve it. If there's anything else useful I can provide, please let me know...

In addition, I created a smaller graph from a subset of the data, limited to 2M nodes and 770K edges, and this didn't show anything like the same latency (basically instantaneous to generate the execution plan, and about 50ms for the query).

Thanks again for all the help so far!

Full-text Search: query stemming in other languages

@rasteiner
ref: RedisGraph/RedisGraph#3105

As far as I know, both the corpus and the query need to be stemmed for a full-text search to work.
While I can specify a language for the index, which applies to the indexed corpus, I don't think I can specify a language for the query, and the embedded RediSearch therefore falls back to English stemming rules for all queries.

Example:

GRAPH.QUERY TEST "CREATE (:Test_it { txt: 'mela' })"
GRAPH.QUERY TEST "CALL db.idx.fulltext.createNodeIndex({ label: 'Test_it', language: 'italian'}, 'txt')"
GRAPH.QUERY TEST "CALL db.idx.fulltext.queryNodes('Test_it', 'mele')"
# nothing found, should have found "mela"
GRAPH.QUERY TEST "CALL db.idx.fulltext.queryNodes('Test_it', 'mels')"
# incorrectly found "mela"

According to snowballstem.org: the Italian stemmer transforms both "mela" (apple), as well as "mele" (apples) into "mel", while "mels" stays "mels".
It looks like the query (passed into db.idx.fulltext.queryNodes) is always stemmed in English, since the English stemmer transforms "mels" (which doesn't mean anything in Italian) into "mel".

This seems to make the stemming functionality rather unusable in other languages.
Sorry if I'm missing something obvious, I haven't found anything in the docs.

Additional context

This behaviour actually mirrors that of RediSearch, but there I can specify the language at query time to get stemming to work:

FT.CREATE idx:Test_it ON HASH PREFIX 1 "Test_it:" LANGUAGE italian SCHEMA txt TEXT 
HSET Test_it:1 txt "Mela"

FT.SEARCH idx:Test_it "mele" RETURN 1 txt
# incorrectly finds nothing

FT.SEARCH idx:Test_it "mels" RETURN 1 txt
# incorrectly finds "mela"

FT.SEARCH idx:Test_it "mele" LANGUAGE italian RETURN 1 txt
# correctly finds "mela"

FT.SEARCH idx:Test_it "mels" LANGUAGE italian RETURN 1 txt 
# correctly finds nothing

Version info

Redis: 6.2.7
RediSearch: 2.4.14
RedisGraph: 2.8.19

GraphQL - Yoga - Apollo - Schema Augmentation - Cypher Autogenerator (can we make it work?)

Created by: @styk-tv
Source: RedisGraph/RedisGraph#347
Has anyone tried using RedisGraph with Apollo/Yoga and Neo4j-graphql-js npm module? Idea here is to take advantage of awesomeness of neo4j-graphq-js module and its ability to autogenerate of queries and mutations. Basically: schema = automatic resolvers (or the ones you want only) = automatic cypher from graphql.

https://grandstack.io/docs/neo4j-graphql-js.html (note: do not get distracted by neo4j, the idea is to come back here and make that component work with RedisGraph)

Thinking about Vector Search

Just to add another possibility to FalkorDB...

I just watched this video:
Workshop: Searching document data with Redis, JSON, and vector-similarity

The second half about Vector Search in Redis was, for me, the clearest explanation of how to use Redis for semantic searching on text (my main use-case). Maybe this is already possible with RedisGraph (except for the lack of support for Nested JSON). So, if FalkorDb could support using the graph nodes for Vector Search that would be fantastic. I guess if not, farming the relevant node text into a RedisJson store could also work?

Anyway, just some thoughts.

Thanks,
Murray

Support for API Bulk Loading

Created by: @mboldisc
Source: RedisGraph/RedisGraph#476
I've been trying to load hundreds of thousands of vertices into RedisGraph using Redis Python APIs.

Each vertex is loaded as follows:
MERGE (n:MyNode{id:'12345'})

Using the basic Redis Python client and asyncio client I see around 200 inserts per second. I also tried pipelining and it had the same performance. Redis consumes one entire CPU during bulk load. It appears to be tapped on the single-core Redis CPU. Redis doesn't support multi-process (out of the box).

I assume there's a reason why the CSV bulk loader script was written using binary data. I'd prefer a solution that doesn't involve CSV records for bulk loading data. Any recommendations? My first thought is to convert the bulk loading script into a simple API. I'd be willing to help if others think this is useful. Other thoughts?

Nested JSON on nodes?

Hi.
I intend to implement FalkorDB as a replacement for RedisGraph as soon as I get through my current backlog! ;-)

I know RedisGraph doesn't support nested json objects on graph nodes because Cypher doesn't support it (as I understand).

I am in the process of working up a way to use RedisJson (which of course does support nested json) in conjunction with RedisGraph / FalkorDB. I am just wondering if nested json objects on graph nodes is something you are considering. I presume not (in order to maintain the Cypher standard) but just thought I would ask. And maybe my assumptions are not correct. ;-)

Thanks for your work.
Murray

Inconsistent query behaviour for fixed length variable length path queries

Created by: @K-Jo
Source: RedisGraph/RedisGraph#321
TL;DR: RedisGraph returns DISTINCT results for fixed length variable length path queries with a single starting point, without mentioning the DISTINCT keyword.

Given

CREATE (n0:A {name: 'n0'}),
       (n00:B {name: 'n00'}),
       (n01:B {name: 'n01'}),
       (n000:C {name: 'n000'}),
       (n0)-[:LIKES]->(n00),
       (n0)-[:LIKES]->(n01),
       (n00)-[:LIKES]->(n000),
       (n01)-[:LIKES]->(n000)

The following query

MATCH (a:A)-[:LIKES*2]->(c) WHERE a.name='n0' RETURN c

Should return twice the node n000. However RedisGraph returns n000 only once

redis:6379> graph.query test "MATCH (a:A)-[:LIKES*2]->(c) WHERE a.name='n0' RETURN c"
1) 1) 1) "c.name"
   2) 1) "n000"

This bug can be explained via the implementation of RedisGraph. The query will execute a conditional traverse for each node with the label A. For each node, RedisGraph will create a Hyper Sparse Vector with a single element and multiply it twice by the adjacency matrix of relationships of type LIKES. The resulting vector will have a Non Zero Value (NZV) at node n000, however it did not keep track of the actual paths that lead to it.

In such, the result of this query is incorrect and behaves as

MATCH (a:A)-[:LIKES*2]->(c) WHERE a.name='n0' RETURN DISTINCT c

Note 1: The behaviour is correct for (non fixed) variable length path queries. e.g.

MATCH (a:A)-[:LIKES*2..3]->(c) WHERE a.name='n0' RETURN c

returns correctly n000 twice

redis:6379> graph.query test "MATCH (a:A)-[:LIKES*2..3]->(c) WHERE a.name='n0' RETURN c"
1) 1) 1) "c.name"
   2) 1) "n000"
   3) 1) "n000"

Note 2: For more than 1 starting point, a fixed length variable length path query returns non distinct values if it the destination node could be reached from different starting points.

MATCH (b:B)-[:LIKES*1]->(c)  RETURN c

This query should return n000 twice since there are two start nodes with the label B.

redis:6379> graph.query test "MATCH (b:B)-[:LIKES*1]->(c)  RETURN c"
1) 1) 1) "c.name"
   2) 1) "n000"
   3) 1) "n000"

How to provide NaN and inf values in CLI?

Created by: @GnaneshKunal
Source: RedisGraph/RedisGraph#601
I have used this dataset: RedisGraph/RedisGraph#570 (comment)

The first row has NaN and the following snippet from bulk_insert.py converts it to NaN in binary representation and stores it.

https://github.com/RedisGraph/redisgraph-bulk-loader/blob/40de2c589516f18ac1fa30825821f3b90ecc93ae/bulk_insert.py#L289-L293

How should it be done in CLI interface with the normal GRAPH.QUERY?

Can you give an example with a small node that has a field whose value is a NaN or inf using either GRAPH.QUERY or GRAPH.BULK in the cli interface?

Error: Encountered unhandled type in inlined properties

ref: RedisGraph/RedisGraph#3113

I execute a query on my empty RedisGraph and get an error.

Redis version:6.2.10
RedisGraph version: v2.10.10
operating system: windows 10
API:Cypher version 9
query: match ()-[{k:size([(a)-[{k:1}]-()|a.k])}]-() return 1

error message:
"Encountered unhandled type in inlined properties."

Steps to reproduce:
match ()-[{k:size([(a)-[{k:1}]-()|a.k])}]-() return 1

Expected behavior:
The query should run successfully.

Actual behavior:
The query failed.

@ljhhuxiaoba

Incorrect results triggered by query partitioning

Version: Docker image falkordb/falkordb:edge
OS: Ubuntu 22.04
API/Driver: Cypher

According to the following query, the 18 should equals to 10+12

lab:6380> GRAPH.QUERY 0 "MATCH (n1:L3)-[]-(n2:L2)-[]-(n3:L1:L2)-[r1:T1]-(n4:L3:L0:L4) RETURN COUNT(*)"
1) 1) "COUNT(*)"
2) 1) 1) (integer) 18
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 1.820675 milliseconds"
lab:6380> GRAPH.QUERY 0 "MATCH (n1:L3)<-[]-(n2:L2)-[]-(n3:L1:L2)-[r1:T1]-(n4:L3:L0:L4) RETURN COUNT(*)"
1) 1) "COUNT(*)"
2) 1) 1) (integer) 10
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 0.738668 milliseconds"
lab:6380> GRAPH.QUERY 0 "MATCH (n1:L3)-[]->(n2:L2)-[]-(n3:L1:L2)-[r1:T1]-(n4:L3:L0:L4) RETURN COUNT(*)"
1) 1) "COUNT(*)"
2) 1) 1) (integer) 12
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 0.759969 milliseconds"
lab:6380> GRAPH.QUERY 0 "MATCH (n)-[r]-(n) RETURN n,r LIMIT 10"
1) 1) "n"
   2) "r"
2) (empty array)
3) 1) "Cached execution: 0"
   2) "Query internal execution time: 0.584519 milliseconds"

The create statement is attached below for your reproduction
1692914926.9295375.log

[Feature] Bulk Loader?

Any plans on either utilizing or building a new bulk loader feature? That's the one thing that would stop us from moving across to FalkorDB anytime soon.

Update label failure

CREATE (:A)
UNWIND range(0, 1000000) as x CREATE (:B)
CREATE (:C)
MATCH (c:C) SET c:A
MATCH (n:A) RETURN count(n)

Expecting last query to return 2

The issue is in graph_hub.c UpdateNodeLabels

 RG_Matrix m = Graph_GetLabelMatrix(gc->g, schema_id);

won't resize the matrix as the graph's matrix sync policy at this stage is NOP

How to create a Graph from a stream of messages(say Kafka) ?

Created by: @kant111
Source: RedisGraph/RedisGraph#416
How to create a Graph from a stream of messages(say Kafka)?

I have a continuous stream of incoming messages into Kafka and I need to form a graph out of it. what is the best way to do it?

I am new to Cypher but one solution I can think of is something like this

while (true) {
    batch_of_records = consumer.poll("100 milliseconds")
    if batch_of_records is not empty {
        vertices = []
        for each_record in batch_of_records {
             a_vertex = (:User {email: each_record.email, userId: each_record.id })
             vertices.append(a_vertex)
        }
       batch_of_vertices = ",".join(vertices)

       CREATE batch_of_vertices // just dump vertices into redis graph

       // create relationships among the existing vertices
       MATCH (a:User {name: "Jack", surname: "Roe"}), 
                     (b:User {name: "Jack", surname: "Smith"})
       CREATE (a) -[r:Knows]-> (b)
   }
}

Essentially the algorithm here is for each batch of records that I am continuously polling from Kafka I create a bunch of vertices using CREATE and then create relationships using MATCH and CREATE

Running a query causes the Docker container to exit abnormally

This issue was imported from the original issue: RedisGraph/RedisGraph#3191
Created by: @ljhhuxiaoba

I execute a query on my empty RedisGraph and get an error.

Redis version:7.2.0
RedisGraph version: v2.12.9
operating system: windows 10
API:Cypher version 9
query:UNWIND NULL AS a0 WITH a0 ORDER BY NULL DESC WHERE false REMOVE (NULL).k RETURN *
error message:"Reached the max retries per request limit (which is 1). Refer to "maxRetriesPerRequest" option for details."

Actual behavior:
The query failed and the redisgraph container exited abnormally with the error message.

Unable to scale read query on redisgraph

Created by: @vvippark
Source: RedisGraph/RedisGraph#389
I am running below query
MATCH (me:Person)-[r:friends]->(he:Person) WHERE ( me.id="X" OR me.id="Y" OR me.id="Z" ) AND
(he.city="Berlin" AND he.status="Married" AND (he.id!="X" )) RETURN he LIMIT 40

I have logged the timing of redis graph queries

benchmarking via h2load
h2load -n 5 -c 1 "https://testing.com/test1"

Here regis graph is taking around 90ms.

h2load -n 5000 -c 1000 "https://testing.com/test1"
Here request/sec is around 4
when doing concurrent request of 1000 same query is taking more than 60secs

But if I execute a normal api without redis graph query
h2load -n 5000 -c 1000 "https://testing.com/test2"
Its request/sec is round 1200.

Export / Import of edges

Created by: @pbakiewicz
Source: RedisGraph/RedisGraph#585
My question is twofold:

  1. Can I make a backup of only edges from my Graph? Or even better can I specify which edges (i.e. of a certain type) I want to dump?

  2. Is there a way to import edges to my Graph. Let's say I have a Graph only with nodes - is there a way to import to that graph edges from a back_up file?

Thanks in advance guys!

Crash encountered with more than one node in the graph

This issue was imported from the original issue: RedisGraph/RedisGraph#3193
Created by: @ljhhuxiaoba

I execute a query on my RedisGraph and get a crash.

Redis version:7.2.0
RedisGraph version: v2.12.9
operating system: windows 10
API:Cypher version 9
query:CREATE (c0) DELETE c0 WITH * MATCH (),() RETURN 1
error message:"Reached the max retries per request limit (which is 1). Refer to "maxRetriesPerRequest" option for details."

To reproduce:
CREATE (),();
CREATE (c0) DELETE c0 WITH * MATCH (),() RETURN 1;

It's worth noting that the database works fine when there is only one node in the graph, but crashes when there are two nodes in the graph.

Actual behavior:
The query failed and the redisgraph container exited abnormally with the error message.

Popoto as UI. Easy.

Created by: @styk-tv
Source: RedisGraph/RedisGraph#479
Not sure if you've met Popoto? JS, NPM-based UI that talks Cypher. Would be good as currently missing bit of UI for browsing graphs. It is currently binding to neo4j only, however since it generates Cypher as result of UI operations it would be interesting to feed that as input and to RedisGraph.

popoto

Anyone wants to explore further? Would be a good addition to Backlog/Integrations

help me create scalable architecture for redis graph?

Created by: @vvippark
Source: RedisGraph/RedisGraph#430
i have redis graph on my production it is running on kubernetes(1 Master and Two Salve). but i am unable to scale redisgraph read query.i am running 'or' query to fetching users friends.
but when i hitting and trying to benchmarking it, i am unable to scale it to 1000 req/sec,
recently it 4 req/sec. I am unable to find out the actual infra (cluster) where can i scale it or unable to find out setting where can i tune my redisgraph to scale 1000 req/sec.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.