Comments (2)
Hi,
5 Gigs of data is quite a bit. I would, in any case, add it in multiple smaller chunks. I am assuming that the data is stored in one or multiple files on disk.
I can think of 3 different ways on how the data could be added:
- Adding it through the command line using the -import argument. I would imagine a script which creates import files and then runs EliasDB's import.
- You could use the REST API. However, for this amount of data it is probably too slow and just the wrong mechanism.
- You could write a Go program which embeds EliasDB. It could read the JSON data in chunks and use transactions to feed it into the datastore. (I think this is the best alternative.)
In general: the power of a graph database lies in its ability to store highly connected data. It sounds as if your data uses primarily keys which might be better stored in a normal key-value store. However, if you do want to store it as a graph (e.g the advancement of the blockchain could be modelled as edges) you would need to form small objects of key value pairs. In EliasDB the keys must be strings while the values can be anything. The value of the "key" and "kind" attribute should also be a string. (It is currently a bug that the "key" attribute value must be a string).
Node values can contain any (and all) JSON data. If you embed EliasDB in an own Go program you can even store slices and object structures. In fact anything which can be serialized with gob.
from eliasdb.
aha!
Okay that was a really huge tip right there about storing slices and object
structres in embedded elias. That's actually what had drawn me towards
it. Of couse I guess I could be getting led back to a non-graph-database
by my next point, because... I suppose if I did something like:
blocknum:returnfromcurl that wouldn't really get me anywhere in terms of it
being a graph database, really.
I'd need to break it down into more granular units like:
blocknum contains userid
blocknum contains trxid
trxid contains operations {
one
two
three
}
or....?
But that's really, really good to know about golang embedded DBs. I'm
definitley a noob when it comes to go.
Jacob Gadikian
E-mail: [email protected]
SKYPE: faddat
Phone/SMS: +84 167 789 6421
On Mon, Oct 3, 2016 at 7:47 PM, Matthias Ladkau [email protected]
wrote:
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#6 (comment), or mute the
thread
https://github.com/notifications/unsubscribe-auth/AGz6iWLss3202ShJx-jdiqhA4dScbM5Pks5qwPl7gaJpZM4KLsKn
.
from eliasdb.
Related Issues (20)
- Race condition in lockfile.go:219 after DiskGraphStorage.Close() HOT 2
- Node deletion while iterating HOT 1
- Test error HOT 1
- Error compiling HOT 1
- Eliasdb status HOT 2
- tutorial demo - "Get Line fails on Term (browser) but not in Terminal CLI HOT 4
- TLS certs and browser trust idea. HOT 2
- Web IDE HOT 1
- Performance question HOT 3
- Thoughts about OSX support & "common" dependencies HOT 9
- Crashing the server while checking out the tutorial HOT 5
- `go mod tidy` error while using embed EliasDB HOT 1
- Question: What visualization tool can integrate with eliasdb? HOT 1
- How well does Eliasdb scale? HOT 2
- Query not returning expected result HOT 2
- Better GraphQL Integration HOT 5
- Wow HOT 2
- Scalability in p2p network HOT 3
- Could not take ownership of lockfile HOT 1
- Encryption ? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eliasdb.