pixelspark / catena Goto Github PK
View Code? Open in Web Editor NEWCatena is a distributed database based on a blockchain, accessible using SQL.
License: MIT License
Catena is a distributed database based on a blockchain, accessible using SQL.
License: MIT License
Post to other peers so they don't have to be mined at the peer that originated them. Needs a check to find out whether a transaction is already on the chain or not.
Only those transactions that are not on the new chain, obviously.
* thread #1, name = 'catena', stop reason = signal SIGABRT
* frame #0: 0x00007ffff4859428 libc.so.6`gsignal + 56
frame #1: 0x00007ffff485b02a libc.so.6`abort + 362
frame #2: 0x00007ffff489b7ea libc.so.6`___lldb_unnamed_symbol220$$libc.so.6 + 730
frame #3: 0x00007ffff48a437a libc.so.6`___lldb_unnamed_symbol277$$libc.so.6 + 1578
frame #4: 0x00007ffff48a853c libc.so.6`cfree + 76
frame #5: 0x00007ffff6ae0cf8 libswiftCore.so`destroyArray value witness for Swift.String + 40
frame #6: 0x00007ffff6969fb9 libswiftCore.so`Swift._ContiguousArrayStorage.__deallocating_deinit + 89
frame #7: 0x00007ffff696dac4 libswiftCore.so`Swift.String.init (stringInterpolation : Swift.Array<Swift.String>...) -> Swift.String + 36
frame #8: 0x00000000005fb78e catena`HeliumLogger.formatEntry(type=info, msg="[Node] Re-broadcasting transaction apRxbg3QQkQo+CIYtDw+gQwk/mNwSmxmBrtIbwKmEs=@1 to peers as it is new", functionName="receive(transaction:from:)", lineNum=153, fileName="/root/Sources/CatenaCore/Node.swift", self=<unavailable>) -> String at HeliumLogger.swift:278
frame #9: 0x00000000005fab7f catena`HeliumLogger.log(type=<unavailable>, msg=Swift.String @ 0x00007fffffffb2c0, functionName=Swift.String @ 0x00007fffffffb2a8, lineNum=<unavailable>, fileName=Swift.String @ 0x00007fffffffb288, self=<unavailable>) -> () at HeliumLogger.swift:239
frame #10: 0x00000000005fbfa6 catena`protocol witness for Logger.log(LoggerMessageType, msg : String, functionName : String, lineNum : Int, fileName : String) -> () in conformance HeliumLogger at HeliumLogger.swift:0
frame #11: 0x0000000000433771 catena`static Log.info(msg=<unavailable>, functionName=Swift.String @ 0x00007fffffffb570, lineNum=<unavailable>, fileName=Swift.String @ 0x00007fffffffb550, self=<unavailable>) -> String, functionName : String, lineNum : Int, fileName : String) -> () at Logger.swift:130
frame #12: 0x0000000000697adf catena`Node.receive(transaction=<unavailable>, peer=<unavailable>, self=<unavailable>, $error=<unavailable>) throws -> () at Node.swift:153
frame #13: 0x000000000042d1be catena`main at main.swift:224
frame #14: 0x00007ffff4844830 libc.so.6`__libc_start_main + 240
frame #15: 0x00000000004254c9 catena`_start + 41
The reason Catena currently doesn't compile on Linux is the WebSocket client library used (Starscream) which relies on CFNetwork. This should be replaced with something that uses (for instance) BlueSocket.
Perhaps with a triggers
table that has on
, condition
(see also #11) and statement
columns (and perhaps also a way to run the trigger as the invoker or as the definer).
Syntax would be 'CREATE TRIGGER' still.
As is also done by Ethereum (see https://github.com/ethereum/wiki/wiki/enode-url-format).
Peers without username can only be passive peers and may not appear in advertised peer lists.
A node should print its node ID and URL (as far as it knows its own IP) when starting.
Support at least:
--init-only should create the peer database and seed the chain, but nothing else. (Useful for Docker images)
Enforce transaction count and block size limits in Miner.
Needs a proper Linux WebSocket Client (Starscream relies on CFNetwork and that is not available on Linux)
Instead of PoW, allow a select set of keys to sign blocks. The genesis block decides on the mechanism used for signing blocks.
The list of keys that are allowed to sign blocks should be stored in a special table.
See also ethereum/EIPs#225
Keep a 'grants' table in the database that validates mutations.
Add a command line switch to disable grant validation for databases that do not need it.
Currently, a node can choose whether to 'replay' SQL transactions on top of the database engine, and select only those tables that it is interested in. The query history however needs to be stored by all nodes for all tables (nodes could throw away blocks after a certain amount of time has passed, but that would make it impossible for new nodes to download the full blockchain from block 0).
It would be preferable if SQL transactions could be split into 'side chains', where each node would only 'follow' the side chains of its interest (and discard the rest).
Starting a sidechain would be akin to creating a new database. The root chain may also handle payments and dropping of sidechains (either upon request, or when a certain time has exceeded).
Nodes mining a sidechain block should perhaps be required to 'prove' they also still have all history blocks (perhaps by requiring a deterministically randomly chosen history block to be included in the signature hash?).
Things to figure out:
Like what Bitcoin does: https://en.bitcoin.it/wiki/Block_timestamp
Expose to queries as $approximateTime.
Add a column to the grant table that (optionally) contains an expression that needs to evaluate to true in order for the grant to be available. The user
column also still exists, but may be set to NULL to always match:
Examples:
insert
, user=NULL, condition=$invoker=X'somehash'
,table=sometable
: equivalent to user=someHashcreate
, user=NULL, condition=$invoker=$miner
, table=NULL: allows miners to create a tableinsert
, user=x, condition=$timestamp < y
, table=z: allows inserts to a table only before a certain date/time (approximate)When index << current index and/or timestamp << current time
Block nonce and height are currently defined as UInt64, but can JSON carry this? If so, it appears the Linux builds fail to serialize/deserialize.
See e.g.
Line 85 in cca41f3
SELECT 'foo' & 'bar';
=> 'foobar'
Add a 'users' table that maintains a counter, incremented for each transaction invoked by a public key. Transactions are only allowed when their count = previous_count + 1
Appears to be an issue with deserializing JSON (which seems to be okay...). Might be an issue with my setup (Ubuntu 16.04 with Swift 3.1).
Options to investigate:
Persist a peer database
When counter > current counter (but below a certain threshold), remember the transaction in the hope that the transactions in between will be fulfilled shortly.
Do not execute queries except those required for validation (e.g. process only grants).
What Bitcoin does against sybil attacks:
Bitcoin makes these attacks more difficult by only making an outbound connection to one IP address per /16 (x.y.0.0). Incoming connections are unlimited and unregulated, but this is generally only a problem in the anonymity case, where you're probably already unable to accept incoming connections.
It should be possible to disable this for private/testing deployments.
Using Bonjour?
Make seed.catena.pixelspark.nl an A record that points to (non-replaying) catena nodes.
Do not advertise peers discovered through DNS seeds to other peers.
Add reply type to 'query' request indicating the the peer does not maintain an index ("passive" action type). Change peer state to .passive and do not query anymore (but do send updates for blocks/txes).
Do not maintain a 'blocks' array in Ledger; instead, use the persisted storage (now SQLHistory) to find blocks.
Do not (tryo to) connect to all peers ever received, but select based on last seen and latency.
Block should be <10MB, transaction <100kB?
If a node receives a block from a peer that it hasn't seen before and it can append it to its chain, it should forward it to all other peers (except to the peer that sent it).
catena/Sources/CatenaSQL/SQLAPIEndpoint.swift
Line 132 in f3c0c86
These lead to errors on Linux - serialize as NSNumber(value: x)
instead.
In the grants table, add an expression
column that optionally contains an expression that restricts any action performed using the grant, so you can:
The expression should be able to contain special variables to check certain things, e.g.:
$invoker
: the public key of the user invoking the transaction. A grant with expression "$invoker=owner
" would only allow modifications to rows that have the public key of the invoker in the 'owner' column. This could be made more complex, e.g. (CASE WHEN $owner=x THEN ... WHEN $owner=y THEN ... ELSE 0 END
).$index
: the index of the block this transaction is part of (useful for allowing transactions only after a certain amount of time)Database should contain a table that holds feature flags (e.g. which signing/key algorithms are used). The table should be created in block 1.
Some ideas: have a $miner variable that contains the public address of a miner, then use that to send extra quota (through grants) using a query stored in 'flags' executed as first query for each block.
Some ideas:
Per-table (or per-row?) expiry date that can be 'topped up' using credits (for which the coins are generated by mining)
Evicting expired CREATE/INSERT/UPDATE queries from the chain completely (requires merkle trees)
Currency system for database privileges, e.g. grants with a quotum on the maximum number of times they are used, and/or the number of bytes stored, etc.
After a certain amount of non-response, reset the peer state to .new and close connections.
SELECT 'escaped '' string';
==> "escaped ' string"
Currently, difficulty is fixed at a low value. This could be managed using a 'flags' table that holds the difficulty and/or contains an expression to calculate the difficulty (based on $index, $averageBlockTime, or something like that).
See also #32
Strace shows:
setsockopt(15, SOL_SOCKET, SO_REUSEPORT, [1], 4) = -1 ENOPROTOOPT (Protocol not available)
Solution is probably to not use SO_REUSEPORT.
This allows to implement encryption and authentication between nodes in the future.
precondition failed: host name 43ff16893f97. should have suffix .local: file /root/.build/checkouts/NetService.git--4183857386241951162/Sources/NetService/Responder.swift, line 6 2
Current stack trace:
0 libswiftCore.so 0x00007f54507c66b0 swift_reportError + 120
1 libswiftCore.so 0x00007f54507e0fc0 _swift_stdlib_reportFatalErrorInFile + 100
2 libswiftCore.so 0x00007f54505db4dc <unavailable> + 1189084
3 libswiftCore.so 0x00007f545077386d <unavailable> + 2861165
4 libswiftCore.so 0x00007f54505dacb6 <unavailable> + 1186998
5 libswiftCore.so 0x00007f5450779780 <unavailable> + 2885504
6 libswiftCore.so 0x00007f54505db0ef <unavailable> + 1188079
7 libswiftCore.so 0x00007f545073a499 <unavailable> + 2626713
8 libswiftCore.so 0x00007f54505dacb6 <unavailable> + 1186998
9 libswiftCore.so 0x00007f54506f7610 specialized _assertionFailure(StaticString, String, file : StaticString, line : UInt, flags : UInt32) -> Never + 144
10 Catena 0x0000000000641a66 <unavailable> + 2366054
11 Catena 0x0000000000641087 <unavailable> + 2363527
12 Catena 0x0000000000640f85 <unavailable> + 2363269
13 Catena 0x00000000006589e0 <unavailable> + 2460128
14 Catena 0x00000000006ab204 <unavailable> + 2798084
15 Catena 0x0000000000696db8 <unavailable> + 2715064
16 Catena 0x0000000000696d0c <unavailable> + 2714892
17 Catena 0x0000000000696e69 <unavailable> + 2715241
18 Catena 0x000000000042b022 <unavailable> + 176162
19 libc.so.6 0x00007f544e4ce740 __libc_start_main + 240
20 Catena 0x00000000004254c9 <unavailable> + 152777
Specifically detail size of fields, endianness, etc.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.