chdb-io / chdb-server Goto Github PK
View Code? Open in Web Editor NEWStandalone chdb server by chdb-go
License: Apache License 2.0
chdb-server's People
Contributors
Stargazers
Watchers
chdb-server's Issues
Implement Statistics
Implement support for query statistics returning processed rows and bytes available in chdb 0.13.0 and higher
Repository Transfer
This community repository contains my own work and was hosted on my personal github and not part of chdb or its codebase. I was convinced to host this (and other) personal repositories with chdb integrations under the chdb-io organization with a written warranty I could fully manage them, and sadly this is no longer the case as of today.
@auxten @alexey-milovidov for fairness and since I never signed any CLA for this code/repo for which I am the principal author and maintainer alongside my friends and colleagues as anyone can see, I am formally and kindly asking you here in public to migrate this repo back to the rightful owner (my username) as soon as possible, so I can continue my (blocked) development for the community and for other projects using this resource.
Note this project actions are linked to external resources I personally own and still contains my private access tokens.
Feel free by all means to fork it and contribute if useful for chdb, although I'm sure you will not need this in the future.
Building docker image on arm fails
Presumably because its trying to strip a file that doesn't exist.
https://github.com/chdb-io/chdb-server/blob/main/Dockerfile#L5
The strip needs to be more arch relaxed
PR inbound
400 response on sending multiline JSONEachRow requests
- Create a db:
cat <<EOF | curl -X POST http://a:b@localhost:8123/ --data-binary @-
CREATE DATABASE a;
EOF
- create a table
cat <<EOF | curl -X POST http://a:b@localhost:8123/?database=a --data-binary @-
CREATE TABLE IF NOT EXISTS time_series (date Date,fingerprint UInt64,labels String, name String)
ENGINE = ReplacingMergeTree(date) PARTITION BY date ORDER BY fingerprint;
EOF
- send one line of data into the table:
cat <<EOF | curl -X POST 'http://a:b@localhost:8123/?query=INSERT%20INTO%20a.time_series(date,%20fingerprint,%20labels,%20name)%20FORMAT%20JSONEachRow' --data-binary @-
{"fingerprint":"15239659101592556481","timestamp_ns":"1699265700000000000","value":null,"string":"FREQ_TEST_0"}
EOF
- send several lines of data into the table
cat <<EOF | curl -v -X POST 'http://a:b@localhost:8123/?query=INSERT%20INTO%20a.time_series(date,%20fingerprint,%20labels,%20name)%20FORMAT%20JSONEachRow' --data-binary @-
{"fingerprint":"15239659101592556481","timestamp_ns":"1699265700000000000","value":null,"string":"FREQ_TEST_0"}
{"fingerprint":"15239659101592556481","timestamp_ns":"1699265700000000000","value":null,"string":"FREQ_TEST_0"}
EOF
AR: p.4 doesn't work with error 400
ER: p.4 should work same as p.3
request Content-Type suppression
$ curl -vv -X POST http://a:b@localhost:8123 --data 'CREATE DATABASE test';
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8123...
* Connected to localhost (127.0.0.1) port 8123 (#0)
* Server auth using Basic with user 'a'
> POST / HTTP/1.1
> Host: localhost:8123
> Authorization: Basic YTpi
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 20
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Thu, 02 Nov 2023 21:23:48 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 2
< Connection: close
<
* Closing connection 0
Ok
$ curl -vv -X POST http://a:b@localhost:8123 --data 'SHOW DATABASES';
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8123...
* Connected to localhost (127.0.0.1) port 8123 (#0)
* Server auth using Basic with user 'a'
> POST / HTTP/1.1
> Host: localhost:8123
> Authorization: Basic YTpi
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 14
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Thu, 02 Nov 2023 21:24:21 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 2
< Connection: close
<
* Closing connection 0
The database doesn't get created because of Content-Type: application/x-www-form-urlencoded in the request.
On the other hand clickhouse server doesn't rely on content-type.
In order to suppress Content-Type header it's enough to use request.get_data() instead of request.data as described in https://stackoverflow.com/questions/55847242/flask-flask-restful-force-correct-override-incorrect-content-type-header-for-po
inserting data into a table
$ curl -vv -X POST http://a:b@localhost:8123 --data 'CREATE DATABASE test';
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8123...
* Connected to localhost (127.0.0.1) port 8123 (#0)
* Server auth using Basic with user 'a'
> POST / HTTP/1.1
> Host: localhost:8123
> Authorization: Basic YTpi
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 20
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Thu, 02 Nov 2023 21:30:46 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Connection: close
<
* Closing connection 0
$ curl -vv -X POST http://a:b@localhost:8123 --data 'CREATE TABLE test.a(a Int8)Engine=MergeTree ORDER BY a';
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8123...
* Connected to localhost (127.0.0.1) port 8123 (#0)
* Server auth using Basic with user 'a'
> POST / HTTP/1.1
> Host: localhost:8123
> Authorization: Basic YTpi
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 54
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Thu, 02 Nov 2023 21:31:45 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Connection: close
<
* Closing connection 0
$ curl -vv -X POST 'http://a:b@localhost:8123?query=INSERT%20INTO%20test.a%20FORMAT%20JSONEachRow' --data '{"a":1}'
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8123...
* Connected to localhost (127.0.0.1) port 8123 (#0)
* Server auth using Basic with user 'a'
> POST /?query=INSERT%20INTO%20test.a%20FORMAT%20JSONEachRow HTTP/1.1
> Host: localhost:8123
> Authorization: Basic YTpi
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Thu, 02 Nov 2023 21:33:07 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 2275
< Connection: close
<
Code: 62. DB::Exception: Syntax error: failed at position 1 ('{'): {"a":1}. Expected one of: Query, Query with output, EXPLAIN, SELECT query, possibly with UNION, list of union elements, SELECT query, subquery, possibly with UNION, SELECT subquery, SELECT query, WITH, FROM, SELECT, SHOW CREATE QUOTA query, SHOW CREATE, SHOW [FULL] [TEMPORARY] TABLES|DATABASES|CLUSTERS|CLUSTER 'name' [[NOT] [I]LIKE 'str'] [LIMIT expr], SHOW, SHOW COLUMNS query, SHOW ENGINES query, SHOW ENGINES, SHOW INDEXES query, EXISTS or SHOW CREATE query, EXISTS, DESCRIBE FILESYSTEM CACHE query, DESCRIBE, DESC, DESCRIBE query, SHOW PROCESSLIST query, SHOW PROCESSLIST, CREATE TABLE or ATTACH TABLE query, CREATE, ATTACH, REPLACE, CREATE DATABASE query, CREATE VIEW query, CREATE DICTIONARY, CREATE LIVE VIEW query, CREATE WINDOW VIEW query, ALTER query, ALTER TABLE, ALTER LIVE VIEW, ALTER DATABASE, RENAME query, RENAME TABLE, EXCHANGE TABLES, RENAME DICTIONARY, EXCHANGE DICTIONARIES, RENAME DATABASE, DROP query, DROP, DETACH, TRUNCATE, UNDROP query, UNDROP, CHECK TABLE, KILL QUERY query, KILL, OPTIMIZE query, OPTIMIZE TABLE, WATCH query, WATCH, SHOW ACCESS query, SHOW ACCESS, ShowAccessEntitiesQuery, SHOW GRANTS query, SHOW GRANTS, SHOW PRIVILEGES query, SHOW PRIVILEGES, INSERT query, INSERT INTO, USE query, USE, SET ROLE or SET DEFAULT ROLE query, SET ROLE DEFAULT, SET ROLE, SET DEFAULT ROLE, SET query, SET, SYSTEM query, SYSTEM, CREATE USER or ALTER USER query, ALTER USER, CREATE USER, CREATE ROLE or ALTER ROLE query, ALTER ROLE, CREATE ROLE, CREATE QUOTA or ALTER QUOTA query, ALTER QUOTA, CREATE QUOTA, CREATE ROW POLICY or ALTER ROW POLICY query, ALTER POLICY, ALTER ROW POLICY, CREATE POLICY, CREATE ROW POLICY, CREATE SETTINGS PROFILE or ALTER SETTINGS PROFILE query, ALTER SETTINGS PROFILE, ALTER PROFILE, CREATE SETTINGS PROFILE, CREATE PROFILE, CREATE FUNCTION query, DROP FUNCTION query, CREATE NAMED COLLECTION, DROP NAMED COLLECTION query, Alter NAMED COLLECTION query, ALTER, CREATE INDEX query, DROP INDEX query, DROP access entity query, GRANT or REVOKE query, REVOKE, GRANT, EXTERNAL DDL query, EXTERNAL DDL FROM, TCL query, BEGIN TRANSACTION, COMMIT, ROLLBACK, SET TRANSACTION SNAPSHOT, Delete query, DELETE, BACKUP or RESTORE query, BACKUP, RESTORE. (SYNTAX_ERROR)
* Closing connection 0
-
The error of the last query gets returned with code 200. Erroneous requests should be responsed with one of error http codes.
-
The server responses an error on a valid request. It is widespread to send data to a server using the
query=INSERT INTO test.a FORMAT JSONEachRowget parameter and formatted data in body. It's a useful feature to support.
Weird behaviour on requests with newlines
$ curl -vv -X POST http://a:b@localhost:8123 --data 'CREATE DATABASE test';
$ cat <<EOF | curl -vv -X POST http://a:b@localhost:8123 --data-binary @-
CREATE TABLE IF NOT EXISTS test.settings (fingerprint UInt64, type String, name String, value String, inserted_at DateTime64(9, 'UTC')) ENGINE = ReplacingMergeTree(inserted_at) ORDER BY fingerprint
EOF
$ cat <<EOF | curl -vv -X POST http://a:b@localhost:8123/?database=test --data-binary @-
INSERT INTO settings (fingerprint, type, name, value, inserted_at) VALUES (cityHash64('update_v3_5'), 'update',
'v3_1', toString(toUnixTimestamp(NOW())), NOW())
EOF
< HTTP/1.1 400 BAD REQUEST
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Fri, 03 Nov 2023 15:40:44 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 226
< Connection: close
<
Code: 62. DB::Exception: Code: 62. DB::Exception: Cannot parse expression of type String here: : While executing ValuesBlockInputFormat: data for INSERT was parsed from query. (SYNTAX_ERROR) (version 23.6.1.1). (SYNTAX_ERROR)
* Closing connection 0
The third request gets error 400 with some misleading message.
On the other hand:
$ cat <<EOF | curl -vv -X POST http://a:b@localhost:8123/?database=test --data-binary @-
INSERT INTO settings (fingerprint, type, name, value, inserted_at) VALUES (cityHash64('update_v3_5'), 'update', 'v3_1', toString(toUnixTimestamp(NOW())), NOW())
EOF
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:8123...
* Connected to localhost (127.0.0.1) port 8123 (#0)
* Server auth using Basic with user 'a'
> POST /?database=test HTTP/1.1
> Host: localhost:8123
> Authorization: Basic YTpi
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Length: 162
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: Werkzeug/3.0.1 Python/3.8.10
< Date: Fri, 03 Nov 2023 15:45:31 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Connection: close
<
* Closing connection 0
When the same request is sent with no newline inside, the data gets ingested successfully.
Newlines in the middle of a request should be omitted by server.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.