Code Monkey home page Code Monkey logo

surrealdb.go's Introduction

surrealdb.go

The official SurrealDB library for Golang.

Go Reference Go Report Card Mentioned in Awesome Go

Getting Started

For instructions on how to follow SurrealDB, follow Installation Guide

Installation

go get github.com/surrealdb/surrealdb.go

Usage

package main
import (
	"github.com/surrealdb/surrealdb.go"

)

type User struct {
	ID      string `json:"id,omitempty"`
	Name    string `json:"name"`
	Surname string `json:"surname"`
}

func main() {
	// Connect to SurrealDB
	db, err := surrealdb.New("ws://localhost:8000/rpc")
	if err != nil {
		panic(err)
	}

	authData := &surrealdb.Auth{
		Database:  "test",
		Namespace: "test",
		Username:  "root",
		Password:  "root",
	}
	if _, err = db.Signin(authData); err != nil {
		panic(err)
	}

	if _, err = db.Use("test", "test"); err != nil {
		panic(err)
	}

	// Define user struct
	user := User{
		Name:    "John",
		Surname: "Doe",
	}

	// Insert user
	data, err := db.Create("user", user)
	if err != nil {
		panic(err)
	}

	// Unmarshal data
	createdUser := make([]User, 1)
	err = surrealdb.Unmarshal(data, &createdUser)
	if err != nil {
		panic(err)
	}

	// Get user by ID
	data, err = db.Select(createdUser[0].ID)
	if err != nil {
		panic(err)
	}

	// Unmarshal data
	selectedUser := new(User)
	err = surrealdb.Unmarshal(data, &selectedUser)
	if err != nil {
		panic(err)
	}

	// Change part/parts of user
	changes := map[string]string{"name": "Jane"}

	// Update user
	if _, err = db.Update(selectedUser.ID, changes); err != nil {
		panic(err)
	}

	if _, err = db.Query("SELECT * FROM $record", map[string]interface{}{
		"record": createdUser[0].ID,
	}); err != nil {
		panic(err)
	}

	// Delete user by ID
	if _, err = db.Delete(selectedUser.ID); err != nil {
		panic(err)
	}
}
  • Step 1: Create a file called main.go and paste the above code
  • Step 2: Run the command go mod init github.com/<github-username>/<project-name> to create a go.mod file
  • Step 3: Run the command go mod tidy to download surreal db
  • Step 4: Run go run main.go to run the application.

Documentation

Full documentation is available at surrealdb doc

## Building

You can run the Makefile helper to run and build the project

make build
make test
make lint

You also need to be running SurrealDB alongside the tests. We recommend using the nightly build, as development may rely on the latest functionality.

Helper functions

Smart Marshal

SurrealDB Go library supports smart marshal. It means that you can use any type of data as a value in your struct. SurrealDB Go library will automatically convert it to the correct type.

// Recommended to use with SmartUnmarshal SmartMarshal
// User struct is a test struct
user, err := surrealdb.SmartUnmarshal[testUser](surrealdb.SmartMarshal(s.db.Create, user[0]))

// Can be used without SmartUnmarshal
data, err := surrealdb.SmartMarshal(s.db.Create, user[0])

Smart Unmarshal

SurrealDB Go library supports smart unmarshal. It means that you can unmarshal any type of data to the generic type provided. SurrealDB Go library will automatically convert it to that type.

// User struct is a test struct
data, err := surrealdb.SmartUnmarshal[testUser](s.db.Select(user[0].ID))

surrealdb.go's People

Contributors

atoo35 avatar dbut2 avatar electwix avatar ffrizzo avatar garrison-henkle avatar icelain avatar ikaio avatar intabulas avatar jhackenberg avatar mumoshu avatar naisofly avatar phughk avatar plally avatar sbshah97 avatar timothystiles avatar timpratim avatar tobiemh avatar tpotjj avatar youkoulayley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

surrealdb.go's Issues

Add readme

I know you have documentation in pkg.go.dev
But Shouldn't this repo have a proper read me file?

Bug: query string is butchered when using Go library

Describe the bug

The characters >c in the query for some reason are being replaced with Ϭ, making it impossible for the surrealdb to parse it.
The error seems to be consistent, I cannot get rid of it.
My code is:

query := "SELECT (SELECT id, name FROM ->own[WHERE time.deleted is NONE]->corporation) AS corporations FROM character:" + req.Msg.CharacterId
log.Printf("query: %s", query)
data, err := db.Query(query, nil)

When Go prints query into the log, it matches the intention, but in the SurrealDB log I see the following:

2023-05-28T03:33:07.540470Z TRACE surrealdb::net: RPC Received: { id: 'mG1GaJASv0zf40jA', method: 'query', params: ['SELECT (SELECT id, name FROM ->own[WHERE time.deleted is NONE]-Ϭorporation) AS corporations FROM character:1d2smxxqd4hexe50zttl', NULL] }

As you can see ->own[WHERE time.deleted is NONE]->corporation has been replaced with ->own[WHERE time.deleted is NONE]-Ϭorporation

Steps to reproduce

  • Run surrealdb in memory surreal start --log trace --user root --pass root memory
  • Create a go file
package main

import (
	"log"

	"github.com/surrealdb/surrealdb.go"
)

func main() {
	db, err := surrealdb.New("ws://localhost:8000/rpc")
	if err != nil {
		panic(err)
	}

	if _, err = db.Signin(map[string]interface{}{
		"user": "root",
		"pass": "root",
	}); err != nil {
		panic(err)
	}

	if _, err = db.Use("test", "test"); err != nil {
		panic(err)
	}
	query := "SELECT ->own[WHERE time.deleted is NONE]->corporation FROM character"
	log.Printf("query: %s", query)
	_, err = db.Query(query, nil)
	if err != nil {
		panic(err)
	}
}
  • run the file go run cmd/test/main.go
  • See the output:
2023/05/27 21:00:09 query: SELECT ->own[WHERE time.deleted is NONE]->corporation FROM character
panic: sending request failed for method 'query': There was a problem with the database: Parse error on line 1 at character 0 when parsing 'SELECT ->own[WHERE time.deleted is NONE]-Ϭorporation FROM character'

goroutine 1 [running]:
main.main()
        /mnt/c/Users/Nikita/narrow/narrow-services/cmd/test/main.go:29 +0x325
exit status 2

Expected behaviour

No error

SurrealDB version

1.0.0-beta.9+20230402.5eafebd for linux on x86_64

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

[Feature Request] Auto Models Migration

I'm looking for something like this:

models := []database.Model{
    &model.Person{},
    &model.Country{},
    &model.City{},
}

for _, model := range models {

    err := db.AutoMigrate(model)
    
    if err != nil {
    
          fmt.Printf("AutoMigrate(%s): %v\n", model.TableName(), err.Error())
    }
}

Thanks

Key path not found

I am assigning this politely to @TimothyStiles ✨ (this is long, but to keep it brief; if there is a roadmap for this package or formal way to submit and RFC; I will fix this and help refactor the package. I am currently using a different for of this repo that correct some other issue until the context changes get merged in, but I'd like to help! Just need a roadmap)

I have found a very cryptic error related to the json package and how its being called in this library. If you run a query like: create $cid set foo = $foo with the args: {"cid":"", "foo":"baz"} the query will execute successfully, but it will NOT unmarshal correctly.

For context: one of my service was throwing erroneous data because of a migration failure (psql) and that data was being blindly accepted by the service running surreal. A common paradigm forming is to push most of these problems to your DB instead of doing form filtering between all micro-services. My gRPC calls between the services just stated that they accepted a string, and did not try to enforce the string length. Typically you let the query fail, and take the reason it failed and generate your error/response from that and push it back. If you get swarmed with requests then 🤷 that's what rate limiting is for. This argument makes more sense when you are putting in 10->20 fields for one entry, and doing form validation everywhere is a waste

End of context,
The query was run successfully on surreal, and it didn't throw an error of "hey, so I can't put something into nothing". So it took the result of nothing and defaults, and tried to unmarshal it netting us the unhelpful message: Key path not found

The error is wrapped a few times and propegated up the error system by encoding/json --> surreal.go --> my lib. It is extremely difficult to unwrap the error with that many layers of glue ✨ on it. Can we please wrap the error or add in a shim for a logger to catch is like in the Stripe library to make this easier to debug? This error is extremely cryptic and took me a week (60->80 hours) to solve. I was taking apart kubernetes and the rest of my tooling to make sure this wasn't my fault

Bug: Unhandled errors in websockets can lead to panics

Describe the bug

https://github.com/surrealdb/surrealdb.go/pull/118/files

Not much detail, but there is a code line. Apparently, when the error isnt handled correctly, the websocket connections can panic.

The TODO is going to be added to the codebase

Steps to reproduce

Unknown, may need to review code

Expected behaviour

No panics, errors handled correctly

SurrealDB version

1.0.0+20231130.a420c821.dirty for macos on aarch64

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Next steps for this repo

Hi,
I am interested in helping develop this library, but there doesn't seem to be much info about it or what the next steps are. Is there somewhere I can be pointed to for things to do, or should I just start writing code and putting up PR requests?

Bug: `Question or idea` Button

Describe the bug

While Creating Issues
The Question or idea button redirects to void

https://github.com/surrealdb/surrealdb.go/blob/main/.github/ISSUE_TEMPLATE/config.yml

Steps to reproduce

  1. go to the issues tab
  2. create a new issue
  3. click on Question or idea labeled button
  4. Get 404

Expected behaviour

to not redirect to void

SurrealDB version

non-related

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Graph traversals don't work with the default json.Marshal usage

The default json.Marshal method automatically escapes the -> pointers so the Surreal server returns a parsing error.

Actually, the way that Websocket.Send checks for an ID before an error was masking the parsing error response. I had to flip the lines around

I added this little bit of code to see if it would work with a more lenient marshaler. and it does

func JSONHack(t interface{}) ([]byte, error) {
	buffer := &bytes.Buffer{}
	encoder := json.NewEncoder(buffer)
	encoder.SetEscapeHTML(false)
	err := encoder.Encode(t)
	return buffer.Bytes(), err
}

func (ws *WebSocket) write(v interface{}) error {
	data, err := JSONHack(v)
	if err != nil {
		return err
	}

	return ws.conn.WriteMessage(websocket.TextMessage, data)
}

Add dependencies to depguard

Add missing dependencies to depguard.

Issue: Our tests are failing for linting due to the dependencies that are not available in depguard.

Fix: Add the missing dependencies to depguard to resolve the failing tests.

For more info, check failing tests of:
#74

surrealdb.Unmarshal invalid SurrealDB response

I'm trying to run an update, ie.,

db.Update("companies:test01", map[string]string{"name": "Test 01"})

I keep getting an error when I try to unmarshal the response:

invalid SurrealDB response

Noticed that https://github.com/surrealdb/surrealdb.go/blob/main/db.go#L37 is asserting that the data value has a list of interfaces.

I only can get it to work by converting the return value of db.Update into a slice by doing this:

c, _ := db.Update("companies:test01", map[string]string{"name": "Test 01"})
var cc []interface{}
cc = append(cc, c)

var company Company
err := surrealdb.Unmarshal(cc, &company)
fmt.Println(err)

Feature: better error return

Is your feature request related to a problem?

most of errors are embedded in the returned json, not actually returned as error type

there isn't clear documentation on how to parse errors from returned json

if a transaction fails, I'd like to rollback something, but not going to rollback for other errors (like parse errors)

Describe the solution

parse returned json in driver, return error type if there is actually an error

Alternative methods

n/a

SurrealDB version

1.0.0-beta.9+20230402.5eafebd for linux on x86_64

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Serialization/Performance

So to start, I was digging through the code, and our issue mainly stems from the fact that we need to decode the main response, to get the RPC ID for handling the callback

I was playing with this jsonparser package earlier today with something else, then I realised we might be able to use it here.

I did a simple benchmark, and the performance just to get the ID is much faster
CleanShot 2022-09-25 at 17 09 34

My idea:

We can use the jsonparser, to get the id, and then we use a struct like:

type RPCRawResponse struct {
	ID       string
	Data    []byte
	Decoded *RPCResponse
}

This "response" can be passed down the channels and such to the end function and only decoded once.

On top of this, we can build a top level "QueryResolver" which uses a generic, and is an opt-in function/struct, which handles context, decoding, checking for errors etc
This will take the byte version and pull information out in small pieces, we can also decode once to the final desired output struct from this, rather than Unmarshal -> get id -> Marshal -> Unmarshal -> check for errors and potentially more...

How do we feel about this?
I hacked together a crappy version of this "QueryResolver" setup in my project already, but i didn't optimise or care much for code quality as i was just playing, but it's semi elegant to use
CleanShot 2022-09-25 at 17 17 46

Beta V9

Hey, I wanted to switch to Beta-V9 everything has gone downhill literally everything breaks.

I create PR #52 for the patch but it is even with the patch, db. create not generation Unique id and creating user:⟨⟩.
I don't know if this is SurrealDB or Golang Driver cause this is Critical for Database Usage.

@phughk Could look into this.
I can even use db right now.

Example fails in Beta 11 - panic: sending request failed for method 'change': Method not found

Describe the bug

Newbie .....

When executing example code it fails on line 66 with following error:

// Change part/parts of user
changes := map[string]string{"name": "Jane"}
if _, err = db.Change(selectedUser.ID, changes); err != nil {
panic(err)
}

panic: sending request failed for method 'change': Method not found

goroutine 1 [running]:
main.main()
/home/mjc4/go/src/testsurrealdb/testsurrealdb.go:67 +0xf45

I believe that is because Change has been replaced with Merge and SDK has not been updated.

Steps to reproduce

Run example code.

Expected behaviour

Only name is changed on record.

SurrealDB version

1.0.0-beta.11+20230905.59718fc for linux on x86_64

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Combinations of characters are wrongly transformed

Hey team, there's something odd with the client. I printed the query right before passing it to the Query method to make sure this wasn't on me:

fmt.Println("Executing query: ", c.query)
resp, err := c.db.Query(c.query, nil)

Later:

Executing query:
                        CREATE club:`MyNewClub` CONTENT {
                                club_name: "MyNewClub",
                                owner: clubber:`d395c589-c8bb-4f21-b38f-705dcc805af1`,
                                settings: {
                                        is_open: false,
                                        vip_chat: false,
                                        battle_mode: false
                                }
                        };
                        RELATE clubber:`d395c589-c8bb-4f21-b38f-705dcc805af1`->owns->club:`MyNewClub` SET time.bought = time::now();
                        SELECT * FROM club:`MyNewClub` FETCH owner;

2023-04-25T06:55:37.056+0200    ERROR   club_server     club/handlers.go:181    could not create club      {"CreateClub": "", "clubName": "MyNewClub", "identityID": "d395c589-c8bb-4f21-b38f-705dcc805af1", "error": "query error: sending request failed for method 'query': There was a problem with the database: Parse error on line 11 at character 3 when parsing 'RELATE clubber:`d395c589-c8bb-4f21-b38f-705dcc805af1`->owns-Ϭlub:`MyNewClub` SET time.bought = time:'"}

I'm running v0.2.1 of the client, and the latest docker image.

The interesting part: ->owns-Ϭlub
I have added a space between the > and the c, and this seems to fix the issue. So somewhere in the go client, it is transforming the >c into Ϭ.

This is the only transformation I've found so far.

Feature: Move and test `unmarshalMapToStruct(data map[string]interface{}, outStruct interface{})`

Is your feature request related to a problem?

A lot of the deserialisation is happening via JSON, even for code that is already deserialised.
There is a non-json marshaller, unmarshalMapToStruct(data map[string]interface{}, outStruct interface{}), in gorilla.go.
It should be moved to a separate package (not protocol implementation related, not API related) and tested.

Describe the solution

Move the function unmarshalMapToStruct(data map[string]interface{}, outStruct interface{}) into a separate package.
Test the funciton independently of protocol implementations.

Alternative methods

There could already be code that does this. It's important that we don't rely on translating map[string]interface{} into json and back into a golang struct though.

SurrealDB version

1.0.0

Contact Details

[email protected] [email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

surrealdb Auth undefined

Describe the bug

Example in Readme incorrect.
Driver is not idiomatic for Go.
context.Context is not passed in the request.
Before you write something - study good examples of drivers (example: go-redis) and don't embarrass yourself

Steps to reproduce

Run example in readme

Expected behaviour

.

SurrealDB version

1.1.0

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Bug: Cannot use type::thing in queries as they arent escaped properly

Describe the bug

It seems that parameters might not be properly passed as parameters to the db or something weird is happening. Maybe the query is incorrect, but it seems right. We need a test for this though to be certain. The tests are to verify parameters and escaping works as expected. We need to consider "thing", "ident", "table", and perhaps several more that need to have formatting handled correctly.

Steps to reproduce

From discord:

res, err := h.DB.Query(`
    BEGIN TRANSACTION;
    LET $WAREHOUSE = (SELECT * FROM rand::uuid());
    CREATE type::thing("warehouse", $WAREHOUSE) CONTENT {
        name: $name,
        desc: $desc,
        logo: $logo,
        owner: $userID,
        isPhysical: $isPhysical,
        capacity: $capacity
    };
    UPDATE $userID SET owns += type::thing("warehouse", $WAREHOUSE);
    RELATE $userID->manages->type::thing("warehouse", $WAREHOUSE) 
    SET roles = ["owner"];
    COMMIT TRANSACTION;
    `, map[string]interface{}{
        "userID":     data.OwnerID,
        "name":       data.Name,
        "desc":       data.Desc,
        "logo":       data.Logo,
        "isPhysical": data.IsPhysical,
        "capacity":   data.Capacity,
    })

Results in

sending request failed for method 'query': There was a problem with the database: Parse error on line 13 at character 1 when parsing 'RELATE $userID->manages->type::thing("warehouse", $WAREHOUSE)

Workaround was

RELATE user:⟨01899675-7c82-7ed2-8c4b-fb1b59556c1f⟩->manages->warehouse:⟨01899675-7c82-7ed2-8c4b-fb1b59556c1e⟩
    CONTENT {
        roles: ["owner"]
    };

Expected behaviour

The original query should have worked. Or the workaround shouldn't have worked.

SurrealDB version

1.0.0-beta.9+20230726.43794700 for macos on aarch64

Contact Details

[email protected], < worldwidepaniel on discord >

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

panic: repeated read on failed websocket connection

Hi this is our first issue in this project. Please correct our style/convention etc as needed.

We consistently encounter panic: repeated read on failed websocket connection from github.com/gorilla/[email protected]/conn.go:1030. The following code demonstrates the failure with a MTTF of ~90mins. The attached test is passive, just waiting for an eventual websocket close followed by a read error. With active use of the connection, the MTTF drops considerably (but we don't have a concise reproduction of that here).

Our reproduction environment is:

  • surrealdb host is running in AWS Elasticbeanstalk on platform Docker running on 64bit Amazon Linux 2/3.5.8
  • application.go running locally on MacOS.

Note: we noticed a dramatic increase in the MTTF when the platform changed from 3.5.3 to 3.5.8. So, the root cause is likely the proxy closing an inactive socket in the case demonstrated by this code. The problem remains - it seems to us the problem is not detected/reported/recoverable in our application using surrealdb.go.

We'll appreciate any guidance.

steps to reproduce

  1. have a surrealdb instance available: { ns : test, db: test, user : root, password : root } or modify connection details.
  2. initialize a project with the attached application.go.
  3. time SURREALDB_URL=ws://<surrealdbhost>/rpc go run .

application.go.txt

Discussions

I was talking with people on the Discord channel for golang drivers.

I thought it could be nice to have discussions for feature requests and talk about ideas.

I know there are issues with that too but think we need to separate them because there will be some ideas without any
standpoint when there is presented to the public then we can talk about the pros and cons and if the public ok with that we can implement it because we usually go with our personal experience but we can listen to others and their opinions about the topic or feature.

Feature: context.Context Support

Is your feature request related to a problem?

This would be a part of a wider change to allow trace information propagation (if possible). Passing in the context is currently not possible and we would need it to propagate the trace information. Seeing how the Surreal server already supports tracing, this would provide a very good method of troubleshooting entire requests.

Describe the solution

The standard way to do so is by using context.Context and passing it as the first argument to the function. eg.

func (db *DB) Create(ctx context.Context, thing string, data any) (any, error)

This would be the preferred way as the SDK has not yet reached v1.0 so it's easier to change the method signature without a v2 release.

Alternative methods

If the method signatures are not to be changed, the Context suffix can be added to each method. This is rather verbose to type and is typically used when a pre-existing API is adding context support.

eg.

func (db *DB) CreateContext(ctx context.Context, thing string, data any) (any, error)

There is no other obvious way to add context information.

SurrealDB version

N/A

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Getting Parsing Error When Use Fetch with map[string]interface

Hi Everyone,

I was selecting the record and fetching it is the field record link with the fetch keyword
but when I try to use it with the parameter it gives
There was a problem with the database: Parse error when parsing 'fetch $fetchstr'
this error I checkout the source code this line JSON Marshaling giving the error.
Also, I tried with not safe method string merge ( fmt.sprintf ) and added my string at the end it worked that way.

Am I doing something wrong?
if that's the case please let me know.

The Code I'm trying to fetch

res, err := db.Globaldb.Query("select * from $table fetch $fetchstr;", map[string]interface{}{
	"table":  ID,
	"fetchstr": fetchstr,
})

UPDATE:

for clear out

The problem I was facing was:

After the FETCH statement, arguments are no longer parsed.

Example:

select * from $table FETCH $fetchstr;
$table will parse but $fetchstr will not parse

Add support for Live Queries

We would like to have live queries be supported by the driver.

The idea behind live queries, is that clients can invoke the query LIVE SELECT * FROM table and they will get a UUID in response. Then when changes happen to the table table, they will get json messages over the websocket protocol, that are associated with the live query uuid provided earlier.

This change is about handling the live query responses, as users should already be able to invoke the raw SurrealQL query. This can be buffered into a channel that can be read from.

example code gives unmarshalling error

I started a local in-memory database according to the installation instructions, ran go mod init, ran go get github.com/surrealdb/surrealdb.go, created a main.go file in the project root, copied and pasted the example code into it, and ran it with 'go run main.go'. I received the following error:

panic: json: cannot unmarshal object into Go value of type []main.User

goroutine 1 [running]:
main.main()
	/home/atd/Projects/surreal_go/main.go:46 +0xa0b
exit status 2

What's going wrong here?

DB.Query hangs when sending an arrow

Description

The DB.Query function causes the program to hang when an arrow operator is used in the query. When set to --log full, the server shows the message is received and possibly sent, but the client never receives the response.

Steps to reproduce

Execute any query with -> or <-. For example:

data, err := db.Query("relate car:123<-owns<-user:321;", nil)

or even more minimal:

data, err := db.Query("->", nil)

Logs

For the first example above, the server logs the following:

[2022-09-18 23:24:49] TRACE tungstenite::protocol Received message {"id":"3184ca119396f9ea","method":"query","params":["relate car:123\u003c-owns\u003c-user:321;",null]}

[2022-09-18 23:24:49] TRACE tokio_tungstenite /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/lib.rs:268 Stream.poll_next
[2022-09-18 23:24:49] TRACE tokio_tungstenite /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/lib.rs:223 WebSocketStream.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/lib.rs:270 Stream.with_context poll_next -> read_message()
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:178 Write.flush
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:180 Write.with_context flush -> poll_flush
[2022-09-18 23:24:49] TRACE tungstenite::protocol Frames still in queue: 0
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:149 Read.read
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:152 Read.with_context read -> poll_read
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat WouldBlock
[2022-09-18 23:24:49] TRACE tokio_tungstenite /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/lib.rs:223 WebSocketStream.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:178 Write.flush
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:180 Write.with_context flush -> poll_flush
[2022-09-18 23:24:49] TRACE tungstenite::protocol Frames still in queue: 0
[2022-09-18 23:24:49] TRACE tokio_tungstenite /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/lib.rs:223 WebSocketStream.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:178 Write.flush
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:180 Write.with_context flush -> poll_flush
[2022-09-18 23:24:49] TRACE tungstenite::protocol Frames still in queue: 1
[2022-09-18 23:24:49] TRACE tungstenite::protocol Sending frame: Frame { header: FrameHeader { is_final: true, rsv1: false, rsv2: false, rsv3: false, opcode: Data(Text), mask: None }, payload: [123, 34, 105, 100, 34, 58, 110, 117, 108, 108, 44, 34, 101, 114, 114, 111, 114, 34, 58, 123, 34, 99, 111, 100, 101, 34, 58, 45, 51, 50, 55, 48, 48, 44, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 34, 80, 97, 114, 115, 101, 32, 101, 114, 114, 111, 114, 34, 125, 125] }
[2022-09-18 23:24:49] TRACE tungstenite::protocol::frame writing frame
<FRAME>
final: true
reserved: false false false
opcode: TEXT
length: 61
payload length: 59
payload: 0x7b226964223a6e756c6c2c226572726f72223a7b22636f6465223a2d33323730302c226d657373616765223a225061727365206572726f72227d7d

[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:167 Write.write
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:169 Write.with_context write -> poll_write
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:178 Write.flush
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:180 Write.with_context flush -> poll_flush
[2022-09-18 23:24:49] TRACE tokio_tungstenite /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/lib.rs:223 WebSocketStream.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:178 Write.flush
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:126 AllowStd.with_context
[2022-09-18 23:24:49] TRACE tokio_tungstenite::compat /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-tungstenite-0.15.0/src/compat.rs:180 Write.with_context flush -> poll_flush
[2022-09-18 23:24:49] TRACE tungstenite::protocol Frames still in queue: 0

Bug: Broken Tests for Fetch

Describe the bug

This bug is a follow up from the below PR. This requires us to fix testcases listed in TestFetch()
This is not a bug in the SDK and a bug in the core engine which needs to be fixed.

#115

Steps to reproduce

  1. Comment out s.T().Skip
  2. Run testcase and ensure it passes successfully

Expected behaviour

The tests listed above are expected to pass.

SurrealDB version

surreal 1.0.0 for MacOS

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Parsing error

for query select * from ->contact->profile;

I got error: There was a problem with the database: Parse error on line 1 at character 0 when parsing 'select * from -Ϭontact->profile;'

somehow >c is encoded to Ϭ

I ran the same query from thunder client, and it returned fine without error, so it doesn't look like it's a problem with the server

here is Link to minimal reproduction repo

Bug: Re-connection to the DB

Describe the bug

When the application is running, and the DB is restarted, then the application panic with the following stack:

panic: repeated read on failed websocket connection

goroutine 50 [running]:
github.com/gorilla/websocket.(*Conn).NextReader(0xc0000e0580)
        /home/admin/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:1030 +0x4f4
github.com/gorilla/websocket.(*Conn).ReadMessage(0x129dfa0?)
        /home/admin/go/pkg/mod/github.com/gorilla/[email protected]/conn.go:1093 +0x2f
github.com/surrealdb/surrealdb.go/internal/websocket.(*WebSocket).read(0xc00039c080, {0xca5720, 0xc0003f4480})
        /home/admin/go/pkg/mod/github.com/surrealdb/[email protected]/internal/websocket/ws.go:140 +0x45
github.com/surrealdb/surrealdb.go/internal/websocket.(*WebSocket).initialize.func1()
        /home/admin/go/pkg/mod/github.com/surrealdb/[email protected]/internal/websocket/ws.go:167 +0xac
created by github.com/surrealdb/surrealdb.go/internal/websocket.(*WebSocket).initialize in goroutine 1
        /home/admin/go/pkg/mod/github.com/surrealdb/[email protected]/internal/websocket/ws.go:160 +0x8e
exit status 2

Steps to reproduce

run an application and then stop surrealdb

Expected behaviour

reconnect or return back an error to the application, and let developer decide what to do

SurrealDB version

1.0.0+20230913.54aedcd for linux on x86_64

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Feature: Replace Gorilla WS with Nhooyr WS

Is your feature request related to a problem?

No, but it is will great when fixing #119

Nhooyr's implementation is more lightweight and idiomatic than Gorilla's

For instance, Turso's Go driver, a popular database library, demonstrates the effectiveness of Nhooyr's WebSockets implementation (https://github.com/tursodatabase/libsql-client-go).
Also golang std ws mention Nhooyr's ws as

more actively maintained WebSocket
ref

Describe the solution

Remove Gorilla WS support and add Nhooyr WS support.
These changes will not affect function signatures and will improve the maintainability of the repository.

Alternative methods

There was an attempt to add Can along with Gorilla but it was rejected. #127. I considered using a soft switch but it is not feasible.

SurrealDB version

non related

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Might be fair to put some under construction to the repo for now

I was trying to connect to the Server using the URL that works via curl.

As a result I got the error malformed ws or wss URL. Then I noticed this SDK is somehow only for Websockets, when I wanted to see how that works the official docs told me that this is 'The documentation for this integration is coming soon.'

To prevent more users from this unnecessary spend time and inconvenience it might be better to put some disclaimer at the repo.

My code has nothing to do with what i am implying but for completeness:

package main

import (
	"fmt"

	surreal "github.com/surrealdb/surrealdb.go"
)

func main() {
	db, err := surreal.New("http://localhost:8012/sql")
	if err != nil {
		panic(err)
	}
	t, err := db.Authenticate("root:root")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Authenticate: %s", t)
	res, err := db.Query("CREATE productManager:KlausIker SET firstName = 'Klaus', lastName = 'Iker';", nil)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Query: %s", res)
}

Returning interface{}/any on most calls is not very flexible

My idea would be to return some sort of raw value that can be unmarshalled to anything (thus removing the need for the strange Unmarshal top-level function)

type RawValue struct { ... }
func (rv RawValue) Unmarshal(target any) { /* use the raw value internals to unmarshal */ }
func Select(what string) (RawValue, error) {...}

I don't know if this idea is good, as most libraries use a method param to unmarshal into, à la

func Select(what string, target any) (RawValue, error) { /* directly unmarshal into target */ }

The advantage of the first option would be that we can add other methods, for example

func (rv RawValue) Subvalue(target any, keys ...string) // Subvalue(v, "user", "name") would unmarshall user.name to v

I can do the implementation of both, but am very open to any alternative as those do no strike me as very good

Feature Request - Compatibility with go sql

This mostly involves these tasks:

  • Register the go sql driver upon import (when simply calling this, but then for SurrealDB: import _ "github.com/go-sql-driver/mysql")
  • Create a "SQL connection string" for SurrealDB - e.g. parsing the username, password, database, namespace, etc. from a string (should be relatively easy)
    • username
    • password
    • URL
    • database name
    • namespace
  • Implement the types/methods mentioned at https://pkg.go.dev/database/sql/driver - mostly around converting to/from standard Go values and SurrealDB database values.

Bug: unexpected EOF on TestSurrealDBSuite/gorilla/TestUpdate

Describe the bug

While working on #119 noticed while TestSurrealDBSuite/gorilla/TestUpdate running getting unexpected EOF

Steps to reproduce

  1. Print or Log the error on pkg/conn/gorilla/gorilla.go:226
  2. Run Tests
  3. Get Error

Expected behaviour

Expected to not get this error

SurrealDB version

1.1.1 for linux on x86_64

Contact Details

[email protected]

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Bug: The session has expired error when connecting to version 1.3.0

Describe the bug

version 1.3.0 has websocket session expire feature, and throw The session has expired error.

Are we supposed to handle this on our own, or this should be handled by driver?

Steps to reproduce

make connection, idle for a while, then make a request

Expected behaviour

no error, and request go through, result successfully returned

SurrealDB version

surreal 1.3.0

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Unmarshal not declared

I am using github.com/surrealdb/surrealdb.go v0.1.1 when I use the command surrealdb.unmarshal i get this error
unmarshal not declared by package surrealdb

Feature: Formatting Query like fmt.Sprintf

Is your feature request related to a problem?

Please add formatting query function.
Like this one.

what := surrealdb.Format("SELECT * FROM user WHERE name = ?;", username")

You need to add like this function because SQL injection vulnerabilities occur frequently.

Describe the solution

Alternative methods

SurrealDB version

Contact Details

No response

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Connection Pooling / Surreal.Instance

I can see there is a Surreal.Instance in JavaScript API and in rust there is Surreal::init() in the Rust API but I could not see the equivalent of it neither in the Docs nor the source code.

How can one avoid creating new connections everytime and use a static struct to use the same connection or even implement connection pool?

Add support for handling diffs (in live query responses)

Add support for handling DIFF type responses in live query notifications.

This is different to handling live query notifications, described here: #80

The notifications from live queries can return either the full document after modification (creation, update, delete), or they may instead send the diff which resembles a git diff to the document.

This ticket is about handling that functionality.

Test websockets can be concurrent

We want to verify the driver can make requests concurrently. We need a test that follows this pattern

conn = openConnection()
chan response = blaa // cant remember syntax
go func() {
   response << "Starting sleep"
   _, err = conn.Execute("sleep 5")
   suite.Require().NoError(err)
   response << "All good, sleep finished"
}
waitForStart << response // I should probably learn the syntax
start = time.Now()
_, err = conn.Update(doSomeUpdate())
end = time.Now()
suite.Require().NoError(err)
suite.Require().LessThan(end-start, Duration.Second(2))
completed << response // still haven't googled
completeTime = time.Now()
suite.Require().MoreThan(completed-start, Duration.Second(5))

Handle slice input with `SmartUnmarshal`

While SmartUnmarshal is useful with its generics support, it needs an explicit handling for handling a slice input. As we can check further into the unmarshalled data in RawQuery, it would be great to add some handling for complex data structure. This comes with a bit of cost with potential recurses, and thus it is probably preferable to keep the existing logic, and add a new function instead.

Documentation: working with record links

Description

I'm not sure if this is an issue shared across all SurrealDB SDKs right now due to how the underlying RPC works, but I'm currently trying to create new records using db.Create and passing an array of model IDs as strings inside my interface. However, it doesn't appear that these relations are getting saved in the database, despite the other fields in my interface being stored. My table is schemafull and configured to have the specific column be of type set<record>.

Is working with relations not currently supported, or is just currently missing from the docs? If you should be able to create relations via string arrays, happy to open a bug report instead.

Is there an existing issue for this?

  • I have searched the existing issues

Code of Conduct

  • I agree to follow this project's Code of Conduct

Empty response on "select"

func get_configs() Config {
	configs := Config{
		user:   "root",
		pass:   "root",
		db:     "test",
		ns:     "test",
		host:   "127.0.0.1",
		scheme: "ws",
		port:   8000,
	}

	return configs
}

func main() {
	_, use_err := db.Use(configs.ns, configs.db)
	data, d_err := db.Select("apps")
	if d_err != nil {
		fmt.Print(d_err)
	}

	fmt.Println(data)

	var apps Results
	json.Unmarshal([]byte(fmt.Sprint(data)), &apps)

	fmt.Println(&apps)

	for i, s := range apps.result {
		fmt.Println(i, s)
	}
}

Data appears in the shell but doesn't when fetched from the GoLang SDK.
image
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.