thda / tds Goto Github PK

While I am using a special character @ in the password it is not working after the first @ character it considers the host name.
like

db, err = sql.Open("tds", "tds://username:pass#word@abcde#[email protected]:1234?charset=utf8")

it is considering "abcde#[email protected]" rather then hostname.com

Please help!

When the connection is interrupted unexpectedly (eg, stateful firewall loses track of the connection) it seems that when database/sql tries to signal to the driver to close the connection it gets stuck in a busy loop causing 100% CPU usage.

Stack trace of the offending goroutine running under go version go1.12.9 linux/amd64:

goroutine 855659 [runnable]:
syscall.Syscall(0x0, 0x7, 0xc0001565a8, 0x1, 0x0, 0x1, 0x0)
        /usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5
syscall.read(0x7, 0xc0001565a8, 0x1, 0x8, 0x0, 0x0, 0x0)
        /usr/local/go/src/syscall/zsyscall_linux_amd64.go:732 +0x5a
syscall.Read(...)
        /usr/local/go/src/syscall/syscall_unix.go:172
internal/poll.(*FD).Read(0xc00014eb00, 0xc0001565a8, 0x1, 0x8, 0x0, 0x0, 0x0)
        /usr/local/go/src/internal/poll/fd_unix.go:165 +0x131
net.(*netFD).Read(0xc00014eb00, 0xc0001565a8, 0x1, 0x8, 0x0, 0xe97160, 0xc000096050)
        /usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0000ac2b8, 0xc0001565a8, 0x1, 0x8, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/net.go:177 +0x69
io.ReadAtLeast(0x7f47d8c79a08, 0xc0004741c0, 0xc0001565a8, 0x1, 0x8, 0x1, 0x0, 0xe97160, 0xc000096050)
        /usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(...)
        /usr/local/go/src/io/io.go:329
github.com/thda/tds/binary.(*Encoder).Read(0xc000156588, 0xc0001565a8, 0x1, 0x8, 0x0, 0xe97160, 0xc000096050)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/binary/encoder.go:78 +0x6b
github.com/thda/tds/binary.(*Encoder).ReadByte(...)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/binary/encoder.go:306
github.com/thda/tds.(*header).read(0xc000156490, 0xc000156588, 0xe97160, 0xc000096050)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/buffer.go:42 +0x1ab
github.com/thda/tds.(*buf).readPkt(0xc000156480, 0xc000499001, 0xe97160, 0xc000096050)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/buffer.go:140 +0x54
github.com/thda/tds.(*buf).processCancel(0xc000156480, 0x0, 0x0)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/buffer.go:554 +0xcf
github.com/thda/tds.(*buf).sendPkt(0xc000156480, 0xeb3401, 0xc0003c8350, 0x0)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/buffer.go:187 +0x1c0
github.com/thda/tds.(*buf).send(0xc000156480, 0xeadfe0, 0xc0002fb620, 0xc0003c830f, 0xc000499180, 0x1, 0x1, 0x0, 0x0)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/buffer.go:398 +0xee
github.com/thda/tds.(*session).Close(0xc0003ff040, 0x0, 0x0)
        /home/jenkins_slave/workspace/tds-test/src/github.com/thda/tds/session.go:342 +0x192
database/sql.(*driverConn).finalClose.func2()
        /usr/local/go/src/database/sql/sql.go:521 +0x49
database/sql.withLock(0xea1e80, 0xc000119280, 0xc000499270)
        /usr/local/go/src/database/sql/sql.go:3097 +0x63
database/sql.(*driverConn).finalClose(0xc000119280, 0x0, 0x4)
        /usr/local/go/src/database/sql/sql.go:519 +0x130
database/sql.(*driverConn).Close(0xc000119280, 0xc000097390, 0xc000097390)
        /usr/local/go/src/database/sql/sql.go:500 +0x138
database/sql.(*DB).putConn(0xc00010e180, 0xc000119280, 0xe97160, 0xc000097390, 0xc000119201)
        /usr/local/go/src/database/sql/sql.go:1250 +0x2d5
database/sql.(*driverConn).releaseConn(...)
        /usr/local/go/src/database/sql/sql.go:421
database/sql.(*DB).pingDC(0xc00010e180, 0xeae020, 0xc000520e40, 0xc000119280, 0xc0004994a8, 0x0, 0x0)
        /usr/local/go/src/database/sql/sql.go:719 +0x84
database/sql.(*DB).PingContext(0xc00010e180, 0xeae020, 0xc000520e40, 0xc000499520, 0xd567e0)
        /usr/local/go/src/database/sql/sql.go:742 +0x170
<truncated>

In this instance, db.PingContext() was called which made the database/sql package realise there was a bad connection in the pool. Relevant bit of code in the database/sql package that kicks off the chain of events: https://github.com/golang/go/blob/release-branch.go1.12/src/database/sql/sql.go#L1250

Subsequent calls to db.PingContext() worked fine using a fresh connection while the offending go routine was still caught up in a busy loop.

I am querying a table I don't know the schema of. So I am using this to gather the table contents:

cols, err := rows.Columns() // Remember to check err afterwards
vals := make([]interface{}, len(cols))
for i, _ := range cols {
	vals[i] = new(sql.RawBytes)
}

for rows.Next() {
	if err := rows.Scan(vals...); err != nil {
        // Check for a scan error.
	// Query rows will be closed with defer.
	log.Fatal(err)
	}
	log.Info(vals)
}

However when I get to a column with type money I get the following error:

unsupported Scan, storing driver.Value type tds.Num into type *sql.RawBytes

First of all, thankyou so much for creating this library! It is the only pure go sybase library I could find

I am trying to call a Stored Procedure with multiple parameters, but it is returning a errors

_, err := c.db.ExecContext(ctx, exec my_proc ?, ?, param1, param2) returns the following response
tds: Prepare failed: Msg: 7332, Level: 15, State: 1 Server: HORIZON, Procedure: gtds1, Line: 1: The untyped variable ? is allowed only in in a WHERE clause or the SET clause of an UPDATE statement or the VALUES list of an INSERT statement

If I take out the exec eg
_, err := c.db.ExecContext(ctx, my_proc ?, ?, param1, param2) it returns this
tds: Prepare failed: Msg: 102, Level: 15, State: 1 Server: HORIZON, Procedure: gtds1, Line: 1: Incorrect syntax near 'my_proc'

If I manually add the params it works fine eg
_, err := c.db.ExecContext(ctx, exec my_proc 'param1', 'param2')
However I need to be able to pass the params through

I have tested this using a perl library (https://metacpan.org/pod/DBD::Sybase#Stored-Procedures-and-Placeholders) against our Sybase server and that works fine so it seems to be an issue with the library?

After scanning successfully a NULL value into tds.Num, what is the public method to determine if is NULL or not? tds.Num has only String() and Rat() methods?

tds.Num{r:big.Rat{a:big.Int{neg:false, abs:big.nat(nil)}, b:big.Int{neg:false, abs:big.nat(nil)}}, precision:0, scale:0, isNull:true}

Hi,

I was doing a quick check of this driver against our sybase ase 16. I encountered a hang on db.Ping then timedout. Before i dig deeper, was wondering if this works against 16?

Thanks

-jaijiv

Hello everyone,
First of all thank's for your work !
This is my issue:

package main

import (
	"database/sql"
	_ "flag"
	"fmt"
	"os"

	_ "github.com/thda/tds"
)

func main() {
	db, err := sql.Open("tds", "tds://username:password@IP:2638/dbnameA?charset=utf8")
	if err != nil {
		fmt.Printf("Failed to open database: %v", err)
		os.Exit(1)
	}
	defer db.Close()
	err = db.Ping()
	if err != nil {
		fmt.Printf("Failed to ping database: %v", err)
		os.Exit(1)
	}
	fmt.Println("ok")
	var result string
	row, _ := db.Query("select db_name()")
	for row.Next() {
		row.Scan(&result)
	}
	fmt.Println(result)
	os.Exit(0)

}

Response :
ok defaultDB

If I specify a db name into the connection string I can do "select x from TableOfDbA" but if I do "select db_name()" the response will be DefaultDB.
If I specify a Db into the connection string, and if this DB doesn't exist, I don't have an error message because a connection is made with the default db. This is a problem because I use this script to monitor all db on the server.
Thank's for your help

Hi,

I want to use a own encoding.

Specifically, golang's japanese.ShiftJIS replaces some characters with HTML reference characters, but I do not want that.
For example, １−２ is encoded to１−２.

I want a way to encode using another library (e.g. iconv).

my password having special characters like "#" and "|", when I'm trying to connect using perl it connecting successfully but golang is giving error : netlib: unexpected EOF while reading message

here is my code snippet
can you please help what is wrong in this, do i need to encode password into something?
`import (
"database/sql"
_ "github.com/thda/tds"
)

func getConn(cred map[string]string) (c *sql.DB) {
Name := cred["NAME"]
Pass := cred["PASS"] // 6#A(co&k|31V
Server := cred["SERVER"]
User := cred["USER"]
Port := cred["PORT"]
// dbConn := fmt.Sprintf("dbname=%s host=%s port=%s user=%s password=%s", Name, Server, Port, User, Pass)
cnxStr := "tds://" + url.QueryEscape(User) + ":" + url.QueryEscape(Pass) + "@" + Server + ":" + Port + "/" + Name + "?charset=utf8"
conn, err := sql.Open("tds", cnxStr)
if err != nil {
fmt.Println("Open Connection failed..", err)
panic(err)
}
conn.SetConnMaxLifetime(time.Minute * 4)
conn.SetMaxIdleConns(0)
defer conn.Close()
if err := conn.Ping(); err != nil {
log.Println("sql.Ping failed:", err.Error())
return nil
}

return conn

}`

I'm not sure what's happening and I'm not sure how to debug this. I'm open to suggestions on how to gather the right data.

I'm connecting to Sybase 11.0.3.3 and thda/tds seems to handle the TDS protocol this server version is using just fine normally. However I have one particular query that causes it to behave oddly, sometimes returning no data and sometimes entering an infinite loop or crashing.

I've simplified the failing query to select convert(real, @@timeticks) x. It does not fail if I do convert(real, 1) or just do select @@timeticks. It does fail with converting other @@ variables to real though. It also doesn't fail if I do convert(float(5), @@timeticks) but does fail if I do convert(float(4) @@timeticks)

I noted "real" is handled as a single precision float, I wonder if the TDS protocol isn't actually sending a double-width float though? Or perhaps the single precision parsing is just not quite right?

Greetings, I discovered this library from the Golang Weekly email newsletter. I am SO excited to see more drivers supporting Sybase (SAP) databases. I do a lot of work on ASE, and have fallen in love with go the last couple years, but it has always been tough to use go with Sybase products.

I've been using the gofreetds package by minus5 https://github.com/minus5/gofreetds, but it's major limitation for my team is that it that particular implementation requires the linux freetds installed on the system, whereas almost my whole team are Windows users.

I have a fork of that repo here https://github.com/andyedison/gofreetds with a couple small tweaks that let Sybase be used as the database/sql package.

Anyways, please let me know how I could help in making this package production ready!

Most SQL driver packages either export the driver struct directly, or export an instance of the driver struct. Note that database/sql package does not provide a way to obtain a driver.Driver for a given driver name outside of the sql package itself.

This is useful as it allows packages like instrumentedsql to wrap a driver to add instrumentation. For example https://github.com/luna-duclos/instrumentedsql/blob/f5ba2826f7af02db3068b0261754cb828a704237/examples/sql_example_test.go#L20 leverages the exported sqlite3.SQLiteDriver struct to create a new driver instance to wrap. This same pattern is used by the postgres driver pq where the struct is pq.Driver, go-mssqldb where the struct is mssqldb.Driver, and mysql as mysql.MySQLDriver. Oddly the go-oci8 library takes the approach of exporting an instance of the driver at oci8.Driver rather than the struct, but this still works fine as long as a sql driver.Driver can be obtained.

I'm not sure which of the two options is the most idiomatic, but more do seem to opt for exporting the struct itself rather than exporting an instance of the driver.

This project looks very promising. Thanks for working on it and sharing.

I'm trying to use this project to connect to a Sybase ASE 15.0.3 database. I have build the connection string as per the instructions. The sql.Open() command works with no error, but when I try db.Ping() or db.Query() I get an exception. I'm new to Go and am having a hard time finding helpful information from the error message. I am using the latest from master. Do you have any pointers? The error is below. Thanks,

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/thda/tds.(*session).login(0xc00055c340, 0xc00040c756, 0x23, 0xc00040c746, 0x6, 0xc00040c74d, 0x8, 0x0, 0x0, 0x0, ...)
/Users/user/Development/Go/src/github.com/thda/tds/session.go:226 +0x1bd6
github.com/thda/tds.newSession(0xc00040c756, 0x23, 0xc00040c746, 0x6, 0xc00040c74d, 0x8, 0x0, 0x0, 0x0, 0x0, ...)
/Users/user/Development/Go/src/github.com/thda/tds/session.go:124 +0x5fa
github.com/thda/tds.NewConn(0xc00040c740, 0x3d, 0x15477d8, 0xc0004a11c8, 0x10421e4)
/Users/user/Development/Go/src/github.com/thda/tds/driver.go:145 +0x124
github.com/thda/tds.(*sybDriver).Open(0x1ebb1e0, 0xc00040c740, 0x3d, 0xc0004a11d8, 0x10352c3, 0x10c6ddd, 0xc0004a11e8)
/Users/user/Development/Go/src/github.com/thda/tds/driver.go:177 +0x39

I am trying to use a prepared statement to run in my Sybase, with the following code:

cnxStr := "tds://xxxxx:[email protected]:2638/xxxx?charset=utf8"
db, err := sql.Open("tds", cnxStr)

fmt.Println("OK1")

if err != nil {
	fmt.Println(err)
}

fmt.Println("OK2")

rows, err := db.Query("SELECT nome_cta, codi_cta FROM bethadba.ctcontas WHERE codi_cta = ?", 1)
fmt.Println("OK3")

When I run this, it get stuck on the db.Query, it never prints out "OK3".

But if I run the same query:

db.Query("SELECT nome_cta, codi_cta FROM bethadba.ctcontas WHERE codi_cta = 1")

It runs perfectly.

I can't give you a lot of details, because it also doesn't return me an error.

Sybase version : 16.0.0.2754

Thank you so much for the help !

Func Close() sends logout packet, then immediately close the socket, without waiting to server's response.
It causes:

• At server: logged error messge A client process exited abnormally, or a network error was encountered.
• At network level: multiple TCP RST packets are sent back to the server, when logout response arrives

Works well on linux, but when I run my code from my macos laptop. I encountered this.

I'm trying to use this project to connect to a Sybase ASE 12.5.4 database.
cnxStr := "tds://sa:@192.168.211.128:5000/test?charset=cp850"
But it didn't work,the error is blow. Do you have any pointers? Thanks
tds: cannot encode to 'cp850' charset

We have an application that occasionally stuck and the stack trace shows that it goes into this branch and eventually at stuck at a network read in go runtime code.
https://github.com/thda/tds/blob/master/buffer.go#L202

May I know the intention of this readPkt call after EOF encountered?

We don't have any timeout params set as part of the conn string, so from the code, it looks like it is going to take defaultLoginTimeout = 20, which is in seconds. but it doesn't look like this timeout is kicking in for us either. The process is stuck for at least 2 hours - very likely at the exact same place.

From our investigation, we can see from the host netstat output that the connection is in ESTABLISHED state and from the Sybase Text Report that the DB server agrees with that.

I will try to post the stack trace, but it is a bit challenging to get it out from firm network. It is essentially coming from newSession(...) in session.go, then login(...), then initState(...)

First of all thanks for writing this driver that has solved alot for me, below is the issue that i am facing with this driver.

Issue:- Our Database server has 3 databases in it with default database as DBAs database that they use for maintainence, Now when i try to connect to my database, Driver always tries to connect to default Database even if i specify database name in connection string and as a result i always get error saying "Invalid UserName or Password" since my Database username and password do not works with default database.

We are using SQL Anywhere 17.0

Can this be looked upon please.

My Code is below:

package main

import (
	"database/sql"
	"fmt"
	"log"
	"net/url"
	"time"
	_ "github.com/thda/tds"
)

func main1() {
	Init()
}

func buildCnxStr() string {
	return "tds://" + url.QueryEscape("<UserName>") + ":" + url.QueryEscape("<Password>") +
		"@" + url.QueryEscape("<HostName>") + ":" + "<DatabaseName>" + "/" + "<DatabaseName>" + "?" + "charset=utf8"
}

func Init() {
	dbConntString := buildCnxStr()
	db, err := sql.Open("tds", dbConntString)
	db.SetMaxOpenConns(4)
	db.SetMaxIdleConns(1)
	db.SetConnMaxLifetime(60 * time.Second)
	if err != nil {
		log.Fatal(err)
	}

	var result string
	row, err := db.Query("select db_name()") // Here i am getting error
	if err != nil {
		fmt.Println("Error is: ", err)
	}
	for row.Next() {
		row.Scan(&result)
	}

	fmt.Println("Database Name is: ", result)
}

Hi , Thanks for creating the only go lib for Sybase.
Just wanted to check if we have the feature of doing batch select and batch insert .
For Ex in Java I can control the fetch size while doing select which can return million records:
DataSource iq;
Connection iqConnection = iq.getConnection();
PreparedStatement stmnt = iqConnection.prepareStatement(selectQuery);
stmnt.setFetchDirection(ResultSet.FETCH_FORWARD);
stmnt.setFetchSize(batchSize); // 10000
ResultSet result = stmnt.executeQuery()

Similarly for doing batch insert.

Is this something which is already present(I couldn't find it) or is it something in your road map?

Many Thanks

Hi!

trying to get the code working and running into this issue.

go: finding github.com/thda/tablewriter v0.0.0-20180207080603-aedbbc0800df
Username for 'https://github.com': 

We can do a go get github.com/thda/tds without any issues.

But when trying to use inside the project and doing go mod tidy, we get that prompt.

Is there is a dependency on the private repo above?

thank!

Code used to insert row(values are having numeric data type in database):
db.Exec("INSERT INTO table_name (col_a, col_b) VALUES (?, ?)", 1.23, 0)

Error Received: tds: error while scanning array of bytes to numeric: tds: could not parse string . to number

Debugged further and found that num.String() in line below is returning value = .

On further debugging into num.String() function, we found that the following IsInt() check is getting failed: