Comments (15)
mattn
commented on August 17, 2024
1
Should be fixed.
from go-oci8.
commented on August 17, 2024
Is it just me, or do the latest patches for support for clob/blob don't work when the data is greater than 4000 bytes?
from go-oci8.
mattn
commented on August 17, 2024
Could you please check rlen is over 4000?
https://github.com/mattn/go-oci8/blob/master/oci8.go#L621
from go-oci8.
commented on August 17, 2024
I added this:
@@ -598,6 +598,7 @@ func (s *OCI8Stmt) Query(args []driver.Value) (rows driver.Rows, err error) {
oci8cols[i].name = string((*[1 << 30]byte)(unsafe.Pointer(np))[0:int(ns)])
oci8cols[i].size = int(lp)
+ fmt.Printf("type:%d name:%s size:%d rlen:%d\n", oci8cols[i].kind, oci8cols[i].name, oci8cols[i].size, oci8cols[i].rlen)
var defp *C.OCIDefine
if tp == C.SQLT_CLOB || tp == C.SQLT_BLOB {
rv = C.OCIDescriptorAlloc(
The output of my code is currently this:
type:1 name:SCHEMA_NAME size:250 rlen:0
type:1 name:SCHEMA_VERSION size:22 rlen:0
type:1 name:XML_TYPE size:250 rlen:0
type:113 name:X.XML_DOCUMENT.GETBLOBVAL(178) size:4000 rlen:0
About to read row
Successfully read row: breadth_track.xsd
About to read row
Successfully read row: breadth_track_FOP.xsl
About to read row
Successfully read row: breadth_track_HTML.xsl
About to read row
Successfully read row: breadth_track_editor_display.xsl
SIGBUS: bus error
PC=0x383366c8
signal arrived during cgo execution
Note that it fails on rows.Next() not on rows.Scan():
66 for rows.Next() {
67 i := new(SchemaXml)
68 content := make([]byte, 50000)
69
70 fmt.Println("About to read row")
71 rows.Scan(&i.Type, &i.Version, &i.Filename, &content)
72 fmt.Printf("Successfully read row: %s\n", i.Filename)
from go-oci8.
commented on August 17, 2024
Here is when I print the size used during rows.Next(), it is always 4000. So for the first 4 results, it works because the actual size is less than 4000 bytes. If is assumedly failing the first time we hit a record with greater than 4000 bytes.
type:1 name:SCHEMA_NAME size:250 rlen:0
type:1 name:SCHEMA_VERSION size:22 rlen:0
type:1 name:XML_TYPE size:250 rlen:0
type:113 name:X.XML_DOCUMENT.GETBLOBVAL(178) size:4000 rlen:0
Preparing with size 4000
About to read row
Successfully read row: breadth_track.xsd
Preparing with size 4000
About to read row
Successfully read row: breadth_track_FOP.xsl
Preparing with size 4000
About to read row
Successfully read row: breadth_track_HTML.xsl
Preparing with size 4000
About to read row
Successfully read row: breadth_track_editor_display.xsl
Preparing with size 4000
SIGBUS: bus error
PC=0x4105aac8
signal arrived during cgo execution
This is having added another extra print statement to the driver:
779 case C.SQLT_BLOB, C.SQLT_CLOB:
780 var bamt C.ub4
781 fmt.Printf("Preparing with size %d\n", rc.cols[i].size)
782 b := make([]byte, rc.cols[i].size)
783 rv = C.OCILobRead(
from go-oci8.
commented on August 17, 2024
Heres my "hack" which allows me to extract all the data successfully. Doing this means I get all the data. With this hack every blob returned is now of length 100,000 instead of 4,000, and every blob is padded with a run of special characters which I can trim:
case C.SQLT_BLOB, C.SQLT_CLOB:
var bamt C.ub4
- b := make([]byte, rc.cols[i].size)
+ fmt.Printf("Preparing with size %d\n", rc.cols[i].size)
+ b := make([]byte, 100000)
+ //b := make([]byte, rc.cols[i].size)
rv = C.OCILobRead(
(*C.OCISvcCtx)(rc.s.c.svc),
(*C.OCIError)(rc.s.c.err),
@@ -785,7 +788,7 @@ func (rc *OCI8Rows) Next(dest []driver.Value) error {
&bamt,
1,
unsafe.Pointer(&b[0]),
- C.ub4(rc.cols[i].size),
+ C.ub4(100000), // C.ub4(rc.cols[i].size),
nil,
nil,
0,
from go-oci8.
mattn
commented on August 17, 2024
Thanks for your description. I'll look into it later.
from go-oci8.
commented on August 17, 2024
@mattn Awesome, thanks for your help. My ugly patch gets things at least working for now. Much appreciated.
from go-oci8.
commented on August 17, 2024
For reference, here is the patch i settled on that works around the problem. It increases the buffer size, and returns the correct number of bytes:
--- a/oci8.go
+++ b/oci8.go
@@ -774,10 +774,10 @@ func (rc *OCI8Rows) Next(dest []driver.Value) error {
buf := (*[1 << 30]byte)(unsafe.Pointer(rc.cols[i].pbuf))[0:rc.cols[i].rlen]
//TODO Handle BCE dates (http://docs.oracle.com/cd/B12037_01/appdev.101/b10779/oci03typ.htm#438305)
//TODO Handle timezones (http://docs.oracle.com/cd/B12037_01/appdev.101/b10779/oci03typ.htm#443601)
dest[i] = fmt.Sprintf("%v-%v-%v %v:%v:%v", ((int(buf[0])-100)*100)+(int(buf[1])-100), int(buf[2]), int(buf[3]), int(buf[4])-1,
case C.SQLT_BLOB, C.SQLT_CLOB:
var bamt C.ub4
- b := make([]byte, rc.cols[i].size)
+ b := make([]byte, rc.cols[i].size*100)
rv = C.OCILobRead(
(*C.OCISvcCtx)(rc.s.c.svc),
(*C.OCIError)(rc.s.c.err),
@@ -785,7 +785,7 @@ func (rc *OCI8Rows) Next(dest []driver.Value) error {
&bamt,
1,
unsafe.Pointer(&b[0]),
- C.ub4(rc.cols[i].size),
+ C.ub4(rc.cols[i].size*100),
nil,
nil,
0,
@@ -793,7 +793,15 @@ func (rc *OCI8Rows) Next(dest []driver.Value) error {
if rv == C.OCI_ERROR {
return ociGetError(rc.s.c.err)
}
- dest[i] = b
+ //if ociGetErrorCode(rc.s.c.err) != C.OCI_SUCCESS {
+ // fmt.Printf("%d ", ociGetErrorCode(rc.s.c.err))
+ // fmt.Printf("%s\n", ociGetError(rc.s.c.err))
+ //}
+ if int(bamt) < rc.cols[i].size*100 {
+ dest[i] = b[0:int(bamt)]
+ } else {
+ dest[i] = b
+ }
case C.SQLT_CHR, C.SQLT_AFC, C.SQLT_AVC:
buf := (*[1 << 30]byte)(unsafe.Pointer(rc.cols[i].pbuf))[0:rc.cols[i].rlen]
switch {
@@ -828,6 +836,20 @@ func ociGetError(err unsafe.Pointer) error {
return errors.New(s)
}
from go-oci8.
mattn
commented on August 17, 2024
I looked some implementation for using OCI8.
- perl driver using
*10size to fetch. - python driver getting specified size from user.
- ruby driver has limited less than 4000bytes.
from go-oci8.
commented on August 17, 2024
This document explains it:
http://docs.oracle.com/cd/A91202_01/901_doc/appdev.901/a89857/oci16m32.htm
It appears you either have to implement a "callback" handler for getting extra data longer than the initial 4000 bytes, or you need to check for an error status that indicates there is more data to read.
from go-oci8.
mattn
commented on August 17, 2024
OK, I'll implement callback in later.
from go-oci8.
ahjdzx
commented on August 17, 2024
Are you implemented the callback?
from go-oci8.
ahdong2007
commented on August 17, 2024
It seems it's not fixed. I can only get the first 4000 characters when working with CLOB.
from go-oci8.
MichaelS11
commented on August 17, 2024
@ahdong2007 Are you on the newest version?
If so and you are still having an issue, I would suggest opening up a new issue with minimal complete SQL, DDL, and Go code that reproduces the issue.
from go-oci8.
Related Issues (20)
- Context cancelation not being honored HOT 6
- error: not found libclntsh.so HOT 2
- [example] set instant client environment by a shell in linux(ubuntu) HOT 1
- OCIEnvCreate error HOT 3
- Can darwin/arm64 golang build this? HOT 1
- is there any fast bulk insert solutions like executemany() in python/node ? HOT 1
- How to connect as sysasm using our driver? HOT 1
- dyld: Library not loaded: @rpath/libclntsh.dylib.12.1
- compiling for linux using mac throws errors HOT 6
- window10 go build Windows to Linux failed，help me plz HOT 7
- Why is the length of lobbuffersize limited to 4000 and can it be increased to 10K or 100k
- Library not loaded: @rpath/libclntshcore.dylib.19.1 HOT 1
- Is there support for Kerberos authentication?
- Is this library maintained? HOT 1
- exit status 0xc0000135, golang with windows HOT 3
- oracle oci8 support mac arm64?,for example mac m1 HOT 4
- How to Execute Stored Procedure？
- Test oracle 19c TAC got error ORA-25412: transaction replay disabled by call to OCITransStart HOT 2
- How to avoid mistakes ORA-29275: 部分多字节字符
- Can the database stop the query statement being executed to cancel the blocking
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-oci8.