oracle / odpi Goto Github PK
View Code? Open in Web Editor NEWODPI-C: Oracle Database Programming Interface for Drivers and Applications
Home Page: https://oracle.github.io/odpi/
License: Other
ODPI-C: Oracle Database Programming Interface for Drivers and Applications
Home Page: https://oracle.github.io/odpi/
License: Other
When a dpiVar is created with DPI_NATIVE_TYPE_UINT64
and DPI_ORACLE_TYPE_NATIVE_INT
, it is defined or bound as SQLT_INT
. It should be SQLT_UIN
.
When a dpiVar is created with DPI_NATIVE_TYPE_UINT64
and DPI_ORACLE_TYPE_NUMBER
, OCINumberFromInt()
and OCINumberToInt
are called with OCI_NUMBER_SIGNED
. It should be OCI_NUMBER_UNSIGNED
.
dpiSubscr_prepareStmt doesn't increment openChildCount in dpiConn.
It should be incremented as dpiConn_prepareStmt does.
Or it should not be decremented when a dpiStmt created by dpiSubscr_prepareStmt is closed.
dpiStmt_getInfo() returns incorrect information about merge
.
I made a test program.
https://gist.github.com/kubo/d8b623072b755edd508896940be86e81
It printed:
SQL: merge info ...
isQuery: 0
isPLSQL: 0
isDDL: 0
isDML: 0
statementType: 16
isReturning: 0
Merge is DML but isDML
is zero and statementType is a value not listed here.
In this example the statement is not a valid SQL but it returns same results even for valid statements.
Well, I cannot say that this is a bug due to odpi. That's because OCI_STMT_MERGE
is not in OCI manual.
I hope that the Oracle team says 16 is the statement type of merge.
I find function:
// prepare a statement and return it for subsequent execution/fetching
int dpiConn_prepareStmt(dpiConn *conn, int scrollable, const char *sql,
uint32_t sqlLength, const char *tag, uint32_t tagLength,
dpiStmt **stmt);
that means when executing a sql, we should call dpiConn_prepareStmt one time.
will you some functions likes:
1.allocate dpiStmt
2.execute sql
3.get affectedrows
4.get resultset
5.close resultset
6.repeat 1-5
7.close dpiStmt
thinks.
Could you add precision and scale members to dpiObjectAttrInfo and dpiObjectTypeInfo?
dpiOracleTypeNum is in dpiObjectAttrInfo, dpiObjectTypeInfo and dpiQueryInfo structures. However precision and scale are only in dpiQueryInfo.
Language bindings may refer these members to determine data types in its language.
For example if a member of an object type is NUMBER(20)
, a language binding may get the data as integer. (I'm making ruby-odpi, which is not a replacement of ruby-oci8. I'm doing it just to know ODPI.)
I prefer to make a structure as follows and put it in dpiObjectAttrInfo, dpiObjectTypeInfo and dpiQueryInfo. But I could not imagine use cases using size and nullOk members except just information for users as in the #22 (comment) example. So I don't adhere to my idea.
struct dpiDataTypeInfo
dpiOracleTypeNum oracleTypeNum;
dpiNativeTypeNum defaultNativeTypeNum;
uint32_t dbSizeInBytes;
uint32_t clientSizeInBytes;
uint32_t sizeInChars;
int16_t precision;
int8_t scale;
int nullOk;
dpiObjectType *objectType;
};
@kubo has been busy (surprise!) and his ruby-odpi driver is taking shape at https://github.com/kubo/ruby-odpi
While preparing the Solaris Userland (https://github.com/oracle/solaris-userland) integration
of ODPI-C, one of my code reviewers asked about the stability of the API.
2.0.2
None
n/a
n/a
12.1 and 12.2, via IPS packages
Solaris 11.4 and 11.3
gcc 5.4
It would be nice if odpi would support ORACLE_HOME
instead of fiddling with LD_LIBRARY_PATH
for those who have the database or full client installed.
If anyone wants another 'bigger' example that uses ODPI-C, the first development branch of the Node.js node-oracledb interface is available, see the release announcement.
The code is currently on the dev-2.0 branch.
hello, I'm new
Could you please write a minimal example connecting to database and run a query select?
Thank you.
Would you have a plan to expose OCINumber as a native type?
It is required for drivers which implement JDBC-like API.
Assume the following code:
ResultSet rs = stmt.executeQuery("select NUMBER_COL from TABLE_NAME");
while (rs.next()) {
... use rs.getInt(1), rs.getLong(1), rs.getDouble(1) or rs.getBigDecimal(1) ...
}
The native type of NUMBER_COL
is not decided at stmt.executeQuery()
. It depends on methods such as getInt
after the column data is fetched. However such API cannot be implemented by ODPI-C, which must declare native types explicitly or implicitly at SQL execution time.
If ODPI-C exposes OCINumber as a native type and provides conversion functions between OCINumber and integer, double and text, drivers can convert the OCINumber to int, long, double and BigDecimal via text after it is fetched.
Do you have any plan to make dpiConn_startupDatabase()
accept pfile as Data Provider for .NET does?
The priority of this feature is extremely low. I sometimes use startup pfile=xxx
on sqlplus to check effect of initialization parameters without changing persistent parameters. However I could not imagine who use it in applications.
I'm trying to write a Go database/sql/driver from scratch (I've created a cx_Oracle-like driver back as goracle, and later joined rana/ora, rewritten as ora.v4, and now trying to rewrite it from scratch using ODPI-C. cx_Oracle's quality is what I'm chasing).
I've stuck right at the beginning: get the Data from a Query.
I use dpiConn_newVar to get a *dpiVar and an array of *dpiData, store them for later use
C.dpiStmt_setFetchArraySize(st.dpiStmt, fetchRowCount)
var info C.dpiQueryInfo
for i := 0; i < colCount; i++ {
if C.dpiStmt_getQueryInfo(st.dpiStmt, C.uint32_t(i+1), &info) == C.DPI_FAILURE {
return nil, st.getError()
}
typ, numTyp := info.oracleTypeNum, info.defaultNativeTypeNum
bufSize := maxArraySize * info.clientSizeInBytes
switch typ {
case C.DPI_ORACLE_TYPE_VARCHAR, C.DPI_ORACLE_TYPE_NVARCHAR, C.DPI_ORACLE_TYPE_CHAR, C.D
PI_ORACLE_TYPE_NCHAR:
bufSize *= 4
}
var dataArr *C.dpiData
if C.dpiConn_newVar(
st.conn.dpiConn, typ, numTyp, maxArraySize,
bufSize, 1, 0, info.objectType,
&r.vars[i], &dataArr,
) == C.DPI_FAILURE {
return nil, st.getError()
}
if C.dpiStmt_define(st.dpiStmt, C.uint32_t(i+1), r.vars[i]) == C.DPI_FAILURE {
return nil, st.getError()
}
r.data[i] = (*((*[maxArraySize]*C.dpiData)(unsafe.Pointer(dataArr))))[:]
}
if C.dpiStmt_addRef(st.dpiStmt) == C.DPI_FAILURE {
return &r, st.getError()
}
then I use dpi_fetch (tried dpi_fetchRows, with no avail)
var found C.int
if C.dpiStmt_fetch(r.dpiStmt, &found, &r.bufferRowIndex) == C.DPI_FAILURE {
return r.getError()
}
if found == 0 {
r.finished = true
return io.EOF
}
for i, col := range r.columns {
var typ C.dpiNativeTypeNum
var d *C.dpiData
if C.dpiStmt_getQueryValue(r.dpiStmt, C.uint32_t(i+1), &typ, &d) == C.DPI_FAILURE {
return r.getError()
}
if d.isNull == 1 {
dest[i] = nil
continue
}
switch typ {
case C.DPI_ORACLE_TYPE_VARCHAR, C.DPI_ORACLE_TYPE_NVARCHAR, C.DPI_ORACLE_TYPE_CHAR, C.D
PI_ORACLE_TYPE_NCHAR, C.DPI_ORACLE_TYPE_LONG_VARCHAR:
b := C.dpiData_getBytes(d)
fmt.Printf("b=%p[%d]\n", b.ptr, b.length)
if b.ptr == nil {
dest[i] = ""
continue
}
dest[i] = C.GoStringN(b.ptr, C.int(b.length))
...
Fails with SIGSEGV and when not, then data's bytes are zero.
any obvious failures?
is dpiStmt_define's position starts from 1?
what is bufferRowIndex? What shall I index it with? The array of dpiData? But then why is that needed in dpiStmt_fetch ?
I read cx_Oracle/src/Cursor.c, and it increments bufferRowIndex right after the fetch - why does this not skip the first value? As I printed out the dpiData array after the fetch, the first element is nil, so this indicates such thing, but I don't know.
Thanks in advance,
Tamás Gulácsi
What version of ODPI-C are you using (see dpi.h)?
2.0.0-beta.4
What error(s) you are seeing?
It's all my application code's fault.
objType := (*C.dpiObjectType)(C.malloc(C.sizeof_void))
if C.dpiConn_getObjectType(c.dpiConn, cName, C.uint32_t(len(name)), (**C.dpiObjectType)(unsafe.Pointer(&objType))) == C.DPI_FAILURE {
C.free(unsafe.Pointer(objType))
return nil, errors.Wrapf(c.getError(), "getObjectType(%q) conn=%p", name, c.dpiConn)
}
result:
$ go install && go test -run=Object
--- FAIL: TestObject (3.58s)
conn.go:147: msg="PrepareContext" shortcut="--GET_OBJECT_TYPE--"
stmt.go:235: msg="QueryContext" args=[]driver.NamedValue{driver.NamedValue{Name:"", Ordinal:1, Value:"test_pkg_obj.int_tab_typ"}}
z_test.go:710: ORA-04043: DPI-1002: invalid dpiStmt handle
getObjectType("test_pkg_obj.int_tab_typ") conn=0x1778470
gopkg.in/goracle%2ev2.(*conn).GetObjectType
/home/gthomas/src/gopkg.in/goracle.v2/obj.go:165
gopkg.in/goracle%2ev2.(*statement).QueryContext
here c.dpiConn is a proper *dpiConn. Why is an "invalid dpiStmt handle" error returned?
Just wanted to know whether OCIThread is supported in ODPI for now, I see no APIs offering multi-threading using OCIThread, don't know whether I am missing something here!
There are suggestions about debug message format: (1) thread ID and (2) date format.
When DPI_DEBUG_LEVEL was set, the following messages were printed on Linux.
ODPI [140274862778112] 2017/09/02 14:13:44.244: ref 0x167c1d0 (dpiPool) -> 1 [NEW]
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x17026b0 (dpiConn) -> 1 [NEW]
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x167c1d0 (dpiPool) -> 2
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x167c1d0 (dpiPool) -> 1
ODPI [140274862778112] 2017/09/02 14:13:44.282: ref 0x16ed170 (dpiVar) -> 1 [NEW]
...
The thread ID 140274862778112
is available to identify a thread but it is a too big number.
That's because it is an address of struct pthread
in the process, not a thread ID in the OS.
If it is a thread ID in the OS, the messages are printed as follows:
ODPI [11318] 2017/09/02 14:36:30.607: ref 0x151a1d0 (dpiPool) -> 1 [NEW]
ODPI [11318] 2017/09/02 14:36:30.696: ref 0x15a06b0 (dpiConn) -> 1 [NEW]
ODPI [11318] 2017/09/02 14:36:30.696: ref 0x151a1d0 (dpiPool) -> 2
ODPI [11318] 2017/09/02 14:36:30.697: ref 0x151a1d0 (dpiPool) -> 1
ODPI [11318] 2017/09/02 14:36:30.697: ref 0x158b170 (dpiVar) -> 1 [NEW]
...
Thread IDs in the OS is retrieved by syscall(SYS_gettid)
on Linux.
Thread IDs in the macOS seems to be retrieved by pthread_threadid_np.
It is available since OS X 10.6 Snow Leopard. The source of the man page about pthread_threadid_np is here but formatted page could not be found in the apple site.
I prefer YYYY-MM-DD
to YYYY/MM/DD
because the former is same with standards.
The latter is widely used in Japan. But I don't know whether it is acceptable in other countries. (I have not lived outside of Japan.)
When -Wextra
is added to CFLAGS, gcc complains many unused parameter warnings on Linux.
$ make
mkdir build
mkdir lib
gcc -c -Iinclude -O2 -g -Wall -fPIC -Wextra src/dpiConn.c -obuild/dpiConn.o
src/dpiConn.c: In function ‘dpiConn__decrementOpenChildCount’:
src/dpiConn.c:222:63: warning: unused parameter ‘error’ [-Wunused-parameter]
int dpiConn__decrementOpenChildCount(dpiConn *conn, dpiError *error)
^
gcc -c -Iinclude -O2 -g -Wall -fPIC -Wextra src/dpiContext.c -obuild/dpiContext.o
src/dpiContext.c: In function ‘dpiContext__initCommonCreateParams’:
src/dpiContext.c:33:58: warning: unused parameter ‘context’ [-Wunused-parameter]
int dpiContext__initCommonCreateParams(const dpiContext *context,
^
src/dpiContext.c:34:50: warning: unused parameter ‘error’ [-Wunused-parameter]
dpiCommonCreateParams *params, dpiError *error)
^
...
Could you suppress these warnings?
Add the following macro to src/dpiImpl.h
.
#ifdef __GNUC__
#define UNUSED __attribute((unused))
#else
#define UNUSED
#endif
And then use the macro after unused parameters as follows:
int dpiConn__decrementOpenChildCount(dpiConn *conn, dpiError *error UNUSED)
While examining the odpi source in preparation for creating an IPS packaged version, I came across the test suite's TestSuiteRunner.c file, which contains these lines:
98 // run executable and return success only if all tests pass
99 result = system(executableName);
This violates https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=2130132, aka " ENV33-C. Do not call system()".
The posix_spawn() function (see http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html) is available on Linux, Mac osx, Solaris and other UNIX-like systems. For MS Windows, one of the functions noted at https://en.wikipedia.org/wiki/Spawn_(computing) would be preferable.
Could you add lfprec and fsprec to dpiQueryInfo?
If they are added, users can know precisions of timestamp and interval data types.
There is no merit for driver developers. Drivers can handle these Oracle data types without precisions. This is just information for users.
ruby-oci8 can print such information as follows:
conn = OCI8.new('username/password')
conn.exec(<<EOS)
create table test_types (
col1 timestamp not null,
col2 timestamp(9),
col3 interval day to second,
col4 interval day(2) to second(3)
)
EOS
cursor = conn.exec('select * from test_types where 1 = 2')
# string representation of column types which could be put in create table statements
cursor.column_metadata[0].data_type_string # => TIMESTAMP NOT NULL
cursor.column_metadata[1].data_type_string # => TIMESTAMP(9)
cursor.column_metadata[2].data_type_string # => INTERVAL DAY TO SECOND
cursor.column_metadata[3].data_type_string # => INTERVAL DAY(2) TO SECOND(3)
I found a bug when I made a test code about #27.
dpiConn_newSubscription
overwrites additional four bytes after the address passed as the last argument.
Look at https://gist.github.com/kubo/394af98c4c541b2e069467a97f0aaec9#file-subscr-test-c-L51-L55.
If the last argument of dpiConn_newSubscription
is uint32_t *
, subscrId.next4byte
must be 0xdeadbeaf at line 53. However it became 0x00000000.
I guess that the type is uint64_t *
so the next four bytes were overwritten with 0x00000000.
When a timestamp with negative time zone minute offset such as -03:30
is fetched, dpiTimestamp.tzMinuteOffset
is negative. However when the fetched timestamp is set to dpiData_setTimestamp
, dpiStmt_execute
fails with ORA-01875: time zone minute must be between -59 and 59
.
I made a test program.
https://gist.github.com/kubo/3d8cfb06a71454d137e3014475e62674
It printed:
Timestamp value:
year: 2017
month: 1
day: 1
hour: 0
minute: 0
second: 0
fsecond 0
tzHourOffset: -3
tzMinuteOffset: -30
ERROR at line 84
dpiStmt_execute(stmt, 0, &numQueryColumns)
ORA-01875: time zone minute must be between -59 and 59
This issue is fixed if ts.tzMinuteOffset
in this line is replaced with abs(ts.tzMinuteOffset)
. However this is strange workaround.
Could you separate LDFLAGS
in Makefile to LDFLAGS
and LIBS
and use $LIBS
at the last position of the link command as follows?
Conventionally LDFLAGS
doesn't include dependent libraries and they are set to LIBS
.
from
LDFLAGS=-L$(OCI_LIB_DIR) -lclntsh -shared
... skip ...
$(LIB_DIR)/$(LIB_NAME): $(OBJS)
$(LD) $(LDFLAGS) $(LIB_OUT_OPTS) $(OBJS)
to
LDFLAGS=-shared
LIBS=-L$(OCI_LIB_DIR) -lclntsh
... skip ...
$(LIB_DIR)/$(LIB_NAME): $(OBJS)
$(LD) $(LDFLAGS) $(LIB_OUT_OPTS) $(OBJS) $(LIBS)
When odpi is compiled on Linux without the above changes, the -L$(OCI_LIB_DIR) -lcltnsh
options is set just after gcc
.
But it takes no effect as follows.
$ make
mkdir build
mkdir lib
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC src/dpiConn.c -obuild/dpiConn.o
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC src/dpiContext.c -obuild/dpiContext.o
... skip ...
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC src/dpiMsgProps.c -obuild/dpiMsgProps.o
gcc -c -Iinclude -I/opt/instantclient/sdk/include -O2 -g -Wall -fPIC src/dpiRowid.c -obuild/dpiRowid.o
gcc -L/opt/instantclient -lclntsh -shared -o lib/libodpic.so build/dpiConn.o build/dpiContext.o build/dpiData.o build/dpiEnv.o build/dpiError.o build/dpiGen.o build/dpiGlobal.o build/dpiLob.o build/dpiObject.o build/dpiObjectAttr.o build/dpiObjectType.o build/dpiPool.o build/dpiStmt.o build/dpiUtils.o build/dpiVar.o build/dpiOracleType.o build/dpiSubscr.o build/dpiDeqOptions.o build/dpiEnqOptions.o build/dpiMsgProps.o build/dpiRowid.o
$ ldd lib/libodpic.so
linux-vdso.so.1 => (0x00007ffd581f0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f788b44d000)
/lib64/ld-linux-x86-64.so.2 (0x00005572bb26f000)
libodpic.so
doesn't depend on libclntsh.so
.
If -L/opt/instantclient -lclntsh
is set at last. libodpic.so
depends on libclntsh.so
.
$ gcc -shared -o lib/libodpic.so build/dpiConn.o build/dpiContext.o build/dpiData.o build/dpiEnv.o build/dpiError.o build/dpiGen.o build/dpiGlobal.o build/dpiLob.o build/dpiObject.o build/dpiObjectAttr.o build/dpiObjectType.o build/dpiPool.o build/dpiStmt.o build/dpiUtils.o build/dpiVar.o build/dpiOracleType.o build/dpiSubscr.o build/dpiDeqOptions.o build/dpiEnqOptions.o build/dpiMsgProps.o build/dpiRowid.o -L/opt/instantclient -lclntsh
$ ldd lib/libodpic.so
linux-vdso.so.1 => (0x00007ffdeb6bd000)
libclntsh.so.12.1 => /opt/instantclient/libclntsh.so.12.1 (0x00007f0b285c5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0b281d8000)
libmql1.so => /opt/instantclient/libmql1.so (0x00007f0b27f61000)
libipc1.so => /opt/instantclient/libipc1.so (0x00007f0b27be3000)
libnnz12.so => /opt/instantclient/libnnz12.so (0x00007f0b274d9000)
libons.so => /opt/instantclient/libons.so (0x00007f0b27293000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0b2708f000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0b26d86000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0b26b68000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f0b2694f000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0b26747000)
/lib64/ld-linux-x86-64.so.2 (0x0000562a4e5cf000)
libaio.so.1 => /lib/x86_64-linux-gnu/libaio.so.1 (0x00007f0b26544000)
libclntshcore.so.12.1 => /opt/instantclient/libclntshcore.so.12.1 (0x00007f0b25fd2000)
Off-topic: When I write Makefile, I won't use EXTRA_CFLAGS
. I'll use CFLAGS += -DDPI_TRACE_REFS
instead.
The changes to the dpiDeqOptions__getAttrValue
function where the valueLength is checked for null cause the dpiDeqOptions_getWait
call to always fail because the valueLength is hardcoded to NULL.
int dpiDeqOptions_getWait(dpiDeqOptions *options, uint32_t *value)
{
return dpiDeqOptions__getAttrValue(options, DPI_OCI_ATTR_WAIT, __func__,
value, NULL);
}
Similar case in dpiMsgProps
int dpiMsgProps_getNumAttempts(dpiMsgProps *props, int32_t *value)
{
return dpiMsgProps__getAttrValue(props, DPI_OCI_ATTR_ATTEMPTS, __func__,
value, NULL);
}
Those two fell out of my tests, but a quick glance at some other functions where a numeric is the out value have the same issue.
Could you add LIBS=-ldl -lpthread
to Makefile
and remove -ldl -lpthread
in test/Makefile
and samples/Makefile
?
When libodpic.so depends on libdl and libpthread, there is no need to link with -ldl -lpthread
to build programs depending on libodpic.so.
How about custom error message when Visual Studio distributable package is not installed?
IMO, oracle/node-oracledb#404 (comment) should be checked in ODPI-C.
When GetLastError() returns ERROR_MOD_NOT_FOUND(The specified module could not be found), do the following.
GetModuleFileName(NULL, buf, sizeof(buf))
and truncate the file name.)PATH
.The following is a sample function which prints bit and dependent DLL files.
It is easy to customize this to fit this issue's need.
// link dbghelp.lib also
#include <dbghelp.h>
static void print_import_dlls(const char *name)
{
HANDLE hFile = CreateFile(name, GENERIC_READ, FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
HANDLE hFileMapping = NULL;
void *base = NULL;
if (hFile == INVALID_HANDLE_VALUE) {
printf("failed to open: %s\n", name);
return;
}
hFileMapping = CreateFileMapping(hFile, NULL, PAGE_READONLY, 0, 0, NULL);
if (hFileMapping == NULL) {
printf("failed to create file mapping: %s\n", name);
goto cleanup;
}
base = MapViewOfFile(hFileMapping, FILE_MAP_READ, 0, 0, 0);
if (base == NULL) {
printf("failed to map file: %s\n", name);
goto cleanup;
}
IMAGE_NT_HEADERS *nt_hdr = ImageNtHeader(base);
if (nt_hdr == NULL) {
printf("not executable nor DLL: %s\n", name);
goto cleanup;
}
printf("%s (%s bit):\n", name, nt_hdr->FileHeader.Machine == IMAGE_FILE_MACHINE_I386 ? "32" : "64");
ULONG uSize = 0;
IMAGE_IMPORT_DESCRIPTOR *desc = (IMAGE_IMPORT_DESCRIPTOR *)ImageDirectoryEntryToDataEx(base, FALSE, IMAGE_DIRECTORY_ENTRY_IMPORT, &uSize, NULL);
while (desc->Characteristics != 0) {
char *name = (char*)ImageRvaToVa(nt_hdr, base, desc->Name, NULL);
printf(" %s\n", name);
desc++;
}
cleanup:
if (base != NULL) {
UnmapViewOfFile(base);
}
if (hFileMapping != NULL) {
CloseHandle(hFileMapping);
}
CloseHandle(hFile);
}
Is it guaranteed that dpiLob_readBytes() reads characters up to the specified amount if it doesn't reach the end of the LOB?
In other words, does the following code work? (Error checks are omitted for simplicity.)
dpiLob *lob = ...;
FILE *fp = ...;
const uint64_t char_size = 8192;
uint64_t byte_size;
char *buf;
uint64_t offset = 1;
dpiLob_getBufferSize(lob, char_size, &byte_size);
buf = malloc(byte_size);
while (1) {
uint64_t read_byte_len = byte_size;
dpiLob_readBytes(lob, offset, char_size, buf, &read_byte_len);
if (read_byte_len == 0) {
break;
}
fwrite(buf, 1, read_byte_len, fp);
offset += char_size;
}
This assume that dpiLob_readBytes() always reads 8192 characters if it doesn't reach the end of the LOB. If this assumption is incorrect, could you change dpiLobReadBytes() or add a new function which returns the number of read characters as OCILobRead2() does? Otherwise, it must be counted in the caller to increment offset
as follows.
offset += number_of_characters_in_buf(buf, read_byte_len);
I am evaluating ODPI-C for use in our project using Master 2.1.0-dev
We have an Oracle custom data type that is a VARRAY of BINARY_DOUBLE[60].
I am following the TestFetchObjects.c example.
Not able to retrieve the array attributes from the object. After executing the query i get the query info and object type info
if (dpiStmt_getQueryInfo(stmt, 29, &queryInfo) < 0)
return dpiSamples_showError();
printf("('%*s', %d, %d, %d, %d, %d, %d)\n",
queryInfo.nameLength,
queryInfo.name,
queryInfo.typeInfo.oracleTypeNum,
queryInfo.typeInfo.sizeInChars,
queryInfo.typeInfo.clientSizeInBytes,
queryInfo.typeInfo.precision,
queryInfo.typeInfo.scale,
queryInfo.nullOk);
if (dpiObjectType_getInfo(queryInfo.typeInfo.objectType, &typeInfo) < 0)
return dpiSamples_showError();
printf("Fetching objects of type %.*s.%.*s\n is collection %d\n number of attributes %d\n",
typeInfo.schemaLength,
typeInfo.schema,
typeInfo.nameLength,
typeInfo.name,
typeInfo.isCollection,
typeInfo.numAttributes);
And this is the output
('VECTOR_COVAR', 2023, 0, 0, 0, 0, 1)
Fetching objects of type DATA_OWNER.SV_VARRAY
is collection 1
number of attributes 0
Notice that the number of attributes is zero. That is not correct. And I am unable to get values.
dpiObjectType_getAttributes()
dpiObjectAttr_getInfo()
dpiObject_getAttributeValue()
I am using Instant Client 12.2 installed locally
My development environment is Linux RHEL6
Does ODPI-C support VARRAY? If so, can you point me to an example?
For those interested in viewing an example of a driver implemented using ODPI-C you can take a look at the Python Oracle Database driver cx_Oracle. You can compare the code between the 5.x branch (direct OCI) and master branch (ODPI-C) to see the differences in the two approaches. I believe you'll see that the code is considerably simpler with ODPI-C.
Please init 0 for last byte while initialize xxx->value.asBytes.ptr. It always display the weird thing at the last character. For example
strIngColval->value.asBytes.ptr[strIngColval->value.asBytes.length] = 0;
==27524== Use of uninitialised value of size 8
==27524== at 0x9D3DE91: r0_des_encp_k8_64 (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9D13958: r0_cipher_cbc8lup_enc (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9D13C3B: r0_cipher_cbc8lup (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9D13C7A: R1_CIPH_CTX_cipher (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9CFEB89: r_ck_cipher_update (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF6DFB: ztca_RSAAdpEncrypt (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9E36FA4: ztcebn (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9E360C8: ztcecrypto_2 (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BDEE0B: ztceenc (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF8914: ztcrbm (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF838F: ztcrbh (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== by 0x9BF8136: ztcrbp (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524== Uninitialised value was created by a stack allocation
==27524== at 0x9BF8586: ztcrsg (in /opt/oracle/instantclient_12_2/libnnz12.so)
==27524==
==27524==
==27524== More than 1000 different errors detected. I'm not reporting any more.
==27524== Final error counts will be inaccurate. Go fix your program!
==27524== Rerun with --error-limit=no to disable this cutoff. Note
==27524== that errors may occur in your program without prior warning from
==27524== Valgrind, because errors are no longer being displayed.
==27524==
Done.
==27524==
==27524== HEAP SUMMARY:
==27524== in use at exit: 1,228,491 bytes in 368 blocks
==27524== total heap usage: 2,725 allocs, 2,357 frees, 2,084,971 bytes allocated
==27524==
==27524== 43 bytes in 1 blocks are definitely lost in loss record 100 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB46A115: slzsetevar (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB46E4DA: lfvSetOHome (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33EB3C: slpmloclfv (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33E6A1: slpmloc (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C047: lpmloadpkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E09A: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D384: lfvinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0x63BD768: kpummpin (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524== by 0x5E486A3: kpuenvcr (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524==
==27524== 192 bytes in 1 blocks are possibly lost in loss record 203 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB31755E: sltsmxi (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB3380F6: lmmhpinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB336FD6: lmmcis (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C589: lpmpali (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33B7F7: lpminitm (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E041: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D384: lfvinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0x63BD768: kpummpin (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524== by 0x5E486A3: kpuenvcr (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524==
==27524== 192 bytes in 1 blocks are possibly lost in loss record 204 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB31755E: sltsmxi (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB3380F6: lmmhpinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB336FD6: lmmcis (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C589: lpmpali (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C13C: lpmloadpkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E09A: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D384: lfvinit (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0x63BD768: kpummpin (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524== by 0x5E486A3: kpuenvcr (in /opt/oracle/instantclient_12_2/libclntsh.so.12.1)
==27524==
==27524== 139,264 bytes in 1 blocks are possibly lost in loss record 297 of 298
==27524== at 0x4C29BE3: malloc (vg_replace_malloc.c:299)
==27524== by 0xB33AB14: lmmstvrt (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB339585: lmmstchnk (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33A909: lmmstsml (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB55516E: lmmstmalloc (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB554885: lmmmalloc (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB336EC5: lmmcis (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33C589: lpmpali (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB33B7F7: lpminitm (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31E041: lfvLoadPkg (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31DD45: lfvSetShlMode (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524== by 0xB31D73F: lfvini1 (in /opt/oracle/instantclient_12_2/libclntshcore.so.12.1)
==27524==
==27524== LEAK SUMMARY:
==27524== definitely lost: 43 bytes in 1 blocks
==27524== indirectly lost: 0 bytes in 0 blocks
==27524== possibly lost: 139,648 bytes in 3 blocks
==27524== still reachable: 1,088,800 bytes in 364 blocks
==27524== suppressed: 0 bytes in 0 blocks
==27524== Reachable blocks (those to which a pointer was found) are not shown.
==27524== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==27524==
==27524== For counts of detected and suppressed errors, rerun with: -v
==27524== ERROR SUMMARY: 1129 errors from 1004 contexts (suppressed: 0 from 0)
-bash-4.2$
dpiObjectAttr_getInfo() returns incorrect dpiOracleTypeNum about NVARCHAR2 attributes.
I made a test program which gets attribute information about udt_ObjectDataTypes.
https://gist.github.com/kubo/17189baff057bb8fdd5797ae80750518
It printed:
STRINGCOL - 2001
UNICODECOL - 2001
FIXEDCHARCOL - 2003
FIXEDUNICODECOL - 2004
INTCOL - 2010
NUMBERCOL - 2010
DATECOL - 2011
TIMESTAMPCOL - 2012
TIMESTAMPTZCOL - 2013
TIMESTAMPLTZCOL - 2014
BINARYFLTCOL - 2007
BINARYDOUBLECOL - 2008
The datatype of UnicodeCol is nvarchar2(60). Its oracleTypeNum should be 2002(DPI_ORACLE_TYPE_NVARCHAR). However it is 2001(DPI_ORACLE_TYPE_VARCHAR).
Hi,
ODPI seems to be a really good initiave :)
I'm the author of OCILIB and I had a quick look at odpi code in order to check if moving OCILIB code to ODPI could be an option :)
1 - Why restrict Oracle client version to 11gR2 and above ?
Lot's of application using Oracle often reliy on heterogenous Oracle versions and often not the latest ones. Imposing this limit is a very limiting from my point of view
2 - Why not permitting runtime loading of OCI shared libraries ?
Can be limiting, especially on Windows platforms
2 - Has it been heavily tested with Environment set to UTF16 ?
OCI had shown a numerous amount of bugs over the years in Unicode mode.
And Some are still present many years after been reported.
2 examples : Retrieving column name form a describe call is leaking, calling OCIObjectSetAttr() and OCIObjectGetAttr() still fail.
In OCILIB, I had to find workarounds (sometimes ugly) for these issues (re-implementing OCIObjectSetAttr/OCIObjectGetAttr for example).
Shall odpi try to "compensate" OCI internal issues or shall it remain a generic wrapper on top of it ?
In latter case, trying to concentrate any driver implementation on odpi seems to me preventing from using OCI in UTF16 mode in real life.
3 - Lack of support for some types such as collections
Nested tables and varrays seems to not be supported.
Do you plan to support them in a short term time frame ?
Do you have any documented plans/roadmap ?
In its current state, some OCI features seems to not be supported yet : Direct Path, full SQL/PLSQL type support, user callback tracing, ....
The only purpose of these questions is to find out if I could make OCILIB based on ODPI or not at some point :)
Best regards,
Vincent
When incorrect-architecture OCI.DLL is in the PATH and correct-architecture OCI.DLL isn't there, GetLastError() returns ERROR_BAD_EXE_FORMAT(%1 is not a valid Win32 application). How about custom error message saying "c:\full\path\of\OCI.DLL is 32-bit, which isn't available in x64 programs" or so in this case?
This will reduce issues such as oracle/node-oracledb#752.
Architecture could be checked by nt_hdr->FileHeader.Machine
in #48 (comment) or by the following function.
#include <windows.h>
static void print_arch(const char *name)
{
FILE *fp = fopen(name, "rb");
IMAGE_DOS_HEADER dos_hdr;
IMAGE_NT_HEADERS nt_hdr;
if (fp == NULL) {
printf("failed to open: %s\n", name);
return;
}
fread(&dos_hdr, sizeof(dos_hdr), 1, fp);
if (dos_hdr.e_magic != IMAGE_DOS_SIGNATURE) {
printf("invalid DOS signature: 0x%x\n", dos_hdr.e_magic);
fclose(fp);
return;
}
fseek(fp, dos_hdr.e_lfanew, SEEK_SET);
fread(&nt_hdr, sizeof(nt_hdr), 1, fp);
fclose(fp);
if (nt_hdr.Signature != IMAGE_NT_SIGNATURE) {
printf("invalid NT signature: 0x%x\n", nt_hdr.Signature);
return;
}
switch (nt_hdr.FileHeader.Machine) {
case IMAGE_FILE_MACHINE_I386:
printf("architecture is x86.\n");
break;
case IMAGE_FILE_MACHINE_AMD64:
printf("architecture is x64.\n");
break;
case IMAGE_FILE_MACHINE_IA64:
printf("architecture is IA64.\n");
break;
default:
printf("invalid architecture: 0x%x\n", nt_hdr.FileHeader.Machine);
break;
}
}
<<<
#include <stdio.h>
#include <dpi.h>
int main()
{
dpiErrorInfo errorInfo;
dpiContext *gContext = NULL;
if (dpiContext_create(DPI_MAJOR_VERSION, DPI_MINOR_VERSION, &gContext, &errorInfo) < 0) {
fprintf(stderr, "ERROR: %.*s (%s : %s)\n", errorInfo.messageLength, errorInfo.message, errorInfo.fnName, errorInfo.action);
return -1;
}
return 0;
}
in the CMakeList.txt:
<<<
cmake_minimum_required(VERSION 3.7)
project(odpi)
set(CMAKE_C_STANDARD 11)
set(SOURCE_FILES
main.c
src/dpiConn.c
src/dpiContext.c
src/dpiData.c
src/dpiDeqOptions.c
src/dpiEnqOptions.c
src/dpiEnv.c
src/dpiError.c
src/dpiGen.c
src/dpiGlobal.c
src/dpiLob.c
src/dpiMsgProps.c
src/dpiObject.c
src/dpiObjectAttr.c
src/dpiObjectType.c
src/dpiOci.c
src/dpiOracleType.c
src/dpiPool.c
src/dpiRowid.c
src/dpiStmt.c
src/dpiSubscr.c
src/dpiUtils.c
src/dpiVar.c
)
add_executable(odpi ${SOURCE_FILES})
When I try to build project I get the following error:
ERROR: DPI-1047: Oracle Client library cannot be loaded: No such file or directory. See https://oracle.github.io/odpi/doc/installation.html for help (dpiContext_create : load library)
How I can load client library?
LONG
corresponds to DPI_ORACLE_TYPE_LONG_VARCHAR.
LONG RAW
corresponds to DPI_ORACLE_TYPE_LONG_RAW.
What Oracle data type corresponds to DPI_ORACLE_TYPE_LONG_NVARCHAR?
Could you set -Wl,-rpath,$(OCI_LIB_DIR)
to LDFLAGS
on Linux and -R,$(OCI_LIB_DIR)
on Solaris in addition to macOS?
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S), Linux)
LDFLAGS += -Wl,-rpath,$(OCI_LIB_DIR)
endif
ifeq ($(UNAME_S), SunOS)
LDFLAGS += -R,$(OCI_LIB_DIR)
endif
If rpath is set, client programs depending on odpi have no need to use LD_LIBRARY_PATH to know the directory containing Oracle client libraries.
Hello,
I what to use the driver in a multithreaded project. So I created a pool and I am acquiring a connection from the pool in order to create a statement and then to execute this statement. However the code crashes when sending multiple request (via an http interface). It never crashes so far when sending the requests one by one.
The method where the crash begins is:
int test_method( ... , *context ) {
if (dpiPool_acquireConnection(pool, NULL, 0, NULL, 0, NULL, &conn) < 0 )
return ShowError(context);
if( conn ) {
dpiConn_release(conn) ;
}
}
and the pool is created like this
dpiErrorInfo errorInfo;
dpiContext *context;
if (dpiContext_create(DPI_MAJOR_VERSION, DPI_MINOR_VERSION, &context, &errorInfo) < 0) {
return -1;
}
dpiCommonCreateParams commonParams;
dpiPoolCreateParams createParams;
dpiPool *pool;
if (dpiContext_initCommonCreateParams(context, &commonParams) < 0) {
std::cout << "cannot init common params" << std::endl;
return 1;
}
commonParams.createMode = DPI_MODE_CREATE_THREADED;
if (dpiContext_initPoolCreateParams(context, &createParams) < 0)
{
std::cout << "cannot init the params" << std::endl;
return 1; //dpiTestCase_setFailedFromError(testCase);
}
createParams.minSessions = 50; //MINSESSIONS;
createParams.maxSessions = 100; //MAXSESSIONS;
createParams.sessionIncrement = 20; //SESSINCREMENT;
if (dpiPool_create(context, "DB", strlen("DB"),
"DB", strlen("DB"), "1.7.7.16:1521/DEV",
strlen("1.7.7.16:1521/DEV"), NULL, &createParams, &pool) < 0)
return 1; //dpiTestCase_setFailedFromError(testCase);
Is there a problem with the way I am creating the pool or the problem is one the way I am acquiring the connection?
I am working with oracle 12c in a Linux machine, and as I said the app works w/o problem when the requests are made sequentially.
Looks like you have new implementations announcements here, so I'll toss mine in the mix.
https://github.com/rustyhorde/mimir
Still a work in progress, but the initial implementation is complete. Needs documentation.
I use the go-goracle/goracle driver for a golang project. This version uses the version 2.0.3 of ODPI-C. The program is running on Centos 7 and I'm use the Instant Client 12.2.0.1 from Oracle.
The program is running as expected, however when I try to stop the program with pressing Ctrl+C
the program is calling my Close()
method as expected but than the program hangs forever.
After a while (approximately 10 minutes) I open a second terminal and send kill -ABRT <pid>
and than the stacktrace below will be printed.
goroutine 62 [syscall, locked to thread]:
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle%2ev2._Cfunc_dpiConn_release(0x46da9c0, 0x0)
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle.v2/_obj/_cgo_gotypes.go:1448 +0x49
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle%2ev2.(*conn).Close.func1(0x46da9c0, 0x2f5d950)
/home/mguggi/work/beats/src/github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle.v2/conn.go:112 +0x60
github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle%2ev2.(*conn).Close(0xc420230300, 0x0, 0x0)
/home/mguggi/work/beats/src/github.com/mguggi/oraclebeat/module/oracle/vendor/gopkg.in/goracle.v2/conn.go:112 +0xa9
database/sql.(*driverConn).finalClose.func2()
/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:467 +0x49
database/sql.withLock(0x405f9e0, 0xc420011ce0, 0xc42019bd38)
/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:2867 +0x65
database/sql.(*driverConn).finalClose(0xc420011ce0, 0xc42019be58, 0xc42019be48)
/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:465 +0x128
database/sql.(finalCloser).(database/sql.finalClose)-fm(0xc42008d108, 0xc420346ea0)
/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:564 +0x2f
database/sql.(*DB).Close(0xc42008d0e0, 0x0, 0x0)
/home/mguggi/public/go-1.9.2/src/database/sql/sql.go:678 +0x31b
github.com/mguggi/oraclebeat/module/oracle/sgastats.(*MetricSet).Close(0xc42008d7c0, 0x406ec20, 0xc42008d7c0)
/home/mguggi/go/src/github.com/mguggi/oraclebeat/module/oracle/sgastats/sgastats.go:110 +0xd7
github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module.(*metricSetWrapper).close(0xc420118d40, 0x0, 0x0)
/home/mguggi/go/src/github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:257 +0x64
github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module.(*Wrapper).Start.func1(0xc4200193e0, 0xc420020ba0, 0xc4202504e0, 0xc420118d40)
/home/mguggi/go/src/github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:111 +0xbe
created by github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module.(*Wrapper).Start
/home/mguggi/go/src/github.com/mguggi/oraclebeat/vendor/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:106 +0x146
I've already opened the issue on the driver repository #33 but in the driver code conn.go
the last call ist the the C function dpiConn_release
.
When I execute the procedure above and create an tcpdump on my interface and port 1521 I receive the following line:
153 31.614966 192.168.12.12 172.16.1.108 TCP 67 57578 → 1521 [PSH, ACK, URG] Seq=2790 Ack=87582 Win=182272 Urg=1 Len=1 TSval=32936936 TSecr=717866813[Malformed Packet: TNS]
The URG flag is set when I pressing Ctrl+C
...
But the curious thing is that this code works when the database is running on the same host. It looks like a vpn tunnel or firewall issue ... maybe???
Additionally the code works when I remove the db.Close()
stuff from the ´Close()` method, but this is probably not the best idea.
Is there a way or a procedure how to activate a trace so that I can further narrow the root cause?
Followers of ODPI-C will have seen questions from @tgulacsi who was working on a Go driver using ODPI-C. He has it up at https://gopkg.in/goracle.v2 with the code at https://github.com/go-goracle/goracle/tree/v2.0.0
Do you have any plan to add soname such as libodpic.so.2
?
libodpic.so
with -hlibodpic.so.2
linker flag.libodpic.so
as libodpic.so.2.0.x
.libodpic.so.2
and libodpic.so
which point to libodpic.so.2.0.x
.libodpic.so.3
.Soname gets it anticipated that customers might want to have incompatible versions such as ODPI-C 2.x and ODPI-C 3.x installed. (the second question in #42)
ODPI-C doesn't seem to support Oracle object subtype properly.
Consider the table created by the following statements.
create type test_inheritance_base as object (
name varchar2(30)
) not final
/
create type test_inheritance_sub under test_inheritance_base (
surname varchar2(30)
)
/
create table test_inheritance (id integer, name test_inheritance_base);
insert into test_inheritance values (1, test_inheritance_base('John'));
insert into test_inheritance values (2, test_inheritance_sub('John', 'Smith'));
commit;
When the name column where id = 1 is fetched, its type is same with the type information retrieved by dpiStmt_getQueryInfo.
But when the name column where id = 2 is fetched, there is no way to know its actual type.
IMO, the problem will be fixed if dpiObject_getObjectType(), which internally uses OCIObjectGetTypeRef and OCIDescribeAny with OCI_OTYPE_REF, is added.
It may be better to add isFinalType member to dpiObjectTypeInfo to check whether the type is a final type. When a column in a query is a final type, there is no need to call dpiObject_getObjectType() for each fetched value. When it isn't a final type, dpiObject_getObjectType() must be called for each value.
HI,
I need some help: I want to bind PL/SQL array types - use of DBMS_OUTPUT.get_lines works, so OUT is OK.
For IN OUT types (TYPE vc_tab_typ IS TABLE OF VARCHAR2(100) INDEX BY PLS_INTEGER; PROCEDURE x(p_vc IN OUT NOCOPY vc_tab_typ)
),
dpiConn_newVar" typ=2001 natTyp=3004 arraySize=2 bufSize=32767 isArray=1
,dpiVar_setFromBytes
pos=1 then pos=2, and then calldpiVar_setNumElementsInArray
n=2dpiStmt_bindByPos
dpiStmt_execute
result is ORA-06513: PL/SQL: index for PL/SQL table out of range for host language array
stmt.go:259: enter="bindVars" args=[]driver.NamedValue{driver.NamedValue{Name:"", Ordinal:1, Value:sql.Out{_Named_Fields_Required:struct {}{}, Dest:(*[]string)(0xc42000c380), In:true}}}
stmt.go:341: msg="bindVars" i=0 in=true out=true value="[]string []string{\"string\", \"bring\"}"
stmt.go:482: msg="newVar" i=0 plSQLArrays=true typ=2001 natTyp=3004 sliceLen=2 bufSize=32767
conn.go:179: C="dpiConn_newVar" conn=&goracle._Ctype_struct_dpiConn{} typ=2001 natTyp=3004 arraySize=2 bufSize=32767 isArray=1 v=(*goracle._Ctype_struct_dpiVar)(nil)
stmt.go:511: msg="set" i=0 j=1 n=2 v="string=\"string\""
stmt.go:670: C="dpiVar_setFromBytes" dv=&goracle._Ctype_struct_dpiVar{} pos=1 p=(*goracle._Ctype_char)(0xc4200126a8) len=6
stmt.go:511: msg="set" i=0 j=2 n=2 v="string=\"bring\""
stmt.go:670: C="dpiVar_setFromBytes" dv=&goracle._Ctype_struct_dpiVar{} pos=2 p=(*goracle._Ctype_char)(0xc420012710) len=5
stmt.go:519: C="dpiVar_setNumElementsInArray" n=2
stmt.go:530: C="dpiStmt_bindByPos" dpiStmt=&goracle._Ctype_struct_dpiStmt{} i=0 v=&goracle._Ctype_struct_dpiVar{}
stmt.go:174: C="dpiStmt_execute" mode=0x20 colCount=0x0
z_test.go:236: ORA-06513: PL/SQL: index for PL/SQL table out of range for host language array
ORA-06512: at line 1
dpiStmt_execute(mode=32 arrLen=1024)
gopkg.in/goracle%2ev2.(*statement).ExecContext
/home/gthomas/src/gopkg.in/goracle.v2/stmt.go:179
I've tried ofsetting the pos at dpiVar_setFromBytes, changing n of setNumElementsInArray, resulting in different errors.
What am I doing wrong?
Thanks in advance,
Tamás Gulácsi
I would like to make a question regarding performance
Below a small "demo" file that is used to measure execution time. I want to update a sequence with nextvval 100K times.
CREATE SEQUENCE "TEST_SEQ"
MINVALUE 1
MAXVALUE 10000000000
INCREMENT BY 1
START WITH 1
CACHE 200
NOORDER
CYCLE;
The sequence is created with the above script, and using the workbench like this
declare local_1 INT;
BEGIN
FOR i IN 1..100000 LOOP
select TEST_SEQ.nextval into local_1 from dual;
END LOOP;
END;
/
it takes less than 2 seconds for the counter to reach 100K.
However the following code wants more than 7 seconds to reach 10K (an order of magnitude less).
I am wondering if there is something better I can do, or if I miss something. Is it possible using the library to reach better number, ie 2K-3K per second?
In order to compile please use
g++ t_main.cpp -std=c++11 -lboost_system -lboost_thread -lodpic -ldl
I used a simple thread pool from boost.
#include <boost/array.hpp>
#include <iostream>
#include <boost/asio/io_service.hpp>
#include <boost/bind.hpp>
#include <boost/thread/thread.hpp>
#include <boost/chrono/thread_clock.hpp>
#include <memory>
#include <chrono>
dpiContext *context;
dpiPool *pool;
int ShowError(dpiContext *context)
{
return 1;
}
int icalc(int x) {
dpiConn *conn;
dpiStmt *stmt;
uint32_t numQueryColumns;
const char *sql = "select TEST_SEQ.nextval from dual";
if (dpiPool_acquireConnection(pool, NULL, 0, NULL, 0, NULL, &conn) < 0 )
return ShowError(context);
if (dpiConn_prepareStmt(conn, 0, sql, strlen(sql), NULL, 0, &stmt) < 0)
return ShowError(context);
if ( dpiStmt_execute(stmt, DPI_MODE_EXEC_DEFAULT, &numQueryColumns) < 0)
return ShowError(context);
if ( dpiStmt_release(stmt) < 0 )
return ShowError(context);
if( conn ) {
dpiConn_release(conn) ;
return 0; //"Hello world";
} else {
return 1; //"err";
}
}
int main(int argc, char** argv)
{
dpiErrorInfo errorInfo;
//dpiContext *context;
if (dpiContext_create(DPI_MAJOR_VERSION, DPI_MINOR_VERSION, &context, &errorInfo) < 0) {
return -1;
}
dpiCommonCreateParams commonParams;
dpiPoolCreateParams createParams;
//dpiPool *pool;
if (dpiContext_initCommonCreateParams(context, &commonParams) < 0) {
std::cout << "cannot init common params" << std::endl;
return 1;
}
commonParams.createMode = DPI_MODE_CREATE_THREADED;
if (dpiContext_initPoolCreateParams(context, &createParams) < 0)
{
std::cout << "cannot init the params" << std::endl;
return 1; //dpiTestCase_setFailedFromError(testCase);
}
createParams.minSessions = 15; //MINSESSIONS;
createParams.maxSessions = 20; //MAXSESSIONS;
createParams.sessionIncrement = 1; //SESSINCREMENT;
if (dpiPool_create(context, "DB", strlen("DB"),
"DB", strlen("DB"), "1.7.7.1:1521/DEV",
strlen("10.70.7.146:1521/L10DEV"), &commonParams, &createParams, &pool) < 0)
return 1;
if (dpiPool_setGetMode(pool, DPI_MODE_POOL_GET_WAIT) < 0)
return 1;
int maxthreads = 50;
boost::asio::io_service ioService;
std::unique_ptr<boost::asio::io_service::work> work( new boost::asio::io_service::work(ioService));
boost::thread_group threadpool; //pool
std::cout << "thread starts with max threads: : " << maxthreads << std::endl;
for (std::size_t i = 0; i < maxthreads; ++i)
threadpool.create_thread(boost::bind(&boost::asio::io_service::run, &ioService));
using namespace boost::chrono;
//thread_clock::time_point start = thread_clock::now();
std::chrono::steady_clock::time_point begin = std::chrono::steady_clock::now();
for(int i = 0 ; i < 10000; i++)
ioService.post(boost::bind(icalc, 42));
work.reset();
threadpool.join_all();
ioService.stop();
std::chrono::steady_clock::time_point end= std::chrono::steady_clock::now();
std::cout << "Time difference = " << std::chrono::duration_cast<std::chrono::seconds>(end - begin).count() <<std::endl;
return 0;
}
Do we misuse the library somehow?
It is a little bit critical for us, since we want to decide if we will use oracle for a high performance server.
Thank you very much in advance.
ODPI-C defines DPI_NUMBER_AS_TEXT_CHARS
as 135 in dpiImpl.h. However it is too short. It must be at least 170.
I made a test code and put it here.
The following is the result.
define as DPI_ORACLE_TYPE_VARCHAR and DPI_NATIVE_TYPE_BYTES
isNull=0x0
len=170
str=-.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000009999999999999999999999999999999999999999
define as DPI_ORACLE_TYPE_NUMBER and DPI_NATIVE_TYPE_BYTES
isNull=0x39393939
len=960051553
str=0x3939393939393939
When a number column UTL_RAW.CAST_TO_NUMBER('7F0202020202020202020202020202020202020202')
is defined as DPI_ORACLE_TYPE_VARCHAR and DPI_NATIVE_TYPE_BYTES, it is fetched as a string whose length is 170.
When the same number is defined as DPI_ORACLE_TYPE_NUMBER and DPI_NATIVE_TYPE_BYTES, isNull
is 0x39393939 and str
is 0x3939393939393939. isNull
must be either 0 or 1. It is apparently overwritten with "9999" (code point of '9' is 0x39). str
is also overwritten with "99999999".
Object type attribute number could be get as double but not as int64 or bytes.
I made a test program.
https://gist.github.com/kubo/aea497d746d3d1603882c86b450accc0
It printed:
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_DOUBLE
=> 1
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_INT64
=> ERROR: DPI-1014: conversion between Oracle type 2010 and native type 3000 is not implemented
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_BYTES
=> ERROR: DPI-1014: conversion between Oracle type 2010 and native type 3004 is not implemented
What I want is:
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_DOUBLE
=> 1
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_INT64
=> 1
Getting ObjectCol.NumberValue as DPI_NATIVE_TYPE_BYTES
=> 1
The Database Change Notification docs seem to indicate that transaction ID should be available on a change message. I would like to request this be plumbed through to the odpi layer (next I'll ask for it in cx_Oracle.
I see DPI_OCI_ATTR_CHDES_XID exists in the headers, so I've been stumbling around in the code seeing if it is populated.
/tmp/cx_Oracle/odpi ((v2.0.0))$ git diff
diff --git a/include/dpi.h b/include/dpi.h
index d8f8a60..6739766 100644
--- a/include/dpi.h
+++ b/include/dpi.h
@@ -587,6 +587,8 @@ struct dpiSubscrMessage {
dpiEventType eventType;
const char *dbName;
uint32_t dbNameLength;
+ const char *TxID;
+ uint32_t TxIDLength;
dpiSubscrMessageTable *tables;
uint32_t numTables;
dpiSubscrMessageQuery *queries;
diff --git a/src/dpiSubscr.c b/src/dpiSubscr.c
index e35c29d..8eb9cdc 100644
--- a/src/dpiSubscr.c
+++ b/src/dpiSubscr.c
@@ -289,6 +289,13 @@ static int dpiSubscr__populateMessage(dpiSubscr *subscr,
DPI_OCI_ATTR_CHDES_DBNAME, "get DB name", error) < 0)
return DPI_FAILURE;
+ // determine transaction id
+ if (dpiOci__attrGet(descriptor, DPI_OCI_DTYPE_CHDES,
+ (void*) &message->TxID, &message->TxIDLength,
+ DPI_OCI_ATTR_CHDES_XID, "get transaction id", error) < 0)
+ return DPI_FAILURE;
+
+
// populate event specific attributes
switch (message->eventType) {
case DPI_EVENT_OBJCHANGE:
This seems to give the transaction ID, but not quite correclty. For a transaxction ID of 4.22.470 I get back 0x80 0x00 0x00 0x00 THEN the transaction 0x04 0x00 0x16 0x00 0x68 0x04 0x00 0x00.
In other words if I look in message->TxID+4 I see what seem like a transaction ID.
So, any chance of getting this added, and any idea why my test returned a weird shifted result?
Dear Anthony,
There is a problem in the folowing function (dpiLob.c:1427):
int dpiOci__lobCreateTemporary(dpiLob *lob, dpiError *error)
{
uint8_t lobType;
int status;
DPI_OCI_LOAD_SYMBOL("OCILobCreateTemporary",
dpiOciSymbols.fnLobCreateTemporary)
if (lob->type->oracleTypeNum == DPI_ORACLE_TYPE_BLOB)
lobType = DPI_OCI_TEMP_BLOB;
else lobType = DPI_OCI_TEMP_CLOB;
status = (*dpiOciSymbols.fnLobCreateTemporary)(lob->conn->handle,
error->handle, lob->locator, DPI_OCI_DEFAULT,
lob->type->charsetForm, lobType, 0, DPI_OCI_DURATION_SESSION);
return dpiError__check(error, status, lob->conn, "create temporary LOB");
}
It creates temporary lob with cache=false, which leads to unnecessary physical io every time such a variable is created.
For example, if we try to pass empty string from python code to plsql procedure with CLOB in parameter using cx_Oracle and ODPI, it would generate physical IO (could be easily seen in 10046 trace).
I believe it should be changed to chache=true, or, which is probably better, make this parameter configurable.
Regards,
Andrey
Hello.
I'm trying to compile the samples and tests and error occurs:
Make
Gcc -c -I ../ include -O2 -g -Wall -obuild / TestContext.TestContext.c
Gcc -c -I ../ include -O2 -g -Wall -obuild / TestLib.o TestLib.c
Gcc -L ../ lib -lodpic -ldl build / TestContext.o -obuild / TestContext build / TestLib.o
Build / TestContext.o: in function dpiTest_100_validMajorMinor ': /odpi/test/TestContext.c:35: undefined reference to
dpiContext_create '
/odpi/test/TestContext.c:38: undefined reference to `dpiContext_destroy '
...
OS: Ubuntu 16.4,
InstantClient: /usr/lib/oracle/12.2/client64/lib
Any tips?
Thank you.
Some issues about FormatMessage used here on Windows
I might have to create separate issues. But some issues are related so I put all in one.
The 6th argument FormatMessageW
is sizeof(wLoadError)
but it should be sizeof(wLoadError)/sizeof(wchar_t)
because the argument is the number of characters, not bytes, allocated for the 5th argument.
Trailing period and carriage return are stripped here. However, when the last character before carriage return is a multi-byte character, the character is broken. It should be stripped only when it is really period.
The error message retrieved by FormatMessageW
depends on the OS language. Could you change the messages to English or add a compile-time option to change it?
The message is changed to English when the 4th argument is MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT)
.
Programs in the samples directory print garbled text in Japanese Windows command prompt when OCI.DLL is not found. If the message language is English, it isn't garbled.
When the message isn't changed to English, could you add the error number to the message?
For example when the error code is 126:
DPI-1047: 64-bit Oracle Client library cannot be loaded: Win32 error 126: "The specified module could not be found". See https://oracle.github.io/odpi/doc/installation.html#windows for help
When an issue is posted with a localized error message and you don't know the language, you cannot read the message. Moreover if it is garbled as messages in Japanese Windows command prompt, nobody read it. If the error number is included in the text, you can know the error as follows.
DPI-1047: 64-bit Oracle Client library cannot be loaded: Win32 error 126: "指定されたモジュールが見つかりません。". See https://oracle.github.io/odpi/doc/installation.html#windows for help
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.