Code Monkey home page Code Monkey logo

elasticsearch-sql-odbc's Introduction

ODBC Driver for Elasticsearch SQL

This is the ODBC driver for Elasticsearch's SQL plugin. The driver implements the Unicode version of the 3.80 API release.

Elastic License Functionality

The driver files in this repository are subject to the Elastic License. The full license can be found in LICENSE.txt. Usage requires a subscription. Files that are not subject to the Elastic License are in the libs directory.

Supported platforms

The currently supported platforms on both x86 and amd64 architectures are:

  • Microsoft Windows 10
  • Microsoft Windows Server 2016

Support for other platforms might be added at a later time.

Running Requirements

On the client side, the ODBC driver requires the OS'es Driver Manager to interface between the clients and the driver itself. On the server side, the requirements follow Elasticsearch SQL's requirements.

Building Requirements

CMake

The project is CMake enabled, which generates the environment-dependent build pipeline. This is a general build requirement.

The building itself is then delegated to the platform-specific tools.

CMake 3.14 or newer is required for building with Visual Studio 2019.

External libraries/headers

The driver makes use of the following libraries/headers:

  • ODBC-Specification
    • this is the project that contains the ODBC specification, including the headers defining the ODBC C API;
  • libcurl
    • the library is used for the HTTP(S) communication with Elasticsearch REST API;
  • c-timestamp
    • library used for parsing ISO 8601 formated timestamps;
  • ujson4c
    • fast scanner library for JSON;
  • tinycbor
    • a small CBOR encoder and decoder library.

The required libraries are added as subtrees to the project, in the libs directory:

   somedirectory\
    |_elasticsearch-sql-odbc
      |_CMakeLists.txt
      |_...
      |_libs
        |_ODBC-Specification
        |_curl
        |_zlib
        |_c-timestamp
        |_ujson4c
        |_tinycbor

Windows

MSVC

Building the driver requires the installation of Microsoft tools. These can be from the Visual Studio pack or with the standalone tools.

Version 2019 Enterprise 16 is used to develop with, older versions should work fine too, with their corresponding modules. The lists of packages for MSVC 2019 are given below.

Required packages:

  • MSBuild
    • as the build tool;
  • Windows 10 SDK
    • headers and libraries;
  • VC++ toolset
    • for the compiler;
  • C++/CLI support
    • for DSN editor's C-to-C# CLI binding;
  • C# support
    • for DSN editor's C# form;
  • F# support
    • for building the MSI package.

Optional packages:

  • CMake Project Wizards
  • C++ profiling tools
  • Git for Windows / GitHub Extentions for VisualStudio

Building

Windows MSVC

Start Visual Studio and menu-access File > Open > "CMake..". Navigate to the checkout folder of the project and select the file CMakeLists.txt. Once the file is read and imported, a new menu top item will appear, CMake. Menu-access CMake > "BuildAll".

Windows CLI

The project contains a BAT script - build.bat - that can run different steps for building the ODBC driver.

Some environment parameters can be set to customized its behavior (see start of script).

The script will take a set of parameters, run build.bat help to see which these are.

Testing

Testing the driver is done with unit tests and integration tests.

Requirements

The unit testing makes use of the Googletest framework. This is being fetched and built at testing time.

The integration testing makes use of a Python application that requires the following packages be installed:

  • Python3, both x86 and amd64 distributions
    • both x86 and x64 driver builds are tested;
  • Python launcher (py)
    • to selectively launch the right Python build;

For each of the two Python releases, the following packages must be installed:

  • pyodbc
    • for ODBC access;
  • requests
    • for HTTP support;
  • psutils
    • for OS process management.

Installation

See: https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-odbc.html

elasticsearch-sql-odbc's People

Contributors

bpintea avatar brianseeders avatar codebrain avatar craigtaverner avatar csoulios avatar debadair avatar droberts195 avatar elastic-backstage-prod[bot] avatar elasticsearchmachine avatar gmarouli avatar jakelandis avatar kingherc avatar mark-vieira avatar mattc58 avatar mpdreamz avatar pugnascotia avatar quux00 avatar rjernst avatar salvatore-campagna avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

elasticsearch-sql-odbc's Issues

BUG: no results with column-wise binding if ES/SQL returns an "empty" cursor

When:

  1. the application binds column-wise to the result set; and
  2. the result set contains a number of rows lower than app's rows array buffer (for the bound column); and
  3. ES/SQL appends to the first and only result set a cursor that (when executed) returns no further rows (i.e. an "empty cursor"), the driver will return no results back to the application.

The reason being that the result for the last cursor execution is returned as the final result code.

provide data source version as returned by the node

SQLGetInfo()'s attributeSQL_DBMS_VER requires Elasticsearch'es version (mostly for error reporting/logging). This is currently statically defined and need to be changed to deliver whatever the root REST API delivers (http://[::1]:9200 / version / number or qualified).

Improve credentials handling

The credentials are currently stored in the registry in plain text, under ODBC mandated UID and PWD keys. This area needs some effort to find better ways to handle this storage.

Config GUI: add new tab for logging

Currently, the only way to configure the tracing of actions the driver is asked to perform is possible by using environment variables. A more practical (and possibly very useful for pre-releases) way of configuring this is through the DSN, per connection. This would require a new tab to the DSN editor:

  • enable/disable logging;
  • allow choose a directory where to write the log fileS;
  • the log level.

ODBC DSN Test Connection crash on earlier versions of Windows

Just filing to share a crash event for those who may run into something similar on unsupported Windows OSes.

image
image

crash_event.xml.zip

Certainly, Windows 2012 R2 does not meet the pre-requisites for the ODBC driver installation. However, we currently allow the ODBC driver to install successfully (without errors) on this OS version (#146).

I am actually able to bring data into Excel on this OS version and it appears to work fine; the piece that crashes is the Test Connection button from the DSN setup.

Build: integrate installer's build.bat with driver's

Installer's build.bat needs to be hooked into driver's, so that building of both can be managed by the top-level build.bat.

This currently needs:

  • driver's build.bat changed so that: (1) it accepts [a] new parameter[s] to take the signing cert and pass files; (2) the package and clean targets call installer's build.bat with release (if signing) and clean params respectively.
  • installer's build.bat clean action should reset installer code: (1) remove any build byproducts; (2) revert any changes made to the code tree (like installer\src\Installer\Properties\AssemblyInfo.cs).
  • installer's build.bat should only bootstrap packet when needed (like when building the installer, but not cleaning), to speed up the build lifecycle. Avoid running paket bootstrap on clean.

Can you use File DSN?

Hi.
I'm setting Elasticsearch SQL ODBC Driver.
At that time I noticed Can you use File DSN?

The following is written as 「three tabs」.
Configuration

The next step is to configure a DSN. You can choose between the following options mapped on the first three tabs of the Administrator application:

However, only「User DSN」and「System DSN」are written in「2.Configure a DSN」.

Thank you for your review.

BETA Todos

Branch from master, backport changes to 6.x

BUG: remove usage of static buffer in cursor conversion for requests

The cursor received from the server as a wide-string is stored as such and converted from UTF8 to multibyte string using a static buffer.
The static buffer, currently sized to 4K is too small to accommodate the cursor generated for large queries.
The solution here is to simply ditch the buffer, given that that the cursor is actually a Base64 encoded string and thus will straight forward convert to an ASCII string.

Config GUI: wait indicator for connection testing

When connection testing, the DSN resolution could take some time (a good few seconds, depending on DHCP/DNS setting etc.), especially visible when the hostname is unresolvable. In this case, it might be worth adding a "working" indicator, if there's a quick solution for it.

TODOs

Status Item PR Brief Notes
core: DSN handling #1 Add support for handling and generating connection strings
core: progress #SQLGetInfo() #3
#14
#18
This function is the main entry point for an app to query driver + data source capabilities * 150+ attributes;
* Bring SQLGetStmtAttr() to the level of Set~;
( ✓ ) core: cover all API calls #14
#64
#80
#98
Implement all API functions used by the reference Apps (Tableau, Excel)
core: remove direct registry access #110 use odbccp32.dll's SQLGetPrivateProfileString() & co.
WIP core: data conversions touch-ups #118
#130
* fail quickly on unsupported, object, binary(?), GUID(?);
* string→ time/date/timestamp review/redo;
* numeric/decimal review/redo;
protocol : data conversions #3
#7
#9
#10
Convert data from ES/SQL SQL to SQL C types Data Type Conversions
protocol : data types loading #5
#6
#100
Conversion between ES/SQL ↔ standard SQL types col base name, table & base table name left empty / for later
protocol : catalog support #6 Fetch catalog info from ES and export it to the application based on https://github.com/elastic/x-pack-elasticsearch/issues/3832
protocol : schema support Fetch schema info from ES and export it to the application
protocol : parameterized execution #11 Support for parameters in a prepared statement
protocol : cluster "sniffing"/detection read cluster members, (potentially) round-robin the requests
protocol : read cluster version/name #84 The data source version is reported upstream to the application
protocol : close scroller, if any #102 If application doesn't fetch all results, scroller is left to time-out
API : array of parameters Support for binding arrays of parameters
API : batching support Support for multiple result-sets
API : data-at-exec "Long" data is typically sent out to the data source at execution time, not preparation/binding time. This item would support this functionality. This is mostly done for statements that add data to the data source, which is currently not possible with ES/SQL -> low prio
arch: SPI/connection pooling Support for DM connection pooling
core : multi-threading #16 Multi-threading capabilities This is basic requirement
core : auth/encryption #23
#24
Support for user authentication over an encrypted connection This is basic requirement
( ✓ ) core : scrollable cursors Support for non-forward-only cursors Requires support in ES/SQL for scrollable cursors.
( ✓ ) arch : Win 7/8 Testing and adapting to Win7 and Win8
transport : CBOR #169
#172
#175
#177
#180
add CBOR encapsulation, besides JSON
tests #7
#9
#10
#96
Frameworks in place for: unit, integration, smoke, functional (Excel, Tableau)
arch : ANSI support Make an ANSI variant of the driver * Not a high prio, it depends on how old the clients for the driver might be;
* extend tstr_t usage;
* review assumption of SQLWCHAR as always wchar_t (vs. unsigned short, problematic with non-Windows pre-C95 compilers)
arch : 32 bit support #63
#74
#76
#79
Support for Windows 32bit platforms
WIP core : clean-up the list of TODO/FIXMEs in code #8
#12
#15
#20
#98
Many mandatory, but not "main path" features are left out as TODOs
arch : iODBC/unixODBC Add support for the *nix platforms
delivery : set-up DLL #13 A new library that takes care of the installation and set-up Installer DLL
This is different from general packaging (see next item)
delivery : Installer elastic/windows-installers#218
#19
https://github.com/elastic/release-manager/issues/423
#22
#38
A program that is launched to install the ES/ODBC driver This can be an .exe, .msi written in any specific environment/language
GUI: user prompting for connection details elastic/windows-installers#218
#32
#38
Implement a GUI for promting the user for the connection details
delivery : 3rd party deps report #17
#126
Generate 3rd party dependencies reports https://github.com/elastic/release-manager/issues/207
( ✓ ) delivery : translation DLL A new library that takes care of encoding translations Not a high prio
core: support for 2.x applications (1.x?) ODBC 2.x applications expect slightly different return codes, though mostly the same behavior * SQL_X_Y, X in [COLUMN(2.x), DESC(3.x)], Y in [LENGTH, PRECISION, SCALE] for API functions ref'd in ColSz,DecDigs,XferLen,DispSz
core: better logging: #4
#39
handler- and connection-dependent logging
( ✓ ) core: escape sequences analyse what to handle in the driver and what in the plug-in Implemented on the server side
docs: driver #55 Documenting the usage of the driver, options, limitations
docs: integration #55
#68
Document setting up ES/ODBC with popular applications (Tableau, Excel)

MSI: Installation Integration tests

Need the following integration tests:


Test 1: Assert correct installation

  1. Install driver
  2. Assert files copied
  3. Assert registry keys are present
  4. Install dependent Elasticsearch version
  5. Configure DSN
  6. Insert document into new index
  7. Assert ODBC connection is successful and data retrieved

Test 2: Assert correct un-installation

  1. [As per Test 1]
  2. Uninstall driver
  3. Assert files removed
  4. Assert registry keys are deleted

Test 3: Assert upgrade is prevented

  1. [As per Test 1]
  2. Attempt upgrade to a later version
  3. Assert upgrade not taken place

Test 4: Assert operating system check

  1. Attempt driver install on unsupported operating system
  2. Assert error message received and not installed

Test 5: Assert C++ dependancy check

  1. Attempt driver install on operating system without C++ runtime installed
  2. Assert error message received and not installed

Test 6: Assert .NET dependancy check

  1. Attempt driver install on operating system without .NET framework installed
  2. Assert error message received and not installed

Using Pester and the supporting PowerShell scripts from the windows-installers project will get us further with less effort. If expedience is a high priority, this may be preferable. The ODBC installer is a much simpler beast than the Elasticsearch installer, making the simpler approach of Pester vs. xUnit/NUnit less of an issue.

Consider Unit tests for editor dialog after moving to an MVVM or MVC architecture.

Docs: note overlap of datatype IDs

The IP datatype uses currently the same ID as text/keyword. This prevents a client from explicitly specifying a value as IP and not just text. Note this limitation, should any developer actually consider using the ODBC API over REST.
Same for geo types, if they will re-use text/keyword ID.

BUG: UJSON4C fails to decode answer

This bug appears sporadically; so far not consistently reproducible.

UJSON4C library's UJDecode() fails to decode received JSON answer; the answer is correct. The failure is always Trailing data.
Sometimes the answer buffer gets corrupted after the function returns, although no writing to it seems to occur in the library.

Log excerpts:

Mon Nov 19 09:57:26 2018 - [DEBUG] post_json()@connect.c:568 [STMT@0x000001CBF9FA6860] POSTing JSON [38] `{"query": "SYS TYPES", "mode": "ODBC"}`.
[...]
Mon Nov 19 09:57:26 2018 - [DEBUG] attach_answer()@queries.c:223 [STMT@0x000001CBF9FA6860] attaching answer [2324]`{"columns":[{"name":"TYPE_NAME","type":"keyword"},{"name":"DATA_TYPE","type":"integer"},{"name":"PRECISION","type":"integer"},{"name":"LITERAL_PREFIX","type":"keyword"},{"name":"LITERAL_SUFFIX","type":"keyword"},{"name":"CREATE_PARAMS","type":"keyword"},{"name":"NULLABLE","type":"short"},{"name":"CASE_SENSITIVE","type":"boolean"},{"name":"SEARCHABLE","type":"short"},{"name":"UNSIGNED_ATTRIBUTE","type":"boolean"},{"name":"FIXED_PREC_SCALE","type":"boolean"},{"name":"AUTO_INCREMENT","type":"boolean"},{"name":"LOCAL_TYPE_NAME","type":"keyword"},{"name":"MINIMUM_SCALE","type":"short"},{"name":"MAXIMUM_SCALE","type":"short"},{"name":"SQL_DATA_TYPE","type":"integer"},{"name":"SQL_DATETIME_SUB","type":"integer"},{"name":"NUM_PREC_RADIX","type":"integer"},{"name":"INTERVAL_PRECISION","type":"integer"}],"rows":[["BYTE",-6,3,"'","'",null,2,false,3,false,false,false,null,0,0,-6,0,10,null],["LONG",-5,19,"'","'",null,2,false,3,false,false,false,null,0,0,-5,0,10,null],["BINARY",-3,2147483647,"'","'",null,2,false,3,true,false,false,null,null,null,-3,0,null,null],["NULL",0,0,"'","'",null,2,false,3,true,false,false,null,null,null,0,0,null,null],["INTEGER",4,10,"'","'",null,2,false,3,false,false,false,null,0,0,4,0,10,null],["SHORT",5,5,"'","'",null,2,false,3,false,false,false,null,0,0,5,0,10,null],["HALF_FLOAT",6,16,"'","'",null,2,false,3,false,false,false,null,0,16,6,0,2,null],["SCALED_FLOAT",6,19,"'","'",null,2,false,3,false,false,false,null,0,19,6,0,2,null],["FLOAT",7,7,"'","'",null,2,false,3,false,false,false,null,0,7,7,0,2,null],["DOUBLE",8,15,"'","'",null,2,false,3,false,false,false,null,0,15,8,0,2,null],["KEYWORD",12,256,"'","'",null,2,true,3,true,false,false,null,null,null,12,0,null,null],["TEXT",12,2147483647,"'","'",null,2,true,3,true,false,false,null,null,null,12,0,null,null],["IP",12,39,"'","'",null,2,false,3,true,false,false,null,null,null,12,0,null,null],["BOOLEAN",16,1,"'","'",null,2,false,3,true,false,false,null,null,null,16,0,null,null],["DATE",93,24,"'","'",null,2,false,3,true,false,false,null,3,3,9,3,null,null],["UNSUPPORTED",1111,0,"'","'",null,2,false,3,true,false,false,null,null,null,1111,0,null,null],["OBJECT",2002,0,"'","'",null,2,false,3,true,false,false,null,null,null,2002,0,null,null],["NESTED",2002,0,"'","'",null,2,false,3,true,false,false,null,null,null,2002,0,null,null]]}`.
Mon Nov 19 09:57:26 2018 - [ERROR] attach_answer()@queries.c:230 [STMT@0x000001CBF9FA6860] failed to decode JSON answer: Trailing data ([2324] `¤² p«U`).
Mon Oct 22 10:00:30 2018 - [DEBUG] post_json()@connect.c:566 [STMT@0x0000018AA177EBC0] POSTing JSON [108] `{"query": "SYS COLUMNS CATALOG '7879474408f44e9ca98d1597e659dfe2' TABLE LIKE 'users' ESCAPE '\\'  LIKE '%'"}`.
[...]
Mon Oct 22 10:00:30 2018 - [DEBUG] attach_answer()@queries.c:222 [STMT@0x0000018AA177EBC0] attaching answer [1665]`{"columns":[{"name":"TABLE_CAT","type":"keyword"},{"name":"TABLE_SCHEM","type":"keyword"},{"name":"TABLE_NAME","type":"keyword"},{"name":"COLUMN_NAME","type":"keyword"},{"name":"DATA_TYPE","type":"integer"},{"name":"TYPE_NAME","type":"keyword"},{"name":"COLUMN_SIZE","type":"integer"},{"name":"BUFFER_LENGTH","type":"integer"},{"name":"DECIMAL_DIGITS","type":"integer"},{"name":"NUM_PREC_RADIX","type":"integer"},{"name":"NULLABLE","type":"integer"},{"name":"REMARKS","type":"keyword"},{"name":"COLUMN_DEF","type":"keyword"},{"name":"SQL_DATA_TYPE","type":"integer"},{"name":"SQL_DATETIME_SUB","type":"integer"},{"name":"CHAR_OCTET_LENGTH","type":"integer"},{"name":"ORDINAL_POSITION","type":"integer"},{"name":"IS_NULLABLE","type":"keyword"},{"name":"SCOPE_CATALOG","type":"keyword"},{"name":"SCOPE_SCHEMA","type":"keyword"},{"name":"SCOPE_TABLE","type":"keyword"},{"name":"SOURCE_DATA_TYPE","type":"short"},{"name":"IS_AUTOINCREMENT","type":"keyword"},{"name":"IS_GENERATEDCOLUMN","type":"keyword"}],"rows":[["7879474408f44e9ca98d1597e659dfe2",null,"users","password",12,"TEXT",0,2147483647,null,null,1,null,null,12,0,2147483647,1,"YES",null,null,null,null,"NO","NO"],["7879474408f44e9ca98d1597e659dfe2",null,"users","password.keyword",12,"KEYWORD",0,2147483647,null,null,1,null,null,12,0,2147483647,1,"YES",null,null,null,null,"NO","NO"],["7879474408f44e9ca98d1597e659dfe2",null,"users","user",12,"TEXT",0,2147483647,null,null,1,null,null,12,0,2147483647,2,"YES",null,null,null,null,"NO","NO"],["7879474408f44e9ca98d1597e659dfe2",null,"users","user.keyword",12,"KEYWORD",0,2147483647,null,null,1,null,null,12,0,2147483647,1,"YES",null,null,null,null,"NO","NO"]]}`.
Mon Oct 22 10:00:30 2018 - [ERROR] attach_answer()@queries.c:228 [STMT@0x0000018AA177EBC0] failed to decode JSON answer (`{"columns":[{"name":"TABLE_CAT","type":"keyword"},{"name":"TABLE_SCHEM","type":"keyword"},{"name":"TABLE_NAME","type":"keyword"},{"name":"COLUMN_NAME","type":"keyword"},{"name":"DATA_TYPE","type":"integer"},{"name":"TYPE_NAME","type":"keyword"},{"name":"COLUMN_SIZE","type":"integer"},{"name":"BUFFER_LENGTH","type":"integer"},{"name":"DECIMAL_DIGITS","type":"integer"},{"name":"NUM_PREC_RADIX","type":"integer"},{"name":"NULLABLE","type":"integer"},{"name":"REMARKS","type":"keyword"},{"name":"COLUMN_DEF","type":"keyword"},{"name":"SQL_DATA_TYPE","type":"integer"},{"name":"SQL_DATETIME_SUB","type":"integer"},{"name":"CHAR_OCTET_LENGTH","type":"integer"},{"name":"ORDINAL_POSITION","type":"integer"},{"name":"IS_NULLABLE","type":"keyword"},{"name":"SCOPE_CATALOG","type":"keyword"},{"name":"SCOPE_SCHEMA","type":"keyword"},{"name":"SCOPE_TABLE","type":"keyword"},{"name":"SOURCE_DATA_TYPE","type":"short"},{"name":"IS_AUTOINCREMENT","type":"keyword"},{"name":"IS_GENERATEDCOLUMN","type":"keyword"}],"rows":[["7879474408f44e9ca98d1597e659dfe2",null,"users","password",12,"TEXT",0,2147483647,null,null,1,null,null,12,0,2147483647,1,"YES",null,null,null,null,"NO","NO"],["7879474408f44e9ca98d1597e659dfe2",null,"users","password.keyword",12,"KEYWORD",0,2147483647,null,null,1,null,null,12,0,2147483647,1,"YES",null,null,null,null,"NO","NO"],["7879474408f44e9ca98d1597e659dfe2",null,"users","user",12,"TEXT",0,2147483647,null,null,1,null,null,12,0,2147483647,2,"YES",null,null,null,null,"NO","NO"],["7879474408f44e9ca98d1597e659dfe2",null,"users","user.keyword",12,"KEYWORD",0,2147483647,null,null,1,null,null,12,0,2147483647,1,"YES",null,null,null,null,"NO","NO"]]}`): Trailing data.
Thu Oct 25 12:21:10 2018 - [DEBUG] post_json()@connect.c:566 [STMT@0x00000273EA3D3040] POSTing JSON [72] `{"query": "SYS TABLES CATALOG LIKE '%' ESCAPE '\\' TYPE 'TABLE','VIEW'"}`.
[...]
Thu Oct 25 12:21:10 2018 - [DEBUG] attach_answer()@queries.c:223 [STMT@0x00000273EA3D3040] attaching answer [775]`{"columns":[{"name":"TABLE_CAT","type":"keyword"},{"name":"TABLE_SCHEM","type":"keyword"},{"name":"TABLE_NAME","type":"keyword"},{"name":"TABLE_TYPE","type":"keyword"},{"name":"REMARKS","type":"keyword"},{"name":"TYPE_CAT","type":"keyword"},{"name":"TYPE_SCHEM","type":"keyword"},{"name":"TYPE_NAME","type":"keyword"},{"name":"SELF_REFERENCING_COL_NAME","type":"keyword"},{"name":"REF_GENERATION","type":"keyword"}],"rows":[["7879474408f44e9ca98d1597e659dfe2","",".kibana","TABLE","",null,null,null,null,null],["7879474408f44e9ca98d1597e659dfe2","",".security-6","TABLE","",null,null,null,null,null],["7879474408f44e9ca98d1597e659dfe2","","employ_name","TABLE","",null,null,null,null,null],["7879474408f44e9ca98d1597e659dfe2","","users","TABLE","",null,null,null,null,null]]}`.
Thu Oct 25 12:21:10 2018 - [ERROR] attach_answer()@queries.c:230 [STMT@0x00000273EA3D3040] failed to decode JSON answer: Trailing data ([775] `6ÈD$æõ`).

building with "package" argument generates a doubly-nested .zip file

Building with .\build.bat type:Release package will produce a .zip file that looks like: esodbc-6.5.0-windows-x86_64.zip/esodbc-6.5.0-windows-x86_64/esodbc-6.5.0-windows-x86_64/esodbc6u.dll, which is incorrect.

Building with .\build.bat type:Release package:-SNAPSHOT will work as expected (no unnecessary 2nd level)

Config GUI: activate Save and Test Connection buttons only if input is present

In case of editing or adding a DSN, the Save and Test Connection should only be activated if there is valid input, as follows:

  • the Save button requires a DSN Name and Hostname filled in;
  • the Test Connection button requires at least the Hostname field filled in. The delux feature here would be do have validation on the field (hostname or IPv4/6), but this only if there is already some ready to use framework functionality (regexps or building a parser here might be inadvisable).

Can't click-through create a File DSN

If the Verify this connection (recommended) tick box is left ticked in the "Advanced..." dialog (or simply don't access the "Advanced..." dialog), the editor will pop up in the "connection" mode and the "Connect" button is never activated.

Instead, it should be pop-up in "save" mode and allow editing the DSN name and description and eventually saving the configuration.

Creating a File DSN will work though if the parameters are provided manually in the "Advanced..." dialog and the connection verification is left out; in this case the editor will simply not be invoked.

MSI: support for upgrades

Currently, side-by-side installation of different driver versions is only supported if they are for different architectures (i.e. x86 and x64 drivers can co-exist).
This ticket tracks the feature to allow the installer do upgrades within the same architecture; for example a 7.0.1 x86 be upgraded to a 7.0.2 x86. (This currently requires uninstalling the 7.0.1 beforehand.)

MSI packaging modifications

  1. It is probably worth renaming the DLL and LIB files from elasticodbc7u.dll / .lib to esodbc7u.dll / .lib. That makes them 8 characters long and removes the need to worry about MSI short filename generation.
  2. The license.txt file... if we can make it a license.rtf (rich text format) file then we can simply include it into the license acceptance window without any modifications.
  3. /* add a sample DSN */
    if (! ConfigDSNW(NULL, ODBC_ADD_DSN, lpszDriver, sample_dsn)) {
    WARN("failed to provision a sample DSN.");
    /* no further error indication though, install succeedes */
    }
    : can be removed and replaced by an ODBCDataSouce target in the MSI and have the DSN created in the installation routine.
  4. DLL with versioning info.

Installer: 32 bit support

Evaluate the effort and safety of a 32bit build of the installer.

Note: some preliminary findings out of a quick check:

  • code-wise, a s/-windows-x86_64/-windows-x86 in installer/src/Installer/Program.cs:56 would be enough to get a 32bit .msi;
  • the target installation should be changed from Program Files to Program Files (x86) [1];
  • the installation eventually fails with error 13, potentially related to how the ODBC setup routines are called?
    error13

[1] The 32bit .msi find the 64bit driver and so it refuses to install -- not sure if the install path might be the only cause -- but this reveals that the name of the driver itself should bear the 32 tag (??).

The code itself can simply be made to work with 32-bit builders, resulting in a 32-bit installer (vs. having a 64-bit installer deliver both 64-bit and 32-bit drivers). Reasons for that could be:

  • low effort of having a 32-bit installer (no significant changes required between the builds);
  • low RM integration effort (besides the current build.bat setup 64 package we'd add build.bat setup 32 package);
  • testing could be run similarly irrespective of underlying platform;
  • the 32-bit installer could theoretically be used on 32-bit OSes (small chance of this occurrence, though);
  • better user-experience: the user would need to make a choice right from the start and thus be made aware of the split (and of otherwise hidden potential issues later).

Improve failing on Elasticsearch error

In case Elasticsearch would return the incorrect JSON data format for the signaled data type (like float for DATETYPE), ensure that the application would be provided with a diagnostic (not just the SQL_ERROR).

installer fails to run on RM VM

@codebrain ,
I had a look on why building the installer fails on the updated RM VM and I guess the following points might need to be addressed:

  • as per error.log, installer\build.bat would need a cd %~dp0 before calling .paket\paket.bootstrapper.exe (so that the .exe can be found);
  • the fix above might potentially need to be applied also to the place where ODBCInstaller.exe is invoked:
PS C:\[...]\elasticsearch-sql-odbc\installer> .\src\Installer\bin\Release\ODBCInstaller.exe 7.0.0-alpha9 C:\[...]\elasticsearch-sql-odbc\builds\

Unhandled Exception: System.Exception: The directory '..\..\packages\WixSharp.wix.bin\tools\bin' could not be found
   at ODBCInstaller.Program.Main(String[] args)

The above does work if called from within C:\[...]\elasticsearch-sql-odbc\installer\src\Installer.

  • not positive on all possible formats, but ${versionQualifier} for a build candidate can look like alpha1 (see above linked error.log). This wouldn't be accepted by the VersionRegex regex. So I guess either the regex or the command line parameter in build.gradle (and corresponding release line) needs changing;
  • the ${versionQualifier} var can also be missing/empty, probably for GA releases?
  • with current code, once the versioning string is accepted, it seems that ODBCInstaller.exe is launched with the path to the build directory of the driver as parameter, which errs, since the driver files aren't there:
PS C:\[...]\elasticsearch-sql-odbc> .\installer\build.bat release esodbc-7.0.0-alpha9 C:\[...]\certificate.p12 C:\[...]\passwd.txt
[...]
./src/Installer/bin/Release/OdbcInstaller 7.0.0-alpha9 C:\[...]\elasticsearch-sql-odbc\builds\

Unhandled Exception: System.InvalidOperationException: Sequence contains no matching element
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source, Func`2 predicate)
   at ODBCInstaller.Program.GetDriverFileInfo(String zipContentsDirectory)
   at ODBCInstaller.Program.Main(String[] args)

which seems to occur in GetDriverFileInfo. Could it be that b0226ab needs further adjustments to no longer expect a .zip file or to have the .zip previously unzipped?

@Conky5 : could you please confirm when versionQualifier might be empty in build.gradle and what format could it have, when not empty?

Docs: add note about SmartScreen warning

Since the MSI signing is not currently done with an EV certificate, the SmartScreen would prompt the user with a warning. Note in the documentation about this, to prepare the user.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.