aws / amazon-documentdb-odbc-driver Goto Github PK

LOG_MSG is legacy code from Ignite. Since now we have different log levels, we should replace calls of LOG_MSG with LOG_<log level>_MSG. There are less than or equal to 85 LOG_MSG calls

AC:

• remove all calls to LOG_MSG
• remove LOG_MSG definition in log.h
• look at LOG_MSG calls case by case, and decide if it should be converted to one of: LOG_DEBUG_MSG, LOG_INFO_MSG, or LOG_ERROR_MSG.

AC:

• cent os build ( docker image based on ubunut)
• docker compose to spin up dev image + mongo container
• linux packaging for debian based (deb) and red hat based (rpm)

- name: build-deb-installer
      if: success()
      run: |
        cd cmake-build64
        cmake ../src
        make -j4
        cpack .
        cd ..
    - name: create-output
      if: success()
      run: |
        mkdir deb-installer
        mkdir build-output
        mkdir test-output
        cp -v ./build/odbc/lib/*.a build-output/
        cp -v ./build/odbc/lib/*.so build-output/
        cp -v ./cmake-build64/*.deb deb-installer/
    - name: upload-build
      if: success()
      uses: actions/upload-artifact@v2
      with:
        name: linux64-build
        path: build-output
    - name: upload-linux64-deb-installer
      if: success()
      uses: actions/upload-artifact@v2
      with:
        name: linux64-deb-installer
        path: deb-installer
  build-linux32:
    runs-on: ubuntu-latest
    steps:
    - name: Setup cmake
      uses: jwlawson/[email protected]
      with:
        cmake-version: '3.21.4'
    - uses: actions/checkout@v2
    - name: run-cppcheck
      run: |
        sudo apt install cppcheck
        sh run_cppcheck.sh
    - name: upload-cppcheck-results
      if: failure()
      uses: actions/upload-artifact@v2
      with:
        name: cppcheck-results
        path: cppcheck-results.log
    - name: get-dependencies
      if: success()
      run: |
        # Need to install i386 versions
        sudo dpkg --add-architecture i386
        sudo apt update
        sudo apt install unixodbc:i386 unixodbc-dev:i386 odbcinst1debian2:i386 libodbc1:i386 libcurl4-openssl-dev:i386 libssl-dev:i386 uuid-dev:i386 cpp:i386 cpp-9:i386 gcc:i386 g++:i386 zlib1g-dev:i386 linux-headers-$(uname -r) gcc-multilib:i386 g++-multilib:i386 g++-9:i386 gcc-9:i386 gcc-9-multilib:i386 g++-9-multilib:i386 binutils:i386 make:i386
    - name: prepare-dsn
      if: success()
      run: |
        sudo cp ./src/Tests/Tests/odbc-linux32.ini /etc/odbc.ini
        sudo cp ./src/Tests/Tests/odbcinst-linux32.ini /etc/odbcinst.ini
        mkdir -p ${{ github.workspace }}/odbc-logs
        export ODBCSYSINI=/etc/
        export ODBCINSTINI=odbcinst.ini
        export ODBCINI=/etc/odbc.ini
    - name: configure-and-build-driver
      if: success()
      run: |
        ./build_linux_release32_deb.sh
    - name: run-tests
      if: success()
      run: |
        ./build/odbc/bin/tests --gtest_output="xml:report.xml"
    - name: prepare-test-results
      if: always()
      run: |
        cp ${{ github.workspace }}/report.xml ${{ github.workspace }}/odbc-logs/
    - name: upload-test-report
      if: failure()
      uses: actions/upload-artifact@v2
      with:
        name: test-results-linux32
        path: ${{ github.workspace }}/report.xml
    - name: build-deb-installer
      if: success()
      run: |
        cd cmake-build32
        cmake ../src
        make -j4
        cpack .
        cd ..
    - name: create-output
      if: success()
      run: |
        mkdir deb-installer
        mkdir build-output
        mkdir test-output
        cp -v ./build/odbc/lib/*.a build-output/
        cp -v ./build/odbc/lib/*.so build-output/
        cp -v ./cmake-build32/*.deb deb-installer/
    - name: upload-build
      if: success()
      uses: actions/upload-artifact@v2
      with:
        name: linux32-build
        path: build-output
    - name: upload-linux32-deb-installer
      if: success()
      uses: actions/upload-artifact@v2
      with:
        name: linux32-deb-installer
        path: deb-installer

Convert date in string format to TIMESTAMP/DATE/TIME

AC:

1. Test for YYY-MM-DD[T]HH:MM:SS[.MMM] can be converted to TIMESTAMP/DATE/TIME
2. Test for YYY-MM-DD[T]HH can be converted to TIMESTAMP/DATE
3. Test for HH:MM:SS[.MMM] can be converted to TIME

Issue: With query to get bigint (e.g., SELECT CAST(9223372036854775807 AS BIGINT)), and call SQLGetData(hstmt, 1, SQL_C_SLONG, &sCustID, 0, &cbCustID); (example is based on Microsoft SQLGetData documentation), SQLGetData returns SQL_SUCCESS instead of returning SQL_ERROR with error state 22003 - Numeric value out of range.

Possible cause from looking at the code: When SQLGetData is called, DocumentDbColumn::PutInt* is called to get numeric data (converted from bsoncxx::type to C type), and then eventually ApplicationDataBuffer::PutNum is called to save the data without any conversion or checks for data size.

boost::optional and std::optional have very similar interfaces, so it should be easy to change the naming. due to licensing issues we should use std::optional instead of boost::optional in the end.

AC:

• change all references of boost::optional to std::optional.
• remove references to <boost/optional>

Hello,

I've noticed that the appearance of symbols/punctuation in the DocumentDB master password field often results in an authentication failure to the database. I have a CloudFormation template which generates the cluster and its master credentials via Secrets Manager. When excluding some illegal characters in the 99-character password using ExcludeCharacters: '"@/#%:=' under GenerateSecretString, I get the bellow errors as seen in ODBC Data Sources when clicking "Test" and in DocumentDB audit logs in CloudWatch respectively.

However when I use ExcludePunctuation: true to remove all symbols instead of only those specified above, generating a password of only letters and numbers, I get no error and receive the expected response and log shown below.

For some reason, the "user" field is blank in the error 18 log shown for the failed authentication, even though the user field is populated when establishing the connection through ODBC Data Sources when using this driver. This appears to be caused by some symbol(s) being present in the password, as the error only occurs when symbols are permitted and does not occur when they are all excluded, but it is unclear which symbol(s) is causing this.

Something to note is that I have a Lambda function which is able to successfully connect to and interact with the database using the connection string and pymongo library, even when the majority of symbols are allowed by using ExcludeCharacters: '"@/#%:=' instead of ExcludePunctuation: true. It is only the ODBC driver which appears to have the issue. The Python Lambda function properly encodes the password before transmitting it in the connection string, using the urllib.parse.quote() function.

Does the driver appropriately encode the database password field before attempting to establishing a connection? I also noticed the ODBC driver misinterpreting the password when the "=" symbol is used, also leading me to believe the issue may be related to character escapement.

Thank you.

Currently, the standard Amazon RDS CA root certificate is not being bunded with the ODBC driver.

Download an SSL certificate for your managed database in Amazon Lightsail | Lightsail Documentation

https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem

An attempt was made to enable this in the driver, without success.

tls_options.ca_file("path-to-file")

However, this did not seem to work as expected.

So currently, we are allowing invalid server certificates

tls_options.allow_invalid_certificates(true)

This task is to resolve the issue of using a CA file, distribute the CA certificate and re-enable server certificate checking.

Track fix for this JDBC issue: [BUG] Exception thrown on CASE WHEN where ELSE literal value is non-numeric #495:

When the JDBC is updated, ensure to update the JDBC_DRIVER_VERSION to retrieve the fixed version of the JDBC driver.

Enhance mac install dependencies script (scripts/install_dependencies_mac.sh) to make reinstalling mongodb server optional to user.

DECIMAL_DIGITS: The total number of significant digits to the right of the decimal point. 

for details, see https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlcolumns-function?view=sql-server-ver15

clang-tidy-check fails due to errors that shouldn't be a problem for our code. For example, even with compile_commands.json generated, clang-tidy-check still is unable to find boost/optional.hpp, which might be due to boost being outside of our project root folder. There might be more errors beside that.
Reference to clang-tidy-check error:

AC:

• enable clang-tidy-check inside github workflow.
• [optional] create a test PR with minor code/comment changes to see that clang-tidy-check does not fail

Hi,

I Install in my computer a x86 And x64 version, and muy ODBC test from odbc config works fine. but when try connect from excel or PowerBI, system returns next error message:

"Specified driver could not be loaded due to system error 126. The specified module could not be found
Amazon DocumentDB, C:\Program Files\Amazon DocumentDB\ODBC Driver\documentdb.odbc.dll"

any idea?

re-enable fieldNull SQL Type check in meta_queries_test.cpp

this is to be done after BUG: NULL Columns passed as VARCHAR to Calcite, ResultSetMetadata for queries reports NULL columns as VARCHAR

Right now, since installing cppcheck via chocolatey requires a reboot (which GitHub Actions is not supporting as far as I know), it is failing on Windows 2022 x32 and x64 builds.  choco install cppcheck -y is returning 3010, which GitHub actions treats as a failure as it is non-zero.
For more details on the failure see here. Maybe we will need to use other packaging or install cppcheck manually.

A cppcheck GitHub Action on the market (could perhaps use this): cppcheck-action

AC: Investigate and reactive the cppcheck in Windows GitHub Actions.

JDBC driver parameter loginTimeoutSec parameter doesn’t exit in mongo client. The parameter most likely to be used is connecTimeoutMS.
Connection String URI Format

AC.

• Fix parameter in JDBC driver
• Fix DSN config parameter in ODBC driver

Itemized ODBC Interface for handling of Character/Byte Lengths in parameters - Amazon - Confluence (atlassian.net)

Create tests for each of these API to test if a smaller buffer than required behaves correctly.

• Return buffer size is smaller than necessary
• Reports a return code of SUCCESS_WITH_INFO
• Resulting size is the number of characters (or bytes) returned not including the null character.
• [Optional] String representation of truncated value is correct.

1. SQLColAttribute
2. SQLGetInfo
3. SQLBindCol
4. SQLBindParameter
5. SQLGetData
6. SQLGetDiagField

Logging the connection string would be useful for debugging, but we need to make sure user’s password is not logged. The suggestion would be parsing the connection string, then log the parsed content.

AC:

• log connection string
• ensure that user’s password is not logged

Implement behaviour to override defaultAuthDB on the connection string.
AC

• expose defaultauthdb on the DSN config window

If multiple threads try to get the logger instance, 2 logger instances might be created at the same time. This may or may not be an actual issue. Note that locks would impair the run time.

Example:

  // pseudo-code
  getLoggerInstance() {
  // get lock
    if (!_logger)
      _logger =  new Logger();
    // unlock
    return _logger;
  }

AC:

• add lock protection to getLoggerInstance

See attributes_test.cpp

Two tests with the scenario ConnectionAttributeLoginTimeout and ConnectionAttributePacketSize.
The attributes SQL_ATTR_LOGIN_TIMEOUT and SQL_ATTR_PACKET_SIZE should be set before establishing the connection.

Note that this tests work fine for Microsoft ODBC driver manager and unixODBC

In api_robustness_test.cpp the following two rainy cases crash on Mac with iODBC driver manager, need to investigate the root cause.

TestSQLDriverConnect and TestSQLError

When I worked on another issue, the following tests have found memory leaks. They are disabled for now as root cause has not been identified.

BOOST_AUTO_TEST_CASE(TestFetchScrollLast, *disabled()) {
  CheckFetchScrollUnsupportedOrientation(SQL_FETCH_LAST);
}
 
BOOST_AUTO_TEST_CASE(TestFetchScrollPrior, *disabled()) {
  CheckFetchScrollUnsupportedOrientation(SQL_FETCH_PRIOR);
}
 
BOOST_AUTO_TEST_CASE(TestFetchScrollFirst, *disabled()) {
  CheckFetchScrollUnsupportedOrientation(SQL_FETCH_FIRST);
}

BOOST_AUTO_TEST_CASE(TestOneRowObject, *disabled()) {
...
}

BOOST_AUTO_TEST_CASE(TestCloseNonFullFetch, *disabled()) {
...
}

BOOST_AUTO_TEST_CASE(TestManyCursors, *disabled()) {
...
}

BOOST_AUTO_TEST_CASE(TestManyCursors2, *disabled()) {
...
}

Notes:

• Using Dr Memory could find the memory leak but the call stack is empty. So I suspect this issue is only seen on Windows and it is a 3rd party library memory leak issue as nothing is found on Linux. Need check more to confirm.

• No memory leak is reported on Linux after open detect_memory_leaks and enable TestFetchScrollLast - ./ignite-odbc-tests --catch_system_errors=false --detect_memory_leaks=1

Complete automating of environment and prerequisites on Linux.

CHAR_OCTET_LENGTH: The maximum length in bytes of a character or binary data type column. For all other data types, this column returns a NULL.

AC:

• implement BinaryDateTimeSub, SqlTypeDateTimeSub under namespace type_traits
• call the function in ColumnMetadataQuery::GetColumn under case ResultColumn::CHAR_OCTET_LENGTH

https://docs.microsoft.com/en-us/sql/odbc/reference/syntax/sqlcolumns-function?view=sql-server-ver15