Code Monkey home page Code Monkey logo

elasticlient's People

Contributors

johniez avatar kwach avatar ladislavmacoun avatar mbumba avatar wlbf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

elasticlient's Issues

Async support

Is there interest in adding the cpr async methods (e.g GetCallback)? It appears all calls are blocking in this lib. I have an event based async runtime that I'd like to add this libraries functionality on to, but it can't use blocking calls for HTTP requests given that they may take a few seconds to execute. Thanks for your thoughts and this lib!

fix(scroll): wrong payload on next batch

Dear maintainer(s),
it seems "scroll API" client does not meet elasticsearch expectations.

As described in https://www.elastic.co/guide/en/elasticsearch/reference/current/scroll-api.html, "scroll_api" shall be a JSON param to provide in the payload.

However, the scroll lib "<elasticlient/scroll.h>" posts next scoll request using "scroll_id" directly in request body without encapsulating it in {"scroll_id": "VALUE"}.
Same case for clear function.

scroll.cc (changes):

diff --git a/src/scroll.cc b/src/scroll.cc
index c939c8d..685dcf0 100644
--- a/src/scroll.cc
+++ b/src/scroll.cc
@@ -183,7 +183,7 @@ bool Scroll::next(Json::Value &parsedResult) {
         std::ostringstream urlPart;
         urlPart << "_search/scroll?scroll=" << impl->scrollTimeout;
         LOG(LogLevel::INFO, "Scroll (next) on %s.", urlPart.str().c_str());
-        if (impl->run(urlPart.str(), scrollParameters.scrollId, parsedResult)) {
+        if (impl->run(urlPart.str(), "{\"scroll_id\": \"" + scrollParameters.scrollId + "\"}", parsedResult)) {
             return true;
         }
     }
@@ -202,7 +202,7 @@ void Scroll::clear() {
     } else {
         try {
             const cpr::Response r = impl->client->performRequest(
-                Client::HTTPMethod::DELETE, "_search/scroll/", scrollParameters.scrollId);
+                Client::HTTPMethod::DELETE, "_search/scroll", "{\"scroll_id\": \"" + scrollParameters.scrollId + "\"}");
             if (r.status_code / 100 != 2) {
                 LOG(LogLevel::WARNING, "Scroll delete failed response text: %s", r.text.c_str());
             }

This modification works fine for me.

Do not hesitate if you need more inputs / discuss's

Regards,
Laurent.

Conan package

Hello. Thank you for the pretty good library for work with ElasticSearch from C++.

But integration into C++ project this library isn't so easy. I suugest to prepare Conan package for the library. What do you think?

Unsupported action found at bulk response.

Hi,

elasticlient lib version: version-02
elastic-search version: 7.17.0

I'm bulk perform fail and without error.
Checking log found a msg: Unsupported 'action' found at bulk response.

deatail log:

Host response text: {
  "took": 0,
  "errors": true,
  "items": [
    {
      "create": {
        "_index": ".ds-game-of-thrones-a-2022.04.06-000032",
        "_type": "_doc",
        "_id": "docId_c",
        "status": 409,
        "error": {
          "type": "version_conflict_engine_exception",
          "reason": "[docId_c]: version conflict, document already exists (current version [1])",
          "index_uuid": "1pbbHdwvQyKv55pxEqUjjg",
          "shard": "0",
          "index": ".ds-game-of-thrones-a-2022.04.06-000032"
        }
      }
    }
  ]
}

Host response size: 406
Unsupported 'action' found at bulk response.

Elasticlient ACID Transaction Support

Can we achieve ACID transactional support using Elasticlient (i.e., commit and rollback operations) .?
For eg :- when a document is added into a index which is in a transaction, we need to add/update the subset of that in the index for real time search. In case any error happens in table persist or elastic search index persist, the entire transaction should roll back. Is that case possible with Elasticlient .? If not, any other way to achieve this .?

Client::search() wont give the same answers as the dev tools in Kibana

I'm facing a problem of coherence in the answers of elasticlient::Client::search() versus thoses displayed by Kibana . Same problem when I use elasticlient::Client::Scroll
On my personal debug config (docker.elastic.co/elasticsearch/elasticsearch-oss:6.2.2 and docker.elastic.co/kibana/kibana-oss:6.2.2) , I observed that the number of total _shards strongly differs in the results of elasticlient::Client::search() and Kibana : 5 _shards with search(), vs 21 in Kibana.

cmake error

Hi, when I run cmake .. in build, it appear error like this:
CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: MHD_INCLUDE_DIR (ADVANCED) used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/src used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test used as include directory in directory /data/denny/github/elasticlient/external/httpmockserver/test MHD_LIBRARY (ADVANCED) linked by target "test-server" in directory /data/denny/github/elasticlient/external/httpmockserver/test linked by target "test-server" in directory /data/denny/github/elasticlient/external/httpmockserver/test linked by target "tests-elasticlient" in directory /data/denny/github/elasticlient/test

Could you tell me how to fix it, thank you

Could NOT find HTTPMOCKSERVER

I have followed every step from the README file but I get the following error:

cmake -DUSE_ALL_SYSTEM_LIBS=YES ..
-- Checking compiler C++11 support.
-- ELASTICLIENT_VERSION_MAJOR: 2
-- ELASTICLIENT_VERSION_MINOR: 1
-- ELASTICLIENT_VERSION_PATCH: 0
-- BUILD_ELASTICLIENT_TESTS: ON
-- BUILD_ELASTICLIENT_EXAMPLE: ON
-- USE_ALL_SYSTEM_LIBS: YES
-- CMAKE_MODULE_PATH: /home/vlaicu/ELASTIC_TEST/elasticlient/cmake/Modules/
-- Could NOT find HTTPMOCKSERVER (missing: HTTPMOCKSERVER_LIBRARY HTTPMOCKSERVER_INCLUDE_DIR)
CMake Error at external/CMakeLists.txt:8 (message):
HTTPMockServer library not found.

-- Configuring incomplete, errors occurred!

I have also done step 2: git submodule update --init --recursive

make error

make error

/usr/bin/ld: not found -lgtest

collect2:error: ld returned 1 exit status

failed to build on opensuse 15.2 (leap) with clang

at current tip of master

externals/elasticlient0.1.0-1_src/elasticlient $ git rev-parse HEAD
3adb172a26baae1a995e810e49fee1688ea44df5

a duplicate case statement...

-- The CXX compiler identification is Clang 6.0.1
-- Check for working CXX compiler: /externals/clang6.0-0/bin/clang++
-- Check for working CXX compiler: /externals/clang6.0-0/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Checking compiler C++11 support.
--   ELASTICLIENT_VERSION_MAJOR: 2
--   ELASTICLIENT_VERSION_MINOR: 1
--   ELASTICLIENT_VERSION_PATCH: 0
--   BUILD_ELASTICLIENT_TESTS: ON
--   BUILD_ELASTICLIENT_EXAMPLE: ON
--   USE_ALL_SYSTEM_LIBS: OFF
--   USE_SYSTEM_JSONCPP: OFF
--   USE_SYSTEM_CPR: OFF
--   USE_SYSTEM_HTTPMOCKSERVER: OFF
--   USE_SYSTEM_GTEST: OFF
-- CMAKE_MODULE_PATH: /externals/elasticlient0.1.0-1_src/elasticlient/cmake/Modules/
-- Checking compiler C++11 support.
-- httpmockserver CMake Options
-- =======================================================
--   USE_ALL_SYSTEM_LIBS: OFF
--   USE_SYSTEM_CPR: OFF
--   USE_SYSTEM_GTEST: OFF
-- =======================================================
-- Found mhd: /usr/include  
-- Found CURL: /usr/lib64/libcurl.so (found version "7.66.0") 
-- Checking compiler flags for C++11 support.
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX0X
-- Performing Test COMPILER_SUPPORTS_CXX0X - Success
-- C++11 is supported.
-- C++ Requests CMake Options
-- =======================================================
--   USE_SYSTEM_CURL: YES
--   BUILD_CPR_TESTS: NO
--   GENERATE_COVERAGE: OFF
--   CPR_CURL_NOSIGNAL: OFF
--   USE_SYSTEM_GTEST: OFF
--   CMAKE_USE_OPENSSL: ON
-- =======================================================
-- Set CURL_FOUND to TRUE.
-- Set CURL_LIBRARIES to /usr/lib64/libcurl.so.
-- Set CURL_INCLUDE_DIRS to /usr/include.
-- Using CURL_INCLUDE_DIRS: /usr/include.
-- Using CURL_LIBRARIES: /usr/lib64/libcurl.so.
-- The C compiler identification is Clang 6.0.1
-- Check for working C compiler: /externals/clang6.0-0/bin/clang
-- Check for working C compiler: /externals/clang6.0-0/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PythonInterp: /usr/bin/python (found version "2.7.17") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- JsonCpp Version: 1.8.4
-- Looking for C++ include clocale
-- Looking for C++ include clocale - found
-- Looking for localeconv
-- Looking for localeconv - found
-- Looking for C++ include sys/types.h
-- Looking for C++ include sys/types.h - found
-- Looking for C++ include stdint.h
-- Looking for C++ include stdint.h - found
-- Looking for C++ include stddef.h
-- Looking for C++ include stddef.h - found
-- Check size of lconv
-- Check size of lconv - done
-- Performing Test HAVE_DECIMAL_POINT
-- Performing Test HAVE_DECIMAL_POINT - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /externals/elasticlient0.1.0-1_src/elasticlient/build
INFO -   stderr: CMake Warning:
  Manually-specified variables were not used by the project:

    BOOST_ROOT
INFO - 
DEBUG - 32 processor(s) detected, using -j31
DEBUG - run_env: {'TERM': 'xterm', 'SHLVL': '2', 'OLDPWD': '/', 'HOSTNAME': '1fdf0a7a43b6', 'MAKELEVEL': '1', 'PWD': '/externals', 'MAKE_TERMERR': '/dev/pts/0', '_': './build.py', 'HOME': '/root', 'PATH': '/usr/local/rvm/rubies/ruby-2.6.5/bin:/usr
/local/rvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'GPG_TTY': '/dev/pts/0', 'MFLAGS': '', 'GEM_HOME': '/usr/local/rvm/rubies/ruby-2.6.5', 'MAKEFLAGS': '', 'MAKE_TERMOUT': '/dev/pts/0'}
INFO - running: cd build && make, unsafe_shell=True, check_rc=build failed, retries=0
INFO -   stdout: make[1]: Entering directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
make[2]: Entering directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
make[3]: Entering directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
make[3]: Leaving directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
make[3]: Entering directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
[  2%] Building CXX object external/httpmockserver/external/cpr/cpr/CMakeFiles/cpr.dir/error.cpp.o
make[3]: Leaving directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
make[2]: Leaving directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
make[1]: Leaving directory '/externals/elasticlient0.1.0-1_src/elasticlient/build'
INFO -   stderr: /externals/elasticlient0.1.0-1_src/elasticlient/external/httpmockserver/external/cpr/cpr/error.cpp:41:14: fatal error: duplicate case value 'CURLE_PEER_FAILED_VERIFICATION'
        case CURLE_SSL_CACERT:
             ^
/usr/include/curl/curl.h:623:26: note: expanded from macro 'CURLE_SSL_CACERT'
#define CURLE_SSL_CACERT CURLE_PEER_FAILED_VERIFICATION
/externals/elasticlient0.1.0-1_src/elasticlient/external/httpmockserver/external/cpr/cpr/error.cpp:25:14: note: previous case defined here
        case CURLE_PEER_FAILED_VERIFICATION:
             ^
1 error generated.
make[3]: *** [external/httpmockserver/external/cpr/cpr/CMakeFiles/cpr.dir/build.make:115: external/httpmockserver/external/cpr/cpr/CMakeFiles/cpr.dir/error.cpp.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:191: external/httpmockserver/external/cpr/cpr/CMakeFiles/cpr.dir/all] Error 2
make[1]: *** [Makefile:141: all] Error 2
INFO - 
ERROR - build failed

fixed in the meantime with this local edit... but i'm sure there's a better way

externals/elasticlient0.1.0-1_src/elasticlient/external/httpmockserver/external/cpr/cpr $ git describe --tags
1.3.0-57-g3d14e61

externals/elasticlient0.1.0-1_src/elasticlient/external/httpmockserver/external/cpr/cpr $ git diff
diff --git a/cpr/error.cpp b/cpr/error.cpp
index 713cb10..7a7f9d6 100644
--- a/cpr/error.cpp
+++ b/cpr/error.cpp
@@ -38,8 +38,8 @@ ErrorCode Error::getErrorCodeForCurlError(std::int32_t curl_code) {
             return ErrorCode::SSL_LOCAL_CERTIFICATE_ERROR;
         case CURLE_SSL_CIPHER:
             return ErrorCode::GENERIC_SSL_ERROR;
-        case CURLE_SSL_CACERT:
-            return ErrorCode::SSL_CACERT_ERROR;
+//        case CURLE_SSL_CACERT:
+//            return ErrorCode::SSL_CACERT_ERROR;
         case CURLE_USE_SSL_FAILED:
             return ErrorCode::GENERIC_SSL_ERROR;
         case CURLE_SSL_ENGINE_INITFAILED:

Json parsing speed too slow

JsonCpp library used to parse scroll results is extremely slow. Scrolling through larger indices show the delay much more. In numbers: what the current implementation iterates in 85 seconds, scroll implemented using rapidjson did in 25 seconds.

Unfortunately JsonCpp is used in Scroll header interface, so the change would break API/ABI. But the speed is worth it.

Possible implementation with rapidjson (as it is header only library):

  • hide it completely inside elasticlient, without publishing it in public interface
  • make the rapidjson parsing public (header only), so each client could compile it again

First option means that parsing has to be done twice. Internally and then for data parsing itself. Otherwise there is a risk of using different versions (binary incompatible) of rapidjson.
The second option allows shared parsing (internall and client's data) without possibility of using different versions of rapidjson (one compiled into elasticlient, another version in client code).

Should move away from deprecated Json::Reader::Reader()

elasticlient/src/bulk.cc:167:18: warning: ‘Json::Reader::Reader()’ is deprecated: Use CharReader and CharReaderBuilder instead [-Wdeprecated-declarations]
167 | Json::Reader reader;

elasticlient/src/scroll.cc:62:18: warning: ‘Json::Reader::Reader()’ is deprecated: Use CharReader and CharReaderBuilder instead [-Wdeprecated-declarations]
62 | Json::Reader reader;
| ^~~~~~

Cannot build on Windows

Environment

  • Windows 11 (OS build : 22000.795) - 64bit
  • Visual Studio 2022
  • CMake 3.23.3 (with gui)
  • vcpkg ( installed on my PC : curl, openssl, zlib, boost, ... )

I did this :

  1. README.md

    • step 1 >>> step 2 >> step 3 ( It's going well so far. )
  2. Pull Request Link

but, not build my solution. errors are

  • Command line error D8021 : invalid numeric argument '/Wextra' -> when I tried no.2 (pull request)

  • c4430 error ( If I erase /wextra from the build option, c4430 appears countless times, resulting in c1003. )

moving a temporary object prevents copy elision

Compiling with clang (Apple LLVM version 10.0.0) gives several warnings:

.../elasticlient/src/client.cc:79:11: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
   return std::move(impl->performRequest(method, urlPath, body));

note: remove std::move call here
   return std::move(impl->performRequest(method, urlPath, body));

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.