Hi, is it possible to run this in local environment?

Hello,

I try to build my react project with http-aws-es but connector.js can't be minify.
I would like to know if you can publish a pre-compiled version.

https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/template/README.md#npm-run-build-fails-to-minify

Thanks in advance

I am totally shocked why my code in node.js doesnt work and with python works like a charm? It is so weird for me :( Could someone take a look please and tell me what I am doing wrong?

Python code:

from elasticsearch import Elasticsearch
import json
esclient = Elasticsearch(['https://61699**.eu-central-1.aws.cloud.es.io/recipes/'],
http_auth=('d**', 'e.**'), scheme="https")

response = esclient.search(index="", body=json.dumps({"query": {"match": {'Title':'Easter Bunny Cupcakes'}}}))
    
print response

Node.js code:

const elasticsearch = require('elasticsearch');
const AWS = require('aws-sdk');

const client = elasticsearch.Client({
  hosts: ''61699**.eu-central-1.aws.cloud.es.io/recipes/',
  connectionClass: require('http-aws-es'),
  amazonES: {
    accessKey: 'd**',
    secretKey: 'e.**',
  },
  awsConfig: new AWS.Config({region: 'eu-central-1'}),
});


client.index({
  index: '',
  type: '_doc',
  body: {
    query: {
      match: {
        Title: 'Easter Bunny Cupcakes',
      },
    },
  },
}, function(err, data) {
  console.log(err);
});

This might be more of a request than an issue.

Does this module work from within the browser? I don't have a NodeJS app and I'm trying to execute this script from within the browser. I'm using Browserify to require and bundle the necessary modules.

var AWS = require('aws-sdk');
var creds = new AWS.Credentials('AKID','Security');
var  es = require('elasticsearch').Client({
    host: 'https://hostname.us-east-1.es.amazonaws.com/',
    connectionClass: require('http-aws-es'),
    amazonES: {
      region: 'us-east-1',
      credentials: creds
    }
  });

es.cluster.health({},function(err,resp,status) {
    console.log("-- Client Health --",resp);
});

I get the following error in the console:

AWS Credentials error: No credentials to load

Is it possible to adapt this awesome module to work from within the browser?

I'm not sure what's causing the issue, but copying here in case it's a problem with this library and not a regression in the sdk aws/aws-sdk-js#2469

-----------------------My Code--------------------------------
import es from 'elasticsearch';
import awsSdk from 'aws-sdk';
import httpAwsEs from 'http-aws-es';
import AWS from '../../config/aws';

const options = {
host: '....................',
connectionClass: httpAwsEs,
awsConfig: new awsSdk.Config({
credentials: new awsSdk.Credentials(AWS.accessKeyId, AWS.secretAccessKey),
region: AWS.region,
}),
httpOptions: {}
/amazonES: {
region: AWS.region,
accessKey: AWS.accessKeyId,
secretKey: AWS.secretAccessKey
}/
};
const EsClient = new es.Client(options);
-----------------------------------------------ERROR-------------------------------
Trace: { Authorization Exception :: {"path":"/","query":{},"statusCode":403,"response":""}
at respond (D:\Workspace\2017.........\server\node_modules\elasticsearch\src\lib\transport.js:307:15)
at checkRespForFailure (D:\Workspace\2017............\server\node_modules\elasticsearch\src\lib\transport.js:266:7)
at IncomingMessage.cleanUp (D:\Workspace\2017....................\server\node_modules\http-aws-es\connector.js:61:9)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)
status: 403,
displayName: 'AuthorizationException',
message: 'Authorization Exception',
path: '/',
query: {},
body: undefined,
statusCode: 403,
response: '',
toString: [Function],
toJSON: [Function] }
at D:/Workspace/2017/................./server/app/controllers/EsController.js:46:13
at respond (D:\Workspace\2017.................\server\node_modules\elasticsearch\src\lib\transport.js:326:9)
at checkRespForFailure (D:\Workspace\2017.................\server\node_modules\elasticsearch\src\lib\transport.js:266:7)
at IncomingMessage.cleanUp (D:\Workspace\2017.................\server\node_modules\http-aws-es\connector.js:61:9)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)

I am using http-aws-es and that is the error. It works well when I change the connectionClass from httpAwsEs to 'http'. Could y guys explain to me the reason why I fail to use http-aws-es in this case?

Is there a reason this still is on ES 15 and hasn't been updated in 2 years?

It seems like if anything, you should update to the code in this fork - https://github.com/minhloi/http-aws-es so you get all the new stuff.

Thoughts?

I have tried to use it with rect-native but it is giving me a lots of errors
first error I got is
error: Error: Unable to resolve module http from .../node_modules/elasticsearch/src/lib/connectors/http.js: http could not be found within the project.

Is it even possible to use it with react-native

Unhandled Promise rejection: Syntax error ; Zone: ; Task: Promise.then ; Value: SyntaxError: Syntax error SyntaxError: Syntax error
at ../../../../http-aws-es/connector.js (http://localhost:4200/vendor.bundle.js:3512:1)
at webpack_require (http://localhost:4200/inline.bundle.js:55:12)
at UserService (eval code:57:9)
at _createClass (eval code:11121:13)
at _createProviderInstance$1 (eval code:11091:13)
at resolveNgModuleDep (eval code:11075:13)
at _createClass (eval code:11121:13)
at createProviderInstance$1 (eval code:11091:13)
at resolveNgModuleDep (eval code:11075:13)
at NgModuleRef.prototype.get (eval code:12308:9)
"Unhandled Promise rejection:"
"Syntax error"
"; Zone:"
""
"; Task:"
"Promise.then"
"; Value:"
{
[functions]: ,
proto: { },
__zone_symbol__currentTask: { },
description: "Syntax error",
message: "Syntax error",
name: "SyntaxError",
ngDebugContext: { },
number: -2146827286,
stack: "SyntaxError: Syntax error
at ../../../../http-aws-es/connector.js (http://localhost:4200/vendor.bundle.js:3512:1)
at webpack_require (http://localhost:4200/inline.bundle.js:55:12)
at UserService (eval code:57:9)
at _createClass (eval code:11121:13)
at _createProviderInstance$1 (eval code:11091:13)
at resolveNgModuleDep (eval code:11075:13)
at _createClass (eval code:11121:13)
at createProviderInstance$1 (eval code:11091:13)
at resolveNgModuleDep (eval code:11075:13)
at NgModuleRef.prototype.get (eval code:12308:9)",
Symbol()_i.sxim7lganrf: undefined,
Symbol(observable)_h.sxim7lganrf: undefined,
Symbol(rxSubscriber)_g.sxim7lganrf: undefined
}
"SyntaxError: Syntax error
at ../../../../http-aws-es/connector.js (http://localhost:4200/vendor.bundle.js:3512:1)
at webpack_require (http://localhost:4200/inline.bundle.js:55:12)
at UserService (eval code:57:9)
at _createClass (eval code:11121:13)
at _createProviderInstance$1 (eval code:11091:13)
at resolveNgModuleDep (eval code:11075:13)
at _createClass (eval code:11121:13)
at createProviderInstance$1 (eval code:11091:13)
at resolveNgModuleDep (eval code:11075:13)
at NgModuleRef.prototype.get (eval code:12308:9)"

any idea ?

Hi,
I prefer to use a singleton connection for connecting to elastic search - We use an IAM role rather than IAM user for security reasons.

I'm receiving a 403 error after the session token expires - Does this library interact with aws-sdk to refresh tokens ? (or) is this something that should be handled outside of this library ?

var AWS = require('aws-sdk');
var config = require('config');
var connectionClass = require('http-aws-es');
var elasticsearch = require('elasticsearch');

AWS.config.getCredentials(function() {

AWS.config.update({
  credentials: new AWS.Credentials(AWS.config.credentials.accessKeyId,AWS.config.credentials.secretAccessKey,AWS.config.credentials.sessionToken),
  region: 'us-east-1'
});

}

)

var client = new elasticsearch.Client({
  host: `${config.get('elasticSearch.host')}`,
  log: 'debug',
  connectionClass: connectionClass,
  amazonES: {
    credentials: new AWS.EnvironmentCredentials('AWS')
  }
   
});

module.exports = client;

Hi Guys can you please fix this dependecy identified by veracode
High Risk Prototype Pollution aws-sdk 2.518.0

thanks

the inclusion of aws-sdk will break in a lambda environment. Lambda injects it's own copy of aws-sdk which it requires you to use. I've made my own copy of this library that allows for the aws-sdk to be injected: https://github.com/cphoover/http-aws-es-di

not the cleanest solution but allows me to use this library in a lambda environment.

Hi,

I'm trying to use the /_bulk operation with ES 5.3 but I keep getting the error "Data must be a string or a buffer". I create my client the exact same way as the example in elastic/elasticsearch-es#274.

The error message doesn't give a lot but here it is:

Elasticsearch ERROR: 2017-10-26T08:18:04Z
  Error: Request error, retrying
  POST http://<my-es-instance>.amazonaws.com/_bulk => Data must be a string or a buffer
      at Log.error (/app/node_modules/elasticsearch/src/lib/log.js:225:56)
      at checkRespForFailure (/app/node_modules/elasticsearch/src/lib/transport.js:258:18)
      at cleanUp (/app/node_modules/http-aws-es/connector.js:58:9)
      at <anonymous>
      at process._tickDomainCallback (internal/process/next_tick.js:228:7)

Using trace logging I can see that my body is there and it gets logged in the error handler callback in elasticsearch-js.

I've tried to serialize the body myself before passing it to the bulk function but I get the same error. I don't pass any other parameters.

I haven't been able to trace down where the error actually occurs but I suspect it's in this.httpClient.handleRequest.

My code works fine without the AwsEsConnector (using a local Elasticsearch).

I'm beginning to suspect this could be a bug in the connector?

Hi,

When I debug with this Plugin and aws-xray-sdk version 2.3.3 this plugin stops working and i can not connect to the es cluster anymore. it works when i dont activate the aws-xray plugin.

I use AWS-SDK 2.329.0 & http-aws-es 6.0.0

Just following the tutorial I got this error

    throw err;
    ^

 Error: Cannot find module 'elasticsearch'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)

Hi,

after an update, I tried to test my source code.
It all looked good, but a few tests suddenly fail each time with the "group" property, which is an string-array or the "latlng" which is a geo_point.

The response is:

Error: Authorization Exception
    at respond (/home/rene/Projekte/SmashoCore/node_modules/elasticsearch/src/lib/transport.js:307:15)
    at checkRespForFailure (/home/rene/Projekte/SmashoCore/node_modules/elasticsearch/src/lib/transport.js:266:7)
    at IncomingMessage.cleanUp (/home/rene/Projekte/SmashoCore/node_modules/http-aws-es/connector.js:60:9)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)
  status: 403,
  displayName: 'AuthorizationException',
  message: 'Authorization Exception',
  path: '/_bulk',
  query: { refresh: 'true' },
  body: '{"index":{"_index":"newsmasho","_type":"User","_id":"User_6v7dr8j92p3jer"}}\n{"__model_name":"User","__model_options":{"created":true},"email":"[email protected]","name":"Rainer","latlng":{"lat":0,"lon":0},"createdAt":1508673246723,"ratingHistory":[],"watchList":[],"groups":["Antiquitäten und Sammler"],"favoriteList":[],"updatedAt":1508673246914}\n',
  statusCode: 403,
  response: '{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method.

npm install http-aws-es
[..................] / idealTree:snapshot: sill idealTree buildDeps
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/elasticsearch
npm ERR! elasticsearch@"^16.7.1" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer elasticsearch@"^15.0.0" from [email protected]
npm ERR! node_modules/http-aws-es
npm ERR! http-aws-es@"^6.0.0" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR! See /root/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-01-19T14_38_27_204Z-debug.log

event if i try --force the error is same.

I upgraded to version 2.0 and I'm unclear as how to to create multiple objects to different regions (with different credentials).

For regular AWS objects I can use things like:

const one = new AWS.S3({ region: 'us-east-1' })
const two = new AWS.S3({ region: 'us-west-1' })

I get an error when trying to connect to ES on AWS.
Using this code supplied in the readme:

var es = require('elasticsearch').Client({
  hosts: 'https://amazon-es-host.us-east-1.es.amazonaws.com',
  connectionClass: require('http-aws-es'),
  amazonES: {
    region: 'us-east-1',
    accessKey: 'AKID',
    secretKey: 'secret'
  }
});

I guess it's a dependency that does not work as intended.

Something changed in the elasticsearch module in version 12.1.1 and up, which is causing request signing errors for the clearScroll() method.

"The request signature we calculated does not match the signature you provided."

Reverting to 12.1.0 fixes the issue.

I think that this commit is the culprit: elastic/elasticsearch-js@97706ce

I setup this library yesterday and its been working really well (thank you).

I have a process running that receives requests and then executes a search. The process initializes an ES client once when it starts and then leverages the keep alive connection.

This was working last night and then this morning every request the process made returned an AuthorizationException with a status of 403.

I'm using the AWS.CredentailProviderChain to get the credentials object which resolves to the EC2MetadataCredentails.

My take on what is happening:

• the client starts up and uses the metadata credentials to sign requests
• the API keys rotate on the metadata server so after a certain period of time, the credentials http-aws-es is using become invalid
• any request made will fail until the process restarts so we can fetch the new metadata credentials

I think the right answer is to take the credentials object, but also a method to refresh credentials. Then if we get a 403, we can attempt to refresh the credentails and try the request again.

Sometimes when I use this library, it prints following messages in the log and then fails.

(node:17597) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Data must be a string or a buffer
(node:17597) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

The actual error messages vary but they all come with the UnhandledPromiseRejectionWarning. I am not sure at which part the promise chain is broken.

Here is an example of how I use the library:

const es = require('elasticsearch');
const awsES = require('http-aws-es');

let client = new es.Client({
  hosts: 'es host',
  connectionClass: awsES,
  amazonES: {
    region: 'us-east-1',
    accessKey: 'my access key',
    secretKey: 'my secret key'
  }
});

client.indices.delete({
    index: 'my_index'
})
.then(() => console.log('finish'))
.catch((err) => console.log(err));

version: 2.0.3

npm run test -- --endpoint https://my-aws-es.eu-west-1.es.amazonaws.com --region eu-west-1

ends with socket error however,

npm run test -- --endpoint http://my-aws-es.eu-west-1.es.amazonaws.com --region eu-west-1
npm run test -- --endpoint https://my-aws-es.eu-west-1.es.amazonaws.com:80 --region eu-west-1

works fine

@TheDeveloper I too was facing the issue User: anonymous is not authorized to perform: es:ESHttpPost and found your package via Stackoverflow. Based on the findings I changed my es-client code in the below way.

const AWS = require('aws-sdk');
const config = require('../../configuration');
const elasticsearch = require('elasticsearch');

const esClient = new elasticsearch.Client({
  host: config.get('elasticsearch.rental.host'),
  log: config.get('elasticsearch.log_level'),
  connectionClass: require('http-aws-es'),
  amazonES: {
    region: 'us-west-2',
    credentials: new AWS.EnvironmentCredentials('AWS'),
  },
});

module.exports = esClient;

But now I am getting No Living connections. 7 hours now and still not able to figure out what's happening. Any ideas what might be going wrong?

I'm trying to get http-aws-es working in a browser application using the elasticsearch-js library. My code is currently looking like this but I'm getting a "TypeError: invalid host" in the browser console.

`
let AWS = require('aws-sdk')

  let options = {
    hosts: ['https://my-es-host.es.amazonaws.com'],
    connectionClass: require('http-aws-es'),
    awsConfig: new AWS.Config({
      credentials: new AWS.Credentials('AKA...', 'secret'),
      region: 'eu-central-1'
    }),
    httpOptions: {}
  }

  let es = require('elasticsearch-browser').Client(options)

  es.ping({
    requestTimeout: 30000
  }, function (error) {
    if (error) {
      console.error('elasticsearch cluster is down!')
    } else {
      console.log('All is well')
    }
  })`

Did anyone get a similar setup working in the browser and what am I doing wrong?

Hello All,

Just noticed that latest client API by Elastic installed via npm install @elastic/elasticsearch does not have connectionClass attribute while creating the client object. This break http-aws-es signing of requests for AWS hosted ES.

Any idea how to support the latest Client API. Is there a fork of this already out there?

Latest ES Client API: https://github.com/elastic/elasticsearch-js

Though the Readme of this library mentions supporting it - but internally uses the legacy library "elasticsearch" in package.json.

Is it ok that aws-sdk in devDependencies instead of Dependencies?

Hi,
I'm trying to add config like you're showing, in my case

AWS.config.update({
  httpOptions: {
    agent: new https.Agent({rejectUnauthorized: true, ca: certs});
  }
});

however, the module makes its own instance of the AWS sdk it seems and my request fails... Do you have a complete example dealing with this? Thanks!

on 2nd look, it may be related to a bug in the aws-sdk itself, see 2nd answer here:

https://stackoverflow.com/questions/39970302/aws-sdk-for-node-js-connection-management

Hello!
After following the usage instructions I am getting the following error when trying to hook up the client configuration:

Uncaught TypeError: AWS.NodeHttpClient is not a constructor at HttpAmazonESConnector.request (connector.js:106) at sendReqWithConnection (transport.js:210) at Object.webpackJsonp../node_modules/elasticsearch/src/lib/utils.js._.applyArgs (utils.js:256) at wrapper (lodash.js:5232) at Item../node_modules/process/browser.js.Item.run (browser.js:153) at drainQueue (browser.js:123)

Any help is appreciated!

Hi again,

After reading through the README and running this code:

AWS.config.update({ region: 'us-east-1', credentials: new AWS.Credentials(data.Credentials.AccessKeyId, data.Credentials.SecretKey) }); var ElasticSearch = require('elasticsearch').Client({ hosts: [{myHost}], //hostURL not actually shown connectionClass: require('http-aws-es') });

I am getting a error that looks like this:

es.js:27 TypeError: Cannot read property '1' of null
at Object. (index.js:1)
at Object../node_modules/http-aws-es/index.js (index.js:9)
at webpack_require (bootstrap d1d16ef…:686)
at fn (bootstrap d1d16ef…:105)
at es.js:19

which seems to be something to do with the index.js file in http-aws-es. Am I supposed to specify what version of node I am using somewhere?

Once running it a second time in the same session, a new error appears:

es.js:27 TypeError: Invalid connectionClass "[object Object]", expected a function or one of xhr, jquery, angular
at Object.webpackJsonp../node_modules/elasticsearch/src/lib/utils.js._.funcEnum (utils.js:358)
at new ConnectionPool (connection_pool.js:33)
at new Transport (transport.js:22)
at EsApiClient (client.js:57)
at Function.Client (client.js:101)
at es.js:17

Any help is appreciated!

Unhandled promise rejection (rejection id: 1): UnrecognizedClientException: The security token included in the request is invalid.

Hi, I try to catch this Exception, but I can't do that.

What should I do in that case, I call the ping method from ES, after that I will return the health for that.

router.route('/health').get(function (req, res, next) {
    let client = ES.Client({
        hosts: [config.elasticSeachUrl],
        connectionClass: require('http-aws-es'),
        awsConfig: new AWS.Config({
            region: config.awsZone
        })
    }).catch((error) => {
        console.log(error)
    });
    client.ping({
            // ping usually has a 3000ms timeout
            requestTimeout: 1000
        })
        .then(() => console.log('finish'))
        .catch((err) => console.log(err));

});

but it always is throwing me the exception but i can't to catch that.

This is how I'm instantiating the client:

const AWS = require('aws-sdk');

AWS.config.update({ region: 'ca-central-1' });

const AWS_ES = require('http-aws-es');
const elasticsearch = require('elasticsearch');

new elasticsearch.Client({
  host: 'https://***.ca-central-1.es.amazonaws.com:80/'
  log: 'trace'
  apiVersion: '5.3'
  connectionClass: AWS_ES,
});

With 2.0.3 of http-aws-es I get this trace:

Elasticsearch INFO: 2017-07-15T11:17:59Z
  Adding connection to https://***.es.amazonaws.com:80/

Elasticsearch DEBUG: 2017-07-15T11:17:59Z
  starting request {
    "method": "GET",
    "path": "/_cat/templates/some-template",
    "query": {
      "format": "json"
    }
  }

Elasticsearch TRACE: 2017-07-15T11:18:00Z
  -> GET https://***.es.amazonaws.com:80/_cat/templates/some-template?format=json

  <- 200
  [
    {
      "name": "some-template",
      "template": "some-template-*",
      "order": "1",
      "version": "1"
    }
  ]

Elasticsearch DEBUG: 2017-07-15T11:18:00Z
  Request complete

With 2.0.4 I get this trace:

Elasticsearch INFO: 2017-07-15T11:10:23Z
  Adding connection to https://***.ca-central-1.es.amazonaws.com:80/

Elasticsearch DEBUG: 2017-07-15T11:10:23Z
  starting request {
    "method": "GET",
    "path": "/_cat/templates/some-template",
    "query": {
      "format": "json"
    }
  }


Elasticsearch TRACE: 2017-07-15T11:10:23Z
  -> GET https://***.ca-central-1.es.amazonaws.com:80/_cat/templates/some-template?format=json

  <- 0


Elasticsearch ERROR: 2017-07-15T11:10:23Z
  Error: Request error, retrying
  GET https://***.ca-central-1.es.amazonaws.com:80/_cat/templates/some-template?format=json => write EPROTO 140737038898112:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:794:

      at Log.error (/Users/ktonon/repos/folio-service/node_modules/elasticsearch/src/lib/log.js:225:56)
      at checkRespForFailure (/Users/ktonon/repos/folio-service/node_modules/elasticsearch/src/lib/transport.js:258:18)
      at HttpAmazonESConnector.<anonymous> (/Users/ktonon/repos/folio-service/node_modules/http-aws-es/node6.js:77:11)
      at ClientRequest.bound (/Users/ktonon/repos/folio-service/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
      at ClientRequest.<anonymous> (/Users/ktonon/repos/folio-service/node_modules/aws-sdk/lib/http/node.js:89:19)
      at emitOne (events.js:101:20)
      at ClientRequest.emit (events.js:188:7)
      at TLSSocket.socketErrorListener (_http_client.js:310:9)
      at emitOne (events.js:96:13)
      at TLSSocket.emit (events.js:188:7)

Elasticsearch TRACE: 2017-07-15T11:10:23Z
  -> GET https://***.ca-central-1.es.amazonaws.com:80/_cat/templates/some-template?format=json

  <- 0


Elasticsearch ERROR: 2017-07-15T11:10:23Z
  Error: Request error, retrying
  GET https://***.ca-central-1.es.amazonaws.com:80/_cat/templates/some-template?format=json => write EPROTO 140737038898112:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:../deps/openssl/openssl/ssl/s23_clnt.c:794:

      at Log.error (/Users/ktonon/repos/folio-service/node_modules/elasticsearch/src/lib/log.js:225:56)
      at checkRespForFailure (/Users/ktonon/repos/folio-service/node_modules/elasticsearch/src/lib/transport.js:258:18)
      at HttpAmazonESConnector.<anonymous> (/Users/ktonon/repos/folio-service/node_modules/http-aws-es/node6.js:77:11)
      at ClientRequest.bound (/Users/ktonon/repos/folio-service/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
      at emitOne (events.js:101:20)
      at ClientRequest.emit (events.js:188:7)
      at TLSSocket.socketErrorListener (_http_client.js:310:9)
      at emitOne (events.js:96:13)
      at TLSSocket.emit (events.js:188:7)
      at onwriteError (_stream_writable.js:346:10)

Elasticsearch TRACE: 2017-07-15T11:10:23Z
  -> HEAD https://***.ca-central-1.es.amazonaws.com:80/

  <- 0


Elasticsearch WARNING: 2017-07-15T11:10:23Z
  Unable to revive connection: https://***.ca-central-1.es.amazonaws.com:80/

Elasticsearch TRACE: 2017-07-15T11:10:23Z
  -> HEAD https://***.ca-central-1.es.amazonaws.com:80/

  <- 0


Elasticsearch WARNING: 2017-07-15T11:10:23Z
  Unable to revive connection: https://***.ca-central-1.es.amazonaws.com:80/

Elasticsearch WARNING: 2017-07-15T11:10:23Z
  No living connections

Elasticsearch WARNING: 2017-07-15T11:10:23Z
  No living connections

AFAIK nothing else changes between the two invocations, its just the version of http-aws-es.

Thanks!

I encounter this error : Cannot find module 'aws-sdk'
It appears there is no dependency to aws-sdk in the package.json.
Since, the connector uses aws-sdk, it must declare the dependency to it.
I can do a PR if you want.

http-aws-es not working with the latest elasticsearch version.
When i try to build image, during the npm install it fails for http-aws-es and elasticsearch with:

npm ERR! code ERESOLVE npm ERR! ERESOLVE could not resolve npm ERR! npm ERR! While resolving: [email protected] npm ERR! Found: [email protected] npm ERR! node_modules/elasticsearch npm ERR! elasticsearch@"^16.7.2" from the root project npm ERR! npm ERR! Could not resolve dependency: npm ERR! peer elasticsearch@"^15.0.0" from [email protected] npm ERR! node_modules/http-aws-es npm ERR! http-aws-es@"^6.0.0" from the root project npm ERR! npm ERR! Conflicting peer dependency: [email protected] npm ERR! node_modules/elasticsearch npm ERR! peer elasticsearch@"^15.0.0" from [email protected] npm ERR! node_modules/http-aws-es npm ERR! http-aws-es@"^6.0.0" from the root project npm ERR! npm ERR! Fix the upstream dependency conflict, or retry npm ERR! this command with --force, or --legacy-peer-deps npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

http-aws-es: 6.0.0
elasticsearch: 16.7.2

This isn't an issue it's more of a recommendation to anyone using this lib.

If you need any functionality in any of the pull requests you can easily just extend the HttpAmazonESConnector class to override the request method and add any functionality you need. This is what I use and it seems to be working for me.

// AsyncHttpAmazonESConnector.js

import HttpAmazonESConnector from 'http-aws-es';

function isPromise(obj) {
  return Boolean(obj)
  && (typeof obj === 'object' || typeof obj === 'function')
  && typeof obj.then === 'function';
}

class AsyncHttpAmazonESConnector extends HttpAmazonESConnector {
  request(params, cb) {
    const credentials = this.amazonES.credentials;

    if (credentials && isPromise(credentials)) {
      credentials
        .then((creds) => {
          if (creds.needsRefresh()) {
            return creds.refreshPromise();
          }
          return creds;
        })
        .then((creds) => {
          this.creds = creds;
          super.request(params, cb);
        })
        .catch((err) => {
          cb(err);
        });
    } else {
      super.request(params, cb);
    }
  }
}

export default AsyncHttpAmazonESConnector;

// esClient.js

import elasticsearch from 'elasticsearch';
import awsSDK from 'aws-sdk';
import AsyncHttpAmazonESConnector from './AsyncHttpAmazonESConnector';

const providerChain = new awsSDK.CredentialProviderChain([
  () => new awsSDK.EnvironmentCredentials('AWS'),
  () => new awsSDK.EnvironmentCredentials('AMAZON'),
  () => new awsSDK.SharedIniFileCredentials(),
  () => ((awsSDK.ECSCredentials.prototype.getECSRelativeUri() !== undefined)
    ? new awsSDK.ECSCredentials()
    : new awsSDK.EC2MetadataCredentials()),
]);

const esClient = new elasticsearch.Client({
  host: ES_HOST,
  connectionClass: AsyncHttpAmazonESConnector,
  amazonES: {
    region: AWS_REGION,
    credentials: providerChain.resolvePromise(),
  },
});

Hope this helps anyone who is stuck.

Requests to for example /_alias seem not to be signed. I always get a 403 from Elasticsearch. Even with an open IAM policy that allows any es:* action on the domain.

{
            "Action": [
                "es:*"
            ],
            "Resource": "arn:aws:es:eu-central-1:xxxxx:domain/my-domain/*",
            "Effect": "Allow"
        }

The es-client method that was invoked is client.indices.existsAlias()

Are there any known issues with that or have I just set the wrong IAM policy?

i was trying to giving access to elasticsearch based on cognito users role .for authenticated role it was working fine .but for unauth role it is getting denied. i added both auth and unauth role

Currently there is no error handling for the function getAWSCredentials(). If it fails to retrieve the AWS credential, it will simply print the following message in the console.

(node:17597) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): 
TypeError: Data must be a string or a buffer
(node:17597) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the 
future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit 
code.

It doesn't throw an error with a complete failure stacktrace and stop the program from running further even though the ES initialization completely fails. Because of this failure, if the user continue to use ES, he will always get a time-out error.

I am using elasticsearch 11.0.1 with http-aws-es 1.1.3, connecting to ES 2.3. The setup work perfectly for updating mapping, searching, etc, so I know that portion is set up correctly.

I am trying to register a snapshot repository on s3, and am following along at http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains.html#es-managedomains-snapshots and have set up all permissions/buckets/etc as instructed. I am using the "snapshot.createRepository" method, as detailed at https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/api-reference-2-3.html#api-snapshot-createrepository-2-3 . I have spent quite some time searching, and only found people asking the same question, or solutions that involve the yml config files, which are inaccessible as this is the hosted service.

Do you have any suggestions? I only ask here, as this is how the permissions are being made available to the ES client, and would think that is also how they would be made available for this.

Code and Error follow.

client.snapshot.createRepository({
	repository: 'backup-s3',
	body: {
		type: 's3',
		settings: {
			bucket: 'interkn-elasticsearch-backup',
			region: 'us-west-2',
			role_arn: 'arn:aws:iam::XXX:role/elasticsearch-backup',
		},
	},
});

{
    "error":{
        "root_cause":[
            {
                "type":"repository_exception",
                "reason":"[backup-s3] failed to create repository"
            }
        ],
        "type":"repository_exception",
        "reason":"[backup-s3] failed to create repository",
        "caused_by":{
            "type":"creation_exception",
            "reason":"Guice creation errors:\n\n1) Error injecting constructor, com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain\n  at org.elasticsearch.repositories.s3.S3Repository.<init>(Unknown Source)\n  while locating org.elasticsearch.repositories.s3.S3Repository\n  while locating org.elasticsearch.repositories.Repository\n\n1 error",
            "caused_by":{
                "type":"amazon_client_exception",
                "reason":"Unable to load AWS credentials from any provider in the chain"
            }
        }
    },
    "status":500
}

I started implementing this module today because we are in the middle of switching our ES over to AWS. I ran into an issue where the AWS Credential object eventually throws an error:
The security token included in the request is expired

According to amazon the sdk should be refreshing this automatically but that doesn't seem to be the case. I'm using almost exactly what is listed in the README of this project for using the AWS Credential object.

Has anyone ran into this problem and/or found a way to resolve it?

Hello! Thank you for keeping up this work!

We are working on a new JavaScript client for Elasticsearch, and you can find the work in elastic/elasticsearch-js#next, the new client will replace the current one, which will be deprecated.
The code has been rewritten from scratch, so your current implementation will not be compatible.

Please let me know if you have any question :)

on my production machine i'm getting an Authorization Exception Error.
On development machine it works perfectly fine with the same credentials. what could cause this error?

Error: Authorization Exception
0|Website  |     at respond 
(/home/www/Klimapartner/node/node_modules/elasticsearch/src/lib/transport.js:307:15)
0|Website  |     at checkRespForFailure 
(/home/www/Klimapartner/node/node_modules/elasticsearch/src/lib/transport.js:266:7)
0|Website  |     at HttpAmazonESConnector.<anonymous> 
(/home/www/Klimapartner/node/node_modules/http-aws-es/node8.js:73:9)
0|Website  |     at IncomingMessage.bound 

...

There 2 examples shown

1. using hard coded keys
   2j using a named profile

Is it possible to use the default profile from an instance profile?

When I try to run this module with node v.7.1.0 i get the following error:

[...]/node_modules/http-aws-es/node8.js:45
  async request(params, cb) {
        ^^^^^^^
SyntaxError: Unexpected identifier
    at Object.exports.runInThisContext (vm.js:78:16)
    at Module._compile (module.js:545:28)
    at Object.Module._extensions..js (module.js:582:10)
    at Module.load (module.js:490:32)
    at tryModuleLoad (module.js:449:12)
    at Function.Module._load (module.js:441:3)
    at Module.require (module.js:500:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> ([...]/node_modules/http-aws-es/index.js:8:20)
    at Module._compile (module.js:573:32)

This module seems to check only for even versions (<4, 4, 6) in the index.js file and automatically includes the version for node 8 for all other cases (i.e. node 5 and 7).

It would be very nice if there was support also for the non-LTS versions. Would it be a problem to change the version check into something like this?

if (nodeMajor < 4) {
  module.exports = require('./legacy');
} else if (nodeMajor < 6) {
  module.exports = require('./node4');
} else if (nodeMajor < 8) {
  module.exports = require('./node6');
} else {
  module.exports = require('./node8');
}

elasticsearch is depricated, this should be using https://www.npmjs.com/package/@elastic/elasticsearch instead. This is not going to work with latest elasticsearch versions.

When using http-aws-es, if you fail to explicitly update your AWS configuration object with a region (e.g. aws.config.update({ region: "us-east-1" })) then searches using elasticsearch/http-aws-es fail with the misleading error message:

The "data" argument must be one of type string, TypedArray, or DataView. Received type undefined

This is accompanied by a stack trace such as:

Elasticsearch ERROR: 2020-03-11T13:02:48Z
  Error: Request error, retrying
  POST https://search-endpoint-01/kibana_sample_data_flights/_search?size=10 => The "data" argument must be one of type string, TypedArray, or DataView. Received type undefined
      at Log.error (myproject/node_modules/elasticsearch/src/lib/log.js:226:56)
      at checkRespForFailure (myproject/node_modules/elasticsearch/src/lib/transport.js:259:18)
      at done (myproject/node_modules/http-aws-es/connector.js:48:7)
      at process._tickCallback (internal/process/next_tick.js:68:7)
      at Function.Module.runMain (internal/modules/cjs/loader.js:834:11)
      at startup (internal/bootstrap/node.js:283:19)
      at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)

Because this error message is so unrelated to the root cause, I wonder if http-aws-es could test for this condition and provide better diagnostics. If not, hopefully the existence of this github issue will at least help the next person who hits this problem.

I suspect the actual validation that triggers this error may be in an underlying crypto module.

When I'm behind my companies http proxy the elastic connections timeout, otherwise it works fine.