pnxtech / hydra-router Goto Github PK

Given the following configuration

"routerToken": "098ebe18-7e1b-4ddd-ae2a-cc6521e5b641", "disableRouterEndpoint": false,

will allow you to access routes, as long as you do not supply a token in request url.

http://localhost:5353/v1/router/list/routes => Route will be handled (BUG?)
http://localhost:5353/v1/router/list/routes?token=someinvalid => Route will be rejected (Working as intended)
http://localhost:5353/v1/router/list/routes?token=098ebe18-7e1b-4ddd-ae2a-cc6521e5b641 => Route will be handled (Working as intended)

The router seems to be accepting the first URL match without guarantee that such service have a running node instance. In my case I just started a hapi-service-test service and the response from the router is this:

{
 "statusCode": 503,
 "statusMessage": "Service Unavailable",
 "statusDescription": "The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay",
 "result": {
   "reason": "HR: [ol0r2s8oml] Unavailable express-service-test instances"
 },
 "tracer": "ol0r2s8oml"
}

Meaning that the hydra-router is considering old routes from the express-service-test service that I have ran in the past.

Regards, Rolando

Hi ,

If post request is sent directly to hydra-express with content-type as application/x-www-form-urlencoded then we are able to get the request body, but if the same request is sent through hydra-router then request body is coming empty.

Use a markdown table and add documentation here: https://www.hydramicroservice.com/docs/tools/hydra-router/

Reported by @taurenk

/Users/tkristich/Projects/fwsp-hydra-router/node_modules/fwsp-logger/lib/PinoLogger.js:146
      throw new Error(`Failed to start pino-elasticsearch: ${err}`);
      ^

Error: Failed to start pino-elasticsearch: Error: spawn pino-elasticsearch ENOENT

Temporary fix is to install pino-elasticsearch via:

$ npm install -g pino-elasticsearch

Need to patch code or add to package.json for a proper fix.

If you run a docker swarm locally (using docker stack) and have a hydra microservice that you're developing / testing outside of the swarm you may notice that the hydra-router dashboard will show your test service as being unavailable. The time difference might be 6-12 seconds.

This is a known docker engine (on mac) time drift issue.

• docker/for-mac#1260
• docker/for-mac#2076

To resolve this just reset your docker engine. You can do that by 1) clicking on the docker for mac icon 2) selecting the reset bomb 3) pressing the Restart button. That will restart the docker engine then you can reload the swarm.

Hi Carlos,

My envisioned design for my API URL structure isn't working out with hydra-router and looking at the code I think it might not be possible.

I have route registered with different microservices like so

{
  "ui-service": [
    "[get]/api/table/portal",
    "......"],
   "datastore-service": [
     "[get]/api/table/:tableName",
     "......."]
}

My intent was that all tables are handled by the datastore-service but I wanted to route requests for /api/table/portal to the ui-service

For users there would be one consistent API structure with no need to understand which service provides the data

In service-router._matchRoute on line 386 I see that we iterate through the routes returning on the first matching route.

That doesn't seem to support my idea. Was just looking for confirmation and your thoughts

Thanks

Hello, first of all I want to thank you for this awesome module. This is a general question, asking for guidance, not really an issue, I apologize if it's not the right place to ask. I couldn't figure this out after going through the entire documentation.

I am using hydra-express to build a microservices architecture, which will consist of ~15 microservices, all running behind an API Gateway. The services will communicate between each other via UMF messages.

I am wondering how one would go about implementing a centralized API gateway, which will accept all API requests and send them to the service responsible for managing them. The idea is to not allow external access to any service other than the API gateway.

Let's say I have a service called emailer and I want to route all traffic which comes to the API Gateway's route /api/emailer/* to the emailer service. From there, the service picks up the request and maps it to the corresponding internal route handler.

I am really looking forward to using this module in production with Docker. Thanks!

Please help me how to re route service. I am running three local redis instance on port 5353, 5352, 5351
if i am stopping port 5353 and if i am going to use port localhost:5353/v1/router/version it should re route to port 5352 or 5351. Is it correct but this is not working if i am wrong, please explain me how to achieve hydra-router routing feature.
Please any one help me.

Is their any other link with examples of hydra-router.
Please let me know if it is present.

It seems that when multiple service instances are no longer available, Hydra Router reports timeout rather than service unavailable. Reported by @artemkochnev

Hi,

Im trying to run docker container with Hydra Router, but get this error:

 docker run -p 5353:5353 --add-host host:MY_HOST --name hydra-router flywheelsports/hydra-router:1.3.3
Redis reconnection attempt 2 of 5...
Redis reconnection attempt 3 of 5...
Redis reconnection attempt 4 of 5...
Redis reconnection attempt 5 of 5...
Error: Max reconnection attempts reached. Check connection to Redis.
    at Promise (/usr/src/app/node_modules/hydra/lib/redis-connection.js:109:18)
    at Promise._execute (/usr/src/app/node_modules/bluebird/js/release/debuggability.js:300:9)
    at Promise._resolveFromExecutor (/usr/src/app/node_modules/bluebird/js/release/promise.js:483:18)
    at new Promise (/usr/src/app/node_modules/bluebird/js/release/promise.js:79:10)
    at Object.until (/usr/src/app/node_modules/hydra/lib/redis-connection.js:107:40)
    at Timeout.setTimeout [as _onTimeout] (/usr/src/app/node_modules/hydra/lib/redis-connection.js:115:30)
    at ontimeout (timers.js:386:14)
    at tryOnTimeout (timers.js:250:5)
    at Timer.listOnTimeout (timers.js:214:5)

Redis is runnign on 6379 on the same host

What am I doing wrong?

Thank you for help.

A.

See here:

https://github.com/flywheelsports/hydra-router/blob/master/servicerouter.js#L73

@taurenk ran into this issue while using a RedisLabs redis instance, which only supports db 0.

Hydra stands on the UMF message spec for req/res as well as for real-time messaging. UMF messages does not consider possible HTTP headers except 'Content-Type' and 'Authorization'. On legacy environments, to integrate the hydra can suppose a problem because HTTP headers are heavily used.

As an example, let's analyze the response headers of an app built with hapi.js:

http://localhost:3000/echo/hello

Connection →keep-alive
Date →Wed, 19 Apr 2017 08:26:40 GMT
Transfer-Encoding →chunked
cache-control →no-cache
content-encoding →gzip
content-type →application/json; charset=utf-8
vary →accept-encoding

Now the response through the hydra-router:

http://localhost:5353/hapi-service-test/v1/welcome

Connection →keep-alive
Date →Wed, 19 Apr 2017 08:27:09 GMT
Transfer-Encoding →chunked
X-Hydra-Tracer →1lkglmzk4pn

As a result, the response from the router is unreadable because there are missing headers like content*

In my opinion, limiting the HTTP headers support on the hydra-router->hydra->UMF will limit the capabilities of the hydra ecosystem as well, at least on existing environments who want to get boosted by using hydra.

Here, I would suggest to introduce a "headers" field on the UMF spec, that could be used for communication headers like HTTP, in consequence it would make hydra-* more compatible on legacy environments.

Looking forward to your feedback.
Regards, Rolando.

Hello @cjus

I've tried to deploy my app into AWS this evening and I can't work out why hydra-router is constantly failing with a SIGTERM signal

Starting service hydra-router:1.4.31 on 10.0.2.8:5353
[2017-11-18T00:45:27.566Z] INFO (1 on 90b5a3641439): Starting service hydra-router:1.4.31 on 10.0.2.8:5353
ts: "2017-11-18T00:45:27.565Z"
serviceName: "hydra-router"
type: "info"
processID: 1
Detected IPv4 IPs:
* lo: 127.0.0.1 255.0.0.0
* lo: 10.0.2.8 255.255.255.255
* eth0: 10.0.2.5 255.255.255.0
* eth1: 172.18.0.3 255.255.0.0
[2017-11-18T00:45:27.580Z] INFO (1 on 90b5a3641439): Starting service hydra-router:1.4.31 on 10.0.2.8:5353
serviceName: "hydra-router"
[2017-11-18T00:45:27.611Z] INFO (1 on 90b5a3641439): HR: hydra-router adding [get]/v1/router/list/:thing
serviceName: "hydra-router"
[2017-11-18T00:45:27.616Z] INFO (1 on 90b5a3641439): HR: hydra-router adding [get]/
serviceName: "hydra-router"

[....]

[2017-11-18T00:46:59.517Z] FATAL (1 on 90b5a3641439): Received SIGTERM
serviceName: "hydra-router"
[2017-11-18T00:46:59.519Z] ERROR (1 on 90b5a3641439): Service is shutting down.
ts: "2017-11-18T00:46:59.519Z"
serviceName: "hydra-router"
type: "error"
processID: 1
[2017-11-18T00:46:59.521Z] ERROR (1 on 90b5a3641439): Service is shutting down.
serviceName: "hydra-router"
ts: "2017-11-18T00:46:59.519Z"
type: "error"

Before that, in the logs, my services were populating their routes happily.

Thanks for any pointers you can give.

SUMMERY:
cant proxy/process file uploads/downloads to or from backend services via hydra-router.

##upload
POST postman (file) -->> hydra-router -->> backend-service

backend-service does not receive the upload file from hydra but when I hit the backend-service directly it works perfectly

##download (get file)
GET postman (filename) -->> hydra-router -->> backend-service

backend-service receive the request, processes/return the result (confirmed by tracing), but hydra-router for some reason cant forward file to the client.

HYDRA-ROUTER CONFIGURATION FILE:
{
"externalRoutes": {},
"routerToken": "",
"disableRouterEndpoint": false,
"debugLogging": true,
"queuerDB": 0,
"requestTimeout": 300,
"forceMessageSignature": false,
"signatureSharedSecret": "*********************************",
"cors": {
"access-control-allow-origin": "",
"access-control-allow-methods": "GET, POST, PUT, DELETE, OPTIONS",
"access-control-allow-headers": "accept, authorization, cache-control, content-type, x-requested-with",
"access-control-allow-credentials": "true",
"access-control-Max-Age": 10
},
"hydra": {
"serviceName": "hydra-router",
"serviceDescription": "Service Router",
"serviceIP": "",
"servicePort": "5353",
"serviceType": "router",
"plugins": {
"logger": {
"logRequests": true,
"redact": [
"password"
]
},
"hydraLogger": {
"logToConsole": true,
"onlyLogLocally": false
},
"dontuse-logger": {
"logRequests": false,
"noFile": true,
"toConsole": false,
"redact": [
"password"
],
"elasticsearch": {
"rotate": "daily",
"host": "host",
"port": 9200,
"index": "hydra"
}
},
"dontuse-loggly": {
"method": "POST",
"protocol": "http",
"hostname": "logs-01.loggly.com",
"port": 80,
"path": "/inputs/{token-here}/tag/http/",
"logToConsole": true,
"onlyLogLocally": false
}
},
"redis": {
"url": "redis://127.0.0.1:6379/15"
}
}
}

BACKEND-SERVICE CODE SNIPPET:

galleryServiceRouter.get("/images/:filename", [cache('5 minutes')], async function (req, res) {

  console.log("getting image: " + req.params.filename);

  let image = await Image.findOne({
        filename: req.params.filename
  });

  if (!image) {
        res.status(400).json({
              error: "Couldnt find image with that filename"
        });
  } else {
        res.writeHead(200, {
              "Content-Type": "image/png"
        });
        res.end(image.data);
        console.log("found image: " + req.params.filename);
  }

});

LOG FROM H-ROUTER:
1611308775 INFO hydra-router | {"tracer":"HR: tracer=2ad5pg2nkaf","url":"/api/v1/gallery-service/images/1a7cdd8f-581f-491a-a8cf-772c76d58f73_large.png","method":"GET","callerIP":"::1","body":{},"host":"localhost:5353","userAgent":"PostmanRuntime/7.26.8"}
1611308775 INFO hydra-router | HR: [2ad5pg2nkaf] Request for http://localhost:5353/api/v1/gallery-service/images/1a7cdd8f-581f-491a-a8cf-772c76d58f73_large.png
1611308775 INFO hydra-router | HR: [2ad5pg2nkaf] Calling remote service {"to":"gallery-service:[get]/api/v1/gallery-service/images/1a7cdd8f-581f-491a-a8cf-772c76d58f73_large.png","from":"aa482ad9b2a5469e99f856edc4d2194f@hydra-router:/","headers":{"user-agent":"PostmanRuntime/7.26.8","accept":"/","postman-token":"275ebdae-5619-44b4-a4e0-2633afe9e130","host":"localhost:5353","connection":"keep-alive","x-hydra-tracer":"2ad5pg2nkaf"},"mid":"8f93c9f6-1e90-407f-a2a5-2390f723aeeb-2ad5pg2nkaf","timestamp":"2021-01-22T09:46:15.784Z","version":"UMF/1.4.6","body":{}}
1611308775 INFO hydra-router | {"to":"gallery-service:[get]/api/v1/gallery-service/images/1a7cdd8f-581f-491a-a8cf-772c76d58f73_large.png","from":"aa482ad9b2a5469e99f856edc4d2194f@hydra-router:/","headers":{"user-agent":"PostmanRuntime/7.26.8","accept":"/","postman-token":"275ebdae-5619-44b4-a4e0-2633afe9e130","host":"localhost:5353","connection":"keep-alive","x-hydra-tracer":"2ad5pg2nkaf"},"mid":"8f93c9f6-1e90-407f-a2a5-2390f723aeeb-2ad5pg2nkaf","timestamp":"2021-01-22T09:46:15.784Z","version":"UMF/1.4.6","body":{}}

LOG FROM BACKEND-SERVICE:
getting image: 1a7cdd8f-581f-491a-a8cf-772c76d58f73_large.png
found image: 1a7cdd8f-581f-491a-a8cf-772c76d58f73_large.png

NB: returns image with all relevant headers
NB: ends response

QUESTION:
when I hit the backend-service directly it works perfectly.
Is there a simple way to get around this problem? or how can it be resolved?

Ensure that missing entries in config are handled correctly with regards to defaults.

Hi there,

Excellent work here, i have followed the https://community.risingstack.com/using-docker-swarm-for-deploying-nodejs-microservices/ guide and currently have hydra-router and the hello-service running in a docker cluster on aws.

All routes on the hydra-router give response as required but a request to any of the hello-service endpoint doesn't get through for reasons i'm not quite sure of. I have checked the docker logs to ensure that the services are indeed running without error. The docker image cjus/hello-service:0.0.2 exposes port 5000 by default and i'm not certain how the dynamic servicePort in the hello-service config file is accessible if the container doesn't expose this explicitly.

Please kindly advice, thank you.

Is it possible or could there potentially be a request to the router which would provide a list of the currently connected websocket clients? As well the ability to send a broadcast message to all currently connected clients?

Additionally, I have been unable to locate a mechanism that would allow a service to send a message to a currently connected client. Is this something that could potentially be implemented or exists and I have been unable to discern?

Really great concept and architecture and I'm looking forward to working with it.

An error in my Node JS application is throwing a hydra-express related exception (and covering up the real issue on my end):

TypeError: this.appLogger.fatal is not a function
    at /usr/src/app/node_modules/hydra-express/index.js:487:26
    at Layer.handle_error (/usr/src/app/node_modules/express/lib/router/layer.js:71:5)
    at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:315:13)
    at /usr/src/app/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/usr/src/app/node_modules/express/lib/router/index.js:335:12)
    at next (/usr/src/app/node_modules/express/lib/router/index.js:275:10)
    at Layer.handle_error (/usr/src/app/node_modules/express/lib/router/layer.js:67:12)
    at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:315:13)
    at /usr/src/app/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/usr/src/app/node_modules/express/lib/router/index.js:335:12)
    at next (/usr/src/app/node_modules/express/lib/router/index.js:275:10)
    at Layer.handle_error (/usr/src/app/node_modules/express/lib/router/layer.js:67:12)
    at trim_prefix (/usr/src/app/node_modules/express/lib/router/index.js:315:13)
    at /usr/src/app/node_modules/express/lib/router/index.js:284:7
    at param (/usr/src/app/node_modules/express/lib/router/index.js:354:14)
    at param (/usr/src/app/node_modules/express/lib/router/index.js:365:14)

Based on the trace, I believe the culprit is here: https://github.com/pnxtech/hydra-express/blob/b4871209d81aa19c1b402c84e0cc6a4593d8a3b4/index.js#L487 and should use this.log('fatal', {msg}) instead.

We are running with the following dependencies:

  "dependencies": {
    "redis": "2.8.0",
    "hydra-express": "1.8.3",
    "hydra-express-plugin-jwt-simple-auth": "0.1.0",
    "hydra-plugin-hls": "0.2.6",
    "node-fetch": "2.6.1"
  },

Having a confusing experience with hydra-router and 2 microservices (service-auth and service-ui)

root@89b2d7b901f2:/# hydra-cli nodes 
[
  {
    "serviceName": "ui-service",
    "serviceDescription": "User Interface services",
    "version": "0.0.1",
    "instanceID": "64cda9caf15e23d23fe7adabe3e96d43",
    "updatedOn": "2017-04-04T22:20:53.551Z",
    "processID": 15,
    "ip": "172.20.0.6",
    "port": 8080,
    "elapsed": 1
  },
  {
    "serviceName": "auth-service",
    "serviceDescription": "Authenticate users",
    "version": "0.0.1",
    "instanceID": "801f6d914aca66e28672e4ae1e321884",
    "updatedOn": "2017-04-04T22:20:53.204Z",
    "processID": 15,
    "ip": "172.20.0.5",
    "port": 8080,
    "elapsed": 2
  },
  {
    "serviceName": "hydra-router",
    "serviceDescription": "Service Router",
    "version": "1.0.12",
    "instanceID": "b8a32ef20fdfd3c2ea6c3882153dbe36",
    "updatedOn": "2017-04-04T22:20:52.711Z",
    "processID": 15,
    "ip": "172.20.0.8",
    "port": 5353,
    "elapsed": 2
  }
]

root@89b2d7b901f2:/# hydra-cli routes
{
  "hydra-router": [
    "[get]/v1/router/list/:thing",
    "[get]/v1/router/refresh/:service",
    "[get]/v1/router/refresh",
    "[get]/v1/router/version",
    "[post]/v1/router/message"
  ],
  "ui-service": [
    "[GET]/_config/ui-service",
    "[get]//"
  ],
  "auth-service": [
    "[get]/auth/login",
    "[get]/auth/",
    "[GET]/_config/auth-service"
  ]
}

With my Docker configuration I have access to each microservice on a port that bypasses the router and I can connect to each route

GET / HTTP/1.1
Cookie: sails.sid=s%3Ak7x4QABY5ecethGYCT91rOV6sAMggOlL.WkVy9oeGQySmnwm6LifA7DjwWptabtypaLlzsNp0SRQ
Host: 192.168.99.100:8082
Connection: close
User-Agent: Paw/3.0.16 (Macintosh; OS X/10.10.4) GCDHTTPRequest


HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
X-Process-Id: 15
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Powered-By: ui-service/0.0.1
Content-Type: application/json
Content-Length: 146
X-Response-Time: 1.983ms
Date: Tue, 04 Apr 2017 22:23:24 GMT
Connection: close

{"statusCode":200,"statusMessage":"OK","statusDescription":"Request succeeded without error","result":{"greeting":"Welcome to Hydra Express! UI"}}

GET /auth HTTP/1.1
Cookie: sails.sid=s%3Ak7x4QABY5ecethGYCT91rOV6sAMggOlL.WkVy9oeGQySmnwm6LifA7DjwWptabtypaLlzsNp0SRQ
Host: 192.168.99.100:8081
Connection: close
User-Agent: Paw/3.0.16 (Macintosh; OS X/10.10.4) GCDHTTPRequest

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
X-Process-Id: 15
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Powered-By: auth-service/0.0.1
Content-Type: application/json
Content-Length: 148
X-Response-Time: 1.735ms
Date: Tue, 04 Apr 2017 22:24:39 GMT
Connection: close

{"statusCode":200,"statusMessage":"OK","statusDescription":"Request succeeded without error","result":{"greeting":"Welcome to Hydra Express! AUTH"}}

GET /auth/login HTTP/1.1
Cookie: sails.sid=s%3Ak7x4QABY5ecethGYCT91rOV6sAMggOlL.WkVy9oeGQySmnwm6LifA7DjwWptabtypaLlzsNp0SRQ
Host: 192.168.99.100:8081
Connection: close
User-Agent: Paw/3.0.16 (Macintosh; OS X/10.10.4) GCDHTTPRequest

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
X-Process-Id: 15
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Powered-By: auth-service/0.0.1
Content-Type: application/json
Content-Length: 132
X-Response-Time: 0.874ms
Date: Tue, 04 Apr 2017 22:25:34 GMT
Connection: close

{"statusCode":200,"statusMessage":"OK","statusDescription":"Request succeeded without error","result":{"greeting":"Login the user"}}

But connecting through the hydra-router I am having no success

GET / HTTP/1.1
Cookie: sails.sid=s%3Ak7x4QABY5ecethGYCT91rOV6sAMggOlL.WkVy9oeGQySmnwm6LifA7DjwWptabtypaLlzsNp0SRQ
Host: 192.168.99.100:81
Connection: close
User-Agent: Paw/3.0.16 (Macintosh; OS X/10.10.4) GCDHTTPRequest

HTTP/1.1 302 Found
Location: 
Date: Tue, 04 Apr 2017 22:26:26 GMT
Connection: close
Transfer-Encoding: chunked

GET /auth HTTP/1.1
Cookie: sails.sid=s%3Ak7x4QABY5ecethGYCT91rOV6sAMggOlL.WkVy9oeGQySmnwm6LifA7DjwWptabtypaLlzsNp0SRQ
Host: 192.168.99.100:81
Connection: close
User-Agent: Paw/3.0.16 (Macintosh; OS X/10.10.4) GCDHTTPRequest

HTTP/1.1 404 Not Found
Content-Type: application/json
Access-Control-Allow-Origin: *
Content-Length: 131
Date: Tue, 04 Apr 2017 22:27:19 GMT
Connection: close

{"statusCode":404,"statusMessage":"Not Found","statusDescription":"The requested resource was not found on the server","result":{}}

So a HTTP status 302 and 404 for the routes - how can I debug this further with hydra-cli?

Thanks for an awesome project

The value for request timeout can be set in the config file, but when the call is created using Hydra core the options.timeout is not present so the default of 30 secs is used. However there are some processes that can take longer than 30 secs.
Am I interpreting correctly this parameter? if that is the case then when this call is done:

hydra.makeAPIRequest(msg, {timeout: this.requestTimeout})

The msg will be passed to

makeAPIRequest(message) {
return super._makeAPIRequest(message);
}

But not the timeout, so the actual value in Hydra core will be NaN therefore the value selected will be:
REQUEST_TIMEOUT = 30000;

socket.setTimeout(options.timeout * 1000 || REQUEST_TIMEOUT, () => {

I am getting below error ....

{"statusCode":404,"statusMessage":"Not Found","statusDescription":"The requested resource was not found on the server","result":{}}

There is a need to block internal service APIs from external access via Hydra-Router. I created this issue ticket to openly discuss options.

In Hydra-Express a set of routes is registered this way:

  hydraExpress.registerRoutes({
    '/v1/hello': require('./routes/hello-v1-routes')
  });

Given two routes /v1/hello/public and /v1/hello/private the goal would be to specify the latter as private and not allow access through the hydra-router. Services behind the router should still be able to access each others private routes.