alltherooms / cached-request Goto Github PK

[15:0:55:379] Message: "Checking min deposite - 863830190"
undefined:0

SyntaxError: Unexpected end of input
at Object.parse (native)
at ReadStream. (/bot2/node_modules/cached-request/lib/cached-request.js:179:35)
at ReadStream.emit (events.js:129:20)
at _stream_readable.js:908:16
at process._tickCallback (node.js:355:11)

Does anyone know what can cause this or how to fix it? It happens sometimes and crashes whole script. Thanks

I like your module because it looks very simple and does exactly what I want. Though I found out one inconsistency, maybe I missed something.

When I perform the first request which is not cached yet, I get a plain text in the body property of the response object in the callback function.

body: 'Text\n'

Though when I perform the second request for the same url which is already cached, I get a Buffer object at the body property.

body: <Buffer 54 65 78 74 0a>

And same for the body argument of the callback function.

I perform the request I passing only the one parameter url:

cachedRequest({
    url: 'http://localhost',
  }, (err, response, body) => {
  console.log(err, response, body);
});

Whats about adding an Option for Custom Cache Keys?

 if(options.customkey) {
	  key = options.customkey;
  } else {
  key = JSON.stringify(this.normalizeOptions(options));
  key = this.hashKey(key);
  }

using 1.0.0 I can't seem to get my requests to cache.
I have the following at the start of my file:

var cachedRequest = require('cached-request')(request);
var cacheDirectory = '/tmp/cache';
cachedRequest.setCacheDirectory(cacheDirectory);
cachedRequest.set('ttl', 3000);

When starting my app, I get cachedRequest.set is not a function although this is in the docs. Looking at the object it doesn't indeed have a set method, but does have a setValue docs need to be updated?

Secondly I declare, as seen above, the cacheDirectory but keep getting the following error:

Error: ENOENT: no such file or directory, open '/tmp/cache/1275300535.json'

Looking at the code:
https://github.com/alltherooms/cached-request/blob/master/lib/cached-request.js#L164

It seems that there is no check in place to see if the folder should be created - a simple mkdir:

//Open headers file
  mkdirp(self.cacheDirectory);
  headersReader = fs.createReadStream(this.cacheDirectory + key + '.json');
...
//Open the response file
  mkdirp(self.cacheDirectory);
  responseReader = fs.createReadStream(self.cacheDirectory + key);

would prevent an anti-pattern and make sure that the dir exists.

Thank you for that great module. I have one feature which I think would be usefull.

When the cache-folder does not exist, create it automatically so that the user does not have to do it.
This would especially be helpful when the cache-folder is in gitignore.

According to the documentation to se the ttl you do:

cachedRequest.set('ttl', 1000);

This fails with cachedRequest.set is not a function. Looking at the source it would appear the right code would be:

cachedRequest.ttl = 1000;

Is this correct?

There's this request-promise-native package, which wraps around request to deliver a Promise interface.
What would be the best approach delivre Promises with cached-request? Somehow, I see them both as mixins in a "class hiearchy", which could be mixed independently.

npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible.

Hello, I don't know if it's implemented (at least I don't see it in docs) but I would want to delete the cache before ttl time ends and make a new cached request again.

Unfortunately if I use this tool, I sometimes, randomly get this error (it does not happen always)

events.js:160
     throw er; // Unhandled 'error' event
     ^

Error: stream.push() after EOF
   at readableAddChunk (_stream_readable.js:156:17)
   at RequestMiddleware.Readable.push (_stream_readable.js:134:10)
   at IncomingMessage.<anonymous> (...\betrieb\places_zip_geo_mashup\node_modules\cached-request\lib\request-middleware.js:59:12)
   at emitOne (events.js:101:20)
   at IncomingMessage.emit (events.js:188:7)
   at IncomingMessage.Readable.read (_stream_readable.js:381:10)
   at flow (_stream_readable.js:761:34)
   at resume_ (_stream_readable.js:743:3)
   at _combinedTickCallback (internal/process/next_tick.js:74:11)
   at process._tickCallback (internal/process/next_tick.js:98:9)

var gRequest = cachedRequest({
            url: photoFetch.data.results,
            ttl: 1000 * 60 * 10 //1 minutes
        }).on("error", function(err) {
            console.log(err);
        });
        gRequest.pipe(res);

Cached file does not seem to be deleting automatically. I am doing it wrong?.

Yes, this is bad API design of theirs, but I just came across such situation an a foreign API that I am using. And this is actually an unexpected error on their side that comes and goes (actually comes quite often). This means that while using cachedRequest (or request too) sometimes I end up inside "success" case, even though the body says that there was an unexpected error.. and as the request becomes cached (haha, as it was 200 response), I cannot escape this error anymore, without deleting the entire cache directory.. not very handy..

Is there any workaround recommendation? Any feature of invalidating the cache for certain request? This would be really useful for me at the moment.. I know I could set ttl to lower value, but that would remove most of the benefits of using this library for me (now my ttl is 24 hours..).

Hi

I was writing some code to clean up the cached responses after a test and it occurred to me that a flag to 'not save' the response would be useful. Any thoughts?

Using

// test.js
const request = require('request')
const cr = require('./lib/index')(request)
cr.setCacheDirectory('/tmp/cache');

const log = (i) => (err, response, body) => {
  if (err) throw err;
  const cached = !!response.headers["x-from-cache"];
  console.log(`${i}: ${response.statusCode}, ${cached}`);
};

cr({url: 'https://www.google.com', ttl: 10000}, log(1))
cr({url: 'https://www.google.com', ttl: 10000}, log(2))

If I execute it after a non-cached response, these executions sometimes will result in nothing printed in the screen, and only after the cache expires I'm able to see results back again.

$ node test.js
1: 200, false
2: 200, false
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
$ node test.js
1: 200, false
2: 200, false
$ node test.js
1: 200, true
2: 200, true

Occasionally the APIs give back HTML errors, yet I have no way of seemingly telling the cache to refresh. This would be a nice enhancement :)

Hi!

Great idea, I'm thinking about taking on the missing helper methods (get, etc) but before I started I thought I'd ask - how come the cache isn't in memory? Would it be possible to have that as an option?

I am using the library to query data (and cache it) from fixer.io. Fixer.io is rate limited. So I would like to throttle requests to the api but ONLY if the data is not read from cache. Is there any way on hooking up into the case when there is cache miss?

Anyway to setup with LRU to limit the disk usage?

Hi,
I'm fetching data from external provider and everything works fine.
From time to time the program terminates with following print:

SyntaxError: Unexpected end of input
at Object.parse (native)
at ReadStream. (/Users/pd/node_modules/cached-request/lib/cached-request.js:189:35)
at emitNone (events.js:72:20)
at ReadStream.emit (events.js:166:7)
at endReadableNT (_stream_readable.js:921:12)
at nextTickCallbackWith2Args (node.js:442:9)
at process._tickCallback (node.js:356:17)

My program:
...

   cachedRequest(
            {
                method: 'POST',
                url: url,
                body: xmlReq,
                headers: {'Content-Type': 'text/xml'},
                ttl: 1200
            },
    function (error, response, body) {
            if (error) {
                console.error("getApiInfo: cachedRequest error: " + error);
                return;
            }
            if (!error && response.statusCode === 200) {
                var lastUpdated = new Date().toLocaleTimeString();
                var stop = Date.now();
                var processTime = (stop - start) / 1000;
                body = JSON.parse(body);
                body.processTime = processTime;
                body.lastUpdated = lastUpdated;
                callback(body, trainId);
            }
    });

...