Code Monkey home page Code Monkey logo

yakbak's Introduction

yakbak

Record HTTP interactions The Node Wayโ„ข. Inspired by ruby's vcr.

Build Status

install

$ npm install yakbak --save-dev

usage

The main idea behind testing HTTP clients with yakbak is:

  1. Make your client's target host configurable.
  2. Set up a yakbak server locally to proxy the target host.
  3. Point your client at the yakbak server.

Then develop or run your tests. If a recorded HTTP request is found on disk, it will be played back instead of hitting the target host. If no recorded request is found, the request will be forwarded to the target host and recorded to disk.

yakbak(host, options)

Returns a function of the signature function (req, res) that you can give to an http.Server as its handler.

var handler = yakbak('http://api.flickr.com', {
	dirname: __dirname + '/tapes'
});

options

  • dirname the path where recorded responses will be written (required).
  • noRecord if true, requests will return a 404 error if the tape doesn't exist
  • hash(req, body) provide your own IncomingMessage hash function

with node's http module

yakbak provides a handler with the same signature that http.Server expects so you can create your own proxy:

var http = require('http');
var yakbak = require('yakbak');

http.createServer(yakbak('http://api.flickr.com', {
	dirname: __dirname + '/tapes'
})).listen(3000);

Now any requests to http://localhost:3000 will be proxied to http://api.flickr.com and recorded to /tapes for future playback.

with express

Need more flexibility? express expects the same function signature, so you can use yakbak just like you would any other middleware:

var express = require('express');
var yakbak = require('yakbak');

var flickr = yakbak('http://api.flickr.com', {
	dirname: __dirname + '/tapes'
});

var upload = yakbak('http://up.flickr.com', {
	dirname: __dirname + '/tapes'
});

express().use(function (req, res, next) {
	if (req.path.indexOf('/services/upload') === 0) {
	  upload(req, res);
	} else {
	  flickr(req, res);
	}
}).listen(3000);

as a standalone response server

Each recorded response is itself a node module with the same handler signature, so if you want to create a server that replays a single response, you can do so easily:

var http = require('http');
var tape = require('./tapes/1117f3d81490d441d826dd2fb26470f9.js');

http.createServer(tape).listen(3000);

on the command line

yakbak also ships with a yakbak utility that will start an HTTP server to play back a given tape.

$ yakbak
Error: file is required
Usage: yakbak <file>
$ yakbak ./tapes/1117f3d81490d441d826dd2fb26470f9.js
Server listening on port 3000
* Connection from 127.0.0.1 port 63669
< GET / HTTP/1.1
< host: localhost:3000
< user-agent: curl/7.43.0
< accept: */*
<
> HTTP/1.1 201 Created
> content-type: text/html
> date: Sat, 26 Oct 1985 08:20:00 GMT
> connection: close
> transfer-encoding: chunked
>
* Connection closed

why not [insert other project here]?

Check out this blog post about why we chose a reverse proxy over other existing approaches to recording HTTP interactions.

license

This software is free to use under the MIT license. See the LICENSE file for license text and copyright information.

yakbak's People

Contributors

commenthol avatar ijpiantanida avatar jeremyruppel avatar smgraywood avatar stephenmathieson avatar tonyskn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yakbak's Issues

How to use it with React Native?

I am hoping to use Yakbak along with Detox (integration testing framework) for React Native.

I understand that since this package is built for node.js applications it assumes access to node.js core modules such as assert and fs etc

I was able to get around this by using node-libs-browser which provides these core node.js modules in a React Native environment.

Although I am running into trouble with fs package. Here is the error I get:

error: bundling failed: Error: Unable to resolve module `fs` from `pillow_app/node_modules/yakbak/lib/record.js`: Module does not exist in the module map

So in the record.js file it does this var fs = require('fs'); although the only way to get access to the native filesystem in React Native is through this var fs = require('react-native-fs');

Hence I was wondering if anyone had any thoughts on how to resolve this or a possible workaround to get Yakbak working in React Native?

Odd behaviour when connecting to an https backend

I'm attempting to create a proxy to an https API that sits behind Cloudflare:

http.createServer(yakbak('https://--domain--', {
dirname: __dirname + '/tapes'
})).listen(3000);

When I do this, and curl the proxy:

curl -i localhost:3000/status

I get an error from cloudflare "The request's Host header does not match the request's TLS SNI Host header."

I've got a code change for the proxy to fix this; updating the headers collection prior to mod.request rather than using preq.setHeader('Host', uri.host), and this fixes it, but wasn't able to reproduce it with a test.

Better tape metadata

I don't think tapes need better names (that's what the dirname is for!) but I do think the tapes themselves could use some better metadata, either in the header comment or exported from the module itself.

  • query string as a hash
  • datetime the tape was recorded
  • possibly who recorded it?

Use template-literal instead of EJS

Template Literal is fastest, smallest and simplest template engine, because it use JS's literal template feature.

It's 55 times faster than EJS, and it also use less CPU and RAM ressources, so it may be a good idea to use it instead of EJS ๐Ÿ˜€

Beginner level error: done is not defined

Hello,
I am a beginner in nodejs and require some help regarding the usage of yakbak. Hope, you don't mind :-)

Situation:
At present, I develop ASP.NET webmethods via Visual studio and manually test them. Once I start a visual studio project, the test url becomes: http://localhost/<random-but-valid-port-number>. This link can be opened via any web browser (e.g. chrome, firefox), for which I think node modules can use it also.

Code I tried:
I have started the visual studio project, and it has started on port 52280. Therefore, going to http://localhost/WebService1.asmx/Ping yields a valid timestamp (code written by me which I want to test). Code I have written in index.js:

// Contents of index.js
try {
    var yakbak = require('yakbak');
    var http = require('http');

    var validPort = 52280;

    var proxy = http.createServer(yakbak('http://api.example.com', {
        dirname: __dirname
    }));
    proxy.listen(validPort, done);

    var options = {
        host: 'http://localhost',
        port: validPort,
        path: '/WebService1.asmx/Ping'
    };

    http.get(options, function (resp) {
        resp.on('data', function (chunk) {
            console.log(chunk);
        });
    }).on("error", function (e) {
        console.log("Got error: " + e.message);
    });

} catch (e) {
    console.log(e.message);
    console.trace();
}

Example
If I try 'WebService1.asmx/Ping' then I would expect 2016-07-18T15:50:04.483+06:00 (current timestamp) as response and this response should be cached so that if I call this url later, then I will get the cached response instead of current timestamp.

Desired output
I want to control the variable options (i.e. path) and fetch response from that particular path. Then, I will compare the result in resp.on which will be same as the example output above. I expect yakbak to mock/cache the response.

Actual output

$ node index.js

done is not defined
Trace
    at Object.<anonymous> (C:\Test\index.js:28:13)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:456:32)
    at tryModuleLoad (module.js:415:12)
    at Function.Module._load (module.js:407:3)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:159:18)
    at node.js:444:3

yakbak sometimes does not record tapes

I sometimes -- often, in fact -- find that when I run tests with no tapes in place, the tests work correctly (falling through to the underlying HTTP service) but no tapes get written. I have no idea what causes this. There are no diagnostics of any kind, just a silent failure -- or, worse, I might characterise it as a silent success, but one that doesn't give me the side-effect that I need.

I realise this is a hopelessly underspecified bug report: it's more of a cry for help, really. I'm posting it in the hope that someone else has had this experience, recognises what I am describing, and can put me on the right path.

Yakbak is dead.. Long live Talkback!! [Replacement Library]

I was banging my head against the wall trying to make Yakbak work for my use case. First off the tapes are in base64 format which makes it really hard to debug the tapes & also this library is not maintained anymore.

I came across a replacement called Talkback which serves the same functionality but was written from scratch by @ijpiantanida! ๐Ÿ‘

Hence this led me to replace my mock Yakbak server with Talkback:

Yakbak Cons:

  • The tapes were in base64 format - made it hard to debug the request/response
  • It would re-record tapes when x-request-id changed - an unintended side-effect
  • Needed express & morgan libraries in addition to yakbak
  • Not maintained anymore

Talkback Pros:

  • Easy setup - took less than 2 mins to replace Yakbak
  • Logging requests by default - even shows whether it served a cached request or is recording a new tape
  • Human-readable tapes - you can see the req/res in JSON5 format
  • Ability to rename tapes - this is great as you can know exactly what your tapes your doing

Hence if you are considereing Yakbak for your use case, I would highly suggest you checkout Talkback and use that instead.

add more/better logging to help ease debugging

I'm currently debugging some tests that fail intermittently. The tests hit yakbak for each request and only sometimes have a matching tape. it'd be extremely useful if setting DEBUG=yakbak* when running my test suite would output more than this:

  yakbak:server req +0ms /some/path?with=some&query=strings

would you take a PR that adds a few more debug()s for things like:

  • the request path
  • the request hash
  • if a tape was found
  • if yakbak will respond to the request

Thank you!

Just started using this in a project. So far this is the most seamless http mocking tool I've used.

Thank you for releasing it ๐Ÿ˜„

Expose the `tapename` function

I would like to be able to human-read the names of the tape, and I thought of monkeypatching the tapename function to have more flexibility.

This is not possible however as this function is not exposed.

Allow tape to be re-recorded if tape file is deleted

If any tapes are deleted while the yakbak server is running with the intention of refreshing them, subsequent requests for that tape will result in retrieving the cached tape module instead of re-recording. This is because require.resolve does not throw an error even if the tape file doesn't exist and the tape module was cleared from the require.cache. It successfully resolves the tape file and doesn't throw any error. This, however, causes the require to be called which then throws a file not found error.

A simple fix is to check if the file exists post-request.resolve and to throw a MODULE_NOT_FOUND exception if it doesn't which causes control to pass into the catch(ModuleNotFoundError, ...) block, and the request to be re-recorded.

PR for this request:
#39

Plans for using native Promises?

Hi Jeremy,

Do you have any plans to use native promises with your library? Greater node 8 they are pretty much faster then js implementations like bluebird.

Today we encountered a strange problem with bluebird which we could not
nail down to a PoC. While recording a tape the response was not finally written
to disk but the Promise already resolved, such require(file) returned an empty response resulting in a "TypeError tape is not a function". Removing bluebird from lib/buffer.js solved the issue for us.
For the urgency we had to move forward, we created the fork at https://github.com/spurreiter/yakbak-native-promise which has bluebird replaced with native promises. Are you interested in a PR?

Allow for rewriting responses before writing tape

Yakbak is great - it's really easy to setup, and just works. It'd be nice if there was a custom hook to modify the response before writing to the tape - specifically in the case where the response returns references to the proxied server.

For example:

  1. Using yakbak server to proxy through to some other server
  2. Request made, html returned with references to js/css/img assets that point to the original server url
  3. Ideally, I can write a function to replace these references with the yakbak proxy so that they are then captured through yakbak as well.

I submitted a PR with an attempt at this here: #27

Provide live call option

Consider providing a value in the options to always make a live call and not use existing tapes. I have a situation where certain API's may need to be called live whereas others may be replayed. A case could be made that the caller ought to not route to the yakbak server those calls that need to be live. However, for my scenario, the caller is running against dev (with the ability to route to the yakbak server as a toggle option via qs param) and yakbak is setup to record/replay against prod so that the dev client code can be tested against prod data. In this case, the dev client wouldn't be configured to call prod API's, so any calls not routed to yakbak would go to dev instead of prod.

Is project alive? => New library

First of all, thank you for building this tool. I've been using it for a long time now.

Unfortunately it looks like the project lost a little bit of attention and a lot of good PRs and issues have been open for quite some time. (I've been actually running of my PR's branch for more than a year now)

I decided to create my own version of "yakbak" called talkback to address some of the issues posted here and add some extra features.

If you're planning on still maintaining this project please close this issue and sorry for the spam, otherwise I think it might be a good successor.

Again, thank you for your efforts!

option to create custom hash/tape names

First off, this package is awesome! Great work here =)

I'm currently moving an enormous codebase (all segment.com integrations) away from hitting 3rd party APIs in tests to use yakbak, but I'm hitting issues with APIs that require cache-busting query parameters (example: google-analytics).

Would you be open to a (backwards compatible) addition to the API that allows users to create their own hash function?

Option to record response in different format

Sometimes, when I record an HTTP interaction, I'll want to modify the body of the response later. Usually, I do not want to re-record the entire interaction, I just want to modify the currently recorded response.

One way to do this would be to modify this so that, instead of:

res.write(new Buffer('cafebeef', 'base64'))

we'd have something like:

var obj = '{"v":2}'
res.write(new Buffer(JSON.stringify(obj), 'utf8'))

It could then be surfaced in a configuration, like:

yakbak('http://myServer:3060', {
  dirname: __dirname + '/../fixtures',
  fixtureEncoding: 'utf8', // default would be 'base64'
}));

Is that functionality that would be beneficial? If so, I'd be happy to add it.

Dynamic responsa

Would you be interested in adding documentation on how to edit a yakbak response to make it dynamic?

I had a usecase where I wanted to test an API client that should cache a response if passed a flag and not otherwise, but obviously yakbak "caches" the server response anyway. So I edited the "tape" to return a dynamic response so I could test that.

If it's something you'd like to add docs for, I'm happy to write up what I did with an example so it's clear?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.