Code Monkey home page Code Monkey logo

drool's Introduction

๐Ÿคค Drool is an automation layer that is used to measure if a set of "clean" actions results in a DOM and or Listener leak.

View the API Docs

Build Status Join the chat at https://gitter.im/samccone/drool


Real World wins

Drool has made it far easier to identify memory leaks in an automated and reproducible way, for example:

Why am I making this?

After running perf/memory tests across multiple todomvc implementations, I found that almost all implementations have significant memory leaks on the most basic of tasks. Worse yet, most of these leaks were introduced at a framework level, or were introduced by "expert/(framework authors)". The question arose in my mind, if people who authored a framework are introducing leaks in the most trivial of applications, how can users be expected to create non-leaking implementations of much more complex applications.

Goals

Ideally Drool will leverage standard interfaces, such as todomvc, to test for leaks at a framework level. The result of which should help framework authors and developers realize that memory leaks are pervasive in the tools that we use.

Chrome devtools is a powerful utility layer for detecting memory issues, yet the fact still stands that most developers do not know how to use the tooling around it to arrive any thing that is directly actionable. Drool aims to be a generic automated abstraction layer, so people can get good "numbers" in a consistent way without having to deep dive into memory profiling.

Running

Ensure that you have at least version 2.26.436421 of chromedriver.

var drool = require('drool');
var assert = require('assert');

var driver = drool.start({
  chromeOptions: 'no-sandbox'
});

drool.flow({
  repeatCount: 100,
  setup: function() {
    driver.get('http://todomvc.com/examples/backbone/');
  },
  action: function() {
    driver.findElement(drool.webdriver.By.css('.new-todo')).sendKeys('find magical goats', drool.webdriver.Key.ENTER);
    driver.findElement(drool.webdriver.By.css('.todo-list li')).click();
    driver.findElement(drool.webdriver.By.css('.destroy')).click();
  },
  assert: function(after, initial) {
    assert.equal(initial.counts.nodes, after.counts.nodes, 'node count should match');
  }
}, driver)
.then(() => driver.quit())
.catch(e => {
  driver.quit();
  throw e;
})

View the API Docs

drool's People

Contributors

artemv avatar brianchung808 avatar janraasch avatar murielsilveira avatar prayagverma avatar samccone avatar tbranyen avatar vxsx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

drool's Issues

Dealing with chrome's undo stack

tl;dr - Typing into an input results in a node count increase that is non-trivial to track down, and presents itself as a node leak in your app.


What

In Chrome, whenever you type into an input, chrome stores a (node) entry in the undo stack.. Which allows a user to then press command z and undo the given action.

Ok seems reasonable, so why is this an issue for people that are interested in doing memory profiling?

If we take a look at UndoStack.cpp

image

image

We can see that we are storing up to 1000 records in the undo stack, which means that if you had a test that involved typing into an input multiple times you would think that there is a node leak in your app. ๐Ÿ˜ฟ

image

A keen observer might wonder though if they were to press the force GC button in devtools, shouldn't all nodes that can be garbage collected be cleaned up? That is a valid thought, and normally would be true, however because we are holding onto these references in the undo stack, a force garbage collection will never collect these items until they are evicted from the stack... But there is no way to force this eviction to happen ๐Ÿ’€


Potential Solution

It would be possible to write a setup step that would create an input and then type into it 1000+ times thus filling the buffer. After the buffer was full the diff between two snapshots would then not be impacted by the undo buffer since it would be at a constant 1000 nodes regardless of the tests typing into new inputs. ๐Ÿ’ฅ

Chrome Side Solutions

If chrome was to expose in the counts trace the number of elements inside of the undo buffer it would be possible to calculate the diff without having to "hack" this solution together. This patch however would be somewhat invasive to the chrome core and may not be worth it.. however it is an interesting metric that could be valuable to a subset of users. cc @paulirish

Another solution would be to clear the undo stack when the user pressed the GC button or manually invoked the gc(), this seems like it would be a pretty simple solution that would resolve these concerns.


xref

Add a delta helper.

Given a set of profile objects, this should output a delta object that contains the diff between the first and last profile.

+100 Nodes
-200 Listeners
+3.4 Mb HeapSize

The latest version of chromedriver is causing some problems for drool

I downloaded the latest version (2.9) of chromedriver from here. I threw that into /usr/bin and tried to run my test. The test can be found here and this is the resulting stack trace:

/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:654
    throw error;
    ^

WebDriverError: unknown error: cannot parse capability: chromeOptions
from unknown error: unrecognized chrome option: perfLoggingPrefs
  (Driver info: chromedriver=2.9.248307,platform=Mac OS X 10.11.3 x86_64)
    at WebDriverError (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/error.js:26:26)
    at Object.checkLegacyResponse (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/error.js:639:15)
    at parseHttpResponse (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/http/index.js:528:13)
    at /Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/http/index.js:466:11
    at ManagedPromise.invokeCallback_ (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:1343:14)
    at TaskQueue.execute_ (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:2868:14)
    at TaskQueue.executeNext_ (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:2851:21)
    at /Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:2775:25
    at /Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:639:7
    at process._tickCallback (node.js:412:9)
From: Task: WebDriver.createSession()
    at acquireSession (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/webdriver.js:107:22)
    at Function.createSession (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/webdriver.js:337:12)
    at Driver (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/chrome.js:778:38)
    at Builder.build (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/builder.js:464:16)
    at Object.start (/Users/charlespeebles/Development/rafael/node_modules/drool/lib/index.js:139:4)
    at Object.<anonymous> (/Users/charlespeebles/Development/rafael/test/drool.js:16:20)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
From: Task: WebDriver.navigate().to(file:///Users/charlespeebles/Development/rafael/test/test.html)
    at Driver.schedule (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/webdriver.js:386:17)
    at Navigation.to (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/webdriver.js:1029:25)
    at Driver.get (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/webdriver.js:797:28)
    at drool.flow.setup (/Users/charlespeebles/Development/rafael/test/drool.js:26:12)
    at TaskQueue.execute_ (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:2868:14)
    at TaskQueue.executeNext_ (/Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:2851:21)
    at /Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:2730:27
    at /Users/charlespeebles/Development/rafael/node_modules/drool/node_modules/selenium-webdriver/lib/promise.js:639:7
    at process._tickCallback (node.js:412:9)
From: Task: <anonymous>
    at executeInFlow (/Users/charlespeebles/Development/rafael/node_modules/drool/lib/index.js:9:24)
    at Object.flow (/Users/charlespeebles/Development/rafael/node_modules/drool/lib/index.js:91:3)
    at Object.<anonymous> (/Users/charlespeebles/Development/rafael/test/drool.js:24:7)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Function.Module.runMain (module.js:447:10)
    at startup (node.js:142:18)
    at node.js:939:3

P.S. - I haven't yet tried older versions of chromedriver as there isn't a recommended version listed for drool.

prewarm issue

Here is a test i added to material-design-lite menu component. It passes with 0 event listener diff.

But if you look at menu.js carefully, the click listener is left on document if menu is closed by clicking a menu item. Lucky that the listener is removed next time the menu is clicked open.

The prewarm process will have a listener left there, each action repeat will heal itself first and then leave a listener. As a result the leak is not detected.

Maybe not really an issue but i think it's good to report.

Replace webdriver with chrome-launcher

I've had a brief look at the code and it seemed to me that there is no reason to have selenium webdriver as a dependency. Is there any specific reason for the dependency, a reason not to update the project to chrome-launcher. Is it merely because no one got around to it?

Very basic input typing causes nodes count to increase

Hi Sam,

I was playing around with the module, and stumbled a leak of nodes no one can explain. Do you have any ideas about?
Consider this template (https://github.com/tenbits/leaky/blob/master/test/input/index.html) and the runner (https://github.com/tenbits/leaky/blob/master/test/input/index.js).
As you can see, we just type a word into the input and clean it on enter. Every iteration causes 1 node-leak on e.target.value = str_EMPTY;

Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36
{"documents":1,"jsEventListeners":1,"jsHeapSizeUsed":1231344,"nodes":13}
{"documents":1,"jsEventListeners":1,"jsHeapSizeUsed":1732484,"nodes":23}
Iterations: 10, Leaks: 10
Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2457.0 Safari/537.36
{"documents":1,"jsEventListeners":1,"jsHeapSizeUsed":2101496,"nodes":13}
{"documents":1,"jsEventListeners":1,"jsHeapSizeUsed":2897152,"nodes":23}
Iterations: 10, Leaks: 10

When we try manually to test the page directly via the DevTools, we get the same result, so it seems to be the browser engine issue.

image

type foo; press enter; type baz; press enter; type qux; press enter; gc; stop;

I was just thinking how reliable is the nodes-count?

Thanks, Alex

I should be able to catch errors in a single place

If a user does an action that causes webdriver or any error to be thrown, the flow exits and exits the process, drool should provide a declarative API to catch these errors.

{.
  ..
  action: function() {},
  catch: function(e) {}
}

perhaps something like this

Cannot read property 'executeScript' of undefined

i guess im doing something wrong with the webdriver

TypeError: Cannot read property 'executeScript' of undefined
  at getCounts (/Users/lgubenis/projects/paper/node_modules/drool/lib/index.js:14:9)
  at Object.flow (/Users/lgubenis/projects/paper/node_modules/drool/lib/index.js:39:3)
  at Object.<anonymous> (/Users/lgubenis/projects/paper/test/index.coffee:7:7)
  at Object.<anonymous> (/Users/lgubenis/projects/paper/test/index.coffee:1:1)
  at Module._compile (module.js:460:26)

Not terminating all processes?

I am trying to run drool via gulp-jasmine. A common problem experienced with gulp-jasmine is it not emiting the "end" event for its stream, as seen here: sindresorhus/gulp-jasmine#49. The basic answer given was that the tests being run in Jasmine are currently leaving processes unterminated, which prevents the 'end' event from being fired, since there are still active listeners. I am currently running the following test, based roughly on the example you provide in the docs. It seems like this exact issue is popping up for me:

var drool = require('drool');
var memUtils = require(process.cwd() + '/src/_test/utilsMem.js');
var config = {
  chromeOptions: 'no-sandbox',
};



describe('ngAvatar component: ', function() {
  var driver;

  beforeEach(function() {
    if (typeof process.env.chromeBinaryPath !== 'undefined') {
      config.chromeBinaryPath = process.env.chromeBinaryPath;
    }

    driver = drool.start(config);
  });

  afterEach(function() {
    driver.quit();
  });

  it('should not leak memory when added, then removed', function(done) {
    drool.flow({
      repeatCount: 10,
      setup: function() {
        return driver.get(memUtils.getTestFile())
          .then(function() {
            return driver.executeScript('myAppTest.start("ngAvatar");');
          })
          .catch(function(e) {
            console.log(e.toString().red);
          })
      },
      action: function() {
        var componentStr = '<ng-avatar img-src="./resources/twitter_prof_1.jpg"></ng-avatar>';
        return driver.executeScript('window.testingScope = myAppTest.insert("#root", \'' + componentStr + '\');')
          .then(function() {
            return driver.executeScript('myAppTest.destroy(window.testingScope);');
          })
          .then(function() {
            return driver.executeScript('myAppTest.reset("#root");');
          })
          .catch(function(e) {
            console.log(e.toString().red);
          })
      },
      beforeAssert: function() {
        return driver.executeScript('myAppTest.digest();')
          .then(function() {
            return driver.sleep(4000);
          })
          .catch(function(e) {
            console.log(e.toString().red);
          })
      },
      assert: function(after, initial) {
        memUtils.log(after, initial);

        expect(after.counts.nodes).not.toBeGreaterThan(initial.counts.nodes);
        expect(after.counts.jsEventListeners).not.toBeGreaterThan(initial.counts.jsEventListeners);
      },
      exit: function() {
        done();
      }
    }, driver);
  });
});

Any thoughts on what the unterminated process could be?

Ps: some more details from the gulp side of things, if it helps: http://stackoverflow.com/questions/37756443/gulp-task-exits-before-stream-can-fire-an-end-event/37756998?noredirect=1#comment62983737_37756998

Getting different results with and without beforeAssert

Hi,

when the definition of action method is this, the node count is a match

action: () => {
        driver.findElement(drool.webdriver.By.css('.first-image-sixtile')).click();
        driver.findElement(drool.webdriver.By.css('.close-area')).click();
    }

but, when i change it to this, the node counts at assertion are different

action: () => {
        driver.findElement(drool.webdriver.By.css('.first-image-sixtile')).click();
    },
beforeAssert: () => {
        driver.findElement(drool.webdriver.By.css('.close-area')).click();
      }

Can you please explain this behavior?

Strange example

The example includes a return statement that always gets run before the first line. I'm not sure if driver.quit() is suppose to be run thought, probably not since that would shut down the driver before the test is finished?

I think that return should be removed.


var drool = require('drool');
var assert = require('assert');

var driver = drool.start({
  chromeOptions: 'no-sandbox'
});

return drool.flow({
  repeatCount: 100,
  setup: function() {
    driver.get('http://todomvc.com/examples/backbone/');
  },
  action: function() {
    driver.findElement(drool.webdriver.By.css('#new-todo')).sendKeys('find magical goats', drool.webdriver.Key.ENTER);
    driver.findElement(drool.webdriver.By.css('#todo-list li')).click();
    driver.findElement(drool.webdriver.By.css('.destroy')).click();
  },
  assert: function(after, initial) {
    assert.equal(initial.nodes, after.nodes, 'node count should match');
  }
}, driver)

driver.quit();

ElementNotVisibleError: element not visible (but which?)

/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/goog/async/nexttick.js:39
  goog.global.setTimeout(function() { throw exception; }, 0);
                                            ^
ElementNotVisibleError: element not visible
  (Session info: chrome=44.0.2403.155)
  (Driver info: chromedriver=2.16.333243 (0bfa1d3575fc1044244f21ddb82bf870944ef961),platform=Mac OS X 10.10.4 x86_64)
  at new bot.Error (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/atoms/error.js:108:18)
  at Object.bot.response.checkResponse (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/atoms/response.js:109:9)
  at /Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:379:20
  at [object Object].promise.ControlFlow.runInFrame_ (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:1857:20)
  at [object Object].goog.defineClass.notify (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:2448:25)
  at [object Object].promise.Promise.notify_ (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:564:12)
  at Array.forEach (native)
  at [object Object].promise.Promise.notifyAll_ (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:553:15)
  at goog.async.run.processWorkQueue (/Users/lgubenis/projects/paper/node_modules/drool/node_modules/selenium-webdriver/lib/goog/async/run.js:125:21)
  at runMicrotasksCallback (node.js:337:7)
  at process._tickCallback (node.js:355:11)

From: Task: <anonymous>
  at executeInFlow (/Users/lgubenis/projects/paper/node_modules/drool/lib/index.js:7:24)
  at Object.flow (/Users/lgubenis/projects/paper/node_modules/drool/lib/index.js:37:3)
  at Object.<anonymous> (/Users/lgubenis/projects/paper/test/index.coffee:7:7)
  at Object.<anonymous> (/Users/lgubenis/projects/paper/test/index.coffee:1:1)
  at Module._compile (module.js:460:26)

Integration with existing selenium tests

We have developed HTML5-AngularJS website. Application's UI tests are written using selenium, where it tests all the workflows of the system. i.e. right from opening the site, logging into it, performing tasks like opening all the forms, performing actions inside those forms and then finally logging out.

Right now we are using chrome development tools to find out memory leaks. But came across drool as we want to automate the memory leak detection. I was going through API docs and see that we have to use something like this driver.findElement(drool.webdriver.By.css('.new-todo') in order to get the elements from the DOM.

So question is, can we use drool with existing selenium test cases ?

Doesn't work with chromedriver v >= 2.29

It doesn't detect leaks starting with chromedriver v 2.29 (checked with 2.30 and 2.31 too - same thing) - at least as per tests. Check Travis builds:

Looks like this is because drool don't get counts data starting from 2.29, cd4614d kinda hides this issue - drool doesn't error in this case but becomes useless. If NPM published version 0.4.0 is used it errors instead:

TypeError: Cannot read property 'message' of undefined
    at /my/project/path/node_modules/drool/lib/index.js:40:27
    at ManagedPromise.invokeCallback_ (/my/project/path/node_modules/selenium-webdriver/lib/promise.js:1343:14)
    at TaskQueue.execute_ (/my/project/path/node_modules/selenium-webdriver/lib/promise.js:2868:14)
    at TaskQueue.executeNext_ (/my/project/path/node_modules/selenium-webdriver/lib/promise.js:2851:21)
    at asyncRun (/my/project/path/node_modules/selenium-webdriver/lib/promise.js:2730:27)
    at /my/project/path/node_modules/selenium-webdriver/lib/promise.js:639:7
    at process._tickCallback (internal/process/next_tick.js:103:7)

License

Hi Sam,

Thanks for creating this library, very nice! But I can't see what is the license applied for it, can you provide one?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.