Code Monkey home page Code Monkey logo

nleak's Introduction

nleak

n-leak CI

Warning NLeak is currently under development and is not ready for production use.

NLeak is an automatical memory detection and diagnosis tool for NodeJS. It is built on top of BLeak [1], a memory leak detection tool for JavaScript applications. NLeak is designed to be a lightweight and extensible tool that can be easily integrated into existing CI/CD pipelines. NLeak is currently under development and is not ready for production use.

Background

Memory leaks are a common cause of system memory exhaustion, which can result in slow performance or system crashes. Identifying and fixing memory leaks in JavaScript can be particularly challenging, as the language is dynamically typed and leaks may differ from those in other languages such as C, C++, and Java. There are currently no automated tools that can effectively detect memory leaks in JavaScript applications executed in the NodeJS runtime environment. The team is working on extending BLeak to NodeJS in order to create a memory detection tool that can automate the process of locating, diagnosing, and ranking JavaScript memory leaks in Node.js applications.

How to use it?

You can simply install NLeak globally and run it on your NodeJS application.

npm install -g nleak

Then, you will need the config file to tell NLeak how to run your application. The config file should be named config.js and should be in the root directory of your application. Please refer to the example folder for an example config file and guest NodeJS application with memory leak.

Once you have the files prepared, you can run NLeak on your application by simply running the following command:

// go to ./example folder
nleak run --config ./config.js --guest-app-entry ./app.js --out ./

As a result, you will get a nleak_result.json file in the current directory. This file contains the memory leak detection result. You can also use the NLeak viewer to visualize the result.

System Overview

Slice 1

The above figure demonstrates the execution process of NLeak. The guest application is executed as a child process in inspection mode, and heap snapshots are captured using the Chrome debugger protocol. In the leak diagnosis phase, guest application rewrites are applied to obtain the reference stack trace for potential memory leak objects. This rewrite enables traceability of object references in the guest application's execution and is seamlessly integrated into the system through a customized module import and compile scheme.

From the client's perspective, they only need to provide a configuration file and their guest application to execute NLeak. The memory leak detection results will be stored in a result file. NLeak also includes a viewer for better interpretation of the results on a web frontend.

How NLeak Works

NLeak utilizes the core algorithm from BLeak [1] to identify memory leaks. This algorithm takes a series of JavaScript runtime heap snapshots during idempotent operations in the guest application. Memory leaks are identified as heap objects that gain more outgoing references across the heap snapshots. Once leak-related objects are gathered, NLeak re-executes the guest application to collect the related JavaScript stack trace. A visual representation of this process can be seen in following sequence diagram.

sequenceDiagram
  autonumber
  actor User
  participant NLeak Core
  participant NLeak Guest
  User->>NLeak Core: Provides config.js and guest application
  Note over NLeak Core, NLeak Guest: Detection Phase
  NLeak Core->>NLeak Guest: Spawn child NodeJS process and run guest app
  loop For each iteration
    NLeak Core->>NLeak Guest: Trigger run actions defined in config
    NLeak Guest-->NLeak Guest: Run action and wait for completion
    NLeak Guest->>NLeak Core: Take heap snapshot and send back
  end
  NLeak Core-->NLeak Core: Comparing mutliple heap snapshots and run detection algorithms
  Note over User: Leak detection intermediate result
  NLeak Core->>User: nleak_result that contains potential leaks with score ranking
  Note over NLeak Core, NLeak Guest: Diagnosis Phase
  NLeak Core-->NLeak Core: Rewrite guest JS source, code instrumentation for LeakPath objects
  NLeak Core->>NLeak Guest: Restart child process to prevent caching
  loop For extra 2 iterations
    NLeak Core->>NLeak Guest: Trigger run actions defined in config
    NLeak Guest->>NLeak Core: Collect stack traces on LeakPath
  end
  NLeak Core-->NLeak Core: Mapping stack trace to original code with source map
  Note over User: Full leak detection and diagnosis result
  NLeak Core->>User: Stack traces and leak location highlighted with source map

NLeak viewer is a tool built in reactjs that allows you to visualize the heap snapshot growth of your application. To use it, simply go to https://nleak-viewer.vercel.app/ and upload your nleak_result.json file.

Once you've uploaded the file, NLeak viewer will generate a chart showing the growth of your heap snapshots over time. You'll also see a summary of the last heap snapshot's size, as well as the leak location with source map.

By using NLeak with the viewer, you can easily identify potential memory leaks in your application and take steps to fix them.

Screen Shot 2022-12-04 at 11 32 36 AM

Development

Local Development

  • npm i
  • npm run dev

The dev script will start nodemon to monitor all TypeScript changes and compile them into JavaScript and put to build folder.

For local testing, currently we support driver tests. Simply run npm run test:driver.

Docker Related Build & CI

Docker is used as CI building environment and testing.

$ docker build . -t nleak_build --platform=linux/amd64
$ docker run -v <path on host>:/home/NLeak --platform=linux/amd64 -it nleak_build:latest

Acknowledgement

[1] J. Vilk and E. D. Berger, β€œBLeak: automatically debugging memory leaks in web applications,” in Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, 2018.

nleak's People

Contributors

dependabot[bot] avatar foreseaz avatar frost-lee avatar proy-11 avatar rosesqy avatar yehwenting avatar yizhou0522 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

mmogylenko

nleak's Issues

Problem: Missing verify `FindLeaks` with BLeak end to end tests

@proy-11 @Rosesqy , thanks for the help on end to end tests!

  • createStandardLeakTest('Catches leaks', 'test', 8);
  • createStandardLeakTest('Catches leaks in closures', 'closure_test', 9);
  • createStandardLeakTest('Catches leaks in closures on dom', 'closure_test_dom', 9);
  • createStandardLeakTest('Catches leaks in closures when event listener is assigned on a property', 'closure_test_dom_on_property', 9);
  • createStandardLeakTest('Catches leaks in closures, even with irrelevant DOM objects', 'closure_test_irrelevant_dom', 9);
  • createStandardLeakTest('Catches leaks in closures, even with disconnected DOM fragments', 'closure_test_disconnected_dom', 10);
  • createStandardLeakTest('Catches leaks in closures, even with disconnected DOM collections', 'closure_test_disconnected_dom_collection', 11);
  • createStandardLeakTest('Catches leaks when object is copied and reassigned', 'reassignment_test', 10);
  • createStandardLeakTest('Catches leaks when object stored in multiple paths', 'multiple_paths_test', 12);
  • createStandardLeakTest('Ignores code that does not grow objects', 'irrelevant_paths_test', 8);
  • createStandardLeakTest('Catches event listener leaks', 'event_listener_leak', 5);
  • createStandardLeakTest('Ignores responsible event listener removal', 'event_listener_removal', 5);
  • createStandardLeakTest('Catches leaks that grow DOM unboundedly', 'dom_growth_test', 3);

Problem: is this agent part affect DiagnosisLeak traces?

https://github.com/plasma-umass/BLeak/blob/60836b01a8be42242f079e53639065feaba74e75/src/lib/bleak_agent.ts#L1002

    // Array modeling
    Array.prototype.push = (function(push) {
      return function(this: Array<any>, ...items: any[]): number {
        try {
          disableProxies = true;
          if (getProxyStatus(this) === ProxyStatus.IS_PROXY) {
            const map: GrowthObjectStackTraces = getProxyStackTraces(this);
            const trace = _getStackTrace();
            for (let i = 0; i < items.length; i++) {
              _addStackTrace(map, `${this.length + i}`, trace);
            }
          }
          return push.apply(this, items);
        } finally {
          disableProxies = false;
        }
      };
    })(Array.prototype.push);

Problem: DiagnosisLeaks runCode error on `typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()`

Child_PID[13652] stdout: $$$INSTRUMENT_PATHS$$$ DONE. Instrumented trees [
  {
    isGrowing: false,
    children: [ [Object] ],
    type: 1,
    indexOrName: 'queueMicrotask'
  }
]

e.result.value:  undefined
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" &&  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false
[DEBUG node_driver] runCode:  typeof(BLeakConfig) !== "undefined" && BLeakConfig.loop[0].check()
e.result.value:  false

Problem: M1 machine NodeJS driver runCode `$$$INSTRUMENT_PATHS$$$` not defined

[DEBUG node_driver] runCode:  $$$INSTRUMENT_PATHS$$$([])
exceptionDetails:  {
  exceptionId: 14,
  text: 'Uncaught',
  lineNumber: 0,
  columnNumber: 0,
  scriptId: '164',
  exception: {
    type: 'object',
    subtype: 'error',
    className: 'ReferenceError',
    description: 'ReferenceError: $$$INSTRUMENT_PATHS$$$ is not defined\n' +
      '    at <anonymous>:1:1',
    objectId: '3939281361161565106.1.2'
  }
}

Works on @Frost-Lee 's Intel machine but not on my M1 machine

Problem: Runtime.evaluate error `{ code: -32000, message: 'Cannot find context with specified id' }`

When running Runtime.evaluate from node driver:

[DEBUG node_driver] runCode:  $$$INSTRUMENT_PATHS$$$([{"isGrowing":true,"children":[],"type":1,"indexOrName":"LEAKOBJ","id":37484},{"isGrowing":false,"children":[{"isGrowing":false,"children":[{"isGrowing":false,"children":[{"isGrowing":false,"children":[{"isGrowing":true,"children":[],"type":1,"indexOrName":"list","id":35134}],"type":1,"indexOrName":"tail"}],"type":4,"indexOrName":"queue"}],"type":3,"indexOrName":"__scope__"}],"type":1,"indexOrName":"queueMicrotask"}])
failed to evaluate expression in child process ProtocolError: Cannot find context with specified id
    at /Users/chenxizh/workspace/practium/nleak/node_modules/chrome-remote-interface/lib/chrome.js:94:35
    at Chrome._handleMessage (/Users/chenxizh/workspace/practium/nleak/node_modules/chrome-remote-interface/lib/chrome.js:257:17)
    at WebSocket.<anonymous> (/Users/chenxizh/workspace/practium/nleak/node_modules/chrome-remote-interface/lib/chrome.js:235:22)
    at WebSocket.emit (node:events:513:28)
    at WebSocket.emit (node:domain:489:12)
    at Receiver.receiverOnMessage (/Users/chenxizh/workspace/practium/nleak/node_modules/ws/lib/websocket.js:1068:20)
    at Receiver.emit (node:events:513:28)
    at Receiver.emit (node:domain:489:12)
    at Receiver.dataMessage (/Users/chenxizh/workspace/practium/nleak/node_modules/ws/lib/receiver.js:517:14)
    at Receiver.getData (/Users/chenxizh/workspace/practium/nleak/node_modules/ws/lib/receiver.js:435:17) {
  request: {
    method: 'Runtime.evaluate',
    params: {
      expression: '$$$INSTRUMENT_PATHS$$$([{"isGrowing":true,"children":[],"type":1,"indexOrName":"LEAKOBJ","id":37484},{"isGrowing":false,"children":[{"isGrowing":false,"children":[{"isGrowing":false,"children":[{"isGrowing":false,"children":[{"isGrowing":true,"children":[],"type":1,"indexOrName":"list","id":35134}],"type":1,"indexOrName":"tail"}],"type":4,"indexOrName":"queue"}],"type":3,"indexOrName":"__scope__"}],"type":1,"indexOrName":"queueMicrotask"}])',
      returnByValue: true
    },
    sessionId: undefined
  },
  response: { code: -32000, message: 'Cannot find context with specified id' }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.