comcast / surf-n-perf Goto Github PK

View Code? Open in Web Editor NEW

90.0 14.0 22.0 1.01 MB

Micro-library for gathering web page performance data

Home Page: http://comcast.github.io/Surf-N-Perf/

License: MIT License

JavaScript 96.39% Ruby 2.06% HTML 1.55%

navigation-timing web-performance javascript user-timing hacktoberfest

surf-n-perf's Introduction

Surf-N-Perf

Micro-library for gathering frontend web page performance data.

Surf-N-Perf provides a simple to use API to gather User Timing and other important performance data in any browser.

Tired of typing window.performance.getEntriesByName('foo')[0].startTime; with your User Timing Polyfill?

With Surf-N-Perf, all you need is surfnperf.getMark('foo')';, and that's just the start!

Check out the JavaScript API to see all of its features and the full documentation for a list of methods & how to use them.

Available as an NPM Module, Ruby Gem, and a Bower package.

Usage

Including the code in your project

There are 2 pieces of code that need to be included in your webpage:

1. The following code must be included as high up in the source code of your base HTML document as possible, ideally right after the opening <head> tag:

<script>
  var SURF_N_PERF = {
    marks: {},
    highResMarks: {}
  };

  SURF_N_PERF.marks.pageStart = (new Date()).getTime();

  if(window.performance) {
    if(window.performance.now) {
      SURF_N_PERF.highResMarks.pageStart = window.performance.now();
    }
    if(window.performance.mark) {
      window.performance.mark('pageStart');
    }
  }

  SURF_N_PERF.visibility = {
    initialState: document.visibilityState,
    stateUpdates: [],
    hiddenProperty: null,
    stateProperty: null,
    eventName: null,
    markChange: function() {
      var markName = 'visibility' + SURF_N_PERF.visibility.stateUpdates.length;

      if (window.performance) {
        if (window.performance.mark) {
          window.performance.mark(markName);
        }

        if (window.performance.now) {
          SURF_N_PERF.highResMarks[markName] = window.performance.now();
        }
      }

      SURF_N_PERF.marks[markName] = new Date().getTime();

      SURF_N_PERF.visibility.stateUpdates.push(document[SURF_N_PERF.visibility.stateProperty]);
    },
  };

  if('hidden' in document) {
    SURF_N_PERF.visibility.hiddenProperty = 'hidden';
    SURF_N_PERF.visibility.stateProperty = 'visibilityState';
    SURF_N_PERF.visibility.eventName = 'visibilitychange';
  } else if('webkitHidden' in document) {
    SURF_N_PERF.visibility.hiddenProperty = 'webkitHidden';
    SURF_N_PERF.visibility.stateProperty = 'webkitVisibilityState';
    SURF_N_PERF.visibility.eventName = 'webkitvisibilitychange';
    SURF_N_PERF.visibility.initialState = document[SURF_N_PERF.visibility.stateProperty];
  }

  SURF_N_PERF.setPageLoad = function() {
    SURF_N_PERF.marks.loadEventEnd = (new Date()).getTime();

    if(window.performance && window.performance.now) {
      SURF_N_PERF.highResMarks.loadEventEnd = window.performance.now();
    }
  };

  SURF_N_PERF.setFirstPaint = function() {
    SURF_N_PERF.marks.firstPaintFrame = (new Date()).getTime();

    if(window.performance && window.performance.now) {
      SURF_N_PERF.highResMarks.firstPaintFrame = window.performance.now();

      if(window.performance.mark) {
        window.performance.mark('firstPaintFrame');
      }
    }
  };

  if(window.addEventListener) {
    if (SURF_N_PERF.visibility.stateProperty) {
      document.addEventListener(SURF_N_PERF.visibility.eventName, SURF_N_PERF.visibility.markChange, false);
    }
    window.addEventListener('load', SURF_N_PERF.setPageLoad, false);
  } else {
    window.attachEvent('onload', SURF_N_PERF.setPageLoad);
  }
  if (window.requestAnimationFrame) {
    window.requestAnimationFrame(SURF_N_PERF.setFirstPaint);
  }
</script>

That provides support for the following:

A "pageStart" mark for browsers that do not support Navigation Timing which can be used to compute durations from when the page first started loading (specifically, this mark falls between the domLoading and domInteractive attributes of Navigation Timing)
"pageStart" marks for browsers that support High Resolution Time and/or User Timing so that "pageStart" can be used as a consistent starting point for duration calculations across all browsers regardless of their supported features
A "loadEventEnd" mark for browsers that do not support Navigation Timing which can be used to compute durations from when the load event of the document is completed (loadEventEnd)
A "loadEventEnd" DOMHighResTimeStamp mark for calculating high resolution durations between a Navigation Timing mark and a user mark in browsers that support High Resolution Time but don't support User Timing
A "firstPaintFrame" mark (available in the best possible format for the browser, either a User Timing Mark, DOMHighResTimeStamp, or DOMTimeStamp) that approximates the Time To First Paint in browsers that support window.requestAnimationFrame.
The initial visibilityState as well as listeners for the "visibilitychange" event, enabling the ability to calculate how much time the page was hidden when you call surfnperf.getHiddenTime(). This is of particular importance as Chrome as of version 57 and Firefox as of version 57 limit the resources assigned to background (hidden) tabs.

2. Then just drop the surfnperf.min.js in your codebase and reference that JavaScript file in your HTML document. If you're using RequireJS or Browserify, it registers itself as 'surfnperf'.

3. (Optional) If you would like access to the Resource Timing helper functions, include resource-timing.js in your codebase and reference that JavaScript file in your HTML document. If you're using RequireJS, it registers itself as 'surfnperf/resource-timing', otherwise it is available on window as window.surfnperfRT.

Storing & Retrieving Performance Data

Details in the JavaScript API page in the wiki

Resource Timing Helper Functions

Documented in the JSDoc-generated Documentation

Ruby Project Integration

Using within a Rails project

The surfnperf Ruby Gem allows you to quickly & easily integrate Surf-N-Perf into your Rails projects. To include the necessary files, add surfnperf to your Gemfile:

gem 'surfnperf'

After a $ bundle install, you'll be able to include the main JavaScript file in your JavaScript manifest by simply adding:

//= require surfnperf

The necessary script for the <head> of your HTML document is also available to you via a partial template that you can include in the appropriate layout file for your page, such as app/views/layouts/application.html.erb by simply adding this line:

<%= render "surfnperf/head" %>

Those 3 lines of code are all your need to get started using Surf-N-Perf in Rails!

Using within a Middleman project

The surfnperf Ruby Gem also allows you to quickly & easily integrate Surf-N-Perf into your Middleman projects. Instructions are similar to the Rails instructions above, with one extra step. Start by adding at least v1.1.0 of surfnperf to your Middleman project's Gemfile:

gem "surfnperf", ">=1.1.0"

After a $ bundle install, you'll be able to include the main JavaScript file in your JavaScript manifest by simply adding:

//= require surfnperf

The necessary script for the <head> of your HTML document is also available to you via a custom defined helper that you can include in the appropriate layout file for your page, such as source/layouts/layout.erb by adding this line:

<%= surfnperf_head %>

You will also have to configure the extension for that helper to be recognized by Middleman by adding this line to your config.rb:

activate :surfnperf

You'll want to do that outside of your build-specific configuration (i.e. outside the configure :build do block) so that it is available when you run $ bundle exec middleman server

Those 4 lines of code are all your need to get started using Surf-N-Perf in Middleman!

Using within other Ruby projects that integrate with Sprockets

Sprockets is what powers the Asset Pipeline in Rails, Middleman, and other Ruby website tools. For these other Ruby projects that use Sprockets, integration is similar to the Rails instructions above, with one extra step:

Add surfnperf to your Gemfile:

gem 'surfnperf'

After a $ bundle install, include surfnperf.js in your JavaScript manifest by adding:

//= require surfnperf

For now, you'll have to manually include the necessary script for the <head> of your HTML document.

Running Tests & Other Development Tools

Tests are written in Jasmine and run with Karma

Install the dependencies by executing this command from the root of your Surf-N-Perf project directory:

$ npm install

If Grunt CLI has not been already installed, go install it.

And then run the tests, JSHint, beautify your code & generate the minified file with:

$ grunt dev

By default, it will run the tests using PhantomJS. You can also run the tests in any browser by going to http://localhost:9876/

The grunt dev process will watch for file updates, so as you modify surfnperf.js or the test files, it will automatically run jshint, jsbeautifier, uglify & the tests. To stop the watch process, press control + C

License

Licensed under the MIT License

surf-n-perf's People

Contributors

Stargazers

Watchers

surf-n-perf's Issues

npm WARN EBADENGINE Unsupported engine

Our package.json says it supports any version of NodeJS greater than or equal to 0.10.x, but that results in a warning when installing in npm v7:

$ npm install
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE   package: '[email protected]',
npm WARN EBADENGINE   required: { node: '0.10.x' },
npm WARN EBADENGINE   current: { node: 'v14.18.0', npm: '7.24.1' }
npm WARN EBADENGINE }

Ideally there is an engines value we can set it to that is really any version of node >0.10.0

Create an npm module

Re-run jshint with each file modification in dev

Currently, 'grunt dev' will line files and run tests. It re-runs the tests on each file modification but it doesn't re-lint.

Upgrade Jasmine

The project currently uses Jasmine 1.3. Upgrade to Jasmine 2.3

http://jasmine.github.io/2.3/upgrading.html has tips on upgrading

Fix "Automated documentation update" script

Travis is failing to execute deploy.sh:

$ test $TRAVIS_PULL_REQUEST == "false" && test $TRAVIS_BRANCH == "master" && bash deploy.sh
/home/travis/build/Comcast/Surf-N-Perf/node_modules/chalk/source/index.js:106
	...styles,
	^^^
SyntaxError: Unexpected token ...
    at createScript (vm.js:56:10)
    at Object.runInThisContext (vm.js:97:10)
    at Module._compile (module.js:549:28)
    at Object.Module._extensions..js (module.js:586:10)
    at Module.load (module.js:494:32)
    at tryModuleLoad (module.js:453:12)
    at Function.Module._load (module.js:445:3)
    at Module.require (module.js:504:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/home/travis/build/Comcast/Surf-N-Perf/node_modules/grunt-legacy-log-utils/index.js:12:13)

I'm guessing updating Node and/or using package-lock.json will address this

Full Middleman Support for Ruby Gem

The Ruby Gem provides access to the main surfnperf.js file via the asset pipeline, but ideally it would also provide an easy way to drop in the script required for the head of the document via a Middleman Extension

With the Middleman 4.0's full release coming soon (see https://rubygems.org/gems/middleman), let's make sure the extension works with both Middleman 3.x and Middleman 4.0.

If you're not already familiar with Bundler and/or RVM, you're going to want to be so you can isolate your Middleman versions

add getFirstContentfulPaint()

should behave similar to getFirstPaint() with the "first-contentful-paint" name instead of "first-paint"

See https://developer.mozilla.org/en-US/docs/Web/API/PerformancePaintTiming for docs

Have Feature Demo work with http and https

http://comcast.github.io/Surf-N-Perf/demo/ works well, but https://comcast.github.io/Surf-N-Perf/demo/ does not load resources loaded as http instead of https. Update the demo to work for either protocol.

Time to First Paint metrics

Take a look at http://www.lognormal.com/blog/2013/11/11/calculating-first-paint/ and https://github.com/lognormal/boomerang/blob/master/plugins/navtiming.js and you'll see there are browser-specific ways to calculate first paint for recent versions of IE, Chrome & Firefox. Let's do the following:

add a getFirstPaint() method to Surf-N-Perf that returns the browser's native calculation, or null if the browser doesn't have one
add a getFirstPaintFrame() method will return an estimated value based on Window.requestAnimationFrame.
- I assume that's going to require adding some code to the block we require in the <head> of the document.
- If getFirstPaintFrame() is called but the data is not available yet, return ~~undefined~~NaN
- If getFirstPaintFrame() is called but the browser does not support Window.requestAnimationFrame, return null

After we have that working:

Add it to the JavaScript API page
Add it to the Demo page

'GitHub' is mispelled in the CONTRIBUTING doc

'GitHub' is spelled 'GiHhub' on line 9 of CONTRIBUTING

Add support for the clearMarks method

https://www.w3.org/TR/user-timing/#dom-performance-clearmarks

If we call clearMarks without an argument, then it should still maintain the marks we recommend setting up in the <head> of the document: 'pageStart', 'loadEventEnd', and 'firstPaintFrame'.

If you specify one of those 3 marks as the argument, then we'll delete it, but that's on you 😉

Grunt mods to support real time dev testing and minification

Modify grunt so that it automatically runs tests and minification as you change specified js files during development.

Surf-N-Perf D3 Visualization

Provide another demo page that provides visualization of Surf-N-Perf data, similar to how Comcast uses it with the data we log and then visualize:

https://speakerdeck.com/johnriv/the-truth-about-your-web-apps-performance?slide=53

Feel free to come up with other visualizations as well.

The data visualized should be Network Time, Server Time & Processing Load Time.

My vision for this is that there would be a parent page with the D3 visuals, and that would include an iframe that would serve up another page, run Surf-N-Perf in that iframe, and then send the Network Time, Server Time & Processing Load Time to the parent page so that D3 can visualize it. After it sends the data to the page and D3 renders it, the iframe would reload to gather a new set of those same data points, so it looks like data is coming in in real time. So you'll end up with something that looks like this:

https://fastly.github.io/epoch/real-time/#area

Ideally, we'd publish it to http://comcast.github.io/Surf-N-Perf/visualize/

Switch from gh-pages to /docs folder for documentation

See https://github.com/blog/2233-publish-your-project-documentation-with-github-pages for details on the new GitHub feature. This should simplify things regarding out documentation build process.

Support both ways for Chrome to report on first paint time

The Chrome team is planning to migrate away from chrome.loadTimes() to navigation timing and expose first paint time through a standard API. Once that new standard API is available, Surf-N-Perf should favor that over chrome.loadTimes().

As of June 2016 this issue is blocked until that API is available. See chromium issue 621512 to track the status.

Create a Ruby gem

Create a surfnperf Ruby Gem that includes the minified source in the vendor directory and allows easy access to the code for the of the document specificed in the README via a partial that Rails or Middleman can use.

Support multiple marks with the same key

Currently getMark returns the High Res mark if the browser supports User Timing instead of the User Timing mark. This is by design (user Timing is just for duration calculations), but results in overwriting values with the same key.

Need a way to have the same key used for multiple marks so we can maintain that User Timing feature if needed.

Related - for more accurate marks when the browser supports both High Res time and User Timing, we should grab the User Timing mark reference to startTime and store that as the high-res mark instead of marking with window.performance.now()

Generate JSDoc Documentation

Fully comment the code using JSDoc syntax and publish the generated documentation to http://comcast.github.io/Surf-N-Perf/

I'd like to publish it similar to how Polymer does:
Example Source: https://github.com/Polymer/core-ajax/blob/master/core-ajax.html
Example Output: https://www.polymer-project.org/docs/elements/core-elements.html#core-ajax

I haven't looked at how they're doing that yet.

Note that I originally planned to use NDoc syntax and only commented some of the code, so the existing comments will have to be modified appropriately.

Single Mark Helper

Since it's currently only possible to have one value for a key (see Issue #54), that results in code like this when you only want to mark a key the first time it happens:

if (!surfnperf.getMark('fetchData')) {
  surfnperf.mark('fetchData');
}

A markOnce helper method to provide that same functionality would be useful.

We should determine a plan around this & #54 before our next release.

Run karma via grunt.

It would be nice to use grunt for running tests, linting code, and minifying the build target.

These tasks could be separated into default and 'build' tasks. The default grunt task could be used in CI; it would lint code & run tests.

Setup Travis CI

Once the repo is publish, set up Travis CI to run the tests and add the "Build Passing" tag

Create a demo page

Show the proper setup of the code and use of the JavaScript API to output the values Surf-N-Perf gathers to the page

Specifically, have the HTML content of the page show the methods below, and use Surf-N-Perf to output the value in the "value_returned_goes_here" part:

surfnperf.eventStart('demo'); // (no output)
surfnperf.setCustom('customKey', 'custom value'); // (no output)
surfnperf.getNetworkTime(); // value_returned_goes_here
surfnperf.getServerTime(); // value_returned_goes_here
surfnperf.getNetworkLatency(); // value_returned_goes_here
surfnperf.getProcessingLoadTime(); // value_returned_goes_here
surfnperf.getFullRequestLoadTime(); // value_returned_goes_here
surfnperf.getCustom('customKey'); // value_returned_goes_here
surfnperf.mark('foo'); // (no output)
surfnperf.duration('pageStart', 'foo'); // value_returned_goes_here
surfnperf.duration('responseEnd','foo'); // value_returned_goes_here
surfnperf.duration('loadEventEnd','bar'); // value_returned_goes_here
surfnperf.eventEnd('demo'); // (no output)
surfnperf.eventDuration('demo'); // value_returned_goes_here
surfnperf.eventDuration('demo',{decimalPlaces:10}); // value_returned_goes_here

Note that the code will have to execute after the "onload" event for the page fires.

Easiest way to include Surf-N-Perf on that page is to link to:
http://rawgit.com/Comcast/Surf-N-Perf/master/surfnperf.min.js

Ideally, we'd publish it to http://comcast.github.io/Surf-N-Perf/demo/

Running npm test after a fresh install doesn't work

$ npm test

> [email protected] test /Users/pselle200/Sites/Surf-N-Perf 
> karma start

sh: karma: command not found
npm ERR! weird error 127 
npm ERR! not ok code 0

Resource Timing Metrics

Here's what I want to build:

a "Resource Timing" add-on with the following functions: (note for all functions below, if the browser does not support resource timing (i.e. window.performance.getEntriesByType does not exist), then every function should return null)
- getOrigins(options) - returns an array of all the origins (e.g. "https://github.com") returned by window.performance.getEntriesByType("resource") with no duplicates. The options passed to it is an object that includes either a "whitelist" key with a single string or an array of origins as the value that will be used to return only those items that are in the list of origins or a "blacklist" key with a single string or an array of origins as the value that will be used to filter out those origins
  - IE may include other non-fetched resources, such as about:blank and javascript: URLs in the ResourceTiming data, so also filter out non-http: and https: protocols.
- getResourcesFromOrigin(origin) - returns an array of all the resources ~~names~~ from the specified origin
- getResourceNamesFromOrigin(origin) - returns an array of all the resource names from the specified origin
- isBufferFull() - returns false (if resource timing is supported), unless the performance.onresourcetimingbufferfull or performance.onwebkitresourcetimingbufferfull event fired, then returns true

For all of these functions below, the "name" argument is full request, e.g. "http://johnriv.github.io/Surf-N-Perf/docs/scripts/linenumber.js". If the argument starts with a "/", then use the page's current origin. For example:
surfnperfRT.getResource("/Surf-N-Perf/docs/scripts/linenumber.js") would return window.performance.getEntriesByName("http://johnriv.github.io/Surf-N-Perf/docs/scripts/linenumber.js", "resource")[0] if it was called from http://johnriv.github.io/Surf-N-Perf/

duration(name, eventA, eventB, options) - returns the diff between 2 resource timing marks for a single resource (name). Takes optional 4th param which can be {decimalPlaces:NUMBER}, just like Surf-N-Perf's. If eventA or eventB's value is 0, return false. Returns undefined if the resource is not found
getResource(name) - shorthand for window.performance.getEntriesByName(name, "resource")[0]
start(name, options) - shorthand for window.performance.getEntriesByName(name, "resource")[0].startTime. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found
end(name, options) - shorthand for window.performance.getEntriesByName(name, "resource")[0].responseEnd. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found
getFullRequestLoadTime(name, options) - returns the 'duration' value for the specified resource from window.performance.getEntriesByName(name, "resource")[0]. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found.
getNetworkTime(name, options) - very similar to Surf-N-Perf's getNetworkTime in that it returns the duration between 'fetchStart' & 'connectEnd' for the resource. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found. Returns false if connectEnd is 0 (3rd-party without a Timing-Allow-Origin: * header)
getServerTime(name, options) - very similar to Surf-N-Perf's getServerTime in that it returns the duration between 'requestStart' & 'responseEnd' for the resource. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found. Returns false if requestStart is 0 (3rd-party without a Timing-Allow-Origin: * header)
getBlockingTime(name,options) - see http://nicj.net/resourcetiming-in-practice/ for logic. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found. Returns false if requestStart is 0 (3rd-party without a Timing-Allow-Origin: * header)
getNetworkDuration(name,options) - see http://www.stevesouders.com/blog/2014/11/25/serious-confusion-with-resource-timing/ for logic. Optional 2nd param for {decimalPlaces:NUMBER}. Returns undefined if the resource is not found. Returns false if requestStart is 0 (3rd-party without a Timing-Allow-Origin: * header)

And here's how it should be built:

create a new file at /lib/resource-timing.js
- add the same copyright notice to that as surfnperf.js
- the Gruntfile.js will need to be updated so uglify creates a /resource-timing.js from /lib/resource-timing.js, jshint checks /lib/resource-timing.js and watch watches /lib/resource-timing.js
- the package.json will need to be updated to include /lib/resource-timing.js & /resource-timing.js in the list of files
The module definition should be defined as follows so that surfnperf is a dependency:

(function(root, factory) {
  if(typeof define === 'function' && define.amd) {
    // For Require.js
    define('surfnperf/resource-timing', ['surfnperf'], factory);
  } else if(typeof exports === 'object') {
    // For Browserify
    module.exports = factory(require('surfnperf'));
  } else {
    // Browser global if not using Require.js or Browserify
    root.surfnperfRT = factory(root.surfnperf);
  }
}(this, function(surfnperf) {
  // code goes here
}));

All code should be 100% tested
All methods should be fully documented in code with JSDoc comments
Demo page should be updated to showcase new methods
README.md & JavaScript API should be updated explaining how to use them

As a reference, these links help explain the value of Resource Timing:
http://nicj.net/resourcetiming-in-practice/
http://www.stevesouders.com/blog/2014/11/25/serious-confusion-with-resource-timing/
https://paul.kinlan.me/detecting-injected-content/

Have Real-time Performance Metrics Graph Demo work with http and https

http://comcast.github.io/Surf-N-Perf/visualize/ works well, but https://comcast.github.io/Surf-N-Perf/visualize/ does not load resources loaded as http instead of https. Update the demo to work for either protocol.

Create a jshint Grunt task

Lint the code on each build with grunt-contrib-jshint.

Resource Timing: `start` & `end` should be `getStart` & `getEnd`

the majority of the other methods start with "get", so those should too

duration can remain without the "get" since that same method exists for the main surfnperf library

Fix 404 on http://comcast.github.io/Surf-N-Perf/demo/

There's a 404 for http://comcast.github.io/Surf-N-Perf/demo/scripts/bootstrap.min.js, which I don't think we even need that JS file for anything

Resource Timing: Accept resource or name as first argument

It's possible you may have the resource already and just want to use the helper methods. In that case, it makes sense to pass the resource to the helper method instead of the name, so have the methods accept either.

Expose Mocks or Mocking functionality

Clients consuming the library will presumably want to write tests validating their logic around using the data obtained from Surf-N-Perf. Since Surf-N-Perf presumably needs to mock the underlying browsers apis, etc for it's own use it would be nice to explicitly and cleanly expose these to clients for their own testing so they don't need to re-roll their own.

This could be in the form of just exposing mock data or providing functions to generate mocks based on some seed.