Assetflow is an asset deployment tool. It supports md5 hash comparison with S3 and enables you to create powerful asset flows easily and fast. It is a Grunt task and applies solid cache-busting techniques transparently.
If you are not familiar with Grunt check out the Grunt's Getting Started guide.
A typical deployment flow using assetflow:
- Scan assets and generate MD5 hashes.
- Create the
manifest.json
file. - Copy all assets and rename them with their md5 hash to a temporary location.
- Perform
HEAD
operations with S3 and compare hashes using S3'sETAG
. - Upload all assets that did not have a matching
ETAG
.
Optionally there are two more tasks you can perform:
- Search & replace any set of assets based on a custom keyword, i.e.
__ASSET(img/logo.jpg)
. - Create the
clientManifest.js
file, a client optimized subset of the manifest.
npm install assetflow --save-dev
- Grunt Task
assets
:: Creates the manifest file and copies your assets to a temp folder. - Grunt Task
assetsReplace
:: Replaces defined keywords in files using the manifest file. - Grunt Task
assetsBundle
:: Create a front-end optimized manifest file. - Grunt Task
assetsS3
:: Compare assets' hashes with S3 and upload new and changed files. - Using Assetflow on Node
The assets
task performs these operations:
- Scans all the defined assets.
- Generates md5 hashes for each asset.
- Create the
manifest.json
file.- If a manifest file exists, it will compare the hashes.
- Copy all new or updated assets and rename them with their md5 hash to a temporary location.
When this task finishes all your assets have been copied to a new temporary folder that you defined. This folder will contain your assets renamed with their own hash, like so:
app.js
--> app-h522md41d.js
The manifest.json
file generated by this task keeps a reference to all your assets so their names can be properly resolved in all environments.
Type: string
Default: manifest.json
Define the location of the manifest file.
Type: string
Default: none
Add the url of your CDN to prepend it to all assets.
Type: string
Default: none
The rel
option will perform directory subtraction on the source to calculate the relative path to the asset. Consider this case:
Your folder of static assets is under assets/
, so the path to your logo would be assets/img/logo.png
which would be accessed by the browser as /img/logo.png
.
Declaring the assets
folder as a rel
path will make sure that all assets have the proper url.
Example
assets: {
options: {
rel: 'assets/'
},
all: {
src: 'assets/**',
dest: 'temp/assets'
}
}
Type: number
Default: none
The md5 hash is 32 bytes long, you don't need all of it, use this option to truncate the hash down to n chars.
Type: string
Default: none
This option will prepend a value to the asset's key. It is mostly used to prepend a slash and make the asset key absolute, for example:
By default, the assets
task will create records in the manifest.json
file as relative web paths: img/logo.png
. If you need the key to be an absolute path then you have to use prepend
.
options: {
prepend: '/'
}
Type: number
Default: 100
The maximum number of concurrent operations, in this case the operations are file copying.
Set this to 0 to disable throttling. Set it to 1 to make all operations serial.
Type: boolean
Default: false
A fancy progress bar.
Type: boolean
Default: false
Print extra debugging information.
Here's an annotated configuration for the assets
task:
assets: {
// global task options
options: {
// don't output debug information
debug: false,
// trancate the hash length to 8 chars.
truncateHash: 8,
// define the location of the manifest file.
manifest: 'temp/manifest.json',
// define the location of the cdn.
cdnurl: 'http://d3s3z9buwru1xx.cloudfront.net/assets/',
// Set maximum number of file copying concurent operations.
maxOperations: 100,
// Show a fancy progress indicator
progress: true,
// Set path to substract so the relative path for the assets can be calculated.
rel: 'lib/'
},
all: {
// local task options
options: {
// change the rel path to test/case
rel: 'assets/'
},
src: [
// all files under folder assets
'assets/**',
// except all files in less folder
'!assets/less/**',
// except all files in handlebars folder
'!assets/handlebars/**'
],
dest: 'temp/assets'
}
}
The assetsReplace
task will search and replace the contents of your assets. It is useful for cases where you don't have the ability of a 'helper' to resolve your assets.
LESS files are a typical example, use a custom keyword to include your assets and run the assetsReplace
task to populate the asset urls in your .less
files. For example if the custom keyword is __ASSET()
:
@bg-dot-light: url(__ASSET(img/pdf-icon-cv.png)) repeat 0 0 #2a2a2a;
After the assetsReplace
task is executed the same line will look like this:
@bg-dot-light: url(http://d3s3z9buwru1xx.cloudfront.net/assets/img/pdf-icon-cv-fk44j2s.png) repeat 0 0 #2a2a2a;
The
assetsReplace
task is based on grunt-string-replace by @erickrdch
Type: string
Default: manifest.json
Define the location of the manifest file.
Type: string
Default: none
Define the keyword that will be searched for replacement. Use the %
char as a placeholder for the asset name. e.g.:
options: {
key: '__ASSET(%)'
}
Type: string
Default: none
Use this to define a regex as a keyword. The type must be string, but the string will be evaluated as a regex. The %
char is a placeholder for the asset name and the reason why this option needs to be a string instead of a regex type.
So take case to double escape what you need escaped, e.g.
// this regex
var reg = /match[\s]space/;
// is represented like that as a string:
var strReg = 'match[\\\s]space';
Type: string
Default: none
This option will prepend a value to the asset's key. It is mostly used to prepend a slash and make the asset key absolute.
Type: boolean
Default: false
Print extra debugging information.
Here's an annotated configuration for the assetsReplace
task:
assetsReplace: {
// global task options
options: {
// don't output debug information
debug: false,
// define the location of the manifest file.
manifest: 'temp/manifest.json',
},
// the less target
less: {
options: {
// define the keyword for this target
key: '__ASSET(%)'
},
files: {
// Search & replace all .less files under the assets/less folder
// and output the result in the temp/less folder
'temp/less/': ['assets/less/**/*.less']
}
},
// the handlebars target
handlebars: {
options: {
// a regex for lax matching {{asset "%"}} allowing for spaces in between
// and single quotes.
keyRegex: '\\\{\\\{[\\\s]*asset[\\\s]+[\\\'\\\"]{1}%[\\\'\\\"]{1}[\\\s]*\\\}\\\}',
// prepend the slash on every asset query
prepend: '/'
},
// all the files from the assets/handlebars folder
src: 'assets/handlebars/**/*.hbs',
// output to temp/handlebars folder
dest: 'temp/handlebars/'
}
}
Files that are produced with the assetsReplace
task should be in your codebase and tracked.
For example, here is a typical setup for less files:
assets-replace.less
assets.less <--- this is auto-generated by Grunt
main.less
restStyles.less
@import "assets"
@import "restStyles"
// ....
//
// ONLY ADD IMAGES TO assets-replace.less
// IMAGES ADDED TO assets.less WILL BE OVERWRITTEN
//
@asset_img_logo: "__ASSET(/assets/logo.png)";
@asset_img_cover: "__ASSET(/assets/cover.jpg)";
This file is generated by the assetsReplace
Grunt task.
//
// ONLY ADD IMAGES TO assets-replace.less
// IMAGES ADDED TO assets.less WILL BE OVERWRITTEN
//
@asset_img_logo: "/assets/logo.png";
@asset_img_cover: "/assets/cover.jpg";
#header{
.logo {
position: relative;
margin: 0;
background: url(@asset_img_logo) bottom center no-repeat;
}
.cover {
background: url(@asset_img_cover) no-repeat #577d9c;
cursor: default;
padding-left: 35px;
background-position: 9px 9px;
}
}
The assetsBundle task will create a compact version of the manifest file, optimized for transferring to the client as a javascript file. The manifest.json
file is pretty large and not suited for getting transfered. Furthermore it might be the case where you have several hundreds, if not thousands of assets but you only need a handful to be consumed by your front-end app. This is the task that enables you to do this.
Type: string
Default: manifest.json
Define the location of the manifest file.
Type: boolean
Default: false
export the assets as an AMD module.
Type: boolean
Default: false
export the assets using commonjs pattern. e.g: module.exports=..
Type: string
Default: ASSETS
Define a namespace to export using the global window
Object. You can use dot notation.
Type: Array
Default: []
An array of strings where you define the asset filename keys you want included.
Type: boolean
Default: false
Print extra debugging information.
Here's an annotated configuration for the assetsBundle
task:
assetsBundle: {
// global task options
options: {
// define the location of the manifest file.
manifest: 'temp/manifest.json',
// export as AMD
amd: true
},
// Export as amd
amd: {
dest: 'temp/bundles/clientManifest.amd.js'
},
// Export using namespaces
ns: {
options: {
// Export assets on this global namespace:
ns: 'app.assets'
},
dest: 'temp/bundles/clientManifest.ns.js'
}
}
The javascript file generated by the assetsBundle
task will export an Object that contains key value pairs to the assets. If you are using AMD to export the assets this is how you'd access them:
var assets = require('assets');
var asset = assets['/img/logo.png'];
// --> https://d3s3z9buwru1xx.cloudfront.net/assets/img/logo-fh422j4f.png
This is a weak part in the API and will surely see changes in the future.
The assetsS3
task will read the manifest.json
file and upload all the assets to S3. Although it's optional, it is highly advised to use the checkS3Head
option which enables md5 hash checking between S3 and your local files.
The
assetsS3
task is based on grunt-S3 by @pifantastic. All options from that task are available in this one too.
These are the options that are only available in the assetsS3
task of Assetflow.
Type: string
Default: manifest.json
Define the location of the manifest file.
Type: boolean
Default: false
Perform md5 hash comparisons between the local assets and the ones in S3. It is highly advisable that you enable this option so you only upload files that have been updated.
Type: string
Default: none
The rel
option will perform directory subtraction on the source to calculate the relative path to the asset.
Type: number
Default: 100
The maximum number of concurrent operations, in this case the operations are network uploads.
Set this to 0 to disable throttling. Set it to 1 to make all operations serial.
Type: boolean
Default: false
A fancy progress bar.
Type: boolean
Default: false
Print extra debugging information.
These options are available by Grunt-S3, sic:
The grunt-s3 task is now a multi-task; meaning you can specify different targets for this task to run as.
The following are the default options available to each target.
- key - (string) An Amazon S3 credentials key
- secret - (string) An Amazon S3 credentials secret
- bucket - (string) An Amazon S3 bucket
- region - (string) An Amazon AWS region (see http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region)
- maxOperations - (number) max number of concurrent transfers - if not specified or set to 0, will be unlimited.
- encodePaths - (boolean) if set to true, will encode the uris of destinations to prevent 505 errors. Default: false
- headers - (object) An object containing any headers you would like to send along with the
transfers i.e.
{ 'X-Awesomeness': 'Out-Of-This-World', 'X-Stuff': 'And Things!' }
- access - (string) A specific Amazon S3 ACL. Available values:
private
,public-read
,public-read-write
,authenticated-read
,bucket-owner-read
,bucket-owner-full-control
- gzip - (boolean) If true, uploads will be gzip-encoded.
- gzipExclude - (array) Define extensions of files you don't want to run gzip on, an array of strings ie:
['.jpg', '.jpeg', '.png']
. - upload - (array) An array of objects, each object representing a file upload and containing a
src
and adest
. Any of the above values may also be overriden. - download - (array) An array of objects, each object representing a file download and containing a
src
and adest
. Any of the above values may also be overriden. - del - (array) An array of objects, each object containing a
src
to delete from s3. Any of the above values may also be overriden. - debug - (boolean) If true, no transfers with S3 will occur, will print all actions for review by user
Here's an annotated configuration for the assetsS3
task:
assetsS3: {
options: {
// no debug info
debug: false,
// enable checking md5 hashes by performing S3 HEAD requests.
checkS3Head: true,
// the manifest file
manifest: 'temp/manifest.json',
// aws credentials
key: 'AWS-KEY',
secret: 'AWS-SECRET',
bucket: 'S3-BUCKET',
// Enable public access
access: 'public-read',
// don't show the fancy progress indicator.
progress: false
},
all: {
// These options override the defaults
options: {
// limit concurent uploads to 100
maxOperations: 100
},
// the 'upload' option key is required
upload: {
// all files from temp/assets
src: 'temp/assets/**',
// a prefix folder on S3
dest: 'assets/',
// the rel option
rel: 'temp/assets/',
// gzip uploaded files
gzip: true,
// excluded these extensions
gzipExclude: ['.jpeg', '.jpg', '.png', '.gif', '.less', '.mp3',
'.mp4', '.mkv', '.webm', '.gz'],
// upload assets with a looong expire Cache-Control header.
headers: {'Cache-Control': 'max-age=31536000, public'}
}
}
}
You can use the Assetflow library on node:
// mind the () in the end!
var assets = require('assetflow')();
assets.config({
manifest: __dirname + '/assetManifest.json'
});
var assetUrl = assets.asset('/img/logo.png');
Like the client API, node's API is weak and may change in the future.
- v0.1.4, 08 May 2013
- Updated Knox so Assetflow will work for node 0.10.x
- Fixed bug when no rel path is used.
- v0.1.2, 17 April 2013
- S3 paths now get normalized using
path.normalize()
.
- S3 paths now get normalized using
- v0.1.1, 19 March 2013
- Added support for commonjs for browsers.
- v0.1.0, Mid March 2013
- Big Bang
Copyright 2012 Verbling (Fluency Forums Corporation)
Licensed under the MIT License