Code Monkey home page Code Monkey logo

pulldown's Introduction

pulldown

A way to get your favourite libraries and scripts from the internet, and fast.

Built with โ™ฅ by @jackfranklin and @phuu.

Build Status

Supports Node 0.10.x and 0.8.x.

The following documentation is for using Pulldown as a CLI tool. If you'd like documentation on the use of Pulldown as a module, please see MODULE_USAGE.md.

Installation

Install pulldown globally using npm:

$ npm install -g pulldown

If you get any errors (particularly if you are on Node 0.8), try updating npm:

$ npm install -g npm

This gives you a pulldown command to use on your command line:

$ pulldown

  Usage: pulldown <identifier>[::<file>] [<identifier>[::<file>], ...] [options]

  An <identifier> can be a URL, a library name or a set.

  Options:

    -o, --output  output directory

    -d, --dry-run  don't actually download the files

    -v, --version  get the current pulldown version

    -n, --noisy  verbose mode, basically

  Example usage:

    pulldown jquery             # Downloads jQuery
    pulldown jquery::jq.js      # Downloads jQuery to jq.js
    pulldown jquery angular.js  # Downloads jQuery and Angular.js
    pulldown backbone           # Downloads jQuery, Underscore.js and Backbone.js
    pulldown backbone -o js     # Downloads same as above, but into js/

Downloading Libraries

$ pulldown jquery
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to jquery.min.js
$ pulldown jquery underscore
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to jquery.min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/underscore/1.0.0/underscore.min.js was downloaded to underscore.min.js

Pulldown works by searching on the cdnjs site for what you asked it for.

Custom Downloads

Of course, not everything you might want to download exists on cdnjs. You can add your own custom downloads to ~/.pulldown.json. This file might look something like this:

{
  "mycustommodule": "http://www.madeup.com/custom/module.js"
}

Then you can run:

$ pulldown mycustommodule
->  Success: http://www.madeup.com/custom/module.js was downloaded to module.js

Pulldown will know where to look. Pulldown will always look in your local ~/.pulldown.json file before searching. This means if you want a particular version of a library you can put it into your local file, and Pulldown will always use that.

Sets on the Server

Pulldown comes with the notion of sets. Sets are a collection of libraries.

For example, a Backbone application typically requires 3 libraries:

  • Backbone
  • jQuery
  • Underscore

Rather than download all three yourself, Pulldown has you covered:

$ pulldown backbone
->  Success: https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.4.4/underscore-min.js was downloaded to underscore-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/backbone.js/1.0.0/backbone-min.js was downloaded to backbone-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to jquery.min.js

We define sets on our server. When you hit our server asking for backbone, we give you back jQuery, Underscore and Backbone. If you think there's more sets we should support, let us know.

Custom Sets

You can define custom sets in your ~/.pulldown.json. They look like this:

{
  "backboneapp": ["jquery", "underscore", "backbone.js"]
}

Here I define a custom set, backboneapp, that will download jQuery, Underscore and Backbone. This is an example we have on the server, so you don't need to, but it's useful for setting up common sets of libraries you use together. Downloading them all becomes as simple as:

$ pulldown backboneapp
->  Success: https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.4.4/underscore-min.js was downloaded to underscore-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/backbone.js/1.0.0/backbone-min.js was downloaded to backbone-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to jquery.min.js

Output Directories

You can tell Pulldown to save what it downloads into a directory, that will be created if it doesn't exist:

$ pulldown backbone -o foo/
->  Success: https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.4.4/underscore-min.js was downloaded to foo/underscore-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/backbone.js/1.0.0/backbone-min.js was downloaded to foo/backbone-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to foo/jquery.min.js

Custom File Names

If you're downloading something that will resolve to a single file, you can choose the name of the file that will be downloaded using two colons:

$ pulldown jquery::foo.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to foo.js

Versions

Pulldown supports finding a specific version of a library, and will do its best to find it. Use identifier@version:

$ pulldown [email protected]
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/1.7.1/jquery.min.js was downloaded to jquery.min.js

Pulldown searches cdnjs for it. If it can't find the right version, it'll give you the latest.

Dry Run

If you want to see what would happen when you run a particular command, append the -d or --dry-run flag. This will not download the files, but will tell you what would happen:

$ pulldown jquery -d
-> Dry Run - no files will be downloaded
-> https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js would have been downloaded to jquery.min.js

Upgrading

$ npm update pulldown -g

Contributing

See the CONTRIBUTING.md file.

Pulldown vs Bower

We get a lot of questions about the similarities and differences of Pulldown compared to Bower.

Short Answer: Bower is a package manager; Pulldown downloads files. Bower configurations can be saved with a project and shared by people working on it; Pulldown cannot โ€“ it is almost configuration-less. Think of it as way to get a copy of your favourite library, fast.

Long Answer: Bower is a far more complex and fully-featured package and, more importantly, dependency manager. If you have a complex project with multiple 3rd party libraries in play, and you want to keep tight control over the versions you're using, Bower is best suited. Use Bower when you wouldn't want to commit all those libraries to Git, but would want to commit a component.json file.

We envisage (and personally use) Pulldown for those times when you just want one or two libraries, and have little care for versioning or dependencies. For example, if I want to prototype a quick Backbone app, I can quickly run $ pulldown jquery underscore backbone.js to get all three. Here it doesn't matter to me which versions I have (Pulldown will always pull in the latest stable version by default) and I'll probably commit those libraries to Git too. Pulldown also has the concept of sets, which is often where people get confused. These two commands are equal:

$ pulldown jquery underscore backbone.js
$ pulldown backbone

The second command will download the Backbone set, which will give you Underscore, jQuery and Backbone. This looks like dependency management - and sort of is - but it's really dumb. All Pulldown is doing here is seeing you asked it for "backbone", and replacing that with "jquery underscore backbone.js", which it then goes and gets separately. Sets are hard coded in the Pulldown API:

"backbone": ["backbone.js", "underscore", "jquery"],

The sets are there purely to allow you to save time and type less. There is no versioning, dependency checks or anything done with sets. They are just a list of libraries to download, nothing more.

Remember:

  • Pulldown: run once, commit libraries to source control.
  • Bower: add package to configuration file, don't commit libraries to source control, run often.

If you've any further questions, please do ask.

Changelog

V1.1.0

  • Deal better and show on command line any API errors and problems with Heroku

V1.0.4

  • Fixed a bug that would show the help and not the version number when you ran $ pulldown -v.

V1.0.3

  • If you downloaded a URL and passed an output name (eg $ pulldown foo.com/bar.js::test.js), it will now save the file to the specified name and not just ignore it (bug #74).

V1.0.2

  • add found: true flag to the results returned from the pulldown.init callback. This enables you to see if a searchTerm was successful by seeing if result.found === true.

V1.0.1

  • Fixed TypeError when you search for something that no longer exists ( Number #70 )
  • Improve handling of errors and the console output.

V1.0.0

  • complete rewrite to separate Pulldown the module and Pulldown the CLI tool. The module is now responsible for searching the API and returning the contents of the library source. The CLI tool saves those to files and parses the command line arguments.
  • Various small bug fixes and tweaks.

V1.0.0rc1

  • you can now run a dry-run to see what will happen with the flag -d or --dry-run: $ pulldown -d jquery (Number #40)
  • if you use 1 semi colon instead of 2 to split URL and output (eg jquery:foo.js), which is incorrect, you'll see a nice error message instead of a JS exception. (Number #33)
  • you can run $ pulldown ls to list the sets we support. (Number #38)
  • bug fixed: when downloading zips, the -o flag was ignored. (Number #41)
  • bug fixed: you can now specify complex paths to download a file to: $ pulldown jquery::foo/bar/baz/jquery.js (Number #42)

V0.6.0

  • you can now programatically require Pulldown: var Pulldown = require("pulldown") (presuming it's in your package.json and installed as a dependency).
  • progress bars are shown when downloading a file

V0.5.0

  • pulldown will now notify you when a new version is out and you should upgrade
  • swapped to using chalk for the coloured output

V0.4.0

  • swapped to using :: for seperating search term with output name
  • support URLs

V0.3.2

  • allow help to be got at through pulldown -h or pulldown --help

V0.3.1

  • added a help command (pulldown)
  • don't add zip extension to output path if it is already there

V0.3.0

  • support downloading and unzipping of ZIP files.

V0.2.6

  • only let the file name be dictated by the user if there's only one file to download

V0.2.5

  • avoid downloading the same file more than once if we can avoid it

V0.2.4

  • show an error if the search term could not be resolved

V0.2.3

  • pass in an output directory to save all files to (pulldown backbone -o foo/)
  • support saving a file to a particular name (pulldown jquery:foo.js)

V0.2.2

  • improve Windows support (thanks @sindresorhus)

V0.2.1

  • make sure URLs in local JSON file are valid

V0.2.0

  • complete rewrite. Too much to document here (see this README for a full briefing on the new pulldown).

V0.1.2

  • updated structure of .pulldownrc to allow for specifying the file name
  • fix mkdir showing error if no directory given

V0.1.1

  • if you try to install something to a folder that doesn't exist, pulldown will now create it

V0.1.0

  • initial release!
  • this is a rewrite and rework of the old nodefetch module, with a better name.

pulldown's People

Contributors

jackfranklin avatar lloydwatkin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pulldown's Issues

Caching

Store files to some cache? Bower does this, and then will just use the cached version if it can.

May well be over kill for us though.

Specify Files Location

Hey thanks for the update.

This is getting pretty close.

The only thing I'd say is missing is specifying where a file is located after a pulldown happens.

IE I'd want to be able to put jquery into a ./vendor folder etc. instead of dumping it into the local folder. If this options already exists, I'm not sure I see it. It's not really possible to do this as far as I can see unless you make a per folder pulldown.json.

Abstract some stuff out

  • make the Pulldown module return string contents of file
  • the CLI should deal with the arguments, where to save the file, and so on.
  • make a CLI thing that does that and then downloads the file

Modularising shit

TypeError

pulldown foasodhasdasd
TypeError: Cannot call method 'split' of undefined

component.json

Do you think this should look for a component.json so that it's compatible with Bower et al?

Integration Tests?

Maybe a Node script that

  1. Uses Node's child_process.exec or similar, calls pulldown.
  2. Saves everything into a test/tmp dir or similar, and wipes that directory on finish?

Would be nice to have some that we can run before pushing a new version.

Local ~/.pulldown.json

It seems pretty reasonable that you'd want a local pulldown.json per project that you're working on otherwise it would get messy as everything is global.

The suggestion would be to scan for a local pulldown.json file in the current directory, then to scan the home_directory, then fetch the default one if there is nothing to be found.

Prompt for update

I don't know if this is possible.

But it would be great if we had some version checker in place so Pulldown could tell the user they should reinstall it.

$ pulldown foo
-> you're using Pulldown 0.4.0, which is outdated. (Latest is 0.5.0).
-> foo was downloaded to foo.com..

(Maybe an API endpoint?)

-c --concat option

Should minify the arguments (most come minified from cdnjs anyway) into one file, keeping the order in which they are passed.

Figure out why Travis fails

It seems to work on 0.10 but when Travis runs the tests on 0.8.25 they break.

Locally I am able to swap to Node 0.8.25, reinstall pulldown and then run it. It works fine, and running npm test gives me all green too.

.pulldownrc destination

If it doesn't exist, it doesn't work.

Need to make sure it exists. A cheeky mkdir -p should do it.

Integration Tests

We really should have some tests that test the integration.

I think we should mock the API layer but nothing else. So mock requests and replace them with some fake files, so a call to http://my.com/jquery/version.js just returns a fixture file with some text in.

Command Line help

Currently none given. Should be accessed through pulldown -h, pulldown --help or just pulldown.

CLI for creating custom things?

$ pulldown new foo:http://foo.com/foo.js
$ pulldown foo
-> foo downloaded

Thinking something like that? Not sure I like the API like that though. Perhaps it should be:

$ pulldown --new foo:http://foo.com/foo.js

We can then save that to the pulldown.json file?

require('pulldown')

Currently this ain't possible, I think becuase there's no main field in the package.json.

Depreciation warning

pulldown jquery:jquery.js

TypeError: Cannot call method 'split' of null
    at Pulldown.getFile (/usr/local/Cellar/node/0.10.10/lib/node_modules/pulldown/pulldown.js:140:40)
    at null.<anonymous> (/usr/local/Cellar/node/0.10.10/lib/node_modules/pulldown/pulldown.js:135:10)
    at Array.forEach (native)
    at Pulldown.downloadFiles (/usr/local/Cellar/node/0.10.10/lib/node_modules/pulldown/pulldown.js:134:8)

Would be nice to tidy that error up. "We now use :: as the separator" etc.

.zip.zip

โ€ฝ pulldown bootstrap backbone
->  Success: http://twitter.github.io/bootstrap/assets/bootstrap.zip was downloaded to bootstrap.zip.zip
->  Success: bootstrap.zip.zip was extracted to bootstrap
->  Success: https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min.js was downloaded to jquery.min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.4.4/underscore-min.js was downloaded to underscore-min.js
->  Success: https://cdnjs.cloudflare.com/ajax/libs/backbone.js/1.0.0/backbone-min.js was downloaded to backbone-min.js

This is all my fault.

https url failing

โ€ฝ pulldown https://raw.github.com/amdjs/backbone/master/backbone-min.js:backbone.min.js
->  Nothing found for https

Makes me think it's grabbing everything before the colon?

Can't download raw URLs

pulldown http://micah.cowan.name/projects/promptjobs/prompt-jobs.sh
-> http://micah.cowan.name/projects/promptjobs/prompt-jobs.sh downloaded to ./http://micah.cowan.name/projects/promptjobs/prompt-jobs.sh.js

This is a regression that obviously crept in during the move over to a CLI.

Also, it adds .js on the end which it shouldn't.

Project dependencies with the .pulldownrc file

Hello,
it would be fine if it is possible to change the name of destination file, something like this

{
    "destination": "vendors",
    "dependencies": [
        {url:"http://code.jquery.com/jquery.min.js",name:"jquery"}
    ]
}

pulldown runs very well under windows 7 :)

Document the API

Now that #52 is closed and it is possible to use Pulldown programatically (in fact, I think we should encourage it), we should document the API and how it work (wiki)?

Pulldown vs Bower

People keep asking me which to use. In my mind, Pulldown's use case is completely different from Bowers - and Pulldown's not designed to compete. I should write this up.

Output progress bars

Using the \r trick

process.stdout.write("Downloading " + data.length + " bytes\r");

Output directory not honoured by zips.

~/dev/tmp/bomb
โฏ pulldown bootstrap -o output
->  Success: http://twitter.github.io/bootstrap/assets/bootstrap.zip was downloaded to output/bootstrap.zip
->  Success: output/bootstrap.zip was extracted to bootstrap

~/dev/tmp/bomb
โฏ ls
bootstrap output

~/dev/tmp/bomb
โฏ tree
.
โ”œโ”€โ”€ bootstrap
โ”‚ย ย  โ””โ”€โ”€ bootstrap
โ”‚ย ย      โ”œโ”€โ”€ css
โ”‚ย ย      โ”‚ย ย  โ”œโ”€โ”€ bootstrap-responsive.css
โ”‚ย ย      โ”‚ย ย  โ”œโ”€โ”€ bootstrap-responsive.min.css
โ”‚ย ย      โ”‚ย ย  โ”œโ”€โ”€ bootstrap.css
โ”‚ย ย      โ”‚ย ย  โ””โ”€โ”€ bootstrap.min.css
โ”‚ย ย      โ”œโ”€โ”€ img
โ”‚ย ย      โ”‚ย ย  โ”œโ”€โ”€ glyphicons-halflings-white.png
โ”‚ย ย      โ”‚ย ย  โ””โ”€โ”€ glyphicons-halflings.png
โ”‚ย ย      โ””โ”€โ”€ js
โ”‚ย ย          โ”œโ”€โ”€ bootstrap.js
โ”‚ย ย          โ””โ”€โ”€ bootstrap.min.js
โ””โ”€โ”€ output
    โ””โ”€โ”€ bootstrap.zip

6 directories, 9 files

List sets

I'd like to enhance the sets we have in the API. Once that's done, it would be good to add a command to list them:

$ pulldown --list-sets
-> Backbone [backbone.js, jquery.js, underscore.js]
-> Ember [ember.js, jquery.js, handlebars.js]
-> and so on

(API up for discussion)

Make pulldown support URLs

Sometimes I want to quickly pull down a URL I know. Should be able to pass pulldown a URL and it should just download it.

Basically a glorified wget.

Write some tests

I'm a bit clueless here as to how to test this thing, but it definitely needs some.

I guess we should test that certain searches produce the network requests we expect?

Thoughts welcome.

Improve sets

Currently it's:

{
  "backbone" : "http://backbonejs.org/backbone-min.js",
  "underscore" : "http://underscorejs.org/underscore-min.js",
  "jquery": "http://code.jquery.com/jquery.min.js",
  "json2": "https://github.com/douglascrockford/JSON-js/raw/master/json2.js",
  "normalize": "https://raw.github.com/necolas/normalize.css/master/normalize.css",
  "raphael": "http://github.com/DmitryBaranovskiy/raphael/raw/master/raphael-min.js",
  "reset": "http://meyerweb.com/eric/tools/css/reset/reset.css",
  "bootstrap": "http://twitter.github.com/bootstrap/assets/bootstrap.zip",
  "sets": {
    "backboneapp": ["backbone", "jquery", "underscore"]
  }

Would be more natural if it was

{
  "backbone" : "http://backbonejs.org/backbone-min.js",
  "underscore" : "http://underscorejs.org/underscore-min.js",
  "jquery": "http://code.jquery.com/jquery.min.js",
  "json2": "https://github.com/douglascrockford/JSON-js/raw/master/json2.js",
  "normalize": "https://raw.github.com/necolas/normalize.css/master/normalize.css",
  "raphael": "http://github.com/DmitryBaranovskiy/raphael/raw/master/raphael-min.js",
  "reset": "http://meyerweb.com/eric/tools/css/reset/reset.css",
  "bootstrap": "http://twitter.github.com/bootstrap/assets/bootstrap.zip",
  "backboenapp": ["backbone", "jquery", "underscore"]
}

Destination folder for unzipping

First of all I want to say - this is awesome library! Thank you for making it :)

It would be very nice if when dealing with zip files it would take into account "output" value. I mean now zips are just unzipped in project folder no matter where "output" in .pulldownrc points. Seems that script tries to download zip file to path defined in "output", not unzip it there.

Sorry for my bad english.

Make it useful as a module

@PhUU I think you'll be interested in this.

Right now if you require in Pulldown, it's always going to download the files you pass it.

new Pulldown().init(["jquery", "backbone"], function() {
// urls downloaded
});

But what I think we want is a way to just get it to return the URLs it would have downloaded, right?

new Pulldown({ returnUrls: true }).init(["jquery", "backbone"], function(jquery, backbone) {
  console.log(jquery); //=> https://cdnjs.com/blah/blah/jquery.js
  console.log(backbone); //=> https://cdnjs.com/blah/blah/backbone.js
});

Right?

Dry run

I would like to be able to do a pulldown 'dry-run' where I try see what would happen, but not actually do it.

pulldown -d/--dry-run jquery

This would also enable a kind of search mode: "hey pulldown, do you have this?". Currently I search cdnjs before using pulldown for a new thing.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.