Code Monkey home page Code Monkey logo

cartodb-nodejs's Introduction

CARTO (formerly CartoDB) client for node.js

This library abstracts calls to CARTO's SQL, Import, and Maps APIs.

NB! It is not official and not maintained. The version in NPM is outdated and does not work. If you want to use a bit updated version, use one directly from GH repo as:

    "cartodb": "github:cartodb/cartodb-nodejs#master",

Install

npm install github:cartodb/cartodb-nodejs#master

Usage

Here's a simple example to make a SQL API call.

var CartoDB = require('cartodb');

var sql = new CartoDB.SQL({user:{USERNAME}})

sql.execute("SELECT * FROM mytable LIMIT 10")
  //you can listen for 'done' and 'error' promise events
  .done(function(data) {
    console.log(data) //data.rows is an array with an object for each row in your result set
  });

SQL Module

Methods

constructor

var sql = new CartoDB.SQL({
  user: {USERNAME},
  api_key: {APIKEY}
});

Note: api_key is only required for a write query.

SQL.execute

SQL.execute(sql [,vars][, options][, callback])

vars - An object of variables to be rendered with sql using Mustache

options - An object for options for the API call. Default is json.

{
  format: 'json'|'csv'|'geojson'|'shp'|'svg'|'kml'|'SpatiaLite'
}

More details about formats : CARTO SQL API docs

callback - A function that the data object will be passed to if the API call is successful.

This will return a promise. .done() and .error() can be chained after SQL.execute().

.done() first argument will be either : an object containing (when using json format) or a raw string (when using all other formats).

var sql = new CartoDB.SQL({user:{USERNAME});
sql.execute("SELECT * from {{table}} LIMIT 5")
  .done(function(data) {
    console.log(`Executed in ${data.time}, total rows: ${data.total_rows}`);
    console.log(data.rows[0].cartodb_id)
  })
  .error(function(error) {
    //error contains the error message
  });
var sql = new CartoDB.SQL({user:{USERNAME}});
sql.execute("SELECT * from {{table}} LIMIT 5", { format: 'csv' })
  .done(function(data) {
    console.log('raw CSV data :');
    console.log(data);
  });

Piping

You can pipe data from the SQL.execute()

var file = require('fs').createWriteStream(__dirname + '/output.json');

var sql = new CartoDB.SQL({user:{USERNAME}, api_key:{APIKEY}});

sql.execute("SELECT * from {{table}} LIMIT 5", {table: 'all_month'})


sql.pipe(file);

Import Module

Methods

file - Import a file from the filesystem - This method is the same as dragging a file (CSV,ZIP,XLS,KML) into the CartoDB GUI. The end result is a table in your account.

This method takes the path to your file and results in a table_name for the newly-created table.

Import.file(filePath, options)

.done() and .error() can be chained after Import.file().

url - Import a file from URL - This method is the same as specifying a publicly accessible url to import in the CartoDB GUI. The end result is a table in your account.

This method takes the URL to your file and results in a table_name for the newly-created table.

Import.url(url, options)

.done() and .error() can be chained after Import.url().

var importer = new CartoDB.Import({user:{USERNAME}, api_key:{APIKEY}});
var path = require('path');

importer
  .file(path.join(__dirname, 'all_week.csv'), {
    privacy: 'public'
  })
  .done(function(table_name) {
    console.log('Table ' + table_name + ' has been created!');
  });

Named Maps Module

Constructor

Pass in a config object with user and api_key

var namedMaps = new CartoDB.Maps.Named({
    user: 'username',
    api_key: 'yourreallylongcartodbapikey'
});

Methods

All methods create a promise, and you can listen for done and _error events.

Named.create() - Create a named map by providing a JSON template

Named Map Template:

{
  "version": "0.0.1",
  "name": "world_borders",
  "auth": {
    "method": "token",
    "valid_tokens": [
      "auth_token1",
      "auth_token2"
    ]
  },
  "placeholders": {
    "color": {
      "type": "css_color",
      "default": "red"
    },
    "cartodb_id": {
      "type": "number",
      "default": 1
    }
  },
  "layergroup": {
    "version": "1.0.1",
    "layers": [
      {
        "type": "cartodb",
        "options": {
          "cartocss_version": "2.1.1",
          "cartocss": "/** category visualization */ #world_borders { polygon-opacity: 1; line-color: #FFF; line-width: 0; line-opacity: 1; } #world_borders[cat=0] { polygon-fill: #A6CEE3; } #world_borders[cat=1] { polygon-fill: #1F78B4; } #world_borders[cat=2] { polygon-fill: #2167AB; } #world_borders[cat=3] { polygon-fill: #5077b5; }",
          "sql": "SELECT *, mod(cartodb_id,4) as cat FROM world_borders"
        }
      }
    ]
  },
  "view": {
    "zoom": 4,
    "center": {
      "lng": 0,
      "lat": 0
    },
    "bounds": {
      "west": -45,
      "south": -45,
      "east": 45,
      "north": 45
    }
  }
}
namedMaps.create({
   template: template
 })
   .on('done', function(res) {
    console.log(res)
  })

Response:

{ template_id: 'world_borders' }

Named.instantiate() - Instantiate a named map to get a layergroupid, passing an options object with the template_id, auth_token (if required), and placeholder params (if needed by your named map template)

namedMaps.instantiate({
  template_id: 'world_borders',
  auth_token: 'auth_token1',
  params: {
    color: '#ff0000',
    cartodb_id: 3
  }
})
  .on('done', function(res) {
    console.log(res)
  })

Response:

{ layergroupid: 'chriswhong@72f19e2f@28aa9b31a7147f1d370f0f6322e16de6:1453321153152',
  metadata: { layers: [ [Object] ] },
  cdn_url:
   { http: 'ashbu.cartocdn.com',
     https: 'cartocdn-ashbu.global.ssl.fastly.net' },
  last_updated: '2016-01-20T20:19:13.152Z' }

Named.update() - Update a Named Map template

namedMaps.update({
  template: template
})
  .on('done', function(res) {
    console.log(res)
  })

Response:

{ template_id: 'world_borders' }

Named.delete() - Delete a named map - pass it an options object with template_id

namedMaps.delete({
  template_id: 'world_borders'
})
  .on('done', function(template_id) {
    console.log(template_id);
  })

Response is the template_id that was just deleted:

world_borders

Named.list() - Get a list of all named maps in your account

namedMaps.list()
  .on('done', function(res) {
    console.log(res);
  });

Named.definition() - Get the current template for a named map

namedMaps.definition({
  template_id: 'world_borders'
})
  .on('done', function(res) {
    console.log(res);
  });

Command-line access

SQL Module: cartodb / cartodb-sql

Options

  -s, --sql string       A SQL query (required).
  -u, --user string      Your CartoDB username.
  -a, --api_key string   Your CartoDB API Key (only needed for write operations).
  -f, --format string    Output format json|csv|geojson|shp|svg|kml|SpatiaLite
  -o, --output string    Output file. If omitted will use stdout.
  -c, --config string    Config file. Use a JSON file as a way to input these arguments. If no username nor config file is provided, it will look for "config.json" by default.
  --sqlFile string       A file containing a SQL query (you can then omit -s).
  -h, --help

Examples

$ cartodb -u nerik -f svg 'SELECT * FROM europe' > europe.svg
$ cartodb -u nerik -f csv 'SELECT cartodb_id, admin, adm_code FROM europe LIMIT 5' -o europe.csv
$ cartodb -c config.json 'UPDATE ...' #hide your api_key in this file !
$ cartodb 'UPDATE ...' # "config.json" will be used for credentials by default

Import Module: cartodb-import

Options

  -f, --file string      Path to a local file to import.
  -l, --url string       URL to import.
  -p, --privacy string   Privacy of the generated table (public|private)
  -u, --user string      Your CartoDB username
  -a, --api_key string   Your CartoDB API Key (only needed for write operations)
  -c, --config string    Config file. Use a JSON file as a way to input these arguments.
  -h, --help

Examples

$ cartodb-import -u nerik --api_key XXX test.csv
$ cartodb-import -c config.json test.csv
$ cartodb-import test.csv # "config.json" will be used for credentials by default
$ cartodb-import --url http://sig.pilote41.fr/urbanisme/aleas_inondation/aleas_sauldre_shp.zip

cartodb-nodejs's People

Contributors

cayetanobv avatar danicarrion avatar danpaz avatar impronunciable avatar jaakla avatar javierarce avatar javisantana avatar jhkennedy4 avatar nerik avatar nmccready avatar patrickjs avatar tokumine avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cartodb-nodejs's Issues

cartodb_url param on CartoDB instantiation

Since I host CartoDB myself I needed to allow for an additional parameter cartodb_url
and change resources.json to reflect the change.

Creating a client can now be run with:

var client = new CartoDB({
  user: secret.USER, 
  api_key: secret.API_KEY,
  /* e.g. http://myusername.cartodb.com */
  cartodb_url: ''
});

master...busla:master

You want a PR?

Error choosing the request method

File: lib/cartodb.js
Line: 119

if(sql.length > 2048)
this.oa.get(.....

I think the correct way it'd be:
if(sql.length < 2048)
this.oa.get(.....

debug msg crashes express

In sql L96
debug('There was an error with the request %s', body.error);

There are cases where we are trying to make a request without internet service and the body message does not have a key error and so it throws a hard error and crashes the node program running. Bter to do a if (body && body.error) to keep this from happening.

On-premise endpoint: UNABLE_TO_VERIFY_LEAF_SIGNATURE

We are tryingto use Busla's forked version to query our on-premise CartoDB instance at https://cartodb.brighterdevelopment.com

We tried a very simple test:

'use strict';
var CartoDB = require('cartodb');
exports.init = function(req, res){
var client = new CartoDB({user:"demo-admin",cartodb_url:"https://cartodb.brighterdevelopment.com", api_key:"OUR_API_KEY_GOES_HERE"});
var outputRows = function(err, data) {
// do something
};
client.on('connect', function() {
client
.query("select * from {table} limit 5", {table: 'ne_10m_populated_places_simple'}, outputRows)
});
client.connect();
};

NOTE: we also tested with the alternative parameter, "api_url", via the Master branch of cartodb-nodejs

We receive this error:

Error: UNABLE_TO_VERIFY_LEAF_SIGNATURE
at SecurePair. (tls.js:1367:32)
at SecurePair.emit (events.js:92:17)
at SecurePair.maybeInitFinished (tls.js:979:10)
at CleartextStream.read as _read
at CleartextStream.Readable.read (_stream_readable.js:340:10)
at EncryptedStream.write as _write
at doWrite (_stream_writable.js:225:10)
at writeOrBuffer (_stream_writable.js:215:5)
at EncryptedStream.Writable.write (_stream_writable.js:182:11)
at write (_stream_readable.js:601:24)

Any help would be HUGELY appreciated.
Our goals is to query our on premise-instance via https.
We are open to any / all approaches.

Thank you!

make it easier

I wanna something like this:

client = CartoDB.connect(credentials);
client.on('connect', function() {
client.sql("select * from blabla");
})

client.on('data', function(results) {
//do something with results
});

client.on('error', fucntion() ...);

Accessing formats

It's easy enough to use the SQL API to do something like user.cartodb.com/api/v2/sql?format=SHP&q=SELECT * FROM tablename

Being able to choose which format you'd like to view your data in is fantastic. How do I do that using this library?

Currently, SELECT * FROM tablename is giving me nice JSON output, but I'd also like the ability to download shapefiles when necessary. Is that possible?

Example provided in README is inaccurate

Calling client.connect() before client.on('connect', func... will not work. Only figured this out after mucking around in the examples where connect is always called last. Improving the quality of the first impression makes it easier for people to start using the library.

README example correction

README.md lines 40-43 says:

client.on('data', function(data) {
    console.log(results.rows);
});

And results are undefined. It should be:

client.on('data', function(data) {
    console.log(data.rows);
});

Cheers!

Favor options object instead of arguments list in method signatures

Instead of :

SQL.execute(sql [,vars][, options][, callback])

Use :

SQL.execute(sql, options);
SQL.execute(sql, {
   vars: vars
   callback: callback
})

Pros :

  • doesn't force the user to specify all args if they only need the last one
  • changes/adds to the API are a no brainer
  • easy set up of defaults in the beginning of the function

411 Length Required

I am getting when attempting to do an insert. Any ideas?

  cartodb About to perform a POST query to https://rossc1.cartodb.com/api/v1/sql +0ms
  cartodb There was an error with the request <html>
      <head><title>411 Length Required</title></head>
      <body bgcolor="white">
      <center><h1>411 Length Required</h1></center>
      <hr><center>nginx</center>
      </body>
    </html>
  +194ms

Error if existing cartodbClient instance is reused by multiple requests

I don't know if this is truly an error, but if a single cartodbClient instance is reused among multiple simultaneous requests, the following error occurs:

_http_outgoing.js:335
throw new Error('Can't set headers after they are sent.');
^

so I can't do this:

var cartodbClient = new CartoDB({user: config.cartodb.USER, api_key: config.cartodb.API_KEY});

app.route('/myroute')
    .post(function(req, res) {
        cartodbClient.on('connect',function()....

but creating a new instance in the post handler works fine-

Template does not support multiple values

The tmpl function should support multiple values e.g.

UPDATE {table} SET {field}= {val} WHERE ({id}>=32 AND {id}<=42) OR {id} IN (26, 28, 66)

Currently only the first {id} is replaced.

This may not be the most efficient way, but you could rewrite:

s = s.replace('{'+p+'}', d[p]);

as

s = s.split('{'+p+'}').join(d[p]);

'new CartoDB' object error

I was looking at the example in the readme and when I run the "var client = new CartoDB({..." , I receive an error saying 'new' object is not a function. Any thoughts?

Having Trouble with returning data in the callback.

The callback function never seems to fire/data never get's returned to the callback. Event handling works fine. I'll try to put together some examples over my break.

Here's an example of how I'm handling my queries without using a callback:

    var id = req.params.id;
    client.query("select * from monroecountysnap where cartodb_id = " + id);
    client.on('error', function (error) {
        console.log("id fail");
    });
    client.on('data', function (data) {
        res.send(data);
    });

I'm not sure if this is deliberate or a bug. If the former, then the documentation should reflect the proper usage. I think we need a few more in depth examples of how to use the library. I'm happy to share my code, I'm just not sure how to share it without exposing my username and password for CartoDB while still allowing other developers access to the data. (Which should probably be it's own separate issue, should I just create a generic account that I don't mind sharing with the world?)

HTTPS

I'm using the nodejs client, but not sure if this issue belongs to here or https://github.com/CartoDB/CartoDB-SQL-API.

When loading my site over HTTPS (eg. deployed on heroku), I am getting message warning me that the SQL request is sent over HTTP. Is there a way to configure so that all requests are sent over HTTPS?


Mixed Content: The page was loaded over HTTPS, but requested an insecure script
'http://jue.cartodb.com/api/v1/map?stat_tag=API&config=%7B%22version%22%3A%2…
_thumbnail%22%2C%22team_name%22%5D%7D%7D%5D%7D&callback=_cdbc_1212937049_1'. 
This request has been blocked; the content must be served over HTTPS.

resources domain needs to be updated

As of today I believe the cartodb.com has been depracated and no longer works to make request and someone will need to update the cartodb.com/api/ to be carto.com/api/ in the /lib/resources.json file and then push the build to npm to allow people to upgrade and patch the problem. This is a BREAKING issue and a hot fix should be released.

Import API Streaming [feature Request]

Why is the import api limited to a fileName and or URL. Why can we not just pass a stream of raw data as an option? I would love this to exist so I can get rid of my own implementation.

Remove Makefile

Adding a shell command in the test section in the package.json is sufficient.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.