Code Monkey home page Code Monkey logo

arangojs's Introduction

Two stylized avocado halves and the product name.

ArangoDB

ArangoDB is a scalable graph database system to drive value from connected data, faster. Native graphs, an integrated search engine, and JSON support, via a single query language. ArangoDB runs on-prem, in the cloud – anywhere.

ArangoDB Cloud Service

The ArangoGraph Insights Platform is the simplest way to run ArangoDB. You can create deployments on all major cloud providers in many regions with ease.

Getting Started

For the impatient:

  • Test ArangoDB in the cloud with ArangoGraph for free.

  • Alternatively, download and install ArangoDB. Start the server arangod if the installer did not do it for you.

    Or start ArangoDB in a Docker container:

    docker run -e ARANGO_ROOT_PASSWORD=test123 -p 8529:8529 -d arangodb
    

    Then point your browser to http://127.0.0.1:8529/.

Key Features of ArangoDB

Native Graph - Store both data and relationships, for faster queries even with multiple levels of joins and deeper insights that simply aren't possible with traditional relational and document database systems.

Document Store - Every node in your graph is a JSON document: flexible, extensible, and easily imported from your existing document database.

ArangoSearch - Natively integrated cross-platform indexing, text-search and ranking engine for information retrieval, optimized for speed and memory.

ArangoDB is available in a free and open-source Community Edition, as well as a commercial Enterprise Edition with additional features.

Community Edition features

  • Horizontal scalability: Seamlessly shard your data across multiple machines.
  • High availability and resilience: Replicate data to multiple cluster nodes, with automatic failover.
  • Flexible data modeling: Model your data as combination of key-value pairs, documents, and graphs as you see fit for your application.
  • Work schema-free or use schema validation for data consistency. Store any type of data - date/time, geo-spatial, text, nested.
  • Powerful query language (AQL) to retrieve and modify data - from simple CRUD operations, over complex filters and aggregations, all the way to joins, graphs, and ranked full-text search.
  • Transactions: Run queries on multiple documents or collections with optional transactional consistency and isolation.
  • Data-centric microservices: Unify your data storage logic, reduce network overhead, and secure sensitive data with the ArangoDB Foxx JavaScript framework.
  • Fast access to your data: Fine-tune your queries with a variety of index types for optimal performance. ArangoDB is written in C++ and can handle even very large datasets efficiently.
  • Easy to use web interface and command-line tools for interaction with the server.

Enterprise Edition features

Focus on solving enterprise-scale problems for mission critical workloads using secure graph data. The Enterprise Edition has all the features of the Community Edition and offers additional features for performance, compliance, and security, as well as further query capabilities.

  • Smartly shard and replicate graphs and datasets with features like EnterpriseGraphs, SmartGraphs, and SmartJoins for lightning fast query execution.
  • Combine the performance of a single server with the resilience of a cluster setup using OneShard deployments.
  • Increase fault tolerance with Datacenter-to-Datacenter Replication and and create incremental Hot Backups without downtime.
  • Enable highly secure work with Encryption 360, enhanced Data Masking, and detailed Auditing.
  • Perform parallel graph traversals.
  • Use ArangoSearch search highlighting and nested search for advanced information retrieval.

Latest Release

Packages for all supported platforms can be downloaded from https://www.arangodb.com/download/.

For what's new in ArangoDB, see the Release Notes in the Documentation.

Stay in Contact

arangojs's People

Contributors

13abylon avatar brabes avatar casdevs avatar ciscoheat avatar ctmakro avatar ctoesca avatar dependabot[bot] avatar dontrelax avatar dothebart avatar fceller avatar graetzer avatar greenkeeperio-bot avatar janwo avatar jsteemann avatar kvs85 avatar lodestone avatar maxkernbach avatar mpv1989 avatar neunhoef avatar oknoah avatar petitchevalroux avatar pluma avatar pluma4345 avatar samrg472 avatar simran-b avatar sleto-it avatar tombarton avatar uatuko avatar xescugc avatar ziink avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

arangojs's Issues

Query is empty

Hi,

I updated the new version 3.7, but I found my code doesn't work any more. After deep inspect, I found the new utils/request can get the correct request body like the following:

Request Body: {"query":"LET nodes = (FOR node IN @@collection RETURN node), includes = (FOR node IN @@collection RETURN node) RETURN {nodes: nodes, includes: includes}","bindVars":{"@collection":"resources"}}

However the arangoDB returns an error, query is empty... I have no idea what's the reason.

{ [ArangoError: query is empty]\n  name: \'ArangoError\',\n  message: \'query is empty\',\n  errorNum: 1502,\n  code: 400,\n  stack: \'ArangoError: query is empty\\n    at new ArangoError (/Invend/nfserver/node_modules/arangojs/lib/error.js:10:13)\\n    at /Invend/nfserver/node_modules/arangojs/lib/connection.js:85:71\\n    at /Invend/nfserver/node_modules/arangojs/lib/util/once.js:7:15\\n    at IncomingMessage.<anonymous> (/Invend/nfserver/node_modules/arangojs/lib/util/request.js:23:9)\\n    at IncomingMessage.emit (events.js:129:20)\\n    at _stream_readable.js:908:16\\n    at process._tickDomainCallback (node.js:381:11)\',\n  isException: true,\n  status: 400 }

Meanwhile, I found the test case for query with Aqb module is missing. Maybe that's the reason.

Best regards,
Kevin

Cursor with undefined result

Hi,

I made some queries to get documents based on AQL. Some of them will return no error and a cursor like this:

{ extra: undefined,
  _connection: 
   { config: 
      { url: 'http://127.0.0.1:8529',
        databaseName: 'n-fuse',
        arangoVersion: 20300,
        agentOptions: [Object],
        headers: [Object] },
     _request: [Function: request] },
  _api: 
   { _connection: { config: [Object], _request: [Function: request] },
     _path: '_api',
     _headers: undefined },
  _result: undefined,
  _hasMore: false,
  _id: undefined,
  _index: 0 }

First of all, I'm sure the collection and relevant document exists, but no document is found.
Second, even the document doesn't exist, the cursor should not have an undefined result, shouldn't it?
Last but not least, I call cursor.hasNext() to check whether I can advance the cursor or not. My code then hangs on at this step, no error, no returned value, and no exception. I believe it's the reason of undefined result. Even if I put another try/catch block here, I still can not see any exception.

p.s. this case does not occur each time...

Best regards,
Kevin

https performance

The driver performance over HTTPS is very poor.
The same query roundtrips are 4-5 times(!) faster with HTTP. In my simple tests the average query RTT was 41.6ms with HTTP, and 196.5ms with HTTPS on a remote server.
Is there any extra option to pass to the driver?
My environment:
arangoDB 2.6.0 (stable) and 2.5.2
iojs 2.3.1
arangojs 3.8.1

The performance of the web GUI is as fast with HTTPS than with HTTP.

Bulk Import does not support JSON array document

Hi,

I found collection.import function does not support JSON array data. I think that's the reason of type parameter in the POST request

That's the error returned:

Error: no JSON array found in second line
      at new ArangoError (/Invend/server_arango/node_modules/arangojs/lib/error.js:12:13)
      at Request._callback (/Invend/server_arango/node_modules/arangojs/lib/connection.js:65:25)
      at Request.self.callback (/Invend/server_arango/node_modules/request/request.js:373:22)
      at Request.EventEmitter.emit (events.js:110:17)
      at Request.<anonymous> (/Invend/server_arango/node_modules/request/request.js:1318:14)
      at Request.EventEmitter.emit (events.js:129:20)
      at IncomingMessage.<anonymous> (/Invend/server_arango/node_modules/request/request.js:1266:12)
      at IncomingMessage.EventEmitter.emit (events.js:129:20)
      at _stream_readable.js:898:16
      at process._tickCallback (node.js:343:11)

According to the documentation https://docs.arangodb.com/HttpBulkImports/ImportingSelfContained.html , the function should has something like the following for enabling parse of two types of JSON documents on the server side

https://github.com/arangodb/arangojs/blob/master/lib/collection.js#L168

type: 'auto'

sort not working when using bindvars

Hey

I seem to not be able to pick up the sort function when using a AQL with a @sort binding

Here's a most basic example

db.query("FOR f IN films SORT @sort LIMIT 3 RETURN f.title", {sort: 'f.title asc'}, callback)

returns
[ 'Test Film 127933', 'Test Film 123403', 'Test Film 31017' ]

but

db.query("FOR f IN films SORT f.title asc LIMIT 3 RETURN f.title", callback)
returns
[ 'Test Film 1', 'Test Film 10', 'Test Film 100' ]

Is there something I'm doing wrong?
Turns out there is - key on the property e.g.

db.query("FOR f IN films SORT f.@sort @order LIMIT 3 RETURN f.title", {sort: title', order: 'asc'}, callback)

Bulk Import bug

Hi,

type of import function should be defaults to "auto". However, if I do not give opts argument, and then import an array of document, I will become such error:

[ArangoError: no JSON string array found in first line]

I believe it's still a bug of request parameter. Please fix.

Bg,
Kevin

documentCollection.save not returning full document

The documentation for save() implies that the callback gets called with the full document (doc.some === 'data' in the documentation's example). However, I only get back the new meta attributes (when using promises, if that makes any difference).

{
  error: false,
  _id: 'whatever/1645465998905',
  _rev: '1645465998905',
  _key: '1645465998905'
}

How to get connection status ?

How can one get the connection and/or auth status (disconnected, connected, authentication failed etc.) with this driver ?

Options of Query API

Hi,

The Query API uses /_api/cursor for executing AQL. However, this interface should support some additional options, such as fullCount which might be useful for me.

Is it possible to extend the arangojs for supporting these options? or how can I count my result before limit clause is applied in the AQL?

Bg,
Kevin

Errors in callbacks shouldn't be caught

db.collection('foo', function(error, collection) {
  if (error) return console.error(error);
  programmer.mistake();
  console.log(collection);
});

Where's my error?

Avoid unnecessary roundtrips

I've been wondering for a while why collection() is an asynchronous method. It seems that it actually makes a roundtrip to the DB. But why?

db.collection('MyThings')
  .then(collection => collection.count())

So to get the number of MyThings it makes two roundtrips even though, theoretically, it should be just one? Or if I want to insert one document each into 10 different collections, I'd end up with 20 roundtrips?

Graph EdgeCollection edge add not working

Looks like the API has been changed in ArangoDB, or something else is wrong. This should fix it:

--- graph.js.orig   2015-02-21 12:08:57.587976294 +0200
+++ graph.js    2015-02-21 12:10:41.895531469 +0200
@@ -127,11 +127,9 @@
   },
   save: function (data, fromId, toId, callback) {
     if (!callback) callback = noop;
-    this._gharial.post(data, {
-      collection: this.name,
-      from: this._documentHandle(fromId),
-      to: this._documentHandle(toId)
-    }, function (err, body) {
+    data['_from'] = this._documentHandle(fromId);
+    data['_to'] = this._documentHandle(toId);
+    this._gharial.post(data, function (err, body) {
       if (err) callback(err);
       else callback(null, body);
     });

There's probably other graph edge functions that are not working, but I haven't checked those.

"unknown path" responses from server on every API call

I'm just starting with arangodb and arangojs, so maybe I'm making a basic mistake, but arangojs is returning "unknown path" for every API call for me. Manual http requests from iojs to the server work fine.

I made an attempt at debugging, and found that the use of path.join in request.js was changing the slashes in the path to backslashes. Changing them back to slashes fixed the problem.

I'm running arangodb 2.6 with a fresh database and stock settings. I'm running arangojs 3.8.1 on iojs 1.8.1, and using the default Database constructor.

Database `autoCreate` not works with 2.6.0

Hi,

After update to v2.6.0, I found the database connection with autoCreate options enabled does not work any more. I made some further inspect. The log told me

In this line, it gets an error with 404, and errorNum 1228.
https://github.com/arangodb/arangojs/blob/master/lib/database.js#L271
Then the code try to create a new database. However, no error is returned and no database is created.
I guess it might be the reason of default username and password for new arangodb. Nevertheless, some errors should be returned here about that at least.

Thx

Bg,
Kevin

https problem?

I'm trying to connect a remote arangodb from iojs. With plain http everything works properly. After switch the endpoint to ssl the driver fails to connect with a message ECONNRESET.
Handmade API calls with builtin https module works, after added certificates with:
require('ssl-root-cas').inject().addFile('.ssl/sub.class1.server.ca.pem');. The WEB GUI and the browserify bundle are working too.
Arango 2.6a3, arangojs 3.8, iojs 2.0.1, 2.2.1, certificate: StartSSL free cert (green padlock in all tested browsers)
I've to configure something, or is this a bug?

db.collection error handling

I'm using ArangoDB 2.6-alpha3, over HTTPS & HTTP authentication turned on with arangojs 3.8.1 driver.
When I invoke the above function with wrong or no pw, the callback called with error = null. When i use promises the success handler will be invoked. However the response has statusCode = 401

db.collection('collectionname',function(err,resp){
console.log('error: ',e);
console.log('response: ',r)
});

I've not checked more API calls for this bug.

collection.import opts argument is not optional as informed in the documentation

When calling collection.import(data, [opts,] callback) the opts argument is not being treated as optional as informed in the documentation.

For example, if you call import like:

collection.import([{name: 'abc'}], function(err){});

The following error is received on the callback:

[TypeError: Cannot read property 'type' of undefined]

My understanding is that the attribute opts.type is being expected in the implementation even when the opts is not being passed (see collection.js:189) :

ld: Boolean(opts.type !== 'array')

not able to get collection

I am using nodejs with Arangodb (databasename="nodetest1" and Collection="User")

var Database = require('arangojs');
var db = Database('http://127.0.0.1:8529');
//var mydb = db.use("nodetest1");
 db.query("FOR x IN User RETURN x", 
  function(err, cursor) {
  if (err) {
    console.log('error 1: %j', err);
  } else {
    cursor.all(function(err2, list) {
      if (err) {
        console.log('error 2: %j', err2);
      } else {
        console.log(list);
      }
    });
  }
});

getting error: [Circular]

Any help on this problem would be greatly appreciated!
Thank you in advance!

autoCreate collection sometimes fails

Hi,

I used database.collection function with autoCreate flag is true. Sometimes, I will get no error and a collection like this:

{ _connection: 
   { config: 
      { url: 'http://127.0.0.1:8529',
        databaseName: 'n-fuse',
        arangoVersion: 20300,
        agentOptions: [Object],
        headers: [Object] },
     _request: [Function: request] },
  _api: 
   { _connection: { config: [Object], _request: [Function: request] },
     _path: '_api',
     _headers: undefined } }

If I use this collection to call collection.import, I'll then got an error

[ArangoError: 'collection' is missing, expecting /_api/import?collection=<identifier>]
  message: '\'collection\' is missing, expecting /_api/import?collection=<identifier>',
  errorNum: 1204,
  code: 400,
  stack: 'ArangoError: \'collection\' is missing, expecting /_api/import?collection=<identifier>\n    at new ArangoError (/Invend/nfserver/node_modules/arangojs/lib/error.js:10:13)\n    at /Invend/nfserver/node_modules/arangojs/lib/connection.js:90:71\n    at /Invend/nfserver/node_modules/arangojs/lib/util/once.js:7:15\n    at IncomingMessage.<anonymous> (/Invend/nfserver/node_modules/arangojs/lib/util/request.js:23:9)\n    at IncomingMessage.emit (events.js:129:20)\n    at _stream_readable.js:908:16\n    at process._tickDomainCallback (node.js:381:11)']

I believe generally the returned collection should like this:

{ _connection: 
   { config: 
      { url: 'http://127.0.0.1:8529',
        databaseName: 'n-fuse',
        arangoVersion: 20300,
        agentOptions: [Object],
        headers: [Object] },
     _request: [Function: request] },
  _api: 
   { _connection: { config: [Object], _request: [Function: request] },
     _path: '_api',
     _headers: undefined },
  id: '2903076799',
  name: 'organizations',
  waitForSync: false,
  isVolatile: false,
  isSystem: false,
  status: 3,
  type: 2 }

I'm not sure whether it's an issue of arangojs or arangoDB server. Please check with this case.

Best regards,
Kevin

Droping a Graph

I have found an error when trying to drop a graph. The code is the following:

  db.dropGraph(graphName)

The stack of the error is the following:

Unhandled rejection ArangoError: TypeError: Cannot read property '_properties' of undefined
  at new ArangoError (/path/to/project/node_modules/arangojs/lib/error.js:10:13)
  at Request._callback (/path/to/project/node_modules/arangojs/lib/connection.js:78:81)
  at Request.self.callback (/path/to/project/node_modules/arangojs/node_modules/request/request.js:368:22)
  at Request.emit (events.js:98:17)
  at Request.<anonymous> (/path/to/project/node_modules/arangojs/node_modules/request/request.js:1219:14)
  at Request.emit (events.js:117:20)
  at IncomingMessage.<anonymous> (/path/to/project/node_modules/arangojs/node_modules/request/request.js:1167:12)
  at IncomingMessage.emit (events.js:117:20)
  at _stream_readable.js:943:16
  at process._tickCallback (node.js:419:13)

The error behaves as following:

  • The error is raised even if the collection exists or not
  • Even though the error the Graph is droped

node.js + foxx session feature

Hi Guys,

I am currently developing an authentication and session scheme for hapi.js with foxx. To identify the request in foxx I have to send the session cookies with every request (of course). My question is how to obtain the session id in first place using the arangojs driver, since node.js wont keep automatically track on the cookies like the browser does. I was looking at the source code and couldn't find any way to access the response headers. So my only solution would be to make an own request at first to store the session id and then use the arangojs driver, but i dont want to build the url on my own. How is it supposed to work?

Thanks in advance!

`autoCreate` option for db.query function

Hi,

Maybe it's only a suggestion. Is it possible to implement some options like autoCreate in db.collection function for db.query function as well. If I use db.query to make some modification query, such as insert, arangoDB instance will always complain about collection not existing.

Bg,
Kevin

API merge with arangosh's JS scripts?

I'm thrilled that a new JS client is being developed for ArangoDB (not too happy with the current client's API, although it has the merit of existing).

I had a very interesting conversation with Lucas on the Google group about the different JS APIs being used with Arango, and most notably about the opportunity/possibility to merge this new development with the JS part of arangosh. I believe that, if possible, we should not design another API here. Arangosh's API is fine and it would be very nice to have the same on this client.

Now I know that the main difference between these two is arangosh being synchronous vs. the JS client being asynchronous. I didn't have much time to dig into arangosh's scripts yet, so I'm still wondering if there's a way to have the methods returning the result in arangosh and a corresponding promise in this client. The connection object is probably the key here, with each method building the HTTP request and the connection processing it either synchronously or asynchronously.

I would be happy to contribute to this new client, at least from a API design perspective and at best with code. But the technical considerations about the feasibility of merging with arangosh would best be managed by someone from the team!

Cheers
Thomas

Request headers for remote connection

I'm trying to a simple test call to look for all collections, with the arangodb being in another server:

config = {
    db: {
        url: 'http://test:[email protected]:8529'
    }
};

Database = require('arangojs');

var db = new Database(config.db);

db.collections(function (err, collections) {
    if (err) {
        return console.error(err);
    }
    console.log(collections);
});

But I keep getting an unknown path error.

'{"error":true,"code":404,"errorNum":404,"errorMessage":"unknown path \'\\\\_db\\\\_system\\\\_api\\\\collection?excludeSystem=true\'"}' } }

I have no trouble accessing the web interface, so I believe it's a headers issue, but I haven't found a workaround.

**Update: I tried installing arangodb in my local machine (windows 8) and I get the same result.

Handle non-API errors (auth, net, etc)

Right now, if the authentication for the internal request fails, collection.document(id, cb) calls the callback without error and a blank string as data for the document. And cursor.all(cb) throws an exception:

foo/node_modules/arangojs/lib/cursor.js:48
      self._index = self._result.length;
                                ^
TypeError: Cannot read property 'length' of undefined
    at foo/node_modules/arangojs/lib/cursor.js:48:33
    at foo/node_modules/arangojs/lib/cursor.js:25:32
    at ArrayCursor.extend._more (foo/node_modules/arangojs/lib/cursor.js:32:25)
    at ArrayCursor.extend._drain (foo/node_modules/arangojs/lib/cursor.js:23:10)
    at ArrayCursor.extend.all (foo/node_modules/arangojs/lib/cursor.js:47:10)

support Foxx apps / arbitrary endpoints

Maybe something generic like:

db.endpoint(path)
  .get([path,] [qs,] cb)
  .put([path,] [data, [qs,]] cb)
  .patch([path,] [data, [qs,]] cb)
  .post([path,] [data, [qs,]] cb)
  .delete([path,] [qs,] cb)

Error: connect EADDRNOTAVAIL - maxsockets being ignored?

Testing performance of saving docs and having the following error thrown every few thousand writes:

{ [Error: connect EADDRNOTAVAIL]
  code: 'EADDRNOTAVAIL',
  errno: 'EADDRNOTAVAIL',
  syscall: 'connect',
  request: 
   { domain: null,
     _events: { response: [Object], error: [Function] },
     _maxListeners: 10,
     output: [],
     outputEncodings: [],
     writable: true,
     _last: false,
     chunkedEncoding: false,
     shouldKeepAlive: true,
     useChunkedEncodingByDefault: true,
     sendDate: false,
     _headerSent: true,
     _header: 'POST /_db/loadtest/_api/document/?collection=load HTTP/1.1\r\ncontent-type: application/json\r\ncontent-length: 169\r\nx-arango-version: 20300\r\nHost: localhost:8000\r\nConnection: keep-alive\r\n\r\n',
     _hasBody: true,
     _trailer: '',
     finished: true,
     _hangupClose: false,
     socket: 
      { _connecting: false,
        _handle: null,
        _readableState: [Object],
        readable: false,
        domain: null,
        _events: [Object],
        _maxListeners: 10,
        _writableState: [Object],
        writable: false,
        allowHalfOpen: false,
        onend: [Function: socketOnEnd],
        destroyed: true,
        bytesRead: 0,
        _bytesDispatched: 0,
        _pendingData: 'POST /_db/loadtest/_api/document/?collection=load HTTP/1.1\r\ncontent-type: application/json\r\ncontent-length: 169\r\nx-arango-version: 20300\r\nHost: localhost:8000\r\nConnection: keep-alive\r\n\r\n{"id":16313,"type":9,"connections":[100,59,26,58],"positions":[{"time":"2015-06-20T10:49:32.749Z","type":9,"position":{"lat":-122.423246,"lng":37.779388,"radius":250}}]}',
        _pendingEncoding: 'utf8',
        parser: [Object],
        _httpMessage: [Circular],
        ondata: [Function: socketOnData],
        _idleNext: null,
        _idlePrev: null,
        _idleTimeout: -1 },
     connection: 
      { _connecting: false,
        _handle: null,
        _readableState: [Object],
        readable: false,
        domain: null,
        _events: [Object],
        _maxListeners: 10,
        _writableState: [Object],
        writable: false,
        allowHalfOpen: false,
        onend: [Function: socketOnEnd],
        destroyed: true,
        bytesRead: 0,
        _bytesDispatched: 0,
        _pendingData: 'POST /_db/loadtest/_api/document/?collection=load HTTP/1.1\r\ncontent-type: application/json\r\ncontent-length: 169\r\nx-arango-version: 20300\r\nHost: localhost:8000\r\nConnection: keep-alive\r\n\r\n{"id":16313,"type":9,"connections":[100,59,26,58],"positions":[{"time":"2015-06-20T10:49:32.749Z","type":9,"position":{"lat":-122.423246,"lng":37.779388,"radius":250}}]}',
        _pendingEncoding: 'utf8',
        parser: [Object],
        _httpMessage: [Circular],
        ondata: [Function: socketOnData],
        _idleNext: null,
        _idlePrev: null,
        _idleTimeout: -1 },
     agent: 
      { domain: null,
        _events: [Object],
        _maxListeners: 10,
        options: [Object],
        requests: {},
        sockets: [Object],
        maxSockets: 5,
        createConnection: [Function] },
     socketPath: undefined,
     method: 'POST',
     path: '/_db/loadtest/_api/document/?collection=load',
     _headers: 
      { 'content-type': 'application/json',
        'content-length': 169,
        'x-arango-version': 20300,
        host: 'localhost:8000' },
     _headerNames: 
      { 'content-type': 'content-type',
        'content-length': 'content-length',
        'x-arango-version': 'x-arango-version',
        host: 'Host' },
     parser: 
      { _headers: [],
        _url: '',
        onHeaders: [Function: parserOnHeaders],
        onHeadersComplete: [Function: parserOnHeadersComplete],
        onBody: [Function: parserOnBody],
        onMessageComplete: [Function: parserOnMessageComplete],
        socket: [Object],
        incoming: null,
        maxHeaderPairs: 2000,
        onIncoming: [Function: parserOnIncomingClient] } } }

I believe it has something to do with no more available sockets on my machine - but thought that {maxSockets: 5} would control that? I have changed maxSockets from 1 to 25 and keep having the same result. I see keepalive is enabled by default so assume new connections are not being fired up?
Below is the script I am using to test with:

var moment = require('moment'),
    opts = {maxSockets: 5},
    Agent = require('http').Agent;

var db = require('arangojs')({
    url: 'http://localhost:8000',
    databaseName: 'loadtest',
    agent: new Agent(opts)
});

var recs = 1000000,
    currentDoc = 0,
    type,
    start = new Date(),
    collection;

function save() {
    type = Math.round(Math.random() * 10) + 1;
    collection
        .save({
            'id': currentDoc,
            'type': type,
            'connections': [
                Math.round(Math.random() * 100) + 1,
                Math.round(Math.random() * 100) + 1,
                Math.round(Math.random() * 100) + 1,
                Math.round(Math.random() * 100) + 1
            ],
            'positions': [
                {
                    'time': new Date(),
                    'type': type,
                    'position': {'lat': -122.423246, 'lng': 37.779388, 'radius': 250}
                }
            ]
        },
        function (err, result) {
            if (err || result.error) {
                return console.error(err);
            }
            console.log('inserted ' + currentDoc + ' of ' + recs);
            if (currentDoc === recs) {
                return console.log('inserted ' + currentDoc + ' in ' + moment.utc(moment(new Date(), "DD/MM/YYYY HH:mm:ss").diff(moment(start, "DD/MM/YYYY HH:mm:ss"))).format("HH:mm:ss"));
            }
            currentDoc++;
            save();
        });
}

db.collection('load', function (err, result) {
    if (err) {
        console.error(err);
        return;
    }
    collection = result;
    save();
});

UPDATE:
Only seems to happen when durability is set to soft and the waitForSync is set to false on the collection - I assume then this is an arangodb issue and not a driver issue? or possibly both?

Empty string convert to null automatically during db write

Hi,

I use arangojs to write some json data in DB like the following:

{ 
  "@id": "http://www.n-fuse.de/app.min.css",
  "@type": "http://www.n-fuse.de/classes/Object",
  "created": '2015-06-24T10:52:49.327Z',
  "size": 7714,
  "contentLanguage": "",
  "cacheControl": "",
  "contentDisposition": "",
  "contentEncoding": "7bit",
  "contentType": "text/css",
  "crc32c": "",
  "etag": "",
  "md5Hash": "",
  "name": "app.min.css",
  "prefix": "",
  "_key": "appmincss"
}

The Aql looks like this:

INSERT @node INTO @@collection LET nodes = NEW RETURN nodes

bindVars:

{ '@collection': 'docroot',
     node: 
      { '@id': 'http://www.n-fuse.de/app.min.css',
        '@type': 'http://www.n-fuse.de/classes/Object',
        created: '2015-06-24T10:52:49.327Z',
        creator: 'http://www.n-fuse.de/users/thomas.hoppe',
        size: 7714,
        contentLanguage: '',
        modified: '2015-06-24T10:52:49.327Z',
        cacheControl: '',
        contentDisposition: '',
        contentEncoding: '7bit',
        contentType: 'text/css',
        crc32c: '',
        etag: '',
        md5Hash: '',
        name: 'app.min.css',
        prefix: '',
        _key: 'appmincss' } }

However, after wrote into db, I found all properties with empty string is null instead. What's the reason?
I tried to use AQL explicitly with ArangoDB, it works. Therefore I guess it's an arangojs issue.

Connection management in express

What is the preferred method of managing connections in express.js? one connection at startup? Looking through the code, I couldn't see any connection pool management, but may have missed it.

Bluebird promises log unhandled rejection errors

Bluebird is being "helpful" by logging rejected promises that don't have rejection handlers.

This leads to console noise when promises are silently discarded in favour of callbacks (e.g. when db.collection or db.edgeCollection with autoCreate=true determines that a collection doesn't exist).

This problem only occurs when bluebird is used as a promise polyfill via global.Promise = require('bluebird').

Documentation is outdated

The documentation shown on the GitHub page (README.md) for this driver is wrong. This is quite frustrating for first time user. At least the source code is pretty well documented (although it would be good to have some kind of cookbook).

Add optional promise support

We will not ship a promise implementation with arangojs. One of the major concerns the 3.0 rewrite addressed was making the library a lot more lightweight -- adding more dependencies (or rolling our own implementations) is not going to happen.

However there is a rightful demand for a promise based API. I think it's possible to support promises as return values without relying on a polyfill or promise library directly for those who don't want to use them.

The most obvious solution would be to check for the existence of a global Promise function when an async function is called. This could be achieved using a simple helper function like this (using ES6 destructuring for readability):

function promisify (callback) {
  if (!callback) callback = noop;
  if (typeof Promise !== 'function') return {cb: callback};
  // ES6 Promise is supported
  var cb;
  var p = new Promise(function (resolve, reject) {
    cb = function (err, res) {
      if (err) reject(err);
      else resolve(res);
      callback(err, res);
    };
  });
  return {cb, p};
}

and used like this:

Database.prototype.createCollection = function (properties, callback) {
  var {cb, p} = promisify(callback);
  // … same logic as before …
  return p;
};

This would be entirely transparent for environments that don't support ES6 promises and result in miminal overhead in those that do. Non-ES6 environments could use a polyfill like es6-promise (require('es6-promise').polyfill();) or a custom ES6-compatible promise implementation like bluebird (global.Promise = require('bluebird');).

Support traversals

I'm not sure whether this is necessary as traversals can already be performed with AQL queries. I can't seem to find any documentation for traversals in ashikawa either.

@moonglum @fceller any thoughts on this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.